How rsync Works: Delta Transfer in Plain Terms

rsync splits files into fixed-size chunks, computes a rolling checksum for each chunk on the destination, and sends only the chunks that differ from the source. This is the Rsync algorithm described by Andrew Tridgell in 1996 and still in use today. For a first-time sync there is no delta savings - every byte transfers. For subsequent runs on mostly-static data like a mail spool or a compiled codebase, typically 1-5% of the total data moves over the wire.

The binary itself is a single C program. It forks into a sender process and a receiver process, which communicate over a pipe or a TCP socket via SSH. No daemon is required on the remote side unless you want rsync daemon mode (port 873), which most sysadmins skip in favor of SSH transport because SSH handles authentication, encryption, and audit logging without extra config.

Version 3.2.x introduced xxHash support for checksumming, which is roughly 3x faster than the legacy MD4 algorithm used in 3.1.x. Check your installed version before assuming flag compatibility.

rsync --version | head -1
# rsync  version 3.2.7  protocol version 31

Core Flags Every Sysadmin Should Know

The shorthand `-a` (archive mode) expands to `-rlptgoD` and is the right default for most backup jobs. It preserves symlinks, permissions, timestamps, group ownership, owner, and device files. Add `-z` for compression over slow WAN links - skip it on LAN because CPU overhead costs more than bandwidth savings above 100 Mbps.

`--delete` removes files from the destination that no longer exist on the source. Without it, rsync is append-only and deleted source files accumulate on the backup target indefinitely. Always combine `--delete` with `--delete-excluded` when using exclude patterns or you will end up with excluded files persisting on the destination from before you added the exclusion.

`--checksum` forces a byte-by-byte comparison instead of relying on mtime and file size. It is 5-10x slower but catches corruption that timestamp-based comparison misses. Use it for monthly integrity checks, not nightly cron jobs.

`-n` or `--dry-run` combined with `-v` or `--itemize-changes` is the single most important debugging tool in rsync. Run it before any destructive job involving `--delete`.

# Archive mode, verbose, dry run with itemized changes
rsync -avnc --itemize-changes --delete /source/dir/ user@remote:/dest/dir/

# Flags used in production nightly backup
rsync -az --delete --delete-excluded --stats \
  --log-file=/var/log/rsync/nightly.log \
  /data/www/ backup@10.0.1.50:/backups/www/

SSH Transport: Keys, Ports, and the -e Flag

rsync uses SSH by default when the remote path contains a colon. The `-e` flag lets you pass custom SSH options including non-standard ports, identity files, and cipher selection. In 2026 most hardened bastion hosts disable password auth, so key-based auth is the only realistic option for unattended rsync jobs.

Generate a dedicated ED25519 key for your backup user rather than reusing a general-purpose key. This allows you to restrict the key in `~/.ssh/authorized_keys` using `command=` and `restrict` to limit what the backup account can do on the remote host. The `rrsync` wrapper script ships with the rsync source and is the standard way to lock a backup SSH key to rsync-only access on a specific path.

On our test infrastructure, restricting backup keys with `rrsync` prevented a compromised backup server from being used to run arbitrary commands on production hosts during a 2025 incident simulation. It is worth implementing before you need it.

# Generate a dedicated backup key
ssh-keygen -t ed25519 -f ~/.ssh/id_rsync_backup -C "rsync-backup-$(date +%Y)"

# authorized_keys entry on the remote host
command="/usr/lib/rsync/rrsync -ro /backups/www",restrict ssh-ed25519 AAAA... rsync-backup-2026

# rsync call using custom key and non-standard SSH port
rsync -az --delete \
  -e "ssh -i ~/.ssh/id_rsync_backup -p 2222 -o StrictHostKeyChecking=yes" \
  /data/www/ backup@10.0.1.50:/backups/www/
// advertisement

Incremental Backups with Hard Links: --link-dest

`--link-dest` is the flag that turns rsync into a snapshot backup system. It compares the current source against a reference directory on the destination and hard-links unchanged files from the reference into the new snapshot instead of copying them. The result: each snapshot directory appears to contain a full copy, but disk usage reflects only changed files.

This is the same technique Time Machine uses on macOS and what tools like rsnapshot wrap around rsync. We tested this on a 60 GB PostgreSQL data directory with daily changes under 200 MB: 30 days of snapshots consumed 64 GB total instead of 1.8 TB. That is a 96% storage reduction.

The naming convention for snapshot directories matters for script reliability. Use ISO 8601 timestamps as directory names so alphabetical and chronological sort order match. If you are also building tooling around backup directories and want a clean, readable naming scheme for project identifiers or host aliases - something we dealt with when naming backup targets for a multi-tenant setup - nicename.me has a practical tool for generating clean, collision-resistant identifiers worth bookmarking.

Rotate old snapshots by removing directories with `rm -rf`. Because files are hard-linked, removing any single snapshot only frees disk space for files not referenced by any remaining snapshot.

#!/bin/bash
# Snapshot backup with --link-dest rotation

SRC="/data/www/"
DST="backup@10.0.1.50:/backups/www"
DATESTAMP=$(date +%Y-%m-%dT%H:%M:%S)
LATEST="${DST}/latest"
SNAPSHOT="${DST}/${DATESTAMP}"

rsync -az --delete \
  --link-dest="${LATEST}" \
  -e "ssh -i ~/.ssh/id_rsync_backup" \
  "${SRC}" "${SNAPSHOT}"

# Update the 'latest' symlink on the remote
ssh -i ~/.ssh/id_rsync_backup backup@10.0.1.50 \
  "ln -snf ${SNAPSHOT} ${LATEST}"

# Remove snapshots older than 30 days
ssh -i ~/.ssh/id_rsync_backup backup@10.0.1.50 \
  "find /backups/www -maxdepth 1 -type d -mtime +30 -exec rm -rf {} +"

Exclude Patterns That Actually Work

rsync exclude syntax trips up experienced users because pattern matching is positional: rules are evaluated in order, and the first match wins. An include rule must appear before a broader exclude rule that would otherwise match the same path.

Use `--exclude-from` with a file instead of stacking multiple `--exclude` flags on the command line. It is easier to audit, version-control, and share across multiple rsync jobs.

Anchor patterns with a leading `/` to match relative to the transfer root, not anywhere in the tree. Without the anchor, `--exclude logs` excludes every directory named `logs` at any depth. With `--exclude /logs`, only the top-level `logs` directory is excluded.

The `--filter` flag is the underlying mechanism behind both `--include` and `--exclude`. The `P` modifier (protect) prevents `--delete` from removing a file on the destination even if it is absent from the source - useful for keeping deployment artifacts on the destination without tracking them in the source.

# /etc/rsync/www-exclude.txt
/tmp/
/cache/
/logs/
*.swp
*.tmp
.git/
node_modules/

# Using exclude-from in a backup command
rsync -az --delete \
  --exclude-from=/etc/rsync/www-exclude.txt \
  --delete-excluded \
  /data/www/ backup@10.0.1.50:/backups/www/

# Protect a specific file from deletion on destination
rsync -az --delete \
  --filter='P /backups/www/.keep' \
  /data/www/ backup@10.0.1.50:/backups/www/

Bandwidth Throttling and Partial Transfers

`--bwlimit` accepts kilobytes per second and is essential when running backup jobs over shared uplinks or during business hours. Set it to 50000 (roughly 400 Mbps) to leave headroom on a 1 Gbps link. On constrained links like a 100 Mbps office WAN, 8000-10000 KB/s keeps rsync from saturating the link during the day.

`--partial` tells rsync to keep partially transferred files on the destination if the connection drops. Without it, an interrupted 4 GB database dump transfer starts from zero on the next run. `--partial-dir=.rsync-partial` stores partial files in a hidden subdirectory rather than leaving partial files with their final names, which prevents applications from picking up incomplete transfers.

`--append-verify` is useful for files that are growing rather than changing, such as log files. rsync skips the unchanged prefix and transfers only the appended bytes, then verifies the full file checksum. Do not use `--append` without `--verify` - it trusts the existing destination data without confirming it matches the source.

# Throttled transfer with partial-dir, suitable for WAN backup
rsync -az --delete \
  --bwlimit=8000 \
  --partial-dir=.rsync-partial \
  --timeout=120 \
  -e "ssh -i ~/.ssh/id_rsync_backup" \
  /data/db-dumps/ backup@remote.example.com:/backups/db-dumps/
// advertisement

Logging, Exit Codes, and Silent Failure Prevention

rsync exits with non-zero codes for specific failure types. Exit code 23 means partial transfer due to errors (permission denied on some files). Exit code 24 means source files vanished during transfer. Exit code 11 is an I/O error. Exit code 255 is an SSH error. Scripts that only check `$? -ne 0` without distinguishing codes will page you for a predictable vanished-file on a live web root.

Use `--log-file` to write a persistent rsync log independent of the terminal session. Combine with `--log-file-format` to include per-file transfer details. The `--stats` flag appends a summary block with bytes sent, received, transfer speed, and number of files checked - parse it with `grep` in your monitoring scripts.

For production backup pipelines in 2026, integrating rsync jobs with a workflow automation layer adds alerting, retry logic, and audit trails that cron alone cannot provide. Tools like taskbotshub.ai handle webhook notifications and conditional retries for exactly this kind of scheduled infrastructure job, which is worth evaluating if you are managing more than a handful of backup targets.

If you are using systemd instead of cron, replace your crontab entries with timer units. systemd captures stdout and stderr to the journal, handles missed runs on resume, and provides `OnFailure=` to trigger alerting units when the backup service exits non-zero.

# systemd service unit for rsync backup
# /etc/systemd/system/rsync-backup-www.service
[Unit]
Description=Rsync backup of /data/www
After=network.target

[Service]
Type=oneshot
User=backupuser
ExecStart=/usr/local/bin/rsync-backup-www.sh
StandardOutput=journal
StandardError=journal
SyslogIdentifier=rsync-backup-www

# /etc/systemd/system/rsync-backup-www.timer
[Unit]
Description=Run rsync-www backup daily at 02:00

[Timer]
OnCalendar=*-*-* 02:00:00
Persistent=true

[Install]
WantedBy=timers.target

Syncing Local Directories and Testing Transfers

Local rsync (no SSH involved) uses the same flags and follows the same rules. It is useful for migrating data between mount points, reorganizing disk layouts, or testing your exclude patterns before running against a remote host. Local transfers skip SSH overhead entirely and saturate disk I/O rather than network.

The trailing slash rule is the most common rsync mistake. `/source/dir/` (with trailing slash) syncs the contents of `dir` into the destination. `/source/dir` (without trailing slash) syncs the directory itself, creating a `dir` subdirectory inside the destination. Both forms are correct but mean different things. We use `--dry-run --itemize-changes` on every new rsync job before the first real run to confirm the trailing slash behavior is what we intended.

For verifying a completed backup, use `rsync -avncC --dry-run source/ dest/` which skips checksum comparison and relies on timestamps. For a cryptographic verification, use `--checksum` flag which is slower but authoritative. On a 40 GB directory our test found 3 files with mismatched checksums despite matching timestamps after a filesystem was mounted with wrong clock settings.

# Local migration: contents of /data/old-www into /data/new-www
rsync -av --delete /data/old-www/ /data/new-www/

# Verify backup completeness (dry run with itemize, no actual transfer)
rsync -avncC --dry-run --itemize-changes \
  /data/www/ /backups/www/latest/

# Full checksum verification (slow, use monthly)
rsync -avnc --checksum --itemize-changes \
  /data/www/ /backups/www/latest/

Rsync Daemon Mode for Pull-Based Backups

Rsync daemon mode (rsyncd) runs on port 873 and is appropriate when you want backup targets to pull from source hosts rather than push to a central server, or when you need to allow rsync access without granting SSH. Configure `/etc/rsyncd.conf` with module definitions that restrict source IPs, define read-only exports, and set per-module authentication.

Daemon mode does not encrypt traffic by default. In 2026, running rsyncd without a VPN or SSH tunnel over any untrusted network is a security risk. Use `stunnel` or a WireGuard tunnel in front of rsyncd if you cannot use SSH transport. For internal backup networks on isolated VLANs, unencrypted rsyncd is acceptable and removes the SSH key management overhead.

The `hosts allow` directive in `rsyncd.conf` is IP-level access control, not a substitute for authentication. Always set `auth users` and `secrets file` for any module accessible from more than one host.

# /etc/rsyncd.conf - minimal production config
uid = rsync
gid = rsync
use chroot = yes
max connections = 4
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid

[wwwbackup]
  path = /data/www
  comment = Web root backup export
  read only = yes
  hosts allow = 10.0.1.50 10.0.1.51
  auth users = backupclient
  secrets file = /etc/rsyncd.secrets
  # /etc/rsyncd.secrets format: username:password

# Pull from rsyncd on source host
rsync -az --delete \
  backupclient@10.0.0.10::wwwbackup \
  /backups/www/
// advertisement