Syncing and Comparing Backup Drives with macOS

Backing up data is essential, but keeping multiple drives in sync and ensuring file integrity can be challenging. macOS Sequoia includes several built-in command-line utilities to manage, compare, and verify files effectively. Theres also Carbon Copy Cloner which you should just go ahead and use without hesitation. Its been around for decades, not quite as long as the 50 year old Unix diff utility but nevertheless it has earned its reputation as a rock solid graphical drive copying / cloning utility for macOS.

TL;DR – The best macOS tools for syncing and comparing backup drives include rsync for file transfer, diff for directory comparison, shasum for integrity checks, and fswatch for real-time monitoring. Automating these processes with cron - an old school but ubiquitous utility that runs commands or shell scripts periodically at fixed times, dates, or intervals can ensure backup reliability.

Carbon Copy Cloner

Yes, I know its not a command line tool, but really you should start here! Carbon Copy Cliner (CCC) is the gold standard. I've been using it for decades. The original author replied to me back about an issue I raised which was more about my understanding of APFS, Apple's 21st century file system, than it was about the tool, so thats an A+ support experience if you ask me. Carbon Copy Cloner is utterly reliable and freqently updated. Im just an end user but I can't recommend Carbon Copy Cloner enough.

Using rsync for Efficient File Synchronisation

rsync is one of the most reliable tools for copying and syncing files. It efficiently transfers only changed data, making it ideal for backups.

Basic Usage

% rsync -av /source/directory/ /destination/directory/ 

-a ensures permissions, timestamps, and symbolic links are preserved.
-v enables verbose output to track progress.

Incremental Backups

To copy only modified files:

% rsync -av --update /source/directory/ /destination/directory/ 

--dry-run flag can be used to preview changes before execution.

Comparing Directories with diff

Checking Directory Differences

Use diff to compare directory contents:

% diff -rq /dir1 /dir2 ``` - `-r` compares directories recursively. - `-q` reports only differing files.

Verifying Integrity with `shasum`

Generating and comparing checksums ensures file integrity.

% shasum -a 256 file1 > file1.sha256 shasum -c file1.sha256 ```

Finding Differences with comm and uniq

Comparing Sorted File Lists

% sh comm -3 <(ls /drive1 | sort) <(ls /drive2 | sort)

comm highlights unique and common entries.

# Identifying Duplicates

To find duplicate file entries:/p>

% sh uniq -d file_list.txt

Spotting Changes with `fswatch`

Monitoring File Modifications

% fswatch -o /backup-drive

fswatch provides real-time monitoring of file changes on macOS.

Advanced File Comparison with `rsync --dry-run`

# Testing rsync Changes

% sh rsync -avn /source/ /destination/

-n a dry-run shows what would have been transferred.

Automating Backup Verification with Cron Jobs and Scripts

# Scheduling a Check

Add a cron job to verify integrity weekly:

0 3 * * 1 shasum -c /backup/checksums.sha256 > /backup/logs.sha256

Summary

rsync, diff and shasum are built in to macOS Sequoia. fswatch is available via Homebrew See Installing wget with Homebrew on macOS. Carbin Copy Cloner is available at https://bombich.com/ These tools are helpful to ensure your synced copy backups are reliable. Automating these checks can reduce manual effort and prevents data loss. You should always have a second backup method in addition to Time Machine - just in case! Backups are only useful immediately after you need them so its really worth the effort to test that they work as expected.