Time Machine Backup to a Windows Share — Do it More Reliably

You can find plenty of instructions on the interwebs for setting up time machine to a network share, even a Windows share.

What you can't easily find is, how to do it reliably.

I have some recommendations.

1. Use cron scripts to just-in-time mount and dismount the time machine share

This is my first and biggest point. Dismounting the backup drive after each backup removes most of the reliability problem of network backup. Before doing this, I rarely got through 3 months without some kind of “the network share got dismounted uncleanly and now it won't mount until I run Disk First Aid on it”.

2. When it comes to data security, if you don't have 3 copies you aren't being serious.

This is a lesson you can take from cloud computing. Both Microsoft and Amazon clouds treat 'at least 3 copies' as the basic level for storing data. That means you want at least 2 independent backup systems for anything on your own machine.

If you combine this thought with the standard “don't put all your eggs in one physical location” motto of backup, you realise that you need a cloud or offsite backup as well as your time machine backup. The simplest free solution for your third copy, if 5GB is enough, is to use iDrive or OneDrive.

3. Buy a copy of Alsoft Disk Warrior

This is optional, and certainly less important than the first two points but, running Disk First Aid or fsck doesn't always work. Sad but true. I typically got a “fsck can't repair it properly” incident about once a year. I had a growing stack of hard disks with a year's worth of backup each, all only mountable readonly.
DiskWarrior has so far been reliable in restoring broken volumes back to fully working state. NB as of 2021 DiskWarrior can't yet repair APFS volumes so stay with HFS+ volumes for your time machines.

Help with cron scripts and multiple backups

Cron Scripts

Here are my script and cron table for mounting a TM drive from the network, requesting a backup, and dismounting the TM drive. It uses wakeonlan to wake the server from sleep, and ping to confirm it's up before trying to mount. It uses osascript to mount the volume because that deals with saving the network password in your keychain.

#! /usr/bin/env sh
#
# ------------------------------------------------------
# defineNamesAndPaths

    smbServer=myServerName
    smbServerfqdn=$smbServer.local
    smbServerMacAddress='your-server-mac-address-here'
    smbVolumeUrl="smb://$smbServerfqdn/D"
    tmDiskImageName='TM2023.sparsebundle'
    tmVolumeMountedAtPath='/Volumes/Time Machine Backups'
    mountedAtMaybePath1="/Volumes/$smbServerfqdn/Backups"
    mountedAtMaybePath2="/Volumes/D/Backups"  
    mountedAtMaybePath3=$(dirname $(find /Volumes -iname $tmDiskImageName -maxdepth 3 2>/dev/null | head -n 1) 2>/dev/null)

    declare -a maybeMountedPaths=("$mountedAtMaybePath1/$tmDiskImageName" "$mountedAtMaybePath2/$tmDiskImageName" "$mountedAtMaybePath3/$tmDiskImageName")

#-------------------------------------------------------
# helpAndExit
    if [[ "$1" == -h || "$1" == *help* ]] ; then
        echo "$0

mount a time machine backup diskimage from the network and kick off a backup.

-unmount : unmount the time machine volume.
-fsck    : attach the volume with -nomount and run fsck

Current settings:

Attach from : $smbVolumeUrl/$tmDiskImageName
Mount at    : $tmVolumeMountedAtPath

This script is admin-editable. Edit the script to set smbServer url
and mac address, the diskimage name, and the mounted path.

These are the current settings in $0 :
smbServer=$smbServer
smbServerfqdn=$smbServerfqdn #Used for ping
smbServerMacAddress=$smbServerMacAddress #Used for wakeonlan if available
smbVolumeUrl=$smbVolumeUrl
mountedAtMaybePath1=$mountedAtMaybePath1
mountedAtMaybePath2=$mountedAtMaybePath2
tmDiskImageName=$tmDiskImageName
tmVolumeMountedAtPath=$tmVolumeMountedAtPath

"
        exit
    fi

#-----------------------------------------------------------
# cron jobs get a very truncated path and can't find ping, diskutil, hdiutil, tmutil ...
    PATH="$PATH:/sbin:/usr/sbin:/usr/bin:/usr/local/bin"

echo '#-------------------------------------------------------'
date
echo "$0 $@"

#-------------------------------------------------------
# unmountAndExitIfRequested
if [[ "$1" == *unmount* ]] ; then
    tmutil status
    if [[ ! -d "$tmVolumeMountedAtPath" ]] ; then 
        echo "$tmVolumeMountedAtPath is already unmounted"
    elif [ -n "$(tmutil status | grep 'Running = 0')" ] ; then 
        echo "unmounting ..."
        /usr/sbin/diskutil unmount "$tmVolumeMountedAtPath"
    else
        echo "not unmounting $tmVolumeMountedAtPath because tm status says still running."
    fi
    exit
fi
# ---

#-------------------------------------------------------
# wakeAndConfirmPingableElseExit
#
# wakeonlan : I got mine from brew install – https://github.com/jpoliv/wakeonlan/blob/master/wakeonlan
# Otherwise try https://ddg.gg/bash%20script%20wakeonlan

    if [[ -x $(which wakeonlan) ]] ; then   wakeonlan $smbServerMacAddress ; fi
    for tried in {1..50} ; do ping -c 1 -t 5 $smbServerfqdn 2>&1 && break ; done
    if (( $tried == 50 )) ; then
        echo "failed to ping $smbServerfqdn. Exiting."
        exit
    fi
    sleep 10
    if (( $tried > 2 )) ; then 
        echo "waiting in case it was a cold-ish start..."
        sleep 10
    fi
# ---

#-------------------------------------------------------
# mount Network Volume but nomount sparsebundle and run fsck
    if [[ "$1" == *fsck* ]] ; then
        echo 'mounting ...'
        osascript -e 'mount volume "'$smbVolumeUrl'"'
        echo 'Trying locations to attach ...'
        ok=0
        for tryLocation in "${maybeMountedPaths[@]}"
        do
            echo trying to mount from $tryLocation …
            hdiutil attach $tryLocation -nomount && ok=1 && break
        done
        if [ $ok -eq 1 ] ; then 
            id=$(diskutil list | grep "Time Machine Backups" | grep -oE "disk\d+s\d+")
            echo Running fsk on /dev/$id
            fsck_hfs -y /dev/$id || exit
        fi
    fi
# --

#-------------------------------------------------------
# mountAndAttach
    echo 'mounting ...'
    osascript -e 'mount volume "'$smbVolumeUrl'"'
    echo 'Trying locations to attach ...'
    ok=0
    for tryLocation in "${maybeMountedPaths[@]}"
    do
        if [ -d $tryLocation ] ; then
            echo trying to mount from $tryLocation …
            hdiutil attach $tryLocation && ok=1 && echo "attached OK" && break
        else 
            echo not found at $tryLocation
        fi
    done
    [ $ok -eq 0 ] && \
           echo "failed to mount $tmDiskImageName at $mountedAtMaybePath1 or $mountedAtMaybePath2 or $mountedAtMaybePath3"
# ---

#-------------------------------------------------------
# requestTimeMachineBackup
    echo 'requesting backup.'
  tmutil startbackup --auto
  echo 'Done.'
# ---

And the crontab lines:

5  10-23/4 * * * /Users/chris/Applications/tmbackupnow.sh >> /Users/chris/Applications/Logs/tmbackupnow.log 2>&1
20,35 10-23/4 * * * /Users/chris/Applications/tmbackupnow.sh -unmount >> /Users/chris/Applications/Logs/tmbackupnow.log 2>&1

This schedule does 3 or 4 backups per working day on top of the local snapshots that time machine does anyway. Possibly this is overkill if you are squeezed for disk space. It tries to mount the TM machine and kick off a backup at 5 minutes past 10am,2pm,6pm,10pm and then tries to dismount the backup disk 15 minutes later and again another 15 minutes later. Adjust the timing to the size & speed of your backup.

At least 3 copies

My third copy, on top of time machine, is syncthing “continuous file synchronization”, which is great. It's like being able to set up a load of open source CloudDrives but using your local network too.

My fourth copy is either github or bitbucket for code; and iDrive or OneDrive for documents and graphics.

My fifth copy will be scripted backups to Azure storage, which seems like the cheapest way to do cloud backups. Meanwhile I'm paying Apple or Microsoft each month for big enough cloud storage.

You’re Only As Good As Your Last Backup

… is a necessary rule of thumb for computer-based knowledge & design workers. But add the lesson of cloud computing:

Backups: If you don't have 3 copies, you aren't serious.

The standard redundancy for cheap cloud storage options is 3 copies. Anything less is reduced redundancy, sold at discount. You should have at least 2 backups, for instance both a home backup disk and a cloud drive or repo.

A big win, when you plan for multiple copies, is that you no longer need any of them to be highly reliable. What matters more is, how fast can you make another copy if one copy goes down?

(Failing to) Copy a Time Machine Backup to a Network Drive with asr

The Apple support page for copying a Time Machine backup disk doesn't cover the scenario when your new backup target disk is on the network. If you try to do it by hand using cp, rsync, ditto or other, you will likely fail with inscrutable errors.

Using asr may work, but failed for me after 1 ½ days, 500GB, possibly because I had some kind of network disconnected. To rely on a network being reliable for 3 days is to ignore the 8 fallacies of distributed computing, but if your TM backup is small enough this could work.

  1. Use Disk Utility -> File -> New Image -> Blank Image … to create a new sparsebundle disk image on your network drive. The arrowed options must be set correctly (well, you don't have to use sparse bundle but it is allegedly designed specifically for efficient use across a network):

2.

2. Mount the new disk image by double-clicking it, and also attach your existing Time Machine backup drive. Then, use  -> About This Mac -> System Report… -> Hardware/Storage and look in the column BSD Name to find the device names on which your Source and Target volumes are mounted:

3. Turn off Time Machine backup. Usually by unticking “Back Up Automatically” in the Time Machine preferences, if there is no On/Off switch.

4. Then, use asr on the command line to copy the device that hosted the volume to the device hosting the new volume. Use caffeinate at the same time to stop the computer sleeping instead of copying. In my case that was:

sudo caffeinate asr restore --source /dev/disk3 --target /dev/disk4s2 --erase --puppetstrings --allowfragmentedcatalog

I got this output, and after a few seconds had to type y to confirm:

XSTA    start   512 client
XSTA setup
Validating target…done
XSTA metadata
Validating source…done
Erase contents of /dev/disk4s2 (/Volumes/LaCie2019)? [ny]:

The --puppetstrings option means what most of us might call --progress although the output is quite limited.

Expect a speed of about 4 days per terabyte. I don't know why. Watching the Network tab in Activity Monitor I can see that data is rarely going faster than 5MB/s. Even writing to a spinning disk across a 20 year old 100Mbps network should go faster than that. I tried adding --buffers 10 --buffersize 100MB, but that still only got me to about 3 days per terabyte.

Anyway …

For me it failed. Sorry I lost the error message. So I went to to Finder drag-n-drop. The first time this failed after a day; the second it succeeded after 3 days. 🤷‍♂️

Use NSSM to install SyncThing as a Windows service

SyncThing does what OneDrive & Google Drive can do but under your control, across your machines, with more options, and without having to touch a 3rd party data snooping provider and without having to pay 3rd party Terabyte rates. I use it on my home network both to synchronise configuration across multiple machines and as an at-home backup solution. It's fast, simple, well-maintained and it works.

NSSM is “the Non-Sucking Service Manager” which has a simple GUI to set up commandline programs like SyncThing as a Windows Service.

Install SyncThing

To use SyncThing as a Service, avoid the GUI options such as SyncTrayzor and go for the GitHub download. Choose a directory to install to, such as your Program Files directory.

SyncTrayzor is great for your working machine, where you only need SyncThing to run when you are logged in. For a server which is hosting backups and redundant copies of your files, you want a Windows service running whenever the machine is up.

Install NSSM

NSSM also has no installer as of early 2020. Download & extract to a Program Files directory.

I then added New-Alias nssm "C:\Program Files\nssm-2\win64\nssm.exe" to my PowerShell profile

Launch NSSM

nssm without parameters will show you the commands you can use. The simplest is to use install & edit to get the GUI:
To show service installation GUI: nssm install [<servicename>]
To show service editing GUI: nssm edit <servicename>

So use:

nssm install SyncThing

And then fill in the boxes by finding the path where you installed SyncThing. I only edited the first three tabs: Application, Details, and Log On. The rest can stay as default.

What about the Parameters? See the SyncThing Docs. This is mine:

-no-console -no-browser -no-restart -gui-address=localhost:8384

-no-console -no-browser are because services run headless.
-no-restart because the Windows Service infrastructure has options for handling restarts.
-gui-address=localhost:8384 to make the gui console only available on localhost, not across the network. You may not want this.

You can now use nssm to start/stop/monitor services, not just the ones you have installed with it.

nssm start SyncThing
nssm status SyncThing

Or, you can use the standard Windows Services gui.

Where is the config?

Nssm just edits the Windows service config, which is visible in the Local Services app, which you can launch from Task Manager -> Services

SyncThing keeps config in the place noted in SyncThing Docs unless you add e.g. -home=D:\MyPath to the startup parameters

Where is the SyncThing Gui?

If you followed my example and used -gui-address=localhost:8384 then open that address in your browser and read all about at https://docs.syncthing.net/intro/gui.html

More Options?

See https://docs.syncthing.net/users/syncthing.html

Yes but I want to manage it across my home network?

  1. Change the startup options to use -gui-address=0.0.0.0:8384.
  2. Add the full path to SyncThing.exe as a firewall exception in your Windows firewall.
  3. Restart the service

This will make the browser interface accessible across the network. Then:

  1. Open the the GUI at localhost:8384.
  2. Open the Settings (under the Actions menu, top right).
  3. Open the GUI panel.
    1. Choose HTTPS
    2. Add a username and password. NB I think these are both case sensitive.