Space Machine made public

by relevantcircuits

Today, I am making the Space Machine shell script publicly available. Space Machine efficiently transfers backups from a Mac to a network attached storage (NAS) device either within a local network or over the internet.
In my previous post on February 26, I outlined my motivation and need for a frequent and fully automated offsite backup. Here, I am explaining the capabilities of the Space Machine shell script and how a backup job is configured and called. I will focus on the user perspective. To learn about the script’s inner workings, I would like to direct you to the script’s code. I added detailed comments there to facilitate modifications for others.

Space Machine simplifies rsync

Space Machine calls the file synchronizer rsync to backup files from a source on one device to a target on another at a set interval. When synchronization is done over a network, especially over the internet, the connection may easily be interrupted. One typical reason for an interruption could for example be the daily disconnection enforced by some internet service providers. When something like this happens, rsync quits before finishing its job. Space Machine then automatically recalls rsync to complete the synchronization.
To speed up synchronization over unstable connections, Space Machine offers the possibility to split up a big backup job into several smaller folder jobs. One folder job could for example be set up for your music and another for your photo library. In this case rsync is called separately for each folder job and completed jobs are ticked off. Should an interruption occur, synchronization resumes with the last unfinished job, saving the time to recheck all the data that has already been transferred.
As an additional feature, Space Machine can also create a folder named according to the current date within the target folder, into which the source folder is copied, instead of synchronized. This can be useful during work on a critical project which is better backed up daily. With date folders it is possible to go back in time to recover prior file versions. This backup over time is similar to Apple’s Time Machine, however, it does take up more storage capacity as the entire data, instead of just changes, are copied each time.

Running a backup

To run a backup job manually just enter ./spaceMachine.sh myBackup.job in a terminal (you may have to make the downloaded script executable first with the command chmod +x spaceMachine.sh).
In my next post, I’ll discuss how a backup job is added as a cron job. With cron, backups are initiated automatically in the background according to a predefined schedule. In fact, Space Machine is designed to be called by cron. When started, the script first assess whether a new backup is due based on the date of the last completed backup and the set interval. Thus, cron could call a backup job hourly, even if the actual backup only runs weekly. Running backups manually may nevertheless be reasonable in certain cases.
The script reads all relevant parameters from the job file. Script output goes to myBackup.log which can be followed (tail -f myBackup.log) to monitor the progress of an ongoing backup run. While being informative for the user, the log file also serves as the scripts working memory – so don’t make an effort to understand all of its content.
Rsync output goes to files with the following format: myBackup.date-time-jXcY.txt (X: folder job number, Y: number of folder job call). 
All output files are stored in the same folder as the script.

Configuring a backup job

The downloaded script is accompanied by a template job file (myBackup.job). You will find detailed comments on the different parameters there. Not all are essential. For some, a default setting is used if the parameter is omitted in the job file.

Real life

I would recommend to create a backup job for each set of folders that are supposed to be backed up at the same interval. For highly critical folders I usually create a dated backup job which creates a complete copy in a new subfolder named according to the current date.
Usually, you do not have to worry whether you are within your home Wifi or someplace else. Depending on your location the script either uses a local address or an internet domain name for the target NAS.
Even while being in a public Wifi network, a Growl notification from Space Machine once in awhile appeared on my desktop announcing the successful completion of a daily backup job. So if your Mac should go astray on your way home, your work will still and already be there on your NAS.

Configuring everything else

In the next posts, I will provide more details on how a Mac and a Synology Diskstation NAS server need to be configured for the script to be fully functional and to work without any user interference. Upcoming topics will include: 
  • Mac and Diskstation 
  • Mac only 
    • mail
      • essential to get notifications by email from a script running on a Mac
    • growlnotify
      • provides growl desktop notifications as an alternative to email messages
  • Diskstation only 
    • installing IPKG and bash
      • IPKG is a packet manager to install additional programs, such as the bash shell, on a Diskstation 
      • bash is needed as Space Machine is written in bash style 
    • nail
      • essential for email notifications from a script running on a Diskstation
    • scheduled power on and off of a remote Diskstation
      • recommended when a remote Diskstation is used for infrequent offsite backups from the local NAS
      • I have two small auxiliary scripts that solve the problem
    • configuring dynamic DNS
      • internet service providers usually do not issue fixed IP addresses to private customers 
      • a dynamic DNS provider updates the IP to which a domain links whenever it changes, so that a server, such as a home NAS, can always be reached via a fixed domain (such as nas.homeip.net) 
For the script to work in a most basic fashion, key-based SSH login is required. If you know how to set this up you can start using Space Machine right after this post. For now, I would also recommend to install growlnotify (and growl) on a Mac. The links above are of great help to configure everything that is needed to take maximum advantage of Space Machine. Some of the above sites have also been my primary resource. I am planning to add some details on the most critical steps in my next posts. 
Although I will limit my explanations to Macs and Synology Diskstation NAS devices, Space Machine should easily be portable to any Linux system and other Linux-based NAS servers.


get updates on twitter @elmarczeko
Copyright (c) 2013, Elmar Czeko
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

About these ads