From WebarchDocs
Jump to: navigation, search

Help with using the Webarchitects backup space.

Accessing the backups

You can ssh to the backup server and find the backups in .zfs/snapshot/ but due to shell restrictions it is easier to use SSHFS to mount the backups on another machine and then browse them.

First set up SSH, generate a key pair for root (this can also be done as a regular user on a local machine but when mounting the backups on to a server to find things it is more convenient to do it as root):

<syntaxhighlight lang="bash"> sudo -i ssh-keygen -t rsa -b 4096 </syntaxhighlight>

Create /root/ssh/.config containing:

Host *
  CompressionLevel 9
  CheckHostIP yes
  StrictHostKeyChecking ask

Host backup
  User XXX

When XXX is your username.

Test the connection:

<syntaxhighlight lang="bash"> ssh backup </syntaxhighlight>

Install SSHFS, for example:

<syntaxhighlight lang="bash"> aptitude install sshfs </syntaxhighlight>

Mount the filesystem:

<syntaxhighlight lang="bash"> mkdir -p /media/backups chmod 700 /media/backups echo "sshfs#backup: /media/backups fuse ro 0 0" >> /etc/fstab mount -a </syntaxhighlight>

Backing up servers

We use a simple bash script and config file to backup files to the Webarchitects backup space.

To use this script first set up SSH as above, then create a /etc/agile/backup file containing a list of directories to backup, one per line, no leading slashes, for example:


Copy the following to /usr/local/bin/agile-backup and chomd 750 it.


<syntaxhighlight lang="bash">

  1. !/usr/bin/env bash
  1. List of local directories to be backed up
  2. is contained in this file, one per line
  3. eg
  4. /etc
  5. /root
  6. /home
  7. /var/backups
  8. /var/spool/cron/crontabs


  1. The hostname of this server, used to create
  2. a directory on the remote server


  1. The server to backup to, this is set in
  2. /root/.ssh/config


  1. This script uses lockfile which comes with procmail

LOCKFILE_BINARY="/usr/bin/lockfile" LOCKFILE="/var/run/lock/$(basename $0).lock"

  1. Write the outcome to this log

LOGFILE="/var/log/$(basename $0)"

  1. Check that the script is being run by root

if "$(id -u)" != "0"  ; then

 echo "You must run '$0' as root or via sudo" 
 exit 1


  1. Test if the lockfile binary can be found

if ! -e "$LOCKFILE_BINARY" ; then

 echo "$LOCKFILE_BINARY not found, please install the procmail package."
 exit 1


  1. if the $LOCKFILE exists then exit
  2. the lockfile is read only
  3. the timeout is set to 2 hours (7200 secs)
  4. if the lockfile is older than this it will be
  5. removed, this need to be improved so the log is updated

$LOCKFILE_BINARY -r 1 -l 7200 $LOCKFILE || exit 23

  1. Do the backup, first test that the file
  2. with the list of files to be backed up
  3. exists

if [[ -f ${BACKUP_LIST} ]]; then

 for dir in $(<${BACKUP_LIST}); do
   # Make the directory on the backup server
   ssh ${SSH_SERVER} mkdir -p ${HOSTNAME}${dir}
   # Copy the files to the backup server
   # Test for the scripts has been run with -v
   # for verbose output
   # bwlimit is set to 13 KBPS which is approx 0.1 megabit
   # so in the event of 10 backups running at the same time
   # on 10 servers the max bandwidth usage would be 1 megabit
   if $1 == "-v" ; then
     rsync -av --delete --bwlimit=13 ${dir}/ ${SSH_SERVER}:${HOSTNAME}${dir}/
     rsync -aq --delete --bwlimit=13 ${dir}/ ${SSH_SERVER}:${HOSTNAME}${dir}/ 2>&1
   # catch errors
   if [ $? != 0 ]; then
     echo "Backup Error at $(date +%c)" >> $LOGFILE
     if $1 == "-v" ; then
       rm -vf $LOCKFILE
       rm -f $LOCKFILE
     exit 23
   } fi


 echo "$BACKUP_LIST doesn't exist"
 echo "Backup Error at $(date +%c)" >> $LOGFILE
 if $1 == "-v" ; then
   rm -vf $LOCKFILE
   rm -f $LOCKFILE
 exit 23


  1. Remove the lock file and update the log

if $1 == "-v" ; then

 rm -vf $LOCKFILE


 rm -f $LOCKFILE

fi echo "Backup Success at $(date +%c)" >> $LOGFILE </syntaxhighlight>

The script can then be run to test it is working:

<syntaxhighlight lang="bash"> agile-backup -v </syntaxhighlight>

If is is then add a crontab via crontab -e:

<syntaxhighlight lang="bash"> 45 00 * * * /usr/local/bin/agile-backup </syntaxhighlight>

Note that it if you want to backup MySQL databases you can use backupninja to dump the MySQL databases as plain text and then backup the directory the dumps are in.

If you use the script about then you can mount the backups onto the server like this:

<syntaxhighlight lang="bash"> mkdir -p /media/backups chmod 700 /media/backups/ mkdir /media/backups/latest mkdir /media/backups/archive echo "sshfs#backup:$(hostname) /media/backups/latest fuse ro 0 0" >> /etc/fstab echo "sshfs#backup:.zfs/snapshot /media/backups/archive fuse ro 0 0" >> /etc/fstab mount -a </syntaxhighlight>