My overly complicated backup script

Every night, my server fires off a backup. This script does a backup of my web server directory, MySQL databases, my check book, and a directory that’s shared over the network to upload to the cloud.

The main backup is called by backup.sh. In that script, there are a few commands to bring down Apache and cron. Then it continues on and calls a few more backup scripts.

1) backup.sh – This sets up the backup and calls the other scripts that perform the backups

#!/bin/bash -x

# Clear out tmp folder, in case previous backup files exist
cd /tmp
rm /tmp/*

# Check mount points
mount /mnt/Warehouse13
mount /mnt/box

# Stop Apache and cron
service apache2 stop
service cron stop

# Create blank email to send
/etc/./new_email.sh

# Call additional backup scripts
/etc/./wwwsync.sh
/etc/./mysqlbackup.sh
/etc/./acctbackup.sh
/etc/./cloudbackup.sh
/etc/./removeold.sh
/etc/./wwwbackup.sh

# Start cron
service cron start

# Start Apache Webserver
service apache2 start

# Send email
/etc/./send_email.sh

2) new_email.sh – Creates a blank email to record messages during the backup

#!/bin/bash -x

# Create blank email to send
touch /tmp/backup.mail
echo 'To: youremail@bogusdomain.com' >> /tmp/backup.mail
echo 'From: server@yourdomain.com' >> /tmp/backup.mail
echo 'Subject: Backup Results '$fileDate'' >> /tmp/backup.mail
echo '' >> /tmp/backup.mail
echo '' >> /tmp/backup.mail

3) wwwsync.sh – Syncs the directory that holds the web documents using rsync. Only uploads new or changed files.

#!/bin/bash

# Section to backup WWW directory syncing only new or changed files
rsync -rtvzc /var/aprsnew/ /mnt/box/srv/aprs/
rsync -rtvzc /mnt/ext/srv/www/ /mnt/box/srv/www/
rsync -rtvzc /mnt/ext/home/ameyer/ /mnt/box/srv/ameyer/
echo 'WWW Directory Sync Complete' >> /tmp/backup.mail

4) mysqlbackup.sh – Performs a complete dump of the MySQL database. Does a MD5 comparison to see if it needs to upload a new version.

#!/bin/bash

# Set date variables
fileDate=$(date +%m-%d-%Y)

# Section to backup MySQL
mysqldump --skip-comments -u<user> -p<pass> --all-databases > /tmp/alldatabases.sql
localSQL=$(md5sum '/tmp/alldatabases.sql' | cut -b-32)
cloudSQL=$(md5sum '/mnt/box/srv/sql/alldatabases.sql' | cut -b-32)
if [ "$localSQL" = "$cloudSQL" ]; then
     echo 'MySQL File is Current - Backup Skipped' >> /tmp/backup.mail
     rm /tmp/alldatabases.sql
else
     mv '/mnt/box/srv/sql/alldatabases.sql' '/mnt/box/srv/sql/alldatabases.'$fileDate'.sql'
     mv '/tmp/alldatabases.sql' '/mnt/box/srv/sql/alldatabases.sql'
     echo 'MySQL Successfully Backed Up' >> /tmp/backup.mail
fi

5) acctbackup.sh – Creates a backup of the checkbook and purges old backups

#!/bin/bash

# Set date variables
fileDate=$(date +%m-%d-%Y)

# Create a copy of the local file appending date 
cp '/mnt/Warehouse13/Accounts/2014/Household_Accounts_2014.xls' '/mnt/Warehouse13/Accounts/2014/backups/Household_Accounts_2014_'$fileDate'.bak'
accountFileCount=$(find /mnt/Warehouse13/Accounts/2014/backups/ -maxdepth 1 -type f -print | wc -l)

# If more than 10 backups exist eliminate excess backups over 10
if [ $accountFileCount -gt 5 ]; then
     find /mnt/Warehouse13/Accounts/2014/backups/ -mtime +5 -exec rm {} \;
     echo 'Old Account Backups Purged' >> /tmp/backup.mail 
fi

# Sync local account folder to online folder 
rsync -rtvzc --delete '/mnt/Warehouse13/Accounts/' '/mnt/box/Accounts/'

# Append to email notification and send notification of completion
echo 'Account Records Backup Complete' >> /tmp/backup.mail

6) cloudbackup.sh – This is a shared network folder that is synced only uploading new or changed files and deleting removed files.

#!/bin/bash

rsync -rtvzcu --delete '/mnt/Warehouse13/cloud_backup/' '/mnt/box/cloud_backup/'
echo 'Sync complete of cloud backup directory' >> /tmp/backup.mail

7) removeold.sh – Removes old backup files

#!/bin/bash

# Creates a count of how many files exist
sqlFileCount=$(find /mnt/box/srv/sql/ -maxdepth 1 -type f -print | wc -l)
wwwBackupCount=$(find /mnt/box/srv/http-weekly/ -maxdepth 1 -type f -print | wc -l)
boxAccountBackupCount=$(find /mnt/box/Accounts/2014/backups -maxdepth 1 -type f -print | wc -l)
localAccountBackupCount=$(find /mnt/Warehouse13/Accounts/2014/backups -maxdepth 1 -type f -print | wc -l)

# If there are more than 10 files, it deletes any old files
if [ $sqlFileCount -gt 10 ]; then
     find /mnt/box/srv/sql -mtime +10 -exec rm {} \;
     echo 'Old MySQL Backups Purged' >> /tmp/backup.mail
fi

if [ $wwwBackupCount -gt 10 ]; then
     find /mnt/box/srv/http-weekly -mtime +90 -exec rm {} \;
     echo 'Old Weekly SRV Backups Purged' >> /tmp/backup.mail
fi

if [ $boxAccountBackupCount -gt 10 ]; then
     find /mnt/box/Accounts/2014/backups -mtime +10 -exec rm {} \;
     echo 'Old Online Bank Record Backups Purged' >> /tmp/backup.mail
fi

if [ $localAccountBackupCount -gt 10 ]; then
     find /mnt/Warehouse13/Accounts/2014/backups -mtime +10 -exec rm {} \;
     echo 'Old Local Bank Record Backups Purged' >> /tmp/backup.mail
fi

8) wwwbackup.sh – Checks if a week has elapsed and performs a complete backup of the html files, splitting them into 50MB chunks to upload to Box.com

#!/bin/bash

# Set date variables
oldDate=$(sed -n 1p /var/date_check.txt)
epochDate=$(date +%s)
dateCheck=$(expr $epochDate - $oldDate)
fileDate=$(date +%m-%d-%Y)

# Compares epoch date to stored date and if more than a week has elapsed, it performs the backup
if [ $dateCheck -gt 604799 ]; then
     tar czf '/tmp/www.'$fileDate'.tar.gz' '/mnt/ext/srv/www'
     tar czf '/tmp/ameyer.'$fileDate'.tar.gz' '/mnt/ext/home/ameyer' 
     split -b 50m '/tmp/www.'$fileDate'.tar.gz' '/tmp/www.'$fileDate'.tar.gz.part-'
     split -b 50m '/tmp/ameyer.'$fileDate'.tar.gz' '/tmp/ameyer.'$fileDate'.tar.gz.part-'
     rm '/tmp/www.'$fileDate'.tar.gz'
     rm '/tmp/ameyer.'$fileDate'.tar.gz'
     mv /tmp/www.* /mnt/Warehouse13/srv/http-weekly/
     mv /tmp/ameyer.* /mnt/Warehouse13/srv/http-weekly/
     echo $epochDate > /var/date_check.txt
     echo 'Complete Backup of WWW was executed and completed' >> /tmp/backup.mail
else
     echo 'Complete Backup of WWW was not scheduled to run' >> /tmp/backup.mail
fi

9) send_email.sh – Sends the email with results and free space information

#!/bin/bash -x

# Send email with backup results and include free memory and free space.
echo '' >> /tmp/backup.mail
echo '' >> /tmp/backup.mail
free >> /tmp/backup.mail
echo '' >> /tmp/backup.mail
echo '' >> /tmp/backup.mail
df >> /tmp/backup.mail
cat /tmp/backup.mail | msmtp -a default youremail@bogusdomain.com

Leave a Reply

Your email address will not be published. Required fields are marked *