Bluetrait

As most of you know I changed over to FreeBSD for my web server a few days ago. Before that I was running Windows 2003. To backup my data I was simply using the built in "NT backup" in Windows with a scheduled task to run once a day. This backup would copy all the important data and replace yesterday’s backup (done nightly). Then once a month I would copy that backup to CD.

Now that worked well for about a year but there were two main problems

1. The backup would overwrite the previous backup. So this protected me against hardware failure but not an attack where someone deleted all the files.

2. The file that NT backup was making started to get far too big. Just before I moved to FreeBSD I was backing up just under 3 GB every night.

The other problem is simply that FreeBSD doesn't have "NT backup" :)

So I decided to rethink the way I do backups.

There are three main folders that store users data on my FreeBSD server (plus a few more for configuration, but I am just looking at backing up user data today)

/usr/home/
/var/db/mysql
/var/mail

The first is the user’s home directory. This contains their main website, log files and maybe a few other things.

The second directory is all the mysql databases. If a user have a mysql database it is stored here.

The last directory is mail that the user hasn’t moved to there home directory

Now instead of dumping all these directories to a single backup file nightly I decided to split the backup procedure.

I now only backup mail and mysql nightly, while I backup the home directory weekly.

The backup files are stored in /usr/backup and are also uploaded via ftp to my other server.

So let’s get down to setting this up.

1. First we’ll create the .tar.gz backup file
2. Then we’ll automatically upload it via ftp
3. Finally we’ll setup crontab to do it for us.

So to create the home directory backup we use this command:

tar cvvpzf /usr/backup/home-`date +'%Y%m%d'`.tar.gz /usr/home

(we will also be using:
tar cvvpzf /usr/backup/mysql-`date +'%Y%m%d'`.tar.gz /var/db/mysql
tar cvvpzf /usr/backup/mail-`date +'%Y%m%d'`.tar.gz /var/mail)

Let me explain what is does.

Tar: which really stands for tape archiver, but it will also write normal files.

cvvpzf = create file, verbose, verbose, preserve-permissions, gzip it, write to file

/usr/backup/home-`date +'%Y%m%d'`.tar.gz

This shows where to save the file to. Notice that we add the date to the end.

An example is:
home-20050319.tar.gz

/usr/home

Shows what to backup.

Okay so why don’t you run that script and make sure it works. You’ll need to have permissions to access each folder (root does). Or if you’re just backing up your personal home directory that will work too.

On a side note: You might like to add --ignore-failed-read to that command. This means the backup continues even if it cannot read a file.

Now we want to automatically upload this to our offsite ftp server. First we need to create a file called .netrc in our home directory.

.netrc is a script that is run when ever you connect to an ftp server.

*make sure the permissions for this are chmod: 600, otherwise it will not work*

So an example of .netrc

You need a blank line at the end of the file.

at the top we have machine ftp.example.com. This means the script will run when you connect to that server. You can also have an IP address there.

Login is your ftp login name, and password is the password for it.

Now we have two scripts here. A daily one and a weekly one. I’ll show how you call these later.

First you need: macdef. This is the command to start the script. Straight after this we have a command name “daily”.

Lcd /usr/backup (lcd stands for local change directory, we have already connected to the ftp server so cd will change the directory on the ftp connection)

put mysql-$1.tar.gz
put mail-$1.tar.gz

This will upload our backup files. Notice the $1? We’ll pass the date of the file to that later. i.e mysql-20050319.tar.gz

Okay once that is setup, save and close it. Remember the permissions for this file.

Once this is done let us setup a script that does it all for us. Create daily.sh and give it execute permissions.

First we have the standard #!/bin/sh

Then we have our tar commands to make the archive (note we’re not doing the home directory in this one)

Then we have: echo "\$ daily `date +'%Y%m%d'`" | ftp ftp.example.com

This echoes out $ daily ‘some date here’ to the ftp server at ftp.example.com.

The .netrc file picks up this command. It will run the daily script. And $1 is assigned the correct date.

When .netrc runs it will upload the correct mysql and mail backup. 

Now create a weekly.sh and put in:

So we have our two scripts. daily.sh and weekly.sh

Go can run ./daily.sh and watched it gzip the files and then upload them. If there are any errors, go back and look over your scripts.

Now the last thing to do is setup crontab to make this happen daily and weekly.

Lets create a file called crontab.

touch crontab.

Edit crontab

Now paste this in

Make sure you edit path-to-file to show where your script is. If it is in your home folder then. ~/daily.sh will do.

The first command runs daily at 3.01am

The second command runs on the 5th day of the week, at 4:15am

Save and close this file.

Now type:

crontab name-of-crontab-file (i.e crontab)

This copies that information into your cron setup. Now you should have two backups, one daily and one weekly.

That should be it! I might post a cut down version for people who want to backup their information off a server that they have shell access to.

A received an email that has some good tips on this