Simple backup script

[Chris Rowson]

I've been tinkering with backups and backup rotation today and I have
come across many wierd and wonderful backup scripts of varying
complexity.

Is there anything wrong with using something simple like this? (except
of course for the lack of validation).

Basically, let's delete anything over 7 days old and then make a new backup.

Chris

#!/bin/bash
find /home/username/backup/*.tar.gz -mtime +7 -exec rm -f {} \;
/bin/tar -cpzf /home/username/backup/`date
+%a"-"%d"-"%b"-"%y"-"`backupfile.tar.gz /var/www

--
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-uk
https://wiki.ubuntu.com/UKTeam/

[Simon Greenwood]

On 2 June 2011 17:28, Chris Rowson <[hidden email]> wrote:

I've been tinkering with backups and backup rotation today and I have
come across many wierd and wonderful backup scripts of varying
complexity.

Is there anything wrong with using something simple like this? (except
of course for the lack of validation).

Basically, let's delete anything over 7 days old and then make a new backup.

Chris

#!/bin/bash
find /home/username/backup/*.tar.gz -mtime +7 -exec rm -f {} \;
/bin/tar -cpzf /home/username/backup/`date
+%a"-"%d"-"%b"-"%y"-"`backupfile.tar.gz /var/www

Nope, that's fine for something simple. Half the world probably backs up like that ;)

s/

--
Twitter: @sfgreenwood

"Is this your sanderling?"

--
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-uk
https://wiki.ubuntu.com/UKTeam/

[bodsda]

I would say it depends on what you mean by 'wrong'

I handle the backups for a local government, so yes, that is wrong, very wrong.

But it depends what you need it for. If you are happy with the retention that that gives you, then that's fine.

The only suggestion I would make is that you should create the backup first, and only if the backup is created successfully should you delete the old one

Hth,
Bodsda
Sent from my BlackBerry® wireless device

-----Original Message-----
From: Chris Rowson <[hidden email]>
Sender: [hidden email]
Date: Thu, 2 Jun 2011 17:28:57
To: British Ubuntu Talk<[hidden email]>
Reply-To: UK Ubuntu Talk <[hidden email]>
Subject: [ubuntu-uk] Simple backup script

I've been tinkering with backups and backup rotation today and I have
come across many wierd and wonderful backup scripts of varying
complexity.

Is there anything wrong with using something simple like this? (except
of course for the lack of validation).

Basically, let's delete anything over 7 days old and then make a new backup.

Chris

#!/bin/bash
find /home/username/backup/*.tar.gz -mtime +7 -exec rm -f {} \;
/bin/tar -cpzf /home/username/backup/`date
+%a"-"%d"-"%b"-"%y"-"`backupfile.tar.gz /var/www

--
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-uk
https://wiki.ubuntu.com/UKTeam/
--
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-uk
https://wiki.ubuntu.com/UKTeam/

[Chris Rowson]

On Jun 2, 2011 5:50 PM, <[hidden email]> wrote:
>
> I would say it depends on what you mean by 'wrong'
>
> I handle the backups for a local government, so yes, that is wrong, very wrong.
>

I know where you're coming from. I think the list has quite a few  public sector members in various capacities ;-)

> But it depends what you need it for. If you are happy with the retention that that gives you, then that's fine.
>

Sorry. I've probably given you the wrong impression. Data from the backup directory would be backed up too another box. That box in turn is backed up elsewhere. 7 days is the local retention only.

> The only suggestion I would make is that you should create the backup first, and only if the backup is created successfully should you delete the old one
>

Again, my fault, but the final script would run validation checks. I was basically asking if people use find rather than over complicating things as I've seen elsewhere. I wouldn't use that script as is.

Chris

--
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-uk
https://wiki.ubuntu.com/UKTeam/

[Neil Greenwood-2]

On 2 June 2011 18:07, Chris Rowson <[hidden email]> wrote:

>
> On Jun 2, 2011 5:50 PM, <[hidden email]> wrote:
>>
>> I would say it depends on what you mean by 'wrong'
>>
>> I handle the backups for a local government, so yes, that is wrong, very
>> wrong.
>>
>
> I know where you're coming from. I think the list has quite a few  public
> sector members in various capacities ;-)
>
>> But it depends what you need it for. If you are happy with the retention
>> that that gives you, then that's fine.
>>
>
> Sorry. I've probably given you the wrong impression. Data from the backup
> directory would be backed up too another box. That box in turn is backed up
> elsewhere. 7 days is the local retention only.
>
>> The only suggestion I would make is that you should create the backup
>> first, and only if the backup is created successfully should you delete the
>> old one
>>
>
> Again, my fault, but the final script would run validation checks. I was
> basically asking if people use find rather than over complicating things as
> I've seen elsewhere. I wouldn't use that script as is.
>
> Chris

I've started using rsnapshot. It does something similar with the local
retention, but each file which is identical (between retained backups)
is hard-linked rather than taking multiple disk blocks.

I set up the config file once (which took 2 attempts and maybe 2.5
hours in total), set up cron and it just runs.


The problem I find with hand-rolled backup scripts (which I've used in
the past) is that you're the one maintaining them...


Neil.

--
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-uk
https://wiki.ubuntu.com/UKTeam/

[Chris Rowson]

> I've started using rsnapshot. It does something similar with the local
> retention, but each file which is identical (between retained backups)
> is hard-linked rather than taking multiple disk blocks.
>
> Neil.
>

Unfortunately I'm backing the Linux box up to a Windows server. This
in turn gets backed up by a centralised backup system.

Chris

--
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-uk
https://wiki.ubuntu.com/UKTeam/

[James Tait]

On 02/06/11 17:28, Chris Rowson wrote:
> I've been tinkering with backups and backup rotation today and I have
> come across many wierd and wonderful backup scripts of varying
> complexity.
>
> Is there anything wrong with using something simple like this? (except
> of course for the lack of validation).

Nothing at all wrong with that.  I used something very similar as the
first step on a set of production servers a few years ago.  I've
recently discovered Déjà Dup, which is a simple front-end to
duplicity(1), which in turn has an impressive range of options.  It may
or may not be suitable for your purposes.

JT
--
---------------------------------------+--------------------------------
James Tait, BSc                        |    xmpp:[hidden email]
Programmer and Free Software advocate  |        Tel: [phone]
---------------------------------------+--------------------------------


--
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-uk
https://wiki.ubuntu.com/UKTeam/

[j0nr]

 On Thu, 2 Jun 2011 17:28:57 +0100, Chris Rowson wrote:
> I've been tinkering with backups and backup rotation today and I have
> come across many wierd and wonderful backup scripts of varying
> complexity.
>
 <snip>

 I use s3sync [0] to just sync my whole (chosen) directory/directories
 to an Amazon bucket once a week via a cron job.

 [0] http://s3sync.net/wiki

--


 Jon Reynolds (j0nr)
 http://www.jcrdevelopments.com

--
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-uk
https://wiki.ubuntu.com/UKTeam/

[Alan Lord (Gmail)]

On 02/06/11 17:28, Chris Rowson wrote:
> I've been tinkering with backups and backup rotation today and I have
> come across many wierd and wonderful backup scripts of varying
> complexity.
>
> Is there anything wrong with using something simple like this? (except
> of course for the lack of validation).

Nothing wrong with that.

 From a research perspective I would recommend that you take a look at a
rather old (but still functional script) I use called automysqlbackup[1].

It has some great demonstrations of various tricks to use in bash
scripting...

Cheers Al

[1]http://sourceforge.net/projects/automysqlbackup/



--
The Open Learning Centre
http://www.theopenlearningcentre.com


--
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-uk
https://wiki.ubuntu.com/UKTeam/

[Tony Travis-3]

On 02/06/11 17:28, Chris Rowson wrote:
> I've been tinkering with backups and backup rotation today and I have
> come across many wierd and wonderful backup scripts of varying
> complexity.
>
> Is there anything wrong with using something simple like this? (except
> of course for the lack of validation).
>
> Basically, let's delete anything over 7 days old and then make a new backup.

Hi, Chris.

I use this ETOH (Extended Towers of Hanoi) backup that I contributed to
NEBC Bio-Linux. It provides a baseline level 0 dump and three months of
incremental backups. You can get it as a deb from NEBC:

   http://nebc.nerc.ac.uk/tools/bio-linux/

Bye,

   Tony.
--
Dr. A.J.Travis, University of Aberdeen, Rowett Institute of Nutrition
and Health, Greenburn Road, Bucksburn, Aberdeen AB21 9SB, Scotland, UK
tel [phone], fax [phone], http://www.rowett.ac.uk
mailto:[hidden email], http://bioinformatics.rri.sari.ac.uk

--
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-uk
https://wiki.ubuntu.com/UKTeam/

[Chris Rowson]

Thanks so much for all of your suggestions guys.

I ended up with something like this which resulted in part from a need to get my backups onto a Windows Server and to get an email when it happens (or if it doesn't).


#!/bin/bash
BACKUPDIR="/home/USERNAME/backup"
WHATTOBACKUP="/var/www"
SERVERNAME="SERVERNAME"
BACKUPADMIN="[hidden email]"
MESSAGE="/tmp/message.txt"

if [ -d $BACKUPDIR -a -d $WHATTOBACKUP ] #make sure the source & dest dirs exist
then
       #backup the directory
       /bin/tar -cpzf $BACKUPDIR/`date +%a"-"%d"-"%b"-"%Y"-"`backup.tar.gz $WHATTOBACKUP
       #then remove anything over 7 days old
       find $BACKUPDIR/*.tar.gz -mtime +7 -exec rm -f {} \;
       #And let us know what happened
       SUBJECT="Backup Completed"
       TO=$BACKUPADMIN
       echo "Backup of $SERVERNAME completed" >> $MESSAGE
       echo "Result of backup" >> $MESSAGE
       echo "`ls -alt` $BACKUPDIR" >> $MESSAGE
       /usr/bin/mail -s "$SUBJECT" "$TO" < $MESSAGE
       rm $MESSAGE
else
       #if backup dir does not exist, tell us
       SUBJECT="Backup Failure"
       TO=$BACKUPADMIN
       echo "Error backing up $SERVERNAME" >> $MESSAGE
       echo "One of the following directories is missing: $BACKUPDIR $WHATTOBACKUP " >> $MESSAGE
       echo "Date: `date`" >> $MESSAGE
       /usr/bin/mail -s "$SUBJECT" "$TO" < $MESSAGE
       rm $MESSAGE
fi


--
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-uk
https://wiki.ubuntu.com/UKTeam/

[Chris Rowson]

I promise to stop spamming the list now! Final draft.

I know the email component should be put into a function but I'm a bad man and didn't do it!

Just added an extra section that checks to make sure that tar didn't throw an error code for some reason when it exited.

Chris


BACKUPDIR="/home/username/backup"
WHATTOBACKUP="/var/www"
SERVERNAME="servername"
BACKUPADMIN="[hidden email]"
MESSAGE="/tmp/message.txt"

if [ -d $BACKUPDIR -a -d $WHATTOBACKUP ] #make sure the source & dest dirs exist
then
        #backup the directory
        /bin/tar -cpzf $BACKUPDIR/`date +%a"-"%d"-"%b"-"%Y"-"`backup.tar.gz $WHATTOBACKUP
        RESULT=$? #grab the tar exit code
        if [ $RESULT -eq 0 ] #tar will return non-zero if an error occurs. If we have a 0 exit code continue
        then

                #then remove anything over 7 days old
                find $BACKUPDIR/*.tar.gz -mtime +7 -exec rm -f {} \;

                #And let us know what happened
                SUBJECT="Backup Completed"
                TO=$BACKUPADMIN
                echo "Backup of $SERVERNAME completed" >> $MESSAGE
                echo "Result of backup" >> $MESSAGE
                echo "`ls -alt` $BACKUPDIR" >> $MESSAGE
                /usr/bin/mail -s "$SUBJECT" "$TO" < $MESSAGE
                rm $MESSAGE
                exit

        else
                #send a mail if tar failed
                SUBJECT="Backup Failure"
                TO=$BACKUPADMIN
                echo "Backup of $SERVERNAME failed" >> $MESSAGE
                echo "Unable to create tar archive. Tar exit code was $RESULT" >> $MESSAGE
                /usr/bin/mail -s "$SUBJECT" "$TO" < $MESSAGE
                rm $MESSAGE
                exit

        fi
else
        #if backup dir does not exist, tell us
        SUBJECT="Backup Failure"
        TO=$BACKUPADMIN
        echo "Error backing up $SERVERNAME" >> $MESSAGE
        echo "One of the following directories is missing: $BACKUPDIR $WHATTOBACKUP " >> $MESSAGE
        echo "Date: `date`" >> $MESSAGE
        /usr/bin/mail -s "$SUBJECT" "$TO" < $MESSAGE
        rm $MESSAGE
fi



--
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-uk
https://wiki.ubuntu.com/UKTeam/

[Sean Miller]

We always used to do a daily backup, but Sundays we did a weekly one.

Daily backups (ie. Mon-Sat) were kept 7 days, Sunday backups were on a 4-week cycle.

And, of course, we actually backed up to somewhere else... not to the same system, as your script appears to do.

You could make it 8-weekly or something, if you think it will take you time to "discover problems", but we tended to find 4 weeks was sufficient.

Sean               

--
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-uk
https://wiki.ubuntu.com/UKTeam/

[Chris Rowson]

And, of course, we actually backed up to somewhere else... not to the same system, as your script appears to do.

It does. But read the rest of the thread and you'll see why, and what happens next ;-)

Chris

--
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-uk
https://wiki.ubuntu.com/UKTeam/

[Avi Greenbury-4]

Chris Rowson wrote:
> Unfortunately I'm backing the Linux box up to a Windows server. This
> in turn gets backed up by a centralised backup system.

I'd be tempted, in that case, to make things simple(r) for yourself if
you can and just rsync what would need backing up onto some share on the
Windows box, and join in the general backup regimen, unless there's
something peculiar to this data that means you need to back it up in
some way that the general backup scheme can't.

In general, the simpler the backup, the simpler the restore.

--
Avi

--
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-uk
https://wiki.ubuntu.com/UKTeam/

[Tony Arnold-3]

Chris,

Sorry if this has been mentioned before but have you tried sbackup?

Regards,
Tony.

On 03/06/11 14:49, Chris Rowson wrote:

> I promise to stop spamming the list now! Final draft.
>
> I know the email component should be put into a function but I'm a bad
> man and didn't do it!
>
> Just added an extra section that checks to make sure that tar didn't
> throw an error code for some reason when it exited.
>
> Chris
>
>
> BACKUPDIR="/home/username/backup"
> WHATTOBACKUP="/var/www"
> SERVERNAME="servername"
> BACKUPADMIN="[hidden email] <mailto:[hidden email]>"
> MESSAGE="/tmp/message.txt"
>
> if [ -d $BACKUPDIR -a -d $WHATTOBACKUP ] *#make sure the source & dest
> dirs exist*
> then
>         *#backup the directory*
>         /bin/tar -cpzf $BACKUPDIR/`date
> +%a"-"%d"-"%b"-"%Y"-"`backup.tar.gz $WHATTOBACKUP
>         RESULT=$? *#grab the tar exit code*
>         if [ $RESULT -eq 0 ] *#tar will return non-zero if an error
> occurs. If we have a 0 exit code continue*
>         then
>
>                *#then remove anything over 7 days old*
>                 find $BACKUPDIR/*.tar.gz -mtime +7 -exec rm -f {} \;
>
>               *  #And let us know what happened*
>                 SUBJECT="Backup Completed"
>                 TO=$BACKUPADMIN
>                 echo "Backup of $SERVERNAME completed" >> $MESSAGE
>                 echo "Result of backup" >> $MESSAGE
>                 echo "`ls -alt` $BACKUPDIR" >> $MESSAGE
>                 /usr/bin/mail -s "$SUBJECT" "$TO" < $MESSAGE
>                 rm $MESSAGE
>                 exit
>
>         else
>                *#send a mail if tar failed*
>                 SUBJECT="Backup Failure"
>                 TO=$BACKUPADMIN
>                 echo "Backup of $SERVERNAME failed" >> $MESSAGE
>                 echo "Unable to create tar archive. Tar exit code was
> $RESULT" >> $MESSAGE
>                 /usr/bin/mail -s "$SUBJECT" "$TO" < $MESSAGE
>                 rm $MESSAGE
>                 exit
>
>         fi
> else
>        *#if backup dir does not exist, tell us*
>         SUBJECT="Backup Failure"
>         TO=$BACKUPADMIN
>         echo "Error backing up $SERVERNAME" >> $MESSAGE
>         echo "One of the following directories is missing: $BACKUPDIR
> $WHATTOBACKUP " >> $MESSAGE
>         echo "Date: `date`" >> $MESSAGE
>         /usr/bin/mail -s "$SUBJECT" "$TO" < $MESSAGE
>         rm $MESSAGE
> fi
>
>

--
Tony Arnold,                        Tel: [phone]
Head of IT Security,                Fax: [phone]
University of Manchester,           Mob: [phone]
Manchester M13 9PL.                 Email: [hidden email]

--
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-uk
https://wiki.ubuntu.com/UKTeam/

[Chris Rowson]

> Chris,
>
> Sorry if this has been mentioned before but have you tried sbackup?
>
> Regards,
> Tony.
>

Hi Tony,

That's a GUI tool isn't it? This is to backup a Linux Server up to a
Windows Server (only you can't see the Windows Server bits).

Chris

--
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-uk
https://wiki.ubuntu.com/UKTeam/

[Chris Rowson]

> Chris Rowson wrote:
>>
>> Unfortunately I'm backing the Linux box up to a Windows server. This
>> in turn gets backed up by a centralised backup system.
>
> I'd be tempted, in that case, to make things simple(r) for yourself if you
> can and just rsync what would need backing up onto some share on the Windows
> box, and join in the general backup regimen, unless there's something
> peculiar to this data that means you need to back it up in some way that the
> general backup scheme can't.
>
> In general, the simpler the backup, the simpler the restore.
>
> --
> Avi
>

Hi Avi,

You're pretty much right. Centralised backup handles Win Server but I
need to backup the data on the Linux box too, hence shifting data. ATM
the Win box uses a script to 'pull' archive files from the Linux box
using pscp.exe. I looked briefly at rsync but it seemed to require
cygwin deps?

Chris

--
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-uk
https://wiki.ubuntu.com/UKTeam/

[Avi Greenbury-4]

Chris Rowson wrote:

> > Chris Rowson wrote:
> You're pretty much right. Centralised backup handles Win Server but I
> need to backup the data on the Linux box too, hence shifting data. ATM
> the Win box uses a script to 'pull' archive files from the Linux box
> using pscp.exe. I looked briefly at rsync but it seemed to require
> cygwin deps?

Yeah if you want to run rsync on the Win box you need cygwin or
similar. But you can rsync on the Linux box from one volume to another
(such as one mounted from the Windows box).  Windows' xcopy, though, is
roughly analogous to an rsync that's not talking to an rsyncd.


I'd do one of two things, depending on which is simpler and what your
network looks like.

I'd want to install Unix Services for Windows on the win server, and
mount over NFS the bit of the Linux box to be backed up.

If that's not possible - if the network's not isolated enough, or
whoever looks after the Win server doesn't want to add stuff to
it or whatever - perhaps configure Samba on the Linux box and do same
over Samba.

--
Avi

--
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-uk
https://wiki.ubuntu.com/UKTeam/

[Chris Rowson]

>
>> > Chris Rowson wrote:
>> You're pretty much right. Centralised backup handles Win Server but I
>> need to backup the data on the Linux box too, hence shifting data. ATM
>> the Win box uses a script to 'pull' archive files from the Linux box
>> using pscp.exe. I looked briefly at rsync but it seemed to require
>> cygwin deps?
>
> Yeah if you want to run rsync on the Win box you need cygwin or
> similar. But you can rsync on the Linux box from one volume to another
> (such as one mounted from the Windows box).  Windows' xcopy, though, is
> roughly analogous to an rsync that's not talking to an rsyncd.
>
>
> I'd do one of two things, depending on which is simpler and what your
> network looks like.
>
> I'd want to install Unix Services for Windows on the win server, and
> mount over NFS the bit of the Linux box to be backed up.
>
> If that's not possible - if the network's not isolated enough, or
> whoever looks after the Win server doesn't want to add stuff to
> it or whatever - perhaps configure Samba on the Linux box and do same
> over Samba.
>
>

In this instance I could mount a share from the Win box and backup
there. Unfortunately it's not always possible - dependent on as you
say - network security.

--
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-uk
https://wiki.ubuntu.com/UKTeam/