Thread: Incremental Back-up with Shell Scripting - HELP

Hi guys, I have to admit that I have been really banging my head agaist a brick wall over this whole back-up thing.

I'm supposed to come up with a shell script using "tar" that will carry out incremental back-ups on a daily basis. The files to be backed up are all located in the /home directory. The backed-up files can actually be saved anywhere for now.

Any help will be truly appreciated.

-W

you can use simple rsync command for incremental backup ..

like. rsync -av /folder_to_backup /destination


tell us more about your requirement so we can suggest you what type of backup is good for you..

Best
Nitin Bhadauria

You can use tools such as rsnapshot

1. Debian / Ubuntu Linux Install and Configure Remote Filesystem Snapshot with rsnapshot Incremental Backup Utility
2. How To Set Red hat / CentOS Linux Remote Backup / Snapshot Server
3. UNIX / Linux: Rsnapshot Restore Backups

You can create two scripts for daily and monthly backup and add them to cron.

monthly_backup.sh
#!/bin/bash
tar vzcfp /backup/home/home.tgz /home/

daily_backup.sh
#!/bin/bash
find /home -newer /backup/home/home.tgz -type f -print | tar zcvpf /backup/home/home_incremental.tgz -T -

You should try "disk array" solution.If does not help you then you should also try cluster technology. may be this gonna be useful for you.

Hi,

I have been searching for this, and came across this post and have a few wuestions:

1. Where do we include the remote host to get the files from? I want for example this:

rsync <options> root@remote-host:/var/www/* /path/to/local/backup/folder

Can i automate this, including the passwords to access the remote host?
Thanks in advance

Ezequiel

1. Where do we include the remote host to get the files from? I want for example this:
rsync <options> root@remote-host:/var/www/* /path/to/local/backup/folder

after installing rsnapshot you need to configure following entry in your "/etc/rsnapshot.conf"

Code:

backup  root@remote-host:/var/www/      remote-host/

Note :- You must be use TAB instead of space

then just Uncomment "cmd_ssh" in "/etc/rsnapshot.conf" and use following command to check your configuration

Code:

rsnapshot configtest

output should be :
Syntax OK

default backup is stored in "/var/cache/rsnapshot/" , you can modify if required in "/etc/rsnapshot.conf" using "snapshot_root" Tag

Can i automate this, including the passwords to access the remote host?

to configure password less login refer this link

Thanks Rahul,

I thought we were using rsync for this.

The scenario is the following:
We have some 27 servers across some cities and i need to backup (ONLY ONCE) certain folders on EACH of those locations.
We ar developing a big project and we are hiring a new developer however, before i give him root access to all those servers i want to have a backup of all the important configuration files/scripts and web server content.

I will login to each of those Nodes manually over ssh in order to get those contents.
We have a secure server that i will use to backup all the data from those servers and i want to do this with rsync over ssh.

So, basically i am just looking forward to have the perfect syntax to backup each folder.
NOTE that each folder has A LOT of files, i want to make sure all the files inside of a folder are copied.

If i need to "tar"the folders and then get them via scp, and you think is a better idea, please advise.

Again, thanks a lot.

Ezequiel

Hi,
try following script

PHP Code:

#!/bin/bash
# Authur :- Rahul Patil
# Pupose :- Backup remote server using rsync 

# Note :- SSH keys required on all server { Password less ssh access required}
 
SSH="/usr/bin/ssh"
ECHO="/bin/echo"
SCP="/usr/bin/scp"
RSYNC="/usr/bin/rsync"
TAR="/bin/tar"
LOGFILE="/var/log/copy.log"

# Path of Local Store
BACKUP_PATH="/path/to/local/backup/folder"

# Path of file/directories of remote server
REMOTE_PATH="/opt/test"

# remote user
USER="root"
 
# list of server seperated by space
# if you have list in file then you can use `cat /path/of/file` 
# for server in `cat /path/of/file`
for servers in 192.168.3.1 192.168.3.2
do
BACKUP_LOCATION="${BACKUP_PATH}/$servers"
echo $'\n\n' >> $LOGFILE
$ECHO "*************{ Backup Started on $server @$(date +%F)@ }**************************************" >> $LOGFILE
$ECHO $servers >> $LOGFILE

# if dir not exist then created them
[[ ! -d ${BACKUP_LOCATION} ]] && mkdir -p ${BACKUP_LOCATION}

# tar remote folder 
$SSH ${USER}@${servers} $TAR -czf /tmp/backup_$(date +%F).tar.gz ${REMOTE_PATH}

# move tar backup from remote server to local path
$RSYNC -av --remove-source-files ${USER}@${servers}:/tmp/backup_$(date +%F).tar.gz ${BACKUP_LOCATION} >> $LOGFILE

$ECHO "*************{ Backup End on $server @$(date +%F)@ }**************************************" >> $LOGFILE
done