nixCraft

meowing · #1 (**permalink**) 02-02-2007, 09:51 PM

I'm running a CentOS 4.4 server. Started using one of your scripts, adapted it to my needs. It works fine started via SSH (putty), but somehow it fails when I run it from crontab.

Here's 1 of the backup-scripts ( /root/io/back.sh ):

Code:

#!/bin/sh
# Full backup day Sunday (others incremental backup)

DIRS="/etc /usr/share/squirrelmail /w"
BACKUP=/tmp/backup.$$
NOW=$(date +"%d-%m-%Y")
INCFILE="/root/io/tar-inc-backup.dat"
DAY=$(date +"%a")
FULLBACKUP="Sun"

### FTP server Setup ###
FTPD="/backups/barbara/incremental"
FTPU="some username"
FTPP="some password"
FTPS="ftp.somehost.net"
NCFTP="/usr/local/bin/ncftp"

### Other stuff ###
EMAILID="@xs4all.nl"

### Backup local ###
[ ! -d $BACKUP ] && mkdir -p $BACKUP || :

### See if it's full backup day ###
if [ "$DAY" == "$FULLBACKUP" ]; then
  FTPD="/backups/barbara/full"
  FILE="fs-full-$NOW.tar.gz"
  tar -zcvf $BACKUP/$FILE $DIRS
  ccrypt -e -k /root/somekey $BACKUP/$FILE
else
  i=$(date +"%Hh%Mm%Ss")
  FILE="fs-i-$NOW-$i.tar.gz"
  tar -g $INCFILE -zcvf $BACKUP/$FILE $DIRS
  ccrypt -e -k /root/soemkey $BACKUP/$FILE
fi

### Dump backup using FTP ###
#Start FTP backup using ncftp
$NCFTP -u"$FTPU" -p"$FTPP" $FTPS<<EOF
mkdir $FTPD
mkdir $FTPD/$NOW
cd $FTPD/$NOW
lcd $BACKUP
mput *
quit
EOF

### Find out if ftp backup failed or not ###
if [ "$?" == "0" ]; then
 rm -f $BACKUP/*
else
 T=/tmp/backup.fail
 echo "Date: $(date)">$T
 echo "Hostname: $(hostname)" >>$T
 echo "Backup failed" >>$T
 mail  -s "BACKUP FAILED" "$EMAILID" <$T
 rm -f $T
fi

and here's part of /etc/crontab where only the last line is significant here :

Code:

SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
HOME=/

# run-parts
56 0,1,3,4,7,9,11,13,15,17,19,21,22,23 * * * root run-parts /etc/cron.hourly
58 2,5,10,14,18,20,22 * * * root run-parts /etc/cron.x
38 5 * * * root run-parts /etc/cron.daily
10 6 * * 1 root run-parts /etc/cron.weekly
35 5 1 * * root run-parts /etc/cron.monthly
# the script runner:
49 5 * * * root /root/io/back.sh >/dev/null 2>&1

Any idea why it works from commandline, but doesn't work as cron job?

I had to manually install ncftp (was not in yum) and I'm transferring encrypted content only (using ccrypt, which I did install via yum)

I get an e-mail (as scripted), about 6 seconds after 5:49, and from looking in my /tmp I could see the tars are ready and encrypted in a folder called /backup xxxxx/ where xxxxx is a random number I have no idea what it is based on. By the way, the folders under /tmp don't get deleted, not even after a succesful backup upload. Its content is deleted after succesful backup.

Concluding it means that it fails at the ftp start, login or transfer part.
How do I best debug this? Is there a way to log exactly what happens?

I'm not a unix/linux novice but I have no idea what is causing this. The ftp backup locations seem to have been up, but unfortunately I can't check their logs.
By the way, I noticed that ncftp is echo-ing a nice banner when the script is started from commandline.

Thanks in advance for any ideas/insight.

nixcraft · #2 (**permalink**) 02-03-2007, 12:52 AM

Try to change

From:

Code:

ncftp -u"$FTPU" -p"$FTPP" $FTPS<<EOF

To:

Code:

$NCFTP -u"$FTPU" -p"$FTPP" $FTPS<<EOF

To debug a script start as follows:

Code:

bash -x /root/io/script.sh

Let me know if you need any further help

meowing · #3 (**permalink**) 02-05-2007, 05:53 PM

The change with NCFTP in the script did not make a difference, unfortunately.

Quote:

Originally Posted by nixcraft

To debug a script start as follows:

Code:

bash -x /root/io/script.sh

That gives this, probably meaning ncftp doesn't understand the command given:
(irrelevant part cut out)

Code:

+ /usr/local/bin/ncftpput -uusername -ppassword ftp.host.net
NcFTPPut 3.2.0

Usages:
  ncftpput [flags] remote-host remote-dir local-files...   (mode 1)
  ncftpput -f login.cfg [flags] remote-dir local-files...  (mode 2)
  ncftpput -c remote-host remote-path-name < stdin         (mode 3)
  ncftpput -C remote-host local-path-name remote-path-name (mode 4)

Flags:
  -u XX  Use username XX instead of anonymous.
  -p XX  Use password XX with the username.
  -P XX  Use port number XX instead of the default FTP service port (21).
  -j XX  Use account XX with the account (deprecated).
  -d XX  Use the file XX for debug logging.
  -e XX  Use the file XX for error logging.
  -U XX  Use value XX for the umask.
  -t XX  Timeout after XX seconds.
  -a     Use ASCII transfer type instead of binary.
  -m     Attempt to mkdir the dstdir before copying.
  -v/-V  Do (do not) use progress meters.
  -f XX  Read the file XX for host, user, and password information.
  -h XX  Connect to host XX.  Useful for overriding host in -f config.file.
  -c     Read locally from stdin and write remotely to specified pathname.
  -C     Similar to -c, except a local pathname is specified.
  -A     Append to remote files instead of overwriting them.
  -z/-Z  Do (do not) try to resume uploads (default: -Z).
  -T XX  Upload into temporary files prefixed by XX.
  -S XX  Upload into temporary files suffixed by XX.
  -DD    Delete local file after successfully uploading it.
  -b     Run in background (submit job to "ncftpbatch" and run).
  -bb    Same as "-b" but queue only (do not run "ncftpbatch").
  -E     Use regular (PORT) data connections.
  -F     Use passive (PASV) data connections (default).
  -y     Try using "SITE UTIME" to preserve timestamps on remote host.
  -B XX  Try setting the SO_SNDBUF size to XX.
  -r XX  Redial XX times until connected.
  -o XX  Specify miscellaneous options (see documentation).
  -W XX  Send raw FTP command XX after logging in.
  -X XX  Send raw FTP command XX after each file transferred.
  -Y XX  Send raw FTP command XX before logging out.
  -R     Recursive mode; copy whole directory trees.

Examples:
  ncftpput -u gleason -p my.password Elwood.probe.net /home/gleason stuff.txt
  ncftpput -u gleason Elwood.probe.net /home/gleason a.txt (prompt for pass)
  ncftpput -a -u gleason -p my.password -m -U 007 Bozo.probe.net /tmp/tmpdir a.txt
  tar cvf - /home | ncftpput -u operator -c Server.probe.net /backups/monday.tar

Library version: LibNcFTP 3.2.0 (August 5, 2006).

This is a freeware program by Mike Gleason (http://www.ncftp.com).
This was built using LibNcFTP (http://www.ncftp.com/libncftp/).
+ '[' 8 == 0 ']'
+ T=/tmp/backup.fail

When I removed the <<EOF from the end of the non-functional command,
it gave me the same, but added to it:
(irrelevant part cut out)

Code:

This was built using LibNcFTP (http://www.ncftp.com/libncftp/).
+ mkdir /backups/barbara/full
mkdir: cannot create directory `/backups/barbara/full': No such file or directory
+ mkdir /backups/barbara/full/05-02-2007
mkdir: cannot create directory `/backups/barbara/full/05-02-2007': No such file or directory
+ cd /backups/barbara/full/05-02-2007
/root/io/back.sh: line 43: cd: /backups/barbara/full/05-02-2007: No such file or directory
+ lcd /tmp/backup.21722
/root/io/back.sh: line 44: lcd: command not found
+ mput back.sh bam.sh sg.sh tar-inc-back.dat tar-inc-backup.dat tar-inc-bakup.dat
/root/io/back.sh: line 45: mput: command not found
+ quit
/root/io/back.sh: line 46: quit: command not found
+ EOF
/root/io/back.sh: line 47: EOF: command not found
+ '[' 127 == 0 ']'
+ T=/tmp/backup.fail

This is logical since there is no ftp login there.
Again, I'm out of ideas here..

nixcraft · #4 (**permalink**) 02-05-2007, 11:00 PM

You need to use /usr/local/bin/ncftp and not the /usr/local/bin/ncftpput
Change From

Code:

NCFTP="/usr/local/bin/ncftpput"

To:

Code:

NCFTP="/usr/local/bin/ncftp"

Don't remove EOF it is a requirement. I hope you have ncftp installed .. let me know

meowing · #5 (**permalink**) 02-06-2007, 08:25 AM

Yes, I think this was the culprit.

I don't even know why I had ncftpput in there, to be honest.

The debugging command shows it's doing exactly what it should do.

I'll let you know if after this morning's cronjob backup I received a failed warning again, but I don't expect one any more.

Thanks!

thunderbirdsei · #6 (**permalink**) 04-12-2007, 12:51 AM

Do I really have to do it this way or is there a way to transfer a folder with all subfolder and the files with FTP?

Code:

# FTP Transfer
	cnftp=”/usr/bin/ncftp”
	FTPU="username"
	FTPP="password"
	FTPA="xx.xx.xx.xx"

	ncftp -u"$FTPU" -p"$FTPP" "$FTPA"<<EOF

	mkdir /daily
	mkdir /daily/gaga1
	mkdir /daily/gaga2

	mkdir /weekly
	mkdir /weekly/gaga1
	mkdir /weekly/gaga2

	mkdir /monthly
	mkdir /monthly/gaga1
	mkdir /monthly/gaga2

	cd /backup/daily/gaga1
	lcd /home/backup/daily/gaga1
	mput *

	cd /backup/daily/gaga2
	lcd /home/backup/daily/gaga2
	mput *

	cd /backup/weekly/gaga1
	lcd /home/backup/weekly/gaga1
	mput *

	cd /backup/weekly/gaga2
	lcd /home/backup/weekly/gaga2
	mput *

	cd /backup/monthly/gaga1
	lcd /home/backup/monthly/gaga1
	mput *

	cd /backup/monthly/gaga2
	lcd /home/backup/monthly/gaga2
	mput *

	quit
	EOF

meowing · #7 (**permalink**) 04-28-2007, 10:20 PM

Darn! The cronjob stopped working again, and I have no idea why.

Here's the crontab concerning this:

Code:

SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
HOME=/

# run-parts
56 0,1,3,4,7,9,11,13,15,17,19,21,22,23 * * * root run-parts /etc/cron.hourly
# 57 1,5,10,14,18,22 * * * root run-parts /etc/cron.x
31 5 * * * root run-parts /etc/cron.daily
56 5 * * 1 root run-parts /etc/cron.weekly
37 5 1 * * root run-parts /etc/cron.monthly
58 5 3,18 * * root /root/io/reboot.sh >/dev/null 2>&1

3 6 2,5,8,11,14,17,20,23,25,28,30,31 * * root /root/io/back.sh >/dev/null 2>&1
3 6 1,4,7,10,13,15,16,19,22,24,27,29 * * root /root/io/bam.sh >/dev/null 2>&1

The last two lines (should) run the 2 scripts. Which are both pretty much exactly like the one in the first post of this thread.

When run by hand, they do fine, but nothing happens when I'm not there.. Also, using
# bash -x /root/io/back.sh
works without any problem whatsoever.

OS is the same (CentOS 4.4), so I have no clue what has changed. Any idea?

nixcraft · #8 (**permalink**) 04-29-2007, 09:25 AM

Make sure crond is running

Code:

/etc/init.d/crond status

meowing · #9 (**permalink**) 04-29-2007, 06:09 PM

Quote:

Originally Posted by nixcraft

Make sure crond is running

Code:

/etc/init.d/crond status

By golly, crond wasn't even in my init.d directory anymore!
How the heck it got out of there and why I did that is beyond me. Probably sleepy. Something has caused me to temporarily move it out.
I first thought: "Yeah right. Of course I have crond running." but then I actually checked what was in the init.d dir, and it wasn't there!

Thanks anyway, it's all fine now.