Thread: shell script to scp last modified file

Hello Everyone,

I have written below shell script to scp last modified log file.

Code:

cd /data/logs/apache/
ls -trl | grep access-icecast.log.* | tail -1 | awk '{print $8}' | while read temp;
do
        echo "This is before scp" > /tmp/before.txt
        scp $temp itmuser@192.168.201.138:/home/itmuser/XLMEDIA_Logs/jetedgede
        echo "This is after scp" > /tmp/after.txt
done

This script works fine when I directly execute it. However when I schedule this script under cron jobs, "scp $temp itmuser@192.168.201.138:/home/itmuser/XLMEDIA_Logs/jetedgede" part of the script does not work.

Can anyone please suggest me what modification is required in the script so that it works when scheduled under cron jobs?


Thank you.

hi,

«it does not work» does not provide much details.
could you output stderr to a logfile to show how it does not work?

may be it's because of unknown path to scp

Code:

cd /data/log/apache/
/usr/bin/stat -c '%Y %n' access.icecast.log.* | /usr/bin/sort -r -k1,1n | { read date filename; /usr/bin/scp "$filename" itmuser@192.168.201.138:/home/itmuser/XLMEDIA_Logs/jetedgede;}

Hi Watael,

Thank you for the reply.

Script works fine if I manually execute it. My problem is, "SCP" statement is not running if I schedule it under cron jobs.

Can you please explain the code which you posted?


Thank you.

Hi Watael,

Kindly find the attached document for the screenshot


Thank you.

crontab issue.zip

Hi,

use absolute path of scp command , it should be /usr/bin/scp in CentOs5,
use "which scp" command and check absolute path of scp

inside crontab, change >/dev/null 2>&1
for >/data/scripts/copylog.log 2>&1
so you'll be able to show us what's happening.

chmod a+rx copylog.sh
add a shebang, on the first line of your script : #!/bin/sh
and call it without `sh`
did you write your script in unix, or linux, or else?

`ls` is painful to parse, so I preferably use `stat`, that outputs what I want.
in my command line, `stat` prints
%Y Time of last modification, seconds since Epoch
%n File name
piped to be sorted numerically in reverse order on first field only.
then `read` gets first line inside two vars : date, and filename (obvious!)
and finallly `scp` filename.
braces are needed because right hand part of a pipe is executed inside a subscript; so the rest of the script would not know about created variables.
such variables remain known inside braces.

Originally Posted by Watael

inside crontab, change >/dev/null 2>&1
for >/data/scripts/copylog.log 2>&1
so you'll be able to show us what's happening.

Like you suggested I have modified crontab and this what I see in the copylog.log file
ssh: connect to host 21 port 22: Invalid argument

Originally Posted by Watael

chmod a+rx copylog.sh
add a shebang, on the first line of your script : #!/bin/sh
and call it without `sh`
did you write your script in unix, or linux, or else?

Modified the permissions of copylog.sh file and included #!/bin/sh on the first line of my script. I am writing this on Ubuntu server.

Originally Posted by Watael

`ls` is painful to parse, so I preferably use `stat`, that outputs what I want.
in my command line, `stat` prints
%Y Time of last modification, seconds since Epoch
%n File name
piped to be sorted numerically in reverse order on first field only.
then `read` gets first line inside two vars : date, and filename (obvious!)
and finallly `scp` filename.
braces are needed because right hand part of a pipe is executed inside a subscript; so the rest of the script would not know about created variables.
such variables remain known inside braces.

I tried running stat command but it's not giving any output.

Code:

root@jetedgede:/data/logs/apache# /usr/bin/stat -c '%Y %n' access.icecast.log.* | /usr/bin/sort -r -k1,1n
/usr/bin/stat: cannot stat `access.icecast.log.*': No such file or directory
root@jetedgede:/data/logs/apache# /usr/bin/stat -c '%Y %n' access.icecast.log.*
/usr/bin/stat: cannot stat `access.icecast.log.*': No such file or directory
root@jetedgede:/data/logs/apache# /usr/bin/stat -c '%Y %n' access.icecast.log.1336532400
/usr/bin/stat: cannot stat `access.icecast.log.1336532400': No such file or directory
root@jetedgede:/data/logs/apache#

Thank you.

Originally Posted by Rahul.Patil

use absolute path of scp command , it should be /usr/bin/scp in CentOs5,
use "which scp" command and check absolute path of scp

I used absolute path in my code, but in vain..

yep, I miscopied filename, it's access-icecast.log.*

also, always quote variables.

there seems to be something wrong with scp/ssh, add a -v option to it, maybe it'll say more about what's wrong

Hi Watael,

Thank you very much for your help.

Originally Posted by Watael

hi,

«it does not work» does not provide much details.
could you output stderr to a logfile to show how it does not work?

may be it's because of unknown path to scp

Code:

cd /data/log/apache/
/usr/bin/stat -c '%Y %n' access-icecast.log.* | /usr/bin/sort -r -k1,1n | { read date filename; /usr/bin/scp "$filename" itmuser@192.168.201.138:/home/itmuser/XLMEDIA_Logs/jetedgede;}

This code works perfectly under cron job as per my requirement.


Regards,
Ravi