nixCraft Linux Forum

nixCraft

Linux / UNIX Tech Support Forum

shell script and cron -zipping a folder then archiving on same server

This is a discussion on shell script and cron -zipping a folder then archiving on same server within the Shell scripting forums, part of the Development/Scripting category; I am wanting to set up a shell script on my Linux server for a cron to run I have ...

Go Back   nixCraft Linux Forum > Development/Scripting > Shell scripting

Linux answers from nixCraft.


Shell scripting You can discuss the shell scripting, request shell scripts and scripting techniques

Reply

 

LinkBack Thread Tools Display Modes
  #1 (permalink)  
Old 11-26-2008, 11:47 PM
Junior Member
User
 
Join Date: Nov 2008
Posts: 10
Thanks: 0
Thanked 0 Times in 0 Posts
Rep Power: 0
amanuensis is on a distinguished road
Default shell script and cron -zipping a folder then archiving on same server

I am wanting to set up a shell script on my Linux server for a cron to run

I have a folder/directory structure

events
fonts
marketing
admin

and within 2 of those folders (events and marketing) there are 60 folders in one and also 60 folder in the other, within those folders are a collection of files and folders.

What I am wanting to do is have a script copy, zip and then save itself to another designated area on the server, sort of like an archive. This has to happen to all the 60 folders in events and also in the marketing folder, it needs to be carried out once a week and also after the first time it has run it has to either write over or replace the previous file from the week before, the one that was saved originally in the designated area.
To summaries, I want to end up with the 60 folders in each of the marketing and events folders, zipped into 60 individual folders in a different location on the same server.
Hope this all makes sense.

If we were taking about just one folder with lots of files and folder inside of it - I am wanting to zip that folder with its contents save it elsewhere on the server - the original must stay where it is and uncompressed.

Any idea anyone.

Thanks in advance for any help with this
Reply With Quote
Sponsored Links
  #2 (permalink)  
Old 11-27-2008, 03:55 AM
nixcraft's Avatar
Never say die
User
 
Join Date: Jan 2005
Location: BIOS
Posts: 1,521
Thanks: 2
Thanked 16 Times in 13 Posts
Rep Power: 10
nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute
Default

It is not hard to write such script. I suggest a better solution called snapshots. rsnapshot is a filesystem snapshot utility. It can take incrementalsnapshots of local and remote filesystems for any number of machines. You can backup those directories to another harddisk on same system or another server. You will also get hourly, montly, daily and weekly snapshots. Can you tell us more about your Linux distro?
__________________
Vivek | My personal blog
Linux Evangelist
+ ADD [SOLVED] thread prefix to your thread when your problem is sorted out by editing your thread.
+ Always use CODE tags for posting system output and commands!
Reply With Quote
  #3 (permalink)  
Old 11-27-2008, 06:39 PM
Junior Member
User
 
Join Date: Nov 2008
Posts: 10
Thanks: 0
Thanked 0 Times in 0 Posts
Rep Power: 0
amanuensis is on a distinguished road
Default

First of all thank you for replying

Perhaps a more full explanation would clarify things.

I have a php script installed that will allow clients to download files to any folder that I have given them permission to, when they go to folder 1 and open it there are around 60 files available for them to download individually, originally if they wanted the whole folder downloaded they would select the folder and then when the download button was pressed, the folder was then zipped, and when zipping was complete it started to download as one zipped file. problem is that if there are a lot of files in the folder the script times out.
I have maxed all the relevant settings in my php.ini file and the problem sill exists.
Downloading each file individually works well if you only require a few files, but when you need the whole 60 it becomes impracatical - so, my work around was to have all the folders that have loads of files in them, zipped and relocated to another place on the same server so as to give the clients the option of downloading one big zip file containing all the files yet still download if required from the original position on the server the individual files.
As new individual files are being added every few weeks and old ones removed, it would need the zipped file to replace the old one with itself as to always have the most recent available for download as one large zip.

Copying the folder with lots of files in it, zipping it and dropping it onto another location on the same server is important, as I would have to set permission to download the new zipped file to whoever wanted access from the script on that server.

Hope I made things a little clearer.

The php script is 'Secureloads'.

Not sure what you mean when you say "Can you tell us more about your Linux distro?"
Reply With Quote
  #4 (permalink)  
Old 11-27-2008, 10:47 PM
nixcraft's Avatar
Never say die
User
 
Join Date: Jan 2005
Location: BIOS
Posts: 1,521
Thanks: 2
Thanked 16 Times in 13 Posts
Rep Power: 10
nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute
Default

Here is a sample shell script:

Code:
#!/bin/bash

# dirs to backups
DIRS="/path/to/events /path/to/marketing"

# Save zip file elsewhere on the server 
DEST="/disk2/archive"



# set to "no" keep old zip file and add new file
DELETE_OLD_ZIP_FILES="yes" 

#Path to binary files
BASENAME=/usr/bin/basename 
ZIP=/usr/bin/zip

for d in $DIRS
do
    zipfile="${DEST}/$(${BASENAME} ${d}).zip"
    echo -n "Creating $zipfile..."
    if [ "$DELETE_OLD_ZIP_FILES" == "yes" ]
    then
        [ -f $zipfile ] && /bin/rm -f $zipfile 
    fi
    # create zip file
    ${ZIP} -r $zipfile $d &>/dev/null && echo "Done!"
done
Save this file to /root/mkserverzip.sh and set cron job as follows to backup it up weekly:
Code:
@weekly /root/mkserverzip.sh
See cron tutorial for more info:
How do I add jobs to cron under Linux or UNIX oses?

Modify script as per your setup.
__________________
Vivek | My personal blog
Linux Evangelist
+ ADD [SOLVED] thread prefix to your thread when your problem is sorted out by editing your thread.
+ Always use CODE tags for posting system output and commands!

Last edited by nixcraft; 11-27-2008 at 10:48 PM. Reason: Thread moved to scripting section
Reply With Quote
  #5 (permalink)  
Old 11-28-2008, 01:17 PM
Junior Member
User
 
Join Date: Nov 2008
Posts: 10
Thanks: 0
Thanked 0 Times in 0 Posts
Rep Power: 0
amanuensis is on a distinguished road
Default

Hi

Will this script keep the events and marketing separate in its destination or will it make it one big zip? If it does make one big zip, could I duplicate the script and include only one of the folders lets say 'events' and then allocate that a separate (by the side of marketing) location.

Is there a queuing system by default with a cron or will I have to do something to run theses consequently?

Many thanks again in advance for your help
Reply With Quote
  #6 (permalink)  
Old 12-02-2008, 12:06 AM
Junior Member
User
 
Join Date: Nov 2008
Posts: 10
Thanks: 0
Thanked 0 Times in 0 Posts
Rep Power: 0
amanuensis is on a distinguished road
Default

Many thanks the script is now working:
One more question if you dont mind
If i wanted to zip a range of folders and save them all to different directories, do i just
repeat the command I will explain:
if I have folder 1, folder 2 and folder 3 all full of files and various folders, and I would want to zip folder one into a different folder on the same server called folder1zipped, and then have folder2 zipped and saved to another folder called folder2zipped etc,etc.
How do I get them to work in sequence and save to the correct folder

Last edited by amanuensis; 12-02-2008 at 12:47 AM.
Reply With Quote
  #7 (permalink)  
Old 12-02-2008, 10:55 PM
nixcraft's Avatar
Never say die
User
 
Join Date: Jan 2005
Location: BIOS
Posts: 1,521
Thanks: 2
Thanked 16 Times in 13 Posts
Rep Power: 10
nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute nixcraft has a reputation beyond repute
Default

Set DIRS and it will take care of everything:


Code:
 DIRS="folder1 folder2 folder3"
__________________
Vivek | My personal blog
Linux Evangelist
+ ADD [SOLVED] thread prefix to your thread when your problem is sorted out by editing your thread.
+ Always use CODE tags for posting system output and commands!
Reply With Quote
  #8 (permalink)  
Old 12-23-2008, 12:19 PM
Junior Member
User
 
Join Date: Nov 2008
Posts: 10
Thanks: 0
Thanked 0 Times in 0 Posts
Rep Power: 0
amanuensis is on a distinguished road
Default Zipping folder within folders

Hi

This may be beyond the capability of a shell script, but I was wondering.
I have a series of folders 50 in total and they reside inside a folder called 'events' inside these folders there are in fact many other files and folders.
Is it possible for a shell script to get hold of all of the folders that reside inside the 'events folder' however deep they are, zip them and put them into the same folder thus keeping the file structure, only change would be the addition of a zip file for all the corresponding folders.

Before:
folder1
folder2
folder3


After
folder1
folder1.zip
folder2
folder2.zip
folder3
folder3.zip

and the same scenario for any other folders inside folder1, folder2 and folder 3.

I suppose I am wanting to see if there is possible to script to select all the folders within one folder (event without having to name them individually in the script (all folders within the events folder to be zipped and remain in the same location and of course when the script runs again, once a day, for it to replace the old zip with the new one)

Hope all this make sense.

Last edited by amanuensis; 12-23-2008 at 12:23 PM.
Reply With Quote
  #9 (permalink)  
Old 12-28-2008, 11:37 PM
Junior Member
User
 
Join Date: Nov 2008
Posts: 10
Thanks: 0
Thanked 0 Times in 0 Posts
Rep Power: 0
amanuensis is on a distinguished road
Default zipping folders within folders

Hi

This may be beyond the capability of a shell script, but I was wondering.
I have a series of folders 50 in total and they reside inside a folder called 'events' inside these folders there are in fact many other files and folders.
Is it possible for a shell script to get hold of all of the folders that reside inside the 'events folder' however deep they are, zip them and put them into the same folder thus keeping the file structure, only change would be the addition of a zip file for all the corresponding folders.

Before:
folder1
folder2
folder3


After
folder1
folder1.zip
folder2
folder2.zip
folder3
folder3.zip

and the same scenario for any other folders inside folder1, folder2 and folder 3.

I suppose I am wanting to see if there is possible to script to select all the folders within one folder (event without having to name them individually in the script (all folders within the events folder to be zipped and remain in the same location and of course when the script runs again, once a day, for it to replace the old zip with the new one)

Hope all this make sense.
Reply With Quote
  #10 (permalink)  
Old 12-31-2008, 02:03 PM
Junior Member
User
 
Join Date: Feb 2008
Posts: 22
Thanks: 0
Thanked 0 Times in 0 Posts
Rep Power: 0
unSpawn is on a distinguished road
Default

Quote:
Originally Posted by amanuensis View Post
Hope all this make sense.
It doesn't to me but if it does to you, oh well... :-] Maybe something like 'find ${SOMEPATH}/* -type d|while read DIRNAME; do zip -r "${DIRNAME}/${DIRNAME//*\//}.zip" "${DIRNAME}"; done'?..
Reply With Quote
Reply

Tags
archive , backup , linux , shell scripting , unix , zip


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads

Thread Thread Starter Forum Replies Last Post
Tomcat shell script cron job to restart server kasimani Shell scripting 4 05-03-2008 07:46 PM
Shell Script to Automatically Delete a File via Cron Job kakarla Shell scripting 2 01-29-2008 08:54 AM
A script for Removing all the files inside a folder and its sub folder vivekv Shell scripting 1 10-25-2007 01:44 PM
Shell script to perform operation on remote server vivekv Shell scripting 3 10-24-2007 12:10 AM
Shell script to change folder directory owner after restore marinm Shell scripting 5 01-23-2007 12:53 PM


All times are GMT +5.5. The time now is 05:43 AM.


Powered by vBulletin® Version 3.8.3 - Copyright ©2000 - 2009, Jelsoft Enterprises Ltd.
Search Engine Optimization by vBSEO 3.3.0
©2005-2009 nixCraft. All rights reserved

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36