Crontab notes

Jani Reinikainen,

Tip #1: Crontab syntax

I never remember the crontab syntax, so here it is for quick reference:

minute   hour   day of month   month   day of week   command
*        *      *              *       *             echo hello world!

Tip #2: ISP Disconnection Problem

One day my ISP gave me a dynamic IP (an IP that changes every time you connect to the Internet). This was not a good solution from my point of view, as I ran a webserver behind this DSL connection. To make things even worse, my ISP disconnects me after X minutes of inactivity (and my IP address changes again). I solved this problem by adding these lines to my crontab:

0 0,2,4,6,8,10,12,14,16,18,20,22 * * * ping -c 3 server1.com >/dev/null 2>&1
0 1,3,5,7,9,11,13,15,17,19,21,23 * * * ping -c 3 server2.com >/dev/null 2>&1

This pings two servers, one each even hour and the other each odd hour, hence preventing the disconnection. This is, of course, not the ideal solution since it generates extra traffic and load and the administrators of the site you're pinging will probably not like this.

Tip #3: File/Directory Dump

I wanted to automatically generate a file which would consist of a list of all files and subdirectories under a certain directory. Since this directory was a FTP upload directory for users, the contents of it changed constantly. I decided to generate a simple text file using ls every six hours. Here is what I inserted into my crontab:

0 0,6,12,18 * * * ls -lhABGR /mnt/data/upload/ > /mnt/data/upload/index.txt

This way the file is always overwritten. The format of this file is similar to this:

-rw-r--r--    1 ftp           63k May 14  2001 07.htm
-rw-r--r--    1 ftp           36k May 14  2001 08.htm
-rw-r--r--    1 ftp           33k May 14  2001 09.htm
-rw-r--r--    1 ftp         1161k May 14  2001 mycar.mpg

See the man page for ls for more information.

Tip #4: Temporary File Upload Directory

I had the need for a directory where FTP users could upload temporary files that would be deleted at a certain point, so that the temporary files wouldn't take up all my disk space, as I noticed that users wouldn't delete them anyway. I decided that one week would be a suitable amount of time for these files, so each sunday at 11pm (when the user load is light), I delete the entire contents of this directory with the cron job:

0 23 * * sun rm -r /mnt/data/temp/*; cp /var/www/docs/dumpheader.html /mnt/data/temp/HEADER.html >/dev/null 2>&1

Note that I also copy over a new Apache header file each time from a "safe" location (i.e. from outside the temporary files' upload directory structure, which is entirely wiped out each time). Of course, if you upload a file on, for example, wednesday, it will be stored for only one day.

Tip #5: Backing up a directory using Tar

Since I do not have RAID harddrives mirroring each other all the time, I had to invent some other way of creating backups of a specific directory. I also decided to compress the data using Tar in order to save space. Here is what I inserted in my crontab:

0 23 * * wed,sun tar -cf >/dev/null 2>&1

I decided that backing up twice per week (Wednesday and Sunday) would be sufficient, since I did not have any mission critical data stored in this directory.

Tip #6: Periodically mirroring a directory

Since I do not have RAID harddrives mirroring each other all the time, I had to invent some other way of creating backups of a specific directory, and besides, this way the server you are uploading to can be located on the other side of the world, if so desired. Here I'm using lftp to upload the files to another server (reverse mirroring). Here is what I inserted in my crontab:

0 23 * * wed,sun lftp -e 'open www.mysecondserver.com -u username,password -p 21 && 
mirror --continue --reverse --delete /my/local/directory /my/remote/directory' >/dev/null 2>&1

I decided that backing up twice per week (Wednesday and Sunday) would be sufficient, since I did not have any mission critical data stored in this directory. Combined with the tip above, you could save some space. See the lftp manual page for more info. You would probably want to use rsync if there are larger amounts of data in question.