Syntax

sync.pl [-a] [-c]

Description

The first version was v0.9. Changes were introduced in the following versions:

• blue font: v1.0 23 May 2004, not compatible with previous versions

Unless in the -c mode (in which case see below), sync.pl backs up a particular directory to a remote location on a regular basis using SCP and SSH protocols. Generally, the program must be run in the directory being backed up. However, if it does not find string UpdateTo in file desktop.ini in the current directory (see also Files), it will try to change the current directory first to %USERPROFILE%/Files, then to %USERPROFILE%/My Documents (under Windows), or to the home directory (under anything else) and back up this new “home” directory. If there is no string UpdateTo in the “home” directory either, the program exits with an error.

sync.pl saves MD5 sums of all files in the directory into file .md5, and each time it is invoked all new files, which do not have the information about them in file .md5, are archived and sent via scp to the remote location. Then, sync.pl invokes a script at the remote location via ssh. This script unarchives new files, moves, copies, and deletes remote files corresponding to locally moved, copied, and deleted files, thus restoring an exact copy of the particular local directory at the remote location.

Gory details about sync.pl:

• determine the directory to back up (current or “home”) and the remote location (UpdateTo in desktop.ini)
• load stored MD5 sums (if any) from file .md5
• calculate actual MD5 sums, place the remote script in the beginning, and save the result into file log.md5 (unless option -a is specified, if the size, modification date, and full filename are not changed since the last update, the stored MD5 sum is taken instead of calculating a new one; this saves significant amount of time; option -a forces the actual calculation of new MD5 sums)
• using tar and gzip, archive all files with unique MD5 sums which are not among stored MD5 sums and save the result into file tar.md5
• also archive information about deleted, moved and copied files (file dc0rc1.md5) and actual (new) MD5 sums (file log.md5) and append the result to file tar.md5
• if there are deleted, moved, copied or new files, upload archive tar.md5 to the remote location and invoke the remote script, which is at the beginning of remote file .md5, or (if this directory is being backed up for the 1st time) at the beginning of new file log.md5
• rename local file log.md5 to .md5, delete local files dc0rc1.md5 and tar.md5

The remote script, which is at the beginning of files .md5 and log.md5, can run in two different modes. First, the script changes the current directory to the one where it is located itself. Then, if it finds file tar.md5 (which happens if it is invoked by sync.pl), it enters the update mode. Otherwise, it enters the check mode.

Gory details about the update mode of the remote script:

• unarchive all files from tar.md5, including dc0rc1.md5 and log.md5, to subdirectory tmpdir.md5
• delete, move and copy all the necessary files (the destination is tmpdir.md5)
• move all files from tmpdir.md5 to the main directory
• rename file log.md5 to .md5, remove all empty directories and files dc0rc1.md5, tar.md5
• execute script .postupdate.md5 if it exists

Gory details about the check mode of the remote script:

• load stored MD5 sums from the end of the script
• calculate actual MD5 sums
• compare new and old MD5 sums; issue a warning if there are “unaccounted files”, i.e. real files which do not have information about them in the script; issue an error if there exist incorrect MD5 sums in the script

If option -c is specified, the script runs in a special, -c mode. It simply creates file .md5 containing MD5 sums of all files in the current directory. This mode is useful to control data integrity. To check integrity, the script located in the beginning of file .md5 (the remote script, although in this case it runs locally) is used.

Files

desktop.ini is a standard file, at least under Windows, located in the directory to back up. In order for the program to work properly, it must contain a line of the following format:

UpdateTo=USERNAME@HOSTNAME:DIRECTORY_[cr]lf

This line specifies the remote location for the directory where the file desktop.ini is located.

.postupdate.md5 is a script located in the directory to back up. It is invoked by the remote script in the update mode after updating the remote directory.

.md5, located in the directory to back up, contains the remote script and MD5 sums of all files in the directory (excluding file .md5 itself). If sync.pl does not find this file, it will be created. The file has the following format:

REMOTE-SCRIPT_lf
__END___lf
SIZE_tabMD5-SUM_tabDATE_tabFILENAME_lf
SIZE_tabMD5-SUM_tabDATE_tabFILENAME_lf
...

FILENAME here is a relative path containing symbols / (not \), without ./ at the beginning. The order of lines depends on the file system (e.g. on NTFS lines are in the alphabetical order).

log.md5 is a temporary file located in the directory to back up. It has the same format as file .md5 and contains new MD5 sums. It is included in archive file tar.md5 and deleted afterwards.

dc0rc1.md5 is a temporary file located in the directory to back up; it is included in archive file tar.md5 and deleted afterwards.

tar.md5 is a temporary archive file in format tar.gz containing all files with new MD5 sums and files log.md5, dc0rc1.md5. sync.pl creates this file, uploads it to the remote location, and removes it. It is located either in directory %TMP% (under Windows) or in the directory to back up.

Compatibility

Tested with perl v5.005_03 and v5.8.0.

Requires the following programs: gzip, tar (with -T), ssh, scp.

The remote script in the update mode requires rm, cp, gzip, tar.