Home > Linux/Unix > How-To Synchronize Files Using “rsync” Command

How-To Synchronize Files Using “rsync” Command


Placing several backups in a safe place and keeping files synchronized either locally or remotely is an urgent necessity for all users whose everyday tasks solely rely on the use of computers. Hard drives get damaged, files get deleted, memory sticks get lost …etc. You have to make synchronized backups of your stuff!

Many backup applications are there. Some are GUI-based, while others run through the command line. Personally, I prefer to use the command line due to its high flexibility. One of the most widely used command for synchronization purposes has been the rsync command.

By the way, one of the reasons behind my migration from Windows to Linux was the lack of the FREE synchronization tools that run under Windows. Anyway, if I start talking about this again I won’t finish, and I already wrote about this in an earlier post. So, let’s start talking about “rsync” command.

rsync

rsync is an open source utility that provides fast incremental file transfer. “rsync” is freely available under the GNU General Public License

Features

Features of rsync are already mentioned in different sites, but this is how I see it:

  • Incremental File Transfer
  • Compression while Transferring
  • Synchronizing Remote Machines
  • Encrypted and Secure
  • Better Alternative to FTP
  • Very Flexible

Uses

If you have a look at the rsync manual you will notice a huge number of arguments, and that might discourage you from using the command. Well, don’t worry! All those arguments are for advanced usages, and as the synchronization process gets more complex you will need to understand more arguments and when to use them; otherwise, you only need to used to little arguments to do you the job. Therefore , you will find yourself running the same command over and over with same arguments all the time.

Here is the Generic Syntax:

rsync [options] Src Dst

Where Src and Dst respectively are:

Src: Source location from where files will be synchronized.
Dst: Destination location to where files will be synchronized.

Src and Dst can hold these location combinations except for Remote to Remote directories:

Src Dst
Local Local
Local Remote
Remote Local
Local: "/directory"
Remote: "username@hostname:"

Note: Remote Directory will be the HOME directory of the username used at the Remote machine by default!

Full Sync

A full synchronization process implies what is in this table will take place:

Command Process Current Directory Sub-Directories
rsync Create Yes Yes
Update
Delete

A very frequently used arguments with “rsync” command:

rsync -avz -u --progress --delete Src Dst -n
-a Archive mode
-v Verbose mode
-z Compress while transferring files. This will make the transferring faster and will save the network bandwidth
-u Update files which are newer in the Src
--progress Show the transfer progress of each file while transferring
--delete Delete files from Dst if they don’t exist in Src
-n Preview mode. Changes won’t take effect till this argument is omitted
Sync, without Sub-Directories

One of the syncing tasks I perform on a daily basis is that I sometimes want to sync a certain directory without syncing the subdirectories. This is how I do it:

rsync -dvz -u --progress --delete Src Dst -n

If you want the same effect that the argument ‘-a’ have, such as preserving links, owner, group …etc. Then, this is another option:

rsync -dlptgoDvz -u --progress --delete Src Dst -n
Arguments Worthy Looking at
--exclude=PATTERN exclude files matching PATTERN
--include=PATTERN don’t exclude files matching PATTERN
--existing skip creating new files on receiver
--ignore-existing skip updating files that exist on receiver

Important Notes

  1. Synchronization process always works from Src to Dst. So, Any change in the Src will be reflected in the Dst.
  2. A probable confusing matter here is the deleted files. Just remember, a deleted file in the Src will be deleted from the Dst, but any file deleted from the Dst will be recreated from the Src while syncing!
  3. As a rule of thumb, always make the necessary modification in the Src then sync it with the Dst.
  4. Be careful with the Trailing Slash. If the Trailing Slash appears, then content of that directory will be synced, otherwise, that directory AND its content will be synced. Try it out and see what happens!

Too Many Arguments!!

If you found any difficulty in using the proper arguments of rsync command due to their numbers, then I suggest you start with GUI rsync Application called Grsync. The good thing about it is that it shows you the arguments that are going to be used to perform the synchronization process!

About these ads
Categories: Linux/Unix Tags: , ,
  1. March 1, 2010 at 7:32 pm

    Good description and the colored tables really help. Nice job.

    • Husain Al-Khamis
      March 1, 2010 at 7:37 pm

      I’m sooooo flattered :)
      Thanks a lot.

  2. Jamie
    September 29, 2010 at 7:20 am

    Lovely article! Thanks for making this.

    • Husain Al-Khamis
      September 30, 2010 at 11:50 am

      I appreciate the comment..
      Thanks a lot :)

  3. Adam
    October 7, 2010 at 10:19 pm

    Great post. Just what I needed to make it usable.

    • Husain Al-Khamis
      October 14, 2010 at 8:22 pm

      Glad that my post has helped :)

  4. January 28, 2011 at 1:51 pm

    Excellent use of color and graphics, very clearly written. A+!

  5. Vivek
    August 17, 2012 at 11:04 am

    Nice article, clear ti understand. Thanks

  6. Michael Longval
    December 7, 2012 at 11:39 pm

    Thanks Husain. I’m using this on my Mac with an external Raid5.

  1. February 28, 2010 at 12:34 am

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: