Guide to copy and sync Files/Directories using rsync with examples

Wondering how to transfer files from one location to another, that can be either on the same server or across servers. Today, we will share a great tool called rsync (Remote Sync) which is convenient

Rsync is a fast and extraordinarily versatile file copying tool. It can copy locally, to/from another host over any remote shell, or to/from a remote rsync daemon. It offers a large number of options that control every aspect of its behavior and permit very flexible specification of the set of files to be copied. It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. Rsync is widely used for backups and mirroring and as an improved copy command for everyday use.

Rsync finds files that need to be transferred using a “quick check” algorithm (by default) that looks for files that have changed in size or in last-modified time. Any changes in the other preserved attributes (as requested by options) are made on the destination file directly when the quick check indicates that the file’s data does not need to be updated.

Some of the additional features of rsync are:

Features

• support for copying links, devices, owners, groups, and permissions
• exclude and exclude-from options similar to GNU tar
• It’s faster than scp (Secure Copy) because rsync uses remote-update protocol which allows to transfer just the differences between two sets of files. First time, it copies the whole content of a file or a directory from source to destination but from next time, it copies only the changed blocks and bytes to the destination.
• a CVS exclude mode for ignoring the same files that CVS would ignore
• can use any transparent remote shell, including ssh or rsh
• Rsync consumes less bandwidth as it uses compression and decompression method while sending and receiving data both ends.
• does not require super-user privileges
• pipelining of file transfers to minimize latency costs
• support for anonymous or authenticated rsync daemons (ideal for mirroring)

General Information about rsync

Rsync copies files either to or from a remote host or locally on the current host (it does not support copying files between two remote hosts).

There are two different ways for rsync to contact a remote system: using a remote-shell program as the transport (such as ssh or rsh) or contacting an rsync daemon directly via TCP. The remote-shell transport is used whenever the source or destination path contains a single colon (:) separator after a host specification. Contacting a rsync daemon directly happens when the source or destination path contains a double colon (::) separator after a host specification, OR when a rsync:// URL is specified (see also the “USING RSYNC-DAEMON FEATURES VIA A REMOTE-SHELL CONNECTION” section for an exception to this latter rule).

As a special case, if a single source arg is specified without a destination, the files are listed in an output format similar to “ls -l”.

As expected, if neither the source or destination path specify a remote host, the copy occurs locally (see also the –list-only option).

Rsync refers to the local side as the “client” and the remote side as the “server”. Don’t confuse “server” with a rsync daemon — a daemon is always a server, but a server can be either a daemon or a remote-shell spawned process.

The syntax for rsync command to copy/sync files and directories
rsync options source destination

Some common options used with rsync commands

-v, –verbose – increase verbosity
-r : copies data recursively
-a : archive mode, archive mode allows copying files recursively and it also preserves symbolic links, file permissions, user & group ownerships and timestamps
-z : compress file data
-h : human-readable output

• Copy/Sync Files and Directory Locally
  Copy/Sync file with name list.txt to /home/deepak/client directory on the same machine
  rsync -zvh /home/basezap/client/list.txt /home/deepak/client/
• Copy/Sync a Directory on Local Computer
  Copy/Sync backup directory for Deepak from /home/basezap/backups/deepak/ to /home/deepak/backup_temp/
  rsync -avzh /home/basezap/backups/deepak/ /home/deepak/backup_temp/
• Copy/Sync Files and Directory to or From a Server
  • Copy a Directory from Local Server to a Remote Server
    Copy/Sync local backups directory to the remote backup server
    rsync -avz /home/basezap/backups/ root@192.99.145.23:/home/remote_backups/
  • Copy/Sync a Remote Directory to a Local Machine
    Copy/Sync backups from the remote backup server to client Deepak account
    rsync -avz root@192.99.145.23:/home/remote_backups/deepak.tar /home/deepak/backup_temp/
• Rsync with non-standard SSH Port
  rsync -arvz -e 'ssh -p <port-number>' --progress --delete username@remote_server_ip:/path/to/remote/folder /path/to/local/folder
  Copy/Sync local backups directory to remote backup server with port 1234
  rsync -arvz -e 'ssh -p 1234' --progress --delete deepak@192.99.145.23:/home/remote_backups/ /home/basezap/backups/