Just some quick syntax for running multiple rsync threads in parallel using find and xargs. In many cases (especially if NFS is involved)  this can greatly speed up the copy operation.

Another approach is to use the GNU Parallel command. Here’s an example piping output of ls into parallel rsync:

The “-j” option for parallel specifies the number of concurrent threads. For CPU-intensive processes, this number should not exceed the number of available CPU cores minus one (to leave something for the OS). In case with rsync, however, the number of parallel threads can be set well in excess of available CPUs, the primary limiting factors being filesystem and network performance.

Leave A Reply

Please enter your comment!
Please enter your name here