Rsync is a powerful command-line utility for Unix/Linux systems, that allows you to synchronize and transfer files between different two systems. One of the key features of Rsync is the ability to exclude files and directories from the synchronization process. This feature can be incredibly useful for a variety of tasks, such as backups, codebase synchronization, and data management.
In this article, we’ll get a basic understating of excluding files and directories with Rsync command line utility. Also includes a few useful examples using this feature.
Exclude files and directories with rsync
The most basic method for excluding files and directories with Rsync is by using the --exclude option followed by the name of the file or directory you want to exclude. For example, if you want to exclude all files with the “.log” extension, you can use the following command:
rsync -av --exclude='*.log' source/ destination/
You can also exclude specific directories by including the entire path, like this:
rsync -av --exclude='path/to/directory' source/ destination/
Exclude all written in text file
Another way to exclude files and directories is by using a separate file called an “exclude file”. This file contains a list of excluded patterns, one per line, and Rsync will read the file and apply all the patterns specified in it.
rsync -av --exclude-from='exclude.txt' source/ destination/
Here `exclude.txt` file containing all the patterns of files or directories to be excluded
The patterns in the exclude file can be a shell glob (e.g., *.log) or a regular expression (if the --exclude-from option is replaced with --exclude-from-file).
You can also use a `.gitignore` file to specify files to be excluded, which is a good way to exclude version control files.
rsync -av --exclude-from='.gitignore' source/ destination/
Include specific files and exclude other
It is also possible to include and exclude files at the same time by using the --include option followed by the name of the file or directory you want to include, and the --exclude option for everything else. For example, the following command will include all files with the “.txt” extension and exclude all other files:
rsync -av --include='*.txt' --exclude='*' source/ destination/
It’s important to note that the order of the --include and --exclude options is significant. If you want to include a specific file or directory and then exclude others, the --include option should come first.
Exclude existing files on destination
Another thing to consider is to exclude files that are already present in the destination and the destination is a backup. You can use --ignore-existing options to achieve this.
rsync --ignore-existing -av source/ destination/
You can read more about it: https://tecadmin.net/rsync-command-to-copy-missing-files-only/
Exclude large files
Another consideration when using Rsync to exclude files and directories is performance. Excluding large files or directories can save a significant amount of time and space when transferring or synchronizing data. For example, if you’re performing a backup, you may want to exclude large media files or backups of backups.
rsync --ignore-existing --size-range='+100M' -av source/ destination/
There are also many other options available with rsync that can be used to further fine-tune the file transfer process. For example, you can use the --exclude option to exclude certain files or directories from the transfer, or the --dry-run option to see what files would be copied without actually performing the transfer.
Wrap Up
Overall, the ability to exclude files and directories with Rsync is a powerful and essential feature that can greatly improve the efficiency, accuracy, and security of your backups, synchronization, and data management tasks. By using the methods and best practices discussed in this article, you can take full advantage of this feature and ensure that your Rsync commands are working as efficiently and effectively as possible.