How to exclude directories from xargs rsync?

So I'm trying to set up a bash script for incremental backups using rsync with xargs to use multiple threads on Ubuntu Server 20.042 LTS. There's a handful of directories that I want to exclude, but I can't seem to figure out the way to do that. Currently I'm piping ls to xargs which then executes rsync with 8 processes:

# Create the archive and begin the backup.  cd /  ls --indicator-style none \    --ignore={"bin","boot","cdrom","data","dev","init*","lib","lib64","lost+found"} \    --ignore={"media","mnt","proc","root","run","sbin","snap","srv","swap.img","sys"} \    --ignore={"tmp","vmlinuz*"} | xargs --max-args 1 --max-procs 8 -I % \    rsync --archive --compress --delete --ignore-missing-args --recursive --verbose \    --link-dest "${backup_link}" "%" \    --exclude={"*tmp*","*.cache"} \    "${backup_path}"  

For some reason, it's still trying to use rsync on the directories that I've ignored with the ls command. I've tried setting rsync to exclude the directories instead:

# Create the archive and begin the backup.  cd /  ls --indicator-style none | xargs --max-args 1 --max-procs 8 -I % \    rsync --archive --compress --delete --ignore-missing-args --recursive \    --exclude={"/bin","/boot","/cdrom","/data/mysql","/data/backup","/dev"} \    --exclude={"/initrd.img","/initrd.img.old","/lib","/lib64","/lost+found","/media"} \    --exclude={"/mnt","/proc","/root","/run","/sbin","/snap","/srv","/swap.img","/sys"} \    --exclude={"**/tmp","/vmlinuz*","*.cache"} \    --link-dest "${backup_link}" "%" "${backup_path}"  

Which still tries to backup the files that I want to exclude.

The reason I'm trying to run multiple threads is that this script is set to execute every 15 minutes via crontab and, without the multiple processes, it's barely taking less than 15 minutes. Later in the script, it then copies that backup (also using rsync) to an SMB server, which is a huge bottleneck with all of the small files.

Is there something that I'm missing here? Or is there a better way to accomplish this? This is a production server, so I'd really like to avoid installing third-party utilities.

Thank you in advance!

https://stackoverflow.com/questions/66695924/how-to-exclude-directories-from-xargs-rsync March 19, 2021 at 01:14AM