On 15.04.2024 01:29, tink via Duplicity-talk wrote:
hey tink,
Hi Edgar!
> Is it possible to use regular expressions for the source and target paths?
not as such
Fair enough, didn't think there was from looking at the doco and glaring at the code.
> The reason I'm asking is that we're currently collecting backup files onto a backup server using rrsync (using hard-links and date-stamps). I've now been asked to make a backup of the backup-sever to an S3 bucket; but the datestamps in the from would mean that I'm pushing a couple of TBs to the cloud every day, which is not ideal.
ahh, i see you want treat the "different" source folders as the same again to profit from incremental backups. why don't you just do so via shell? let's assume
your sources are
bkp.20240401
bkp.20240402
...
which is a foreseeable naming scheme, you could place that folder as source in your cron command to do the backup (make sure that your rsync finished already of course)
use for current date
date +%Y%m%d
for current date.
or list the latest backup (alphabetically)
ls -1d bkp.*/ | tail -n1
I wish it were that simple =|
It's more like:
/backups/backup/ansible.host/etc/d-2024-0415.05:23:04
/backups/backup/ansible.host/opt/ansible/d-2024-0415.05:23:01
/backups/backup/ansible.host/root/d-2024-0415.05:23:01
/backups/backup/ansible.host/usr/local/d-2024-0415.05:23:05
/backups/backup/ansible.host/var/backups/d-2024-0415.05:23:02
/backups/backup/portal.host/root/d-2024-0415.06:33:14
/backups/backup/portal.host/var/backups/d-2024-0415.06:33:14
/backups/backup/portal.host/etc/d-2024-0415.06:33:15
/backups/backup/portal.host/usr/local/d-2024-0415.06:33:16
/backups/backup/confluence-static.host/etc/d-2024-0415.05:27:57
/backups/backup/confluence-static.host/root/d-2024-0415.05:27:53
/backups/backup/confluence-static.host/usr/local/d-2024-0415.05:27:58
/backups/backup/confluence-static.host/var/backups/d-2024-0415.05:27:55
/backups/backup/dashboards.host/etc/d-2024-0415.05:28:01
/backups/backup/dashboards.host/root/d-2024-0415.05:27:58
/backups/backup/dashboards.host/usr/local/d-2024-0415.05:28:04
/backups/backup/dashboards.host/var/backups/d-2024-0415.05:27:59
/backups/backup/dbx.host/etc/d-2024-0415.05:28:14
/backups/backup/dbx.host/opt/d-2024-0415.05:28:05
/backups/backup/dbx.host/root/d-2024-0415.05:28:08
/backups/backup/dbx.host/usr/local/d-2024-0415.05:28:16
/backups/backup/dbx.host/var/backups/d-2024-0415.05:28:10
/backups/backup/splunk2.host/root/d-2024-0415.06:34:17
/backups/backup/splunk2.host/var/backups/d-2024-0415.06:34:21
/backups/backup/splunk2.host/etc/d-2024-0415.06:34:29
/backups/backup/splunk2.host/usr/local/d-2024-0415.06:34:32
/backups/backup/easyrsa-ca.host/etc/d-2024-0415.05:28:28
/backups/backup/easyrsa-ca.host/home/d-2024-0415.05:28:20
/backups/backup/easyrsa-ca.host/opt/d-2024-0415.05:28:30
/backups/backup/easyrsa-ca.host/root/d-2024-0415.05:28:25
/backups/backup/easyrsa-ca.host/usr/local/d-2024-0415.05:28:30
/backups/backup/easyrsa-ca.host/var/backups/d-2024-0415.05:28:26
/backups/backup/api.host/root/d-2024-0415.06:31:03
/backups/backup/api.host/var/backups/d-2024-0415.06:31:03
/backups/backup/api.host/etc/d-2024-0415.06:31:08
/backups/backup/api.host/usr/local/d-2024-0415.06:31:10
/backups/backup/backup2.host/root/d-2024-0415.06:27:46
/backups/backup/backup2.host/var/backups/d-2024-0415.06:27:46
/backups/backup/backup2.host/etc/d-2024-0415.06:27:48
/backups/backup/backup2.host/usr/local/d-2024-0415.06:27:49
/backups/backup/backup2.host/backups/configs/d-2024-0415.06:27:47
So the date-stamp can (fairly flexibly) appear anywhere in the directory structure.
To work with that I'd have to have create a job per directory, which would be a
pain considering that generating that list using find (to get the latest date only) takes
considerable time in its own right.
.NOTE: duplicity will probably complain that you switched sources. as you intend to do so you can use `--allow-source-mismatch` to ignore that.
does that help? sunny regards ..ede/duply.net
But yes, thank you, it does help; I can hack up something along your initial
thoughts, it just won't be pretty. =} regex in the parameter would have been SO much simpler ;)