signature.ascDescription: Message signed with OpenPGP using GPGMail
|
| From: | Mark Grandi |
| Subject: | Re: [Duplicity-talk] Backup succeeding but not writing manifest / sigtar |
| Date: | Mon, 29 Feb 2016 21:50:28 -0700 |
| I am backing up root, because on Mac OS X, almost all of the applications are stored in /Applications , (shared between all users), so I am backing up root, and excluding all of the other directories besides /Users , but that doesn't seem to stop Duplicity from trying to access them anyway, and failing due to not having the read permissions. Anyway, I looked into my problem myself since I had a free afternoon, and found myself honestly quite concerned with what some of the code does, or rather doesn't do. For instance, with Par2Backend, if for some reason the par2 command doesn't return 0, then it just silently doesn't create any par2 files without any notice to the user. In LocalBackend, if any of the methods in there that have try/except blocks, in the except blocks, they are either 'pass' or just 'return False', and often the caller doesn't care about the return result anyway, so any errors that occur during moving/copying files are silently discarded and not presented to the user, nor does it invoke any of the retry logic in Duplicity. I mentioned the two above things, because I had trouble compiling the par2 binary on my Mac, and upon googling, I found a Mac OS X app that is a GUI frontend to par and par2, and it appears that the author created their own fork of par2, that works on mac (and has improvements such as using grand central dispatch, the website is here: https://gp.home.xs4all.nl/Site/MacPAR_deLuxe.html). Anyway, it sees this version of par2 does not like symlinks, and the Par2Backend class uses symlinks when creating the par2 files. Discovering why the par2 files were not being created required me stepping into the debugger, because no errors are logged to the user. Once I changed the symlink to a hardlink, and modified the command line being passed to par2 somewhat, I was stuck on the issue that I originally posted: the difftar files + par2 files were being moved to the correct spot, but not the manifest or sigtar files and their par2 counterparts. Stepping into the debugger, i found that the difftar files are being written with Par2Backend wrapping LocalBackend.put(), but the sigtar and manifest files are wrapping LocalBackend._move(). However, LocalBackend._move() is using Path.rename() instead of Path.move() ! It even says in the docstring for Path.move(): "Like rename but destination may be on different file system". So there lies my problem, my backup destination was on a different file system, but due to LocalBackend._move() calling the wrong underlying Path method, and swallowing exceptions and only returning false (that Par2Backend doesn't even check), it was not moving the sigtar and manifest files + par2 files because of "[Errno 18] Cross-device link". The patch to fix this is just simply replacing LocalBackend._move() with: def _move(self, source_path, remote_filename): target_path = self.remote_pathdir.append(remote_filename) try: source_path.setdata() # maybe only needed in Par2Backend? source_path.move(target_path) return True except OSError as e: return False This does not solve the concerns I mentioned with swallowing exceptions however. ~Mark
|
signature.asc
Description: Message signed with OpenPGP using GPGMail
| [Prev in Thread] | Current Thread | [Next in Thread] |