[rdiff-backup-users] Large backup crashes when changing permissions

[George Patterson]

Hi All,


I have been landed with the task of backing up a directory tree of about 1G and 
have struck a problem with crashing rdiff-backup.

It could be related to the size of the directory tree as 178MB was backed up 
out of 1000MB. Looks like I might have to learn Python
If this is relevant, I am using cstream to restrict the bandwidth usage. 
rdiff-backup --test-server works correctly.

Network details and software versions.
Machine being backed up
OS: FreeBSD 4.10
Python: 2.4
librsync: 0.9.6
rsync: 2.6.3
Rdiff-backup: 0.12.7

Backup Server
OS: FreeBSD 5.3
Python: 2.4
librsync: 0.9.6
rsync: 2.6.3
Rdiff-backup: 0.12.7

Backup user's umask: 22

I also tried it without the throttling with cstream to no avail, had the same 
crash after 200 odd MB.

Thanks

George Patterson

Find below the command used and the output given by the error. (I sanitised the 
server address)


%rdiff-backup --remote-schema 'cstream -t 10k | ssh %s '\''sudo rdiff-backup 
--server'\'' | cstream -t 100k' \
address@hidden::/usr/server/websites/ /usr/backup/server/servername/websites/
address@hidden's password: 
UpdateError controlpanel2/log_customlog Updated mirror temp file 
/usr/backup/server/servername/websites/controlpanel2/rdiff-backup.tmp
.3279 does not match source
UpdateError controlpanel2/log_errorlog Updated mirror temp file 
/usr/backup/server/servername/websites/controlpanel2/rdiff-backup.tmp.
3280 does not match source
Traceback (most recent call last):
  File "/usr/local/bin/rdiff-backup", line 23, in ?
    rdiff_backup.Main.Main(sys.argv[1:])
  File "/usr/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 254, 
in Main
    take_action(rps)
  File "/usr/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 226, 
in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 271, 
in Backup
    backup.Mirror(rpin, rpout)
  File "/usr/local/lib/python2.4/site-packages/rdiff_backup/backup.py", line 
36, in Mirror
    DestS.patch(dest_rpath, source_diffiter)
  File "/usr/local/lib/python2.4/site-packages/rdiff_backup/backup.py", line 
210, in patch
    ITR(diff.index, diff)
  File "/usr/local/lib/python2.4/site-packages/rdiff_backup/rorpiter.py", line 
275, in __call__
    if self.finish_branches(index) is None:
  File "/usr/local/lib/python2.4/site-packages/rdiff_backup/rorpiter.py", line 
227, in finish_branches
    to_be_finished.end_process()
  File "/usr/local/lib/python2.4/site-packages/rdiff_backup/backup.py", line 
549, in end_process
    rpath.copy_attribs(self.dir_update, self.base_rp)
  File "/usr/local/lib/python2.4/site-packages/rdiff_backup/rpath.py", line 
157, in copy_attribs
    rpout.chmod(rpin.getperms())
  File "/usr/local/lib/python2.4/site-packages/rdiff_backup/rpath.py", line 
629, in chmod
    self.conn.os.chmod(self.path, permissions)
OSError: [Errno 1] Operation not permitted: 
'/usr/backup/server/servername/websites/docroot/HotSaNIC/Documentation/module-howto/CVS'
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound 
method GzipFile.__del__ of <gzip open file '/usr/back
up/server/servername/websites/rdiff-backup-data/file_statistics.2005-01-19T10:07:11+10:30.data.gz',
 mode 'wb' at 0x8302728 0x8310c4c>>
 ignored
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound 
method GzipFile.__del__ of <gzip open file '/usr/back
up/server/servername/websites/rdiff-backup-data/error_log.2005-01-19T10:07:11+10:30.data.gz',
 mode 'wb' at 0x83026e0 0x8310b2c>>
 ignored
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound 
method GzipFile.__del__ of <gzip open file '/usr/back
up/server/servername/websites/rdiff-backup-data/mirror_metadata.2005-01-19T10:07:11+10:30.snapshot.gz',
 mode 'wb' at 0x8302770 0x8310c
8c>> ignored
%Traceback (most recent call last):
  File "/usr/local/bin/rdiff-backup", line 23, in ?
    rdiff_backup.Main.Main(sys.argv[1:])
  File "/usr/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 254, 
in Main
    take_action(rps)
  File "/usr/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 224, 
in take_action
    connection.PipeConnection(sys.stdin, sys.stdout).Server()
  File "/usr/local/lib/python2.4/site-packages/rdiff_backup/connection.py", 
line 347, in Server
    self.get_response(-1)
  File "/usr/local/lib/python2.4/site-packages/rdiff_backup/connection.py", 
line 309, in get_response
    try: req_num, object = self._get()
  File "/usr/local/lib/python2.4/site-packages/rdiff_backup/connection.py", 
line 225, in _get
    raise ConnectionReadError("Truncated header string (problem "
rdiff_backup.connection.ConnectionReadError: Truncated header string (problem 
probably originated remotely)

%umask                                                                          
                                                                 22