[rdiff-backup-users] Re: "OverflowError: long int too large to convert to int"

[Erik Forsberg]

Maik Schreiber <address@hidden> writes:

> > A few days ago, I started getting these errors once for each backup of
> > my home directory structure:
> >
> > OverflowError: long int too large to convert to int
> 
> I'm seeing the same. It looks like rdiff-backup has problems with large 
> files. 
> I had a 3.2 GB file lying around on which it choked. After deleting it 
> everything went as normal again.

Hmm.. It turned out that the file (/home/ingwa/.xsession-errors) on
the backup destination was 6.9Gigabytes. Not really what you'd expect
from that particular file..

Anyway, how do I recover? My initial idea was to remove the file on
the destination directory, and then do:

mv rdiff-backup.tmp.8385 .xsession-errors

(rdiff-backup.tmp.8385 was obviously a .xsession-errors file in
disguise).

I then ran rdiff-backup --check-destination, but got the following
error:

address@hidden root]# rdiff-backup --check-destination /export/home/
Traceback (most recent call last):
  File "/usr/bin/rdiff-backup", line 24, in ?
    rdiff_backup.Main.Main(sys.argv[1:])
  File "/usr/lib/python2.2/site-packages/rdiff_backup/Main.py", line 250, in Mai
n
    take_action(rps)
  File "/usr/lib/python2.2/site-packages/rdiff_backup/Main.py", line 232, in tak
e_action
    elif action == "check-destination-dir": CheckDest(rps[0])
  File "/usr/lib/python2.2/site-packages/rdiff_backup/Main.py", line 618, in Che
ckDest
    dest_rp.conn.regress.Regress(dest_rp)
  File "/usr/lib/python2.2/site-packages/rdiff_backup/regress.py", line 70, in R
egress
    for rf in iterate_meta_rfs(mirror_rp, inc_rpath): ITR(rf.index, rf)
  File "/usr/lib/python2.2/site-packages/rdiff_backup/rorpiter.py", line 279, in
 __call__
    last_branch.fast_process(*args)
  File "/usr/lib/python2.2/site-packages/rdiff_backup/regress.py", line 233, in 
fast_process
    if rf.metadata_rorp.isreg(): self.restore_orig_regfile(rf)
  File "/usr/lib/python2.2/site-packages/rdiff_backup/regress.py", line 255, in 
restore_orig_regfile
    tf.write_from_fileobj(rf.get_restore_fp())
  File "/usr/lib/python2.2/site-packages/rdiff_backup/restore.py", line 456, in 
get_restore_fp
    Rdiff.write_patched_fp(current_fp, delta_fp, new_fp)
  File "/usr/lib/python2.2/site-packages/rdiff_backup/Rdiff.py", line 65, in wri
te_patched_fp
    rpath.copyfileobj(librsync.PatchedFile(basis_fp, delta_fp), out_fp)
  File "/usr/lib/python2.2/site-packages/rdiff_backup/rpath.py", line 58, in cop
yfileobj
    inbuf = inputfp.read(blocksize)
  File "/usr/lib/python2.2/site-packages/rdiff_backup/librsync.py", line 76, in 
read
    self._add_to_outbuf_once()
  File "/usr/lib/python2.2/site-packages/rdiff_backup/librsync.py", line 85, in 
_add_to_outbuf_once
    if not self.infile_eof: self._add_to_inbuf()
  File "/usr/lib/python2.2/site-packages/rdiff_backup/librsync.py", line 95, in 
_add_to_inbuf
    new_in = self.infile.read(blocksize)
  File "/usr/lib/python2.2/gzip.py", line 163, in read
    self._read(readsize)
  File "/usr/lib/python2.2/gzip.py", line 227, in _read
    self._read_eof()
  File "/usr/lib/python2.2/gzip.py", line 243, in _read_eof
    isize = read32(self.fileobj)
  File "/usr/lib/python2.2/gzip.py", line 27, in read32
    return struct.unpack("<l", input.read(4))[0]
struct.error: unpack str size does not match format

Now I instead tried to remove the .xsession-errors completely, which
gave me the following:

address@hidden root]# rdiff-backup --check-destination /export/home
Traceback (most recent call last):
  File "/usr/bin/rdiff-backup", line 24, in ?
    rdiff_backup.Main.Main(sys.argv[1:])
  File "/usr/lib/python2.2/site-packages/rdiff_backup/Main.py", line 250, in Mai
n
    take_action(rps)
  File "/usr/lib/python2.2/site-packages/rdiff_backup/Main.py", line 232, in tak
e_action
    elif action == "check-destination-dir": CheckDest(rps[0])
  File "/usr/lib/python2.2/site-packages/rdiff_backup/Main.py", line 618, in Che
ckDest
    dest_rp.conn.regress.Regress(dest_rp)
  File "/usr/lib/python2.2/site-packages/rdiff_backup/regress.py", line 70, in R
egress
    for rf in iterate_meta_rfs(mirror_rp, inc_rpath): ITR(rf.index, rf)
  File "/usr/lib/python2.2/site-packages/rdiff_backup/rorpiter.py", line 279, in
 __call__
    last_branch.fast_process(*args)
  File "/usr/lib/python2.2/site-packages/rdiff_backup/regress.py", line 233, in 
fast_process
    if rf.metadata_rorp.isreg(): self.restore_orig_regfile(rf)
  File "/usr/lib/python2.2/site-packages/rdiff_backup/regress.py", line 261, in 
restore_orig_regfile
    rf.mirror_rp.write_from_fileobj(rf.get_restore_fp())
  File "/usr/lib/python2.2/site-packages/rdiff_backup/restore.py", line 450, in 
get_restore_fp
    current_fp = self.get_first_fp()
  File "/usr/lib/python2.2/site-packages/rdiff_backup/restore.py", line 465, in 
get_first_fp
    if not first_inc.isinccompressed(): return first_inc.open("rb")
  File "/usr/lib/python2.2/site-packages/rdiff_backup/rpath.py", line 802, in op
en
    else: return open(self.path, mode)
IOError: [Errno 2] No such file or directory: '/export/home/ingwa/.xsession-erro
rs'

I guess now that've broken the golden rule: "DO NOT TAMPER WITH THE
DESTINATION DIRECTORY", I'm damned?

\EF
-- 
Erik Forsberg                Telephone: +46-13-21 46 00
Cendio AB                    Web: http://www.cendio.com