[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[rdiff-backup-users] OverflowError reading get_sigs() response in very l
From: |
Charles Duffy |
Subject: |
[rdiff-backup-users] OverflowError reading get_sigs() response in very large tree |
Date: |
Sat, 13 Jan 2007 16:18:20 +0000 |
User-agent: |
Thunderbird 1.5.0.9 (Windows/20061207) |
Running a backup with a very large number of files (about 150,000) on
both ends, one of the RPC requests returns a response larger than
PY_SSIZE_T_MAX:
Traceback (most recent call last):
File "/usr/bin/rdiff-backup", line 23, in ?
rdiff_backup.Main.Main(sys.argv[1:])
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line
286, in Main
take_action(rps)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line
256, in take_action
elif action == "backup": Backup(rps[0], rps[1])
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line
306, in Backup
backup.Mirror_and_increment(rpin, rpout, incdir)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/backup.py", line
49, in Mirror_and_increment
dest_sigiter = DestS.get_sigs(dest_rpath)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/connection.py",
line 445, in __call__
return apply(self.connection.reval, (self.name,) + args)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/connection.py",
line 365, in reval
result = self.get_response(req_num)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/connection.py",
line 314, in get_response
try: req_num, object = self._get()
File "/usr/lib64/python2.3/site-packages/rdiff_backup/connection.py",
line 239, in _get
data = self._read(length)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/connection.py",
line 208, in _read
return self.inpipe.read(length)
OverflowError: requested number of bytes is more than a Python string
can hold
This is, needless to say, a Bad Thing. There are certainly a few fixes
possible (such as chunking the signatures -- perhaps by directory)...
but if anyone can think of something better (or is familiar with
upstream Python changes which might impact this issue, or otherwise has
something to add/suggest), I'm very much all ears.
- [rdiff-backup-users] OverflowError reading get_sigs() response in very large tree,
Charles Duffy <=