Re: [Duplicity-talk] Fwd: Re: Duplicity process killed

[Chris Vanden Berghe]

Hi Edgar,

I have run duplicity with file:// instead of gs:// and it yields the
same result.

nice /usr/bin/duplicity --verbosity 9 --full-if-older-than 1M
--archive-dir=/root/Backup/cache/ --include-globbing-filelist
/root/Backup/backup.list --exclude '**' / file:///home/chrisvdb/btsync/temp

1) first run (with empty ~chrisvdb/btsync/temp): runs for an hour or so
to end with 'killed'
2) second run (with 257 files totaling 51GB in ~chrisvdb/btsync/temp):
runs for a few min or so to end with 'killed'

I have attached the console output (of the second run) and the
backup.list file.

Regards,
Chris.

---
On 15/07/2016 18:59, edgar.soldin--- via Duplicity-talk wrote:
> ok
>  http://docs.pythonboto.org/en/latest/releasenotes/v2.41.0.html
> does not list any memory specific fix.
> remember now you already use duplicity 0.7.08 .. so no updates necessary.
>
> next would probably be patching duplicity to output memory usage when in max 
> verbosity. don't see anybody doing that unfortunately, do you offer :) ?  
> i'll cc Ken the maintainer, maybe he has another idea?
>
> to make sure it is duplicity and not the backend code, you could do a backup 
> w/ identical settings to a local file:// target and see if the second run 
> sports the same symptoms you experience now.
>
> additionally can you send me a _complete_ terminal output of the killed 
> duplicity run. you may want obfuscate strings in it beforehand.
>
> ..ede/duply.net
>
> On 15.07.2016 12:37, edgar.soldin--- via Duplicity-talk wrote:
>> ok. before we go further - please make sure you have the latest duplicity 
>> release and latest boto, if not install them locally (sent you the mini 
>> howto earlier).
>>
>> can you run a verify or does that also die assuming you run a backup command?
>>
>> ..ede
>>
>> On 15.07.2016 11:45, Chris Vanden Berghe via Duplicity-talk wrote:
>>> Scrolling down was too obvious, I guess ;-) I have attached a screenshot
>>> of what I see a second before Duplicity is killed.
>>>
>>> The source files are ~82 GB. The last successful backup:
>>>
>>> # /usr/bin/duplicity collection-status --archive-dir=/root/Backup/cache/
>>> gs://bobo-backup/
>>> Local and Remote metadata are synchronized, no sync needed.
>>> Last full backup date: Mon Jul  4 17:11:39 2016
>>> Collection Status
>>> -----------------
>>> Connecting with backend: BackendWrapper
>>> Archive dir: /root/Backup/cache/01f81df3eb37282f76c93fb89a3708f5
>>>
>>> Found 0 secondary backup chains.
>>>
>>> Found primary backup chain with matching signature chain:
>>> -------------------------
>>> Chain start time: Mon Jul  4 17:11:39 2016
>>> Chain end time: Mon Jul  4 17:11:39 2016
>>> Number of contained backup sets: 1
>>> Total number of contained volumes: 254
>>>  Type of backup set:                            Time:      Num volumes:
>>>                 Full         Mon Jul  4 17:11:39 2016               254
>>> -------------------------
>>> No orphaned or incomplete backup sets found.
>>>
>>> Regards,
>>> Chris.
>>>
>>> ---
>>> On 15/07/2016 15:36, edgar.soldin--- via Duplicity-talk wrote:
>>>>> On 15.07.2016 05:39, Chris Vanden Berghe via Duplicity-talk wrote:> Hi 
>>>>> Edgar,
>>>>>> I was unsuccessful in getting useful data from htop. It seems like tree
>>>>>> view and sorting on memory usage features are mutually exclusive in
>>>>>> htop... so when I enable tree view I no longer see the duplicity process
>>>>>> (off screen) and when I sort on memory I do not see Duplicity's children
>>>>>> (if any).
>>>>> hehe, read into how htop works. you can scroll the list w/ the cursor 
>>>>> keys.
>>>>>
>>>> sort by tree _and_ memory are mutually exclusive. so you have to sort by 
>>>> tree and scroll to duplicity, make the terminal big enough to see all it's 
>>>> sub processes and look out for the culprit yourself.
>>>>
>>>> btw. how big is your backup. size of data, number/size of volumes per 
>>>> full/chain.
>>>>
>>>> ..ede
>>>>
>>>>
>>>> _______________________________________________
>>>> Duplicity-talk mailing list
>>>> address@hidden
>>>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>>
>>>
>>> _______________________________________________
>>> Duplicity-talk mailing list
>>> address@hidden
>>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>>
>> _______________________________________________
>> Duplicity-talk mailing list
>> address@hidden
>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk

duplicity.txt
Description: Text document

backup.list
Description: Text document