Re: [Duplicity-talk] Duplicity 4.2.1 on CentOS 4.7 causing resource shortage on VPS

[Marcus Clements]

2009/3/4 Kenneth Loafman <address@hidden>:
> Marcus Clements wrote:
>> Hi,
>>
>> I am happily using Duplicity 4.2.1 on my CentOS 4.7 VPS as a scheduled
>> nightly backup. Here are the statistics from last night (a typical
>> night):
>>
>> --------------[ Backup Statistics ]--------------
>> StartTime 1236141154.58 (Wed Mar  4 04:32:34 2009)
>> EndTime 1236142613.88 (Wed Mar  4 04:56:53 2009)
>> ElapsedTime 1459.30 (24 minutes 19.30 seconds)
>> SourceFiles 274433
>> SourceFileSize 11725383817 (10.9 GB)
>> NewFiles 439
>> NewFileSize 257124568 (245 MB)
>> DeletedFiles 235
>> ChangedFiles 1347
>> ChangedFileSize 447798267 (427 MB)
>> ChangedDeltaSize 0 (0 bytes)
>> DeltaEntries 2021
>> RawDeltaSize 133981593 (128 MB)
>> TotalDestinationSizeChange 276740588 (264 MB)
>> Errors 0
>> -------------------------------------------------
>>
>> I use munin to monitor the performance of the server in general and
>> there are no unusual or troubling resource problems reported there.
>> However in the Virtuozzo control panel, under Quality of Service,
>> every night during the backup the VPS runs out of lockedpages, which
>> are (from the Virtuozzo manual) "The memory not allowed to be swapped
>> out (locked with the mlock() system call), in pages."
>> This server is relatively new and not under much load currently but
>> that is likely to change in the coming months. I've talked to the
>> hosting provider, who doubled the lockedpages parameter a couple of
>> days ago in response to my request but QoS still repeatedly reports
>> "Yellow Zone", "Red Zone" and "Black Zone" resource shortages for
>> lockedpages.
>>
>> I'm concerned that the backup will make the server run slowly while it
>> is in operation. Is there a change I can make to the settings in
>> Duplicity or elsewhere in the system so that the process uses less of
>> the lockedpages resource?
>
> You're running an ancient version of duplicity, so I would first suggest
> you upgrade to the latest, 0.5.10, before doing much more analysis.  You
> can get the newest version at http://duplicity.nongnu.org/.
>
> As to mlock'ed pages -- duplicity does not explicitly lock any pages
> into memory, so I am going to have to ask what command line you used,
> and what is the target filesystem (Linux or Windows), and any other
> details you can give (Python version, etc.).
>
> ...Thanks,
> ...Ken
>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>

Thanks for the fast response. I know the version is ancient but
no-ones rolled an RPM for RHEL 4 that I can find and I was hoping not
to get into satisfying the dependencies by hand or upgrading the OS.
If this is vital for the smooth running of duplicity on this server
then maybe I will have to. I thought there might be a simple change to
volume size or other parameter maybe... <hopes>

Target filesystem is a shared webspace provided by Dreamhosters which
seems to be ext3

Python version (from rpm -q) on my vps is python-2.3.4-14.7.el4

I run a bash script with cron which contains:

repDate=`date +%Y%m%d`;

/usr/bin/duplicity --exclude /mnt --exclude /proc --exclude /dev
--exclude /cdrom --exclude /floppy --exclude /boot
scp://address@hidden//home/someone/duplicity_backup >>
/root/BackupLogs/Incremental_Backup_Log_${repDate}.log

thanks

Marcus