Re: [Duplicity-talk] Source of message 'Attempt 1 failed. OSError: Insufficient free space on the filesystem'

[edgar . soldin]

On 12.07.2024 00:07, Jeffrey Walton wrote:
> On Thu, Jul 11, 2024 at 5:46 PM edgar.soldin--- via Duplicity-talk
> <duplicity-talk@nongnu.org> wrote:
> > On July 11, 2024 10:45:21 PM GMT+02:00, Jeffrey Walton via Duplicity-talk 
> > <duplicity-talk@nongnu.org> wrote:
> >
> > answering inline
> >
> > > Our nightly backup script is failing. According to journalctrl, we see:
> > >
> > > Jul 11 20:25:25 localhost daily-backup[11141]: Wiki database size is
> > > 78,637,039 bytes
> > > Jul 11 20:27:00 localhost daily-backup[11323]: Local and Remote
> > > metadata are synchronized, no sync needed.
> > > Jul 11 20:27:00 localhost daily-backup[11323]: Last inc backup left a
> > > partial set, restarting.
> > > Jul 11 20:27:00 localhost daily-backup[11323]: Last full backup date:
> > > Mon Jan  1 05:12:16 2024
> > > Jul 11 20:27:00 localhost daily-backup[11323]: RESTART: The first
> > > volume failed to upload before termination.
> > > Jul 11 20:27:00 localhost daily-backup[11323]:          Restart is
> > > impossible...starting backup from beginning.
> > > Jul 11 20:28:37 localhost daily-backup[11323]: Local and Remote
> > > metadata are synchronized, no sync needed.
> > > Jul 11 20:28:37 localhost daily-backup[11323]: Last full backup date:
> > > Mon Jan  1 05:12:16 2024
> > > Jul 11 20:28:57 localhost daily-backup[11323]: Attempt 1 failed.
> > > OSError: Insufficient free space on the filesystem.
> > > Jul 11 20:29:29 localhost daily-backup[11323]: Attempt 2 failed.
> > > OSError: Insufficient free space on the filesystem.
> > > Jul 11 20:30:02 localhost daily-backup[11323]: Attempt 3 failed.
> > > OSError: Insufficient free space on the filesystem.
> > > Jul 11 20:30:34 localhost daily-backup[11323]: Attempt 4 failed.
> > > OSError: Insufficient free space on the filesystem.
> > > Jul 11 20:31:06 localhost daily-backup[11323]: Giving up after 5
> > > attempts. OSError: Insufficient free space on the filesystem.
> > > Jul 11 20:31:06 localhost daily-backup[11141]: Failed to backup VM to
> > > Remote Backup Storage
> > > Jul 11 20:31:06 localhost systemd[1]: daily-backup.service: Main
> > > process exited, code=exited, status=1/FAILURE
> > >
> > > My question is, is the message "Attempt 1 failed. OSError:
> > > Insufficient free space on the filesystem" (and friends) coming from
> > > Duplicity?
> >
> > yes
>
> Thank you sir. My Google-foo was off. I could not locate a discussion
> of the message.

no biggie

> > > And if it is Duplicity, is the message referring to the local
> > > filesystem or remote filesystem?
> >
> > local
>
> Oh, that is interesting. I'm putzing around the remote filesystem using sshfs.

hmm, in that case your "remote" is local as well, so could be the reason as well

> > > Thanks in advance.
> >
> > 1. which duplicity version?
>
> $ lsb_release -a
> Distributor ID: Ubuntu
> Description:    Ubuntu 20.04.6 LTS
> Release:        20.04
> Codename:       focal
>
> $ duplicity --version
> duplicity 0.8.12

generally unmaintained and very old. consider upgrading.

> > 2. what does 'df -h' say?
>
> # local machine (not backup server)
> $ df -h
> Filesystem             Size  Used Avail Use% Mounted on
> udev                   938M     0  938M   0% /dev
> tmpfs                  194M  680K  194M   1% /run
> /dev/mapper/vg00-lv01   77G   26G   48G  35% /
> tmpfs                  970M     0  970M   0% /dev/shm
> tmpfs                  5.0M     0  5.0M   0% /run/lock
> tmpfs                  970M     0  970M   0% /sys/fs/cgroup
> /dev/vda1              456M  156M  266M  37% /boot
> tmpfs                  194M     0  194M   0% /run/user/0

don't see anything suspicious there like a full /tmp partition or such

> > 3. what's the output of 'duplicity collection-status ...'?
>
> Using the sftp:// url from
> <https://github.com/weidai11/website/blob/master/systemd/bitvise-backup#L178>,

okay, that's NOT sshfs, that's sftp and can't raise that OSError afaik.

> the [long] output is available at
> <https://cryptopp.com/cryptopp-backup.txt>.

i see you started shortening chains to 2 months. good decision! otherwise 
nothing obvious there.

please run the backup from cmd line with debug verbosity ('-v9') and send me 
the _complete_ output privately. check it private information you may want to 
obfuscate first.

sunny regards ..ede