[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Bug-wget] [bug #49429] Option to update a mirror of a section of a web

From: Dale Worley
Subject: [Bug-wget] [bug #49429] Option to update a mirror of a section of a web site
Date: Sat, 17 Dec 2016 20:33:01 -0000
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:34.0) Gecko/20100101 Firefox/34.0


                 Summary: Option to update a mirror of a section of a web site
                 Project: GNU Wget
            Submitted by: worley
            Submitted on: Mon 24 Oct 2016 07:42:37 PM GMT
                Category: Feature Request
                Severity: 3 - Normal
                Priority: 5 - Normal
                  Status: None
                 Privacy: Public
             Assigned to: None
         Originator Name: worley
        Originator Email: 
             Open/Closed: Open
         Discussion Lock: Any
                 Release: 1.18
        Operating System: None
         Reproducibility: None
           Fixed Release: None
         Planned Release: None
              Regression: None
           Work Required: None
          Patch Included: None



Once a copy of a web site has been made with --mirror, it would be useful to
have a --mirror-update option that could be used to cause wget to check and
update the mirror copy as necessary.

This is rather messy to do.  Among other things, wget would need to maintain a
status file in which the mapping from URLs to local files is kept, and the
entries in it need to be retained forever (in case a previously-deleted URL
becomes live again and some local page's converted link to that URL remains).

Given that such a status file is needed, it might as well be used to log "work
yet to be done" in the update and periodically be written to disk to
checkpoint the wget run.

Related tracker items are:
http://savannah.gnu.org/bugs/?49226  Add possibility to append to existing
WARC file
http://savannah.gnu.org/bugs/?34415  Add option to delete local
files/directories if they do not exist on the server
http://savannah.gnu.org/bugs/?33372  Quick resume for mirroring operation.
https://savannah.gnu.org/bugs/?25340  --mirror and --convert-links mixing


Reply to this item at:


  Message sent via/by Savannah

reply via email to

[Prev in Thread] Current Thread [Next in Thread]