[Bug-wget] [bug #49429] Option to update a mirror of a section of a web site

[Dale Worley]

URL:
  <http://savannah.gnu.org/bugs/?49429>

                 Summary: Option to update a mirror of a section of a web site
                 Project: GNU Wget
            Submitted by: worley
            Submitted on: Mon 24 Oct 2016 07:42:37 PM GMT
                Category: Feature Request
                Severity: 3 - Normal
                Priority: 5 - Normal
                  Status: None
                 Privacy: Public
             Assigned to: None
         Originator Name: worley
        Originator Email: 
             Open/Closed: Open
         Discussion Lock: Any
                 Release: 1.18
        Operating System: None
         Reproducibility: None
           Fixed Release: None
         Planned Release: None
              Regression: None
           Work Required: None
          Patch Included: None

    _______________________________________________________

Details:

Once a copy of a web site has been made with --mirror, it would be useful to
have a --mirror-update option that could be used to cause wget to check and
update the mirror copy as necessary.

This is rather messy to do.  Among other things, wget would need to maintain a
status file in which the mapping from URLs to local files is kept, and the
entries in it need to be retained forever (in case a previously-deleted URL
becomes live again and some local page's converted link to that URL remains).

Given that such a status file is needed, it might as well be used to log "work
yet to be done" in the update and periodically be written to disk to
checkpoint the wget run.

Related tracker items are:
http://savannah.gnu.org/bugs/?49226  Add possibility to append to existing
WARC file
http://savannah.gnu.org/bugs/?34415  Add option to delete local
files/directories if they do not exist on the server
http://savannah.gnu.org/bugs/?33372  Quick resume for mirroring operation.
https://savannah.gnu.org/bugs/?25340  --mirror and --convert-links mixing
poorly?





    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?49429>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/