Re: [Bug-wget] [PATCH] Update HSTS info

[Ander Juaristi]

No worries ;-)

As I told Tim off list, I'm working on the improvements.

Sent from my smartphone. Excuse my brevity.

---- Darshit Shah escribió ----

>Hi Ander,
>
>Could you please take a look at dkg's suggestions and recreate a patch?
>
>On Mon, Aug 10, 2015 at 9:36 PM, Daniel Kahn Gillmor
><address@hidden> wrote:
>> Hi Ander--
>>
>> Thanks for this!  having the concise, human-readable description of the
>> intended behavior is really useful.
>>
>> a couple comments on the proposed documentation below:
>>
>> On Sat 2015-08-08 13:50:17 -0400, Ander Juaristi wrote:
>>
>>> Those two patches add two extra debug traces for HSTS, and most 
>>> importantly, update the documentation to include information about HSTS.
>>  [...]
>>> address@hidden address@hidden
>>> +By default, Wget stores its HSTS database in @file{~/.wget-hsts}.
>>> +You can use @samp{--hsts-file} to override this. Wget will use
>>> +the supplied file as the HSTS database. Such file must conform to the
>>> +correct HSTS database format used by Wget. If Wget cannot parse the 
>>> provided
>>> +file, the behaviour is unspecified.
>>
>> There is no pointer or reference here to what the "correct HSTS database
>> format used by Wget" is.  providing a brief pointer (e.g. "look at file
>> doc/FOO in the wget source" or "see https://example.org/wget-hsts-db";)
>> would be useful.
>
>I agree. We should supply a sample hsts config file for the users,
>similar to the wgetrc file that is available in the tarball.
>
>>
>>> +Be aware though, that Wget may modify the provided file if any change 
>>> occurs
>>> +between the HSTS policies requested by the remote servers and those in the
>>> +file. When Wget exists, it effectively updates the HSTS database by 
>>> rewriting
>>> +the database file with the new entries.
>>
>> Is it possible that a user would want to decouple reading from
>> (respecting) an HSTS file and writing to it?  some examples:
>>
>>  * A wget process might have no ability to modify the filesystem (e.g.
>>    a constrained wget -O- in a pipeline), but wants to respect a
>>    system-provided HSTS ruleset.  This might want to read an hsts file
>>    without trying to write to it (or at least without raising an error
>>    on an attempted modification).
>>
>This also allows system admins to create a hsts config file in a
>location where the current user does not have write-access.
>
>>  * A wget process used as part of network testing suite might want to
>>    disobey any system-provided HSTS rules (it needs to emulate behavior
>>    of "ignorant" clients), but might also want to collect the HSTS rules
>>    it sees for later review.  This might want to update an hsts file
>>    without trying to respect its contents.
>>
>> I don't know if these use cases warrant the additional option complexity
>> of accomodating them, but it'd be good to make that decision explicitly
>> (sorry if that's already been done and i've just missed the discussion.
>>
>>> +Care is taken not to override possible changes made by other Wget 
>>> processes at
>>> +the same time over the HSTS database. Before dumping the updated HSTS 
>>> entries
>>> +on the file, Wget will re-read it and merge the changes.
>>
>> This sounds like there might still be a race condition where one
>> process's changes get clobbered.  can we make any atomicity guarantees
>> for users who might be concerned about this?
>>
>>> +Using a custom HSTS database and/or modifying an existing one is 
>>> discouraged.
>>> +For more information about the potential security threats arised from such 
>>> practice,
>>> +see section 14 "Security Considerations" of RFC 6797, specially section 
>>> 14.9
>>> +"Creative Manipulation of HSTS Policy Store".
>>
>> I'd normally characterize these threats as "privacy concerns", not
>> necessarily "security concerns".  users of wget might understand them
>> most closely as offering "cookie-equivalent" mechanisms for some
>> http/https clients.
>>
>> sorry these questions and considerations are in text form and not as
>> patches.  feel free to take from them whatever you might find useful.
>>
>> Regards,
>>
>>         --dkg
>>
>
>
>
>-- 
>Thanking You,
>Darshit Shah