Re: [Bug-wget] Google Summer of Code 2016

[Tim Ruehsen]

Hi Ander,

that would really be much work since many of the dependencies have to be 
implemented first. But maybe a some point in time, we have all the 
dependencies handled by libwget !? At that point, I would suggest a separate 
tool. Maybe Wget evolves into an ecosystem of web tools one day, who knows.

It is likely that the webdriver specs will change in many points until they 
become final.

Tim

On Sunday 06 March 2016 22:20:28 Ander Juaristi wrote:
> I just wanted to share with you another idea I was thinking on some time
> now: WebDriver [1].
> 
> It's basically a protocol/API to communicate with UAs. It's intended to
> be UA-agnostic, so any client should be able to use WebDriver to
> communicate with a compliant UA. From the standard:
> 
>      "WebDriver is a remote control interface that enables introspection
> and control of user agents. It provides a platform- and language-neutral
> wire protocol as a way for out-of-process programs to remotely instruct
> the behaviour of web browsers."
> 
> There are some requirements not at all supported in wget, such as XPath
> DOM traversal, so at first glance I can't give an estimate on whether
> how much time would be needed for this. It will not be too short, sure,
> but might be too big for a GSoC.
> 
> Regards,
> - AJ
> 
> [1] https://www.w3.org/TR/webdriver/
> 
> El 03/03/2016 a las 11:21, Tim Ruehsen escribió:
> > Just more ideas for you, Kushagra:
> > 
> > There are many command line options from Wget still missing in Wget2, you
> > should have a look at
> > https://github.com/rockdaboot/wget2/wiki anyways - feel free to work on
> > the
> > wiki yourself (e.g. fork the wiki pages:
> > https://help.github.com/articles/adding-and-editing-wiki-pages-locally/ or
> > let me know and I'll give you write access).
> > 
> > You can search the Wget bug tracker
> > (https://savannah.gnu.org/bugs/?group=wget) for wishlist items.
> > My favorite is https://savannah.gnu.org/bugs/?45803.
> > Special popen(2|3) functions/code is already in libwget/ directory.
> > E.g., that would allow Wget2 to be used as part of a recursive website
> > malware checker.
> > 
> > The authorization code in the test suite is not complete/not implemented -
> > I once tested authorization (MD5, MD5-sess) 'by hand' with my local
> > Apache. But a automated test is badly needed.
> > 
> > We thought of a statistic module (very basic code exists) for spider mode
> > to output diagnostics very detailed. Missing pages, response times,
> > server load (e.g. using the RTT/ping time), etc.
> > 
> > Tim
> > 
> > On Wednesday 02 March 2016 10:51:02 Kushagra Singh wrote:
> >> Hi,
> >> 
> >> Thanks for the quick reply. I went through the repository and the issues,
> >> and found a couple of things I would like to work on.
> >> 
> >> I have a couple of questions about Wget2. Is it a complete rewrite of the
> >> Wget project, available at git://git.savannah.gnu.org/wget.git, or are we
> >> using existing code and extending functionality? I guess it is the second
> >> one because I saw `libwget` in the repo. However if such is the case,
> >> then
> >> how do we change existing functions in wget? For example, implementing
> >> [2]
> >> would require making changes to the file cookies.c, which is present in
> >> /src in the wget repo, but not in /src in the wget2 repo.
> >> 
> >> I was looking at #43 [1], and have already submitted a patch for
> >> consideration for the first suggestion [2]. The second suggestion
> >> mentioned
> >> [3] is one of the things I'd like to work on, however this is not
> >> something
> >> which will take three months :)
> >> 
> >> Another project I am interested in, is implementing FTPS. I saw this
> >> listed
> >> under one of the ideas of GSoC 2015, but I'm not sure whether it was
> >> implemented, as I didn't see it under 'Development Status' in the wget2
> >> readme on Github.
> >> 
> >> Also, in #67 [4], we are talking about adhering to some specific parts of
> >> RFC 7230. I'm not sure which all parts would be right, as the discussion
> >> thread mentions that it won't be good to stick to each point of the RFC.
> >> WDYT?
> >> 
> >> 
> >> [1] https://github.com/rockdaboot/wget2/issues/43
> >> [2] https://tools.ietf.org/html/draft-west-leave-secure-cookies-alone-04
> >> [3] https://tools.ietf.org/html/draft-west-cookie-prefixes-05
> >> [4] https://github.com/rockdaboot/wget2/issues/67
> >> 
> >> On Tue, Mar 1, 2016 at 9:57 PM, Giuseppe Scrivano <address@hidden> 
wrote:
> >>> Kushagra Singh <address@hidden> writes:
> >>>> Hi,
> >>>> 
> >>>> Will we be taking part in GSoC this year? I would really like to work
> >>>> on
> >>> 
> >>> a
> >>> 
> >>>> project related to Wget this summer. Any specific ideas that are of
> >>>> importance to the community presently?
> >>> 
> >>> yes, we will be take part in GSoC.  I think we would like to see more
> >>> work happening on wget2, at the moment there is a list of issues on
> >>> 
> >>> github that can be useful to you to pick some ideas to work on:
> >>>    https://github.com/rockdaboot/wget2/issues
> >>> 
> >>> Could you take a look at it?  Do you see anything interesting that you
> >>> would like to work on?
> >>> 
> >>> Regards,
> >>> Giuseppe