Re: [Bug-wget] wget2.0 / niwt / refactoring

[Micah Cowan]

On 08/13/2012 03:06 AM, Daniel Stenberg wrote:
> On Mon, 13 Aug 2012, Tim Ruehsen wrote:
> 
>> But we should not forget about a monolithic, backward-compatibel (to
>> wget 1.x) wget 2.0. We all agree, it is time to redesign wget's code
>> architecture to have a clean codebase for new features to implement,
>> to increase readability/hackability, to increase modularity and to
>> increase code reusability (wget-derived libraries). Not to forget
>> current C language / library conventions and features like C99 and
>> Posix compatibility. On older systems wget 1.x would stay the
>> preferred version.
> 
> You should know I talk of my own project here, but I would like to
> remind people that when considering an overhaul of a (portable) HTTP/FTP
> project, there might be reasons to think about letting the transfers be
> handled by libcurl. It already supports more protocols, more SSL
> flavours and more features than the wget transfer engine and it exists
> as a completely free and open, proven and well documented library with a
> stable API today. Most of mget's "Not Yet Implemented" features are
> already supported by libcurl.
> 
> This said, I realize it is not kosher for a GNU project since libcurl is
> not GNU and I believe it will be a primary factor that will prevent this
> from happening. I still believe it would be smart from a technical
> perspecitive.

I don't know about it not being kosher. There's certainly no rule
against using Free Software that comes from outside GNU.

That said, when a GNU alternative is available, RMS often prefers it be
used instead. The promoted (but not required) policy to consider Guile
first for extension/configuration language is an example. And he had
previously said that once GNUTLS support in wget were complete, that
Wget should consider dropping OpenSSL support. There may well be more
than simply "prefer GNU" in that one, considering that OpenSSL requires
a license exemption for the GPL (which slowed us down considerably when
we moved to GPL 3.0, as they wanted to take time to use the right
language for the exemption with the new license, but didn't really get
around to it for a while - volunteer legal organizations tend to be
overworked I think :) ).

> Also, this is not a new idea and it has been turned down before:
>   http://daniel.haxx.se/blog/2007/10/27/wget-going-libcurl/ and
>   http://www.mail-archive.com/address@hidden/msg01129.html

Well, "turned down" may not be accurate. "Didn't get to it" may be more
likely. Certainly, it was considered to be a "Wget 2.x" feature as
opposed to "Wget 1.x", considering the major overhaul it represents, but
I agree that it'd be a smart way to instantly expand wget's protocol
support, expand the testing base for a major piece of wget's
functionality (to the union of curl's and wget's current user bases),
and push off maintenance of much of wget's to be someone else's
maintenance problem ;)

> So this is basically a set of patches on top of the stock Wget? If it
> is, have those patches been posted here? If it isn't, how exactly does
> it relate to wget other than being a similar tool?

It shares no code with current Wget, AFAICT. What I think he's
proposing, is to turn his new Mget into the foundation for a new Wget
2.0, pulling features one-by-one from current Wget 1.x, but redesigning
them to be cleaner, until you have a Wget 2.x, functionally equivalent
to Wget 1.x, but completely rewritten.

Personally, I'm inclined to favor an approach that builds Wget 2.x from
a Wget 1.x base, and in particular its bevy of test cases (if they're up
to date? I don't know if they are), to rewrite things bit by bit to be
cleaner, taking advantage of the tests to ensure feature stability. I
have yet to see a true, complete rewrite-from-scratch of a codebase that
didn't result in loss of features, so I suspect a rewritten Wget 2.x
would either fall short of its backward-compatibility goal, or it would
never reach completion.

A piece-wise improved Wget 1.x has the advantage of never being
functionally incomplete, while at the same time being able to take
immediate advantage of any design improvements. IMO, a 30% cleaned-up
Wget is more useful than an 80% completed Wget-rewrite.

-mjc