[Bug-wget] [GSoC Update] Week 2

[Didik Setiawan]

================================= SUMMARY ======================================
My public fork of Wget2 project is available here [1]. I will continuously keep
pushing my work so anyone interested can track me there. Feel free to
participate in the discussions going on merge request with my mentors. Your
feedback are highly appreciated.


=============================== INTRODUCTION ===================================
The purpose of this project is to use Libmicrohttpd as test suite for Wget2.  I
plan to do this by do some changes on function wget_test_start_server() also
wget_test_stop_server() on src/libtest.c of Wget2. With this approach, I don't
need to change existing test suite which call the internal server code through
functions mentioned above. I've count there are 36 test file which use
wget_test_start_server(). I must ensure all the test passed.
And for installation prerequisite, I must ensure that Libmicrohttpd are included
when building Wget2 binary. Then I need to modify configure.ac. I will give
proper warning about this requirement. There is a section in README.md where I
must explain to user to provide Libmicrohttpd to make all test running
correctly.
With Libmicrohttpd I can add new test using feature that not yet implemented in
old server code, but ready on Libmicrohttpd, such as HTTP authentication [2] and
concurrent request checking. 

Mentors:
Darshit Shah <address@hidden>
Ander Juaristi <address@hidden>


================================== UPDATES =====================================
Things which were done in this week:

 * I have finished modify configure.ac to include Libmicrohttpd into Wget2. I
   keep my work in this branch [3] of my repository.
 * I have ensured that all make check passed on several testing machine
   including: Debian/GCC, Fedora/Clang, MingW64 and OSX.
   Fix from previous week:
   - Previous work just ask to install Libmicrohttpd as requirement, but not
     include it when build Wget2 binary. Based on discussion with Christian
     Grothoff and Tim Rühsen, Libmicrohttpd still need to provided as
     prerequisite for Wget2. For some operating systems, I need to provide
     contrib script to resolve this issue.
 * Started working on wget_test_start_server(). Workflow to resolve this:
   - Disable initial process for HTTP server socket.
   - Disable _http_server_thread, instead call new function which call
     Libmicrohttpd.
   - Create _http_server() function, wrapper for Libmicrohttpd. There is also
     function ahc_eco() which use to create proper HTTP response.


================================= NEXT STEPS ===================================
Things which would be done in the coming week:

 * Finished on wget_test_start_server() in order to call Libmicrohttpd as
   service for wget_test(). Problems and questions need to be resolved:
   - Decide what the best threading model for Libmicrohttpd. Currently using
     MHD_USE_INTERNALLY_POLLING_THREAD which use external select. I still check
     the comparison with legacy code that use Wget2 API wget_thread_start.
   - http_server_port still hardcoded.
   - In ahc_eco() of Libmicrohttpd, urls data still using static checking for
     matching with requested urls. In other word, it's hardcoded. Need to be
     changed to dynamic method to accomodate variadic data.
   - https still not touched yet.
   - What to do with FTP and FTPS functions? Since Libmicrohttpd just provide
     service for HTTP. Do we need keep the function for FTP{s}, or removing it?
   - Last check failed when the test try to resolve URL with question mark.
     E.g: "/subdir1/subpage1.html?query&param", when I debug, it return just
     "/subdir1/subpage1.html" so the result is 404 not found. I also check using
     logging example source code provided in Libmicrohttpd tutorial [4]. When I
     access using http client such as Wget2 and Firefox, the result is still the
     same. The URL result omit the query part. Need to confirm to Libmicrohttpd
     side about this, whether it is intended behaviour or not.
 * Make sure all test suite running correctly.


[1]: https://gitlab.com/dstw/wget2
[2]: 
https://www.gnu.org/software/libmicrohttpd/manual/libmicrohttpd.html#microhttpd_002ddauth
[3]: https://gitlab.com/dstw/wget2/tree/use-mhd
[4]: https://www.gnu.org/software/libmicrohttpd/tutorial.html#logging_002ec

Regards,
Didik Setiawan