[Bug-wget] [GSoC Update] Week 8

[Didik Setiawan]

================================= SUMMARY ======================================
My public fork of Wget2 project is available here [0]. I will continuously keep
pushing my work so anyone interested can track me there. Feel free to
participate in the discussions going on merge request with my mentors. Your
feedback are highly appreciated.


=============================== INTRODUCTION ===================================
The purpose of this project is to use Libmicrohttpd as test suite for Wget2.  I
plan to do this by do some changes on function wget_test_start_server() also
wget_test_stop_server() on src/libtest.c of Wget2. With this approach, I don't
need to change existing test suite which call the internal server code through
functions mentioned above.
I've count there are 38 test file which use wget_test_start_server(). I must
ensure all the test passed when I've done the transition.
For installation prerequisite, I make that Libmicrohttpd support will needed
when we need to run `make check` or run test suite. Then I need to modify
configure.ac. I will give proper warning about this requirement. There is a
section in README.md where I must explain to user to provide Libmicrohttpd to
make all test running correctly.
With Libmicrohttpd I can add new test using feature that not yet implemented in
old server code, but ready on Libmicrohttpd, such as HTTP digest authentication
and concurrent request checking. 

Mentors:
Darshit Shah <address@hidden>
Ander Juaristi <address@hidden>


================================== UPDATES =====================================
Things which were done until this week:

 * Started working on wget_test_start_server() to use Libmicrohttpd as HTTP
   server. Workflow to resolve this:
   - Remove initial process for HTTP server socket
   - Create _http_server_start() function, wrapper for Libmicrohttpd. There is
     also function answer_to_connection() which use to create proper HTTP
     response
   - Use select method (MHD_USE_SELECT_INTERNALLY) for threading model in
     Libmicrohttpd to get better compatibility
   - http_server_port seized automatically using Libmicrohttpd function by
     passing MHD_DAEMON_INFO_BIND_PORT or MHD_DAEMON_INFO_LISTEN_FD parameter to
     MHD_get_daemon_info() 
   - Using iteration to parse urls data in answer_to_connection(). This
     guarantee we can pass any variadic data to Libmicrohttpd and prevent
     segmentation fault 
   - Fix answer_to_connection() function to create proper HTTP response:
     - Handle arguments (query string) on URL
     - Handle redirection
     - Handle directory creation
     - Handle URL with IRI object
     - Handle URL with IDN hostname
     - Handle URL with query string which contains space
     - Handle If-Modified-Since header
     - Handle Byte Range request
     - Handle HTTP basic authentication 
     - Handle HTTPS
 * Fix compiler warning -Wunused-param caused by MHD API
 * Use persistent connection (Connection: keep-alive) instead multiple
   connection (Connection: close). This prevent hiding several problems of the
   connection between HTTP server and client


================================= NEXT STEPS ===================================
Things which would be done in the coming week:

 * Fix connection handler using chunked tranfer encoding
 * Fix CI testing on Gitlab MingW64 and Travis OSX
 * Add HTTP digest authentication test


[0]: https://gitlab.com/gnuwget/wget2/tree/tmp-use-mhd

Regards,
Didik Setiawan