Re: [Bug-wget] [GSoC Proposal] Python3-based FTP Server for Test Suit and Test Suit Refactoring

[Darshit Shah]

On Fri, Mar 7, 2014 at 3:40 PM, 陈子杭 (Zihang Chen) <address@hidden>wrote:

> Oh, right. Didn't come up with the second way. I'll take that into
> consideration. Thanks.
> BTW, how should the final proposal look like? Should I make a pdf file out
> of latex or just plain text? I'm a first-time GSoCer.
>
> You'd have to use HTML. Since the proposals need to be sent through the
melange website which has a WYSWYG editor I guess
There's time for that though. Proposals open on 10th March. Check out the
format then and send in one.

>
>
> 2014-03-07 22:35 GMT+08:00 Darshit Shah <address@hidden>:
>
>
>>
>>
>> On Fri, Mar 7, 2014 at 3:32 PM, 陈子杭 (Zihang Chen) <address@hidden>wrote:
>>
>>> Thanks for your comments! :D
>>>
>>>
>>> 2014-03-07 22:17 GMT+08:00 Darshit Shah <address@hidden>:
>>>
>>> Hi Zihang,
>>>>
>>>> The proposal seems good.
>>>>
>>>>
>>>> On Fri, Mar 7, 2014 at 11:39 AM, 陈子杭 (Zihang Chen) <address@hidden>wrote:
>>>>
>>>>> * this is only a draft, so it's not written according to the GNU GSoC
>>>>> template. When discussion is done, I'll get a formal one done (with
>>>>> schedule etc.) *
>>>>>
>>>>> Briefing
>>>>> ====
>>>>>
>>>>> As is known, a new Python3-based test suite was implemented to replace
>>>>> the
>>>>> old one. And a configurable HTTP server utilizing `http.server` and
>>>>> `http.BaseHTTPRequestHandler` was also implemented. Thus the HTTP
>>>>> server
>>>>> can be configured to respond certain headers according to the
>>>>> configuration
>>>>> in separate test cases.
>>>>>
>>>>> However, a configurable FTP server was not required to be implemented,
>>>>> which breaks the consistency of the test suite of wget. So it's
>>>>> necessary
>>>>> that a configurable FTP server be built.
>>>>>
>>>>> Initial Design
>>>>> ====
>>>>>
>>>>> The test suite consists of two parts: `WgetTest.py` (module that
>>>>> actually
>>>>> runs the tests), servers (configurable HTTP and FTP server). When
>>>>> running
>>>>> test cases, `HTTPTest` instances collect configurations from test
>>>>> cases and
>>>>> start the servers with them so that the HTTP handler will handle the
>>>>> incoming requests according to the configurations (also known as
>>>>> rules).
>>>>>
>>>>> About the FTP server, I think the handler architecture can be applied
>>>>> here:
>>>>> implement multiple handlers corresponding to the FTP commands and
>>>>> register
>>>>> them in server instances automatically. When registering handlers, the
>>>>> server should also inform the handlers about the configurations. When
>>>>> new
>>>>> FTP command arrives, the server invoke the corresponding handler, and
>>>>> then
>>>>> the handler look up the rules itself and do the expected actions. When
>>>>> doing real FTP actions, I intend to use third-party FTP library like
>>>>> the
>>>>> current FTP server (I fear sockets so best not directly touch them).
>>>>>
>>>>> There's two things that are unclear:
>>>>> * the necessity of implementing all the FTP command. I presume that
>>>>> wget
>>>>> wouldn't utilize all of them.
>>>>>
>>>> If you're using a library, e.g. pyftpdlib, I don't see why you'd have
>>>> to implement
>>>> any commands on your own.
>>>>
>>> If the server is configurable, I think I'll have to intercept the
>>> commands so I can apply rules on them. When I say implement FTP command, I
>>> mean creating handle for the command, and invoke library in the handle. I
>>> think that's what mislead you probably.
>>>
>> Oh. Okay then. In that scenario, you could probably add the command
>> handlers
>> iteratively as and when they are required by a test case.
>>
>> Another way would be to look at Wget's FTP client implementation and see
>> what is the
>> superset of commands that it may send. If you're not comfortable with C,
>> I would help
>> you out with that.
>>
>>>    Simply use the library for it.
>>>>
>>>>> * what will the configurations be, do I need to design the tests
>>>>> myself? Or
>>>>> should I also port the Perl scripts to Python3?
>>>>>
>>>> Porting the existing Perl tests should give you a good idea of the
>>>> various
>>>> configurations that will be needed. Looking up the Wget source is also
>>>> often
>>>> a good idea. Either ways, some of us can help out with identifying and
>>>> writing
>>>> test cases when the time comes.
>>>>
>>>>
>>>>> Test Suite Refactoring
>>>>> ====
>>>>>
>>>>> While reading source, I don't find the test suite pythonic. Multiple
>>>>> parts
>>>>> of it seems unmaintainable. If the test suite can be refactored into a
>>>>> better coded one,  maintainability can further be achieved.
>>>>>
>>>>> Yes. I agree the current python based test suite might advantage from
>>>> a
>>>> few refactorings. There are a few places where the test suite is quite
>>>> un-pythonic
>>>> and fixing those would be a nice idea.
>>>>
>>>> Examples of unpythonicness:
>>>>> * docstrings out of classes or methods
>>>>> * mutable objects as default argument values
>>>>> * multiple use of operator is (controversial)
>>>>> * writing `_replace_substring (WgetTest.py:119)` instead of utilizing
>>>>> the
>>>>> format specification mini-language
>>>>>
>>>>> Examples of unmaintainability:
>>>>> * referencing attributes which are defined in subclasses
>>>>> (`WgetTest.py:61`
>>>>> references `self.servers` which is defined in its subclass
>>>>> `WgetTest.py:254`)
>>>>> * inappropriate variable names (in `WgetTest.py:254` `self.servers`
>>>>> actually becomes an integer while it regarded normally as a list of
>>>>> servers)
>>>>>
>>>> Yes, the variable names are horrible aren't they? That should be
>>>> refactored. (Shouldn't
>>>> take much time ).
>>>>
>>>>> * no intuitive, self-declarative module hierarchy
>>>>>
>>>> The module hierarchy is slightly complicated. Making it more intuitive
>>>> would help.
>>>>
>>>>>
>>>>> About me
>>>>> ====
>>>>> My name's Zihang Chen. I'm a 3rd undergraduate student from China
>>>>> (timezone: UTC+8). Please forgive me for any mistaken uses of the
>>>>> English
>>>>> language. I have been using Python for 3 years. I mostly code for
>>>>> myself
>>>>> (assignment, interest etc.) Never worked for any open source
>>>>> organizations
>>>>> before, so I'm thrilled to apply for GSoC. You can visit my GitHub
>>>>> page here<https://github.com/mad4alcohol>
>>>>>
>>>>> .
>>>>>
>>>>> Looking forward to your comments. Thanks very much.
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Thanking You,
>>>> Darshit Shah
>>>>
>>>>
>>>
>>>
>>> --
>>> Regards,
>>> Chen Zihang,
>>> Computer School of Wuhan University
>>> ---
>>> 此致
>>> 陈子杭
>>> 武汉大学计算机学院
>>>
>>>
>>
>>
>> --
>> Thanking You,
>> Darshit Shah
>>
>>
>
>
> --
> Regards,
> Chen Zihang,
> Computer School of Wuhan University
> ---
> 此致
> 陈子杭
> 武汉大学计算机学院
>
>


-- 
Thanking You,
Darshit Shah