Re: [Bug-wget] [GSoC Proposal] Python3-based FTP Server for Test Suit and Test Suit Refactoring

[Zihang Chen]

Oh, right. Didn't come up with the second way. I'll take that into
consideration. Thanks.
BTW, how should the final proposal look like? Should I make a pdf file out
of latex or just plain text? I'm a first-time GSoCer.



2014-03-07 22:35 GMT+08:00 Darshit Shah <address@hidden>:

>
>
>
> On Fri, Mar 7, 2014 at 3:32 PM, 陈子杭 (Zihang Chen) <address@hidden>wrote:
>
>> Thanks for your comments! :D
>>
>>
>> 2014-03-07 22:17 GMT+08:00 Darshit Shah <address@hidden>:
>>
>> Hi Zihang,
>>>
>>> The proposal seems good.
>>>
>>>
>>> On Fri, Mar 7, 2014 at 11:39 AM, 陈子杭 (Zihang Chen) <address@hidden>wrote:
>>>
>>>> * this is only a draft, so it's not written according to the GNU GSoC
>>>> template. When discussion is done, I'll get a formal one done (with
>>>> schedule etc.) *
>>>>
>>>> Briefing
>>>> ====
>>>>
>>>> As is known, a new Python3-based test suite was implemented to replace
>>>> the
>>>> old one. And a configurable HTTP server utilizing `http.server` and
>>>> `http.BaseHTTPRequestHandler` was also implemented. Thus the HTTP server
>>>> can be configured to respond certain headers according to the
>>>> configuration
>>>> in separate test cases.
>>>>
>>>> However, a configurable FTP server was not required to be implemented,
>>>> which breaks the consistency of the test suite of wget. So it's
>>>> necessary
>>>> that a configurable FTP server be built.
>>>>
>>>> Initial Design
>>>> ====
>>>>
>>>> The test suite consists of two parts: `WgetTest.py` (module that
>>>> actually
>>>> runs the tests), servers (configurable HTTP and FTP server). When
>>>> running
>>>> test cases, `HTTPTest` instances collect configurations from test cases
>>>> and
>>>> start the servers with them so that the HTTP handler will handle the
>>>> incoming requests according to the configurations (also known as rules).
>>>>
>>>> About the FTP server, I think the handler architecture can be applied
>>>> here:
>>>> implement multiple handlers corresponding to the FTP commands and
>>>> register
>>>> them in server instances automatically. When registering handlers, the
>>>> server should also inform the handlers about the configurations. When
>>>> new
>>>> FTP command arrives, the server invoke the corresponding handler, and
>>>> then
>>>> the handler look up the rules itself and do the expected actions. When
>>>> doing real FTP actions, I intend to use third-party FTP library like the
>>>> current FTP server (I fear sockets so best not directly touch them).
>>>>
>>>> There's two things that are unclear:
>>>> * the necessity of implementing all the FTP command. I presume that wget
>>>> wouldn't utilize all of them.
>>>>
>>> If you're using a library, e.g. pyftpdlib, I don't see why you'd have to
>>> implement
>>> any commands on your own.
>>>
>> If the server is configurable, I think I'll have to intercept the
>> commands so I can apply rules on them. When I say implement FTP command, I
>> mean creating handle for the command, and invoke library in the handle. I
>> think that's what mislead you probably.
>>
> Oh. Okay then. In that scenario, you could probably add the command
> handlers
> iteratively as and when they are required by a test case.
>
> Another way would be to look at Wget's FTP client implementation and see
> what is the
> superset of commands that it may send. If you're not comfortable with C, I
> would help
> you out with that.
>
>>    Simply use the library for it.
>>>
>>>> * what will the configurations be, do I need to design the tests
>>>> myself? Or
>>>> should I also port the Perl scripts to Python3?
>>>>
>>> Porting the existing Perl tests should give you a good idea of the
>>> various
>>> configurations that will be needed. Looking up the Wget source is also
>>> often
>>> a good idea. Either ways, some of us can help out with identifying and
>>> writing
>>> test cases when the time comes.
>>>
>>>
>>>> Test Suite Refactoring
>>>> ====
>>>>
>>>> While reading source, I don't find the test suite pythonic. Multiple
>>>> parts
>>>> of it seems unmaintainable. If the test suite can be refactored into a
>>>> better coded one,  maintainability can further be achieved.
>>>>
>>>> Yes. I agree the current python based test suite might advantage from a
>>> few refactorings. There are a few places where the test suite is quite
>>> un-pythonic
>>> and fixing those would be a nice idea.
>>>
>>> Examples of unpythonicness:
>>>> * docstrings out of classes or methods
>>>> * mutable objects as default argument values
>>>> * multiple use of operator is (controversial)
>>>> * writing `_replace_substring (WgetTest.py:119)` instead of utilizing
>>>> the
>>>> format specification mini-language
>>>>
>>>> Examples of unmaintainability:
>>>> * referencing attributes which are defined in subclasses
>>>> (`WgetTest.py:61`
>>>> references `self.servers` which is defined in its subclass
>>>> `WgetTest.py:254`)
>>>> * inappropriate variable names (in `WgetTest.py:254` `self.servers`
>>>> actually becomes an integer while it regarded normally as a list of
>>>> servers)
>>>>
>>> Yes, the variable names are horrible aren't they? That should be
>>> refactored. (Shouldn't
>>> take much time ).
>>>
>>>> * no intuitive, self-declarative module hierarchy
>>>>
>>> The module hierarchy is slightly complicated. Making it more intuitive
>>> would help.
>>>
>>>>
>>>> About me
>>>> ====
>>>> My name's Zihang Chen. I'm a 3rd undergraduate student from China
>>>> (timezone: UTC+8). Please forgive me for any mistaken uses of the
>>>> English
>>>> language. I have been using Python for 3 years. I mostly code for myself
>>>> (assignment, interest etc.) Never worked for any open source
>>>> organizations
>>>> before, so I'm thrilled to apply for GSoC. You can visit my GitHub
>>>> page here<https://github.com/mad4alcohol>
>>>>
>>>> .
>>>>
>>>> Looking forward to your comments. Thanks very much.
>>>>
>>>
>>>
>>>
>>> --
>>> Thanking You,
>>> Darshit Shah
>>>
>>>
>>
>>
>> --
>> Regards,
>> Chen Zihang,
>> Computer School of Wuhan University
>> ---
>> 此致
>> 陈子杭
>> 武汉大学计算机学院
>>
>>
>
>
> --
> Thanking You,
> Darshit Shah
>
>


-- 
Regards,
Chen Zihang,
Computer School of Wuhan University
---
此致
陈子杭
武汉大学计算机学院