Re: Base tests on Windows

[Fred Kiefer]

I redirected this mail to the GNUstep developer list, as it makes more 
sense there.

On 29.03.2011 12:31, Riccardo Mottola wrote:
> I run the tests on windows and they don't look so bright. I think we
> should try to care about them! I run windows without ICU.
>
> 5461 Passed tests
> 58 Failed tests
> 11 Dashed hopes
> 9 Skipped sets
> 1 Failed file

I get similar results for MinGW, but don't get the failed file.

> $ grep Failed tests.log
> Failed test: test00.m:88 ... +dateWithString:calendarFormat:locale:
> objects to long timezone

Although this test relies on some internal behaviour of GNUstep, this 
isn't the reason for the failure. What goes wrong is that [NSTimeZone 
timeZoneWithName: @"Any string"] will return a value and not nil as 
expected. We should add a test for invalid time zone names to our test 
code and correct the MS Windows implementation to return nil, if the 
time zone name isn't valid.
The whole time zone classes look like they could do with a bit more 
testing. Also the keyed coding is missing and the initWithCode: method 
will leak self when called on a non place holder class. But all of this 
may have to wait till after the release.

> Failed test: general.m:44 ... newly created file is owned by current user
> Failed test: general.m:353 ... @encode(const char*) makes 'r*' type
> encoding

I didn't get these error.

> Failed test: ../NSString/NSString_tests.h:309 ... character set
> encoding/decoding works

This fail seems to be caused by missing support for the BIG5 character 
encoding. This could be ignored, but the reason for this failure again 
turns out the be more interesting. The configure fails to detect the 
iconv lib. This is caused by a compile error for our test code that 
report "undefined reference __errno". If you google for that error most 
postings state that this is caused by using a wrong library, one that 
isn't suited for MinGW. Adam, could you please look into this? Where 
does our libiconv come from and could it be replaced by a different one?
This may be a specific problem of my machine and already fixed for a 
newer MinGW release of GNUstep.

> Failed test: test00.m:102 ... NSDecimalNumber numberWithInt: works
> Failed test: test00.m:107 ... NSDecimalNumber doubleValue works
> Failed test: test00.m:111 ... NSDecimalNumber charValue works
> Failed test: test00.m:113 ... NSDecimalNumber intValue works
> Failed test: test00.m:115 ... NSDecimalNumber integerValue works
> Failed test: test00.m:117 ... NSDecimalNumber longValue works
> Failed test: test00.m:119 ... NSDecimalNumber longLongValue works
> Failed test: test00.m:121 ... NSDecimalNumber shortValue works
> Failed test: test00.m:123 ... NSDecimalNumber unsignedCharValue works
> Failed test: test00.m:125 ... NSDecimalNumber unsignedIntValue works
> Failed test: test00.m:127 ... NSDecimalNumber unsignedIntegerValue works
> Failed test: test00.m:129 ... NSDecimalNumber unsignedLongValue works
> Failed test: test00.m:133 ... NSDecimalNumber unsignedShortValue works

I don't understand what goes on here. The big difference to most other 
systems is of course that on Windows we don't have gmp so the fall back 
implementation of NSDecimal gets used instead. Contrary to the inline 
documentation that code is less tested than the gmp implementation. 
Still if I switch to that code on my Linux system all the tests pass. We 
will have to debug this on Windows and this may take some time.
In the first case what happens is that we get 199 returned when we 
initialised the NSDecimalNumber with 200. No idea how this could happen.

> Failed file: test00.m aborted without running all tests!
> Failed test: test01.m:229 ... Proxy GSFinePoint
> Failed test: thread.m:182 ... A loop with an input source will block
> Failed test: socket_cs.m:324 ... Local tcp (blocking open)
> Failed test: socket_cs.m:354 ... Local socket
> Failed test: socket_cs.m:385 ... Local socket (blocking open)

I did not look into these tests, nor into the URL tests below as I 
expect the test code to be rather Unix specific.

> Failed test: NSString_tests.h:309 ... character set encoding/decoding works
> Failed test: NSString_tests.h:309 ... character set encoding/decoding works

See above. This is just the same tests repeated. No idea why we are 
doing this.

> Failed test: order.m:23 ... UTF-16 BE OK
> Failed test: order.m:35 ... UTF-16 LE OK
> Failed test: order.m:48 ... UTF-32 BE OK
> Failed test: order.m:61 ... UTF-32 LE OK

Again these tests fail because we cannot use iconv for the conversion. 
We could think about adding direct implementations for these encodings 
into GNUstep.

> Failed test: test02.m:251 ... /home/../nicola stringByStandardizingPath
> == /nicola
> Failed test: test02.m:255 ... /here/and/there/../../nicola
> stringByStandardizingPath == /here/nicola
> Failed test: test02.m:259 ... /here/../../nicola
> stringByStandardizingPath == /nicola
> Failed test: test02.m:289 ... foo->bar absolute symlink expanded by
> stringByResolvingSymlinksInPath
> Failed test: test02.m:293 ... foo->bar relative symlink expanded by
> stringByResolvingSymlinksInPath
> Failed test: test02.m:302 ... /.. stringByStandardizingPath == /
> Failed test: basic.m:78 ... Path of file URL /usr is /usr
> Failed test: basic.m:80 ... File URL /usr is file://localhost/usr/
> Failed test: basic.m:82 ... resourceSpecifier of /usr is //localhost/usr/

Looks like the whole path name handling is broken on Windows.

> Failed test: basic.m:199 ... path quoting
> Failed test: basic.m:206 ... complex -path
> Failed test: basic.m:214 ... complex -absoluteString
> Failed test: basic.m:215 ... complex -relativeString
> Failed test: basic.m:216 ... complex -description
> Failed test: basic.m:247 ... can load file URL with achor
> Failed test: test00.m:121 ... respond with keepalive 0 (pause 0) OK
> Failed test: test01.m:56 ... keep-alive test 0 OK
> Failed test: test01.m:56 ... keep-alive test 1 OK
> Failed test: test01.m:56 ... keep-alive test 2 OK
> Failed test: test01.m:56 ... keep-alive test 3 OK
> Failed test: test01.m:56 ... keep-alive test 4 OK
> Failed test: test01.m:56 ... keep-alive test 5 OK
> Failed test: test01.m:56 ... keep-alive test 6 OK
> Failed test: test01.m:56 ... keep-alive test 7 OK
> Failed test: test01.m:56 ... keep-alive test 8 OK
> Failed test: test01.m:56 ... keep-alive test 9 OK
> Failed test: test01.m:44 ... Got the correct data from a 200 - status load
> Failed test: test01.m:46 ... 200 - status: Handle load succeeded
> Failed test: test01.m:52 ... 401 - status: Handle load not loaded
> (unanswered auth challenge)
> Failed test: test01.m:58 ... 404 - status: Handle load not loaded (resource
> not found)
>
> Did somebody else try to run the tests on windows? How do they fare?

I did :-)
And I am rather surprised how well we are doing there. The NSTimeZone 
difference needs to be fixed and we need to look into the iconv problem 
at some time. But none of this, nor the NSDecimal bugs should stop a 
release.

Fred