Re: [Gcl-devel] trying to finalize the windows issues ...

[Camm Maguire]

Greetings!

"Mike Thomas" <address@hidden> writes:

> Hi Camm.
> 
> Sorry for the abrupt, piecemeal, disjoint work on this stuff but I am really
> pressed for time on all fronts at the moment.
> 
> Your attempts to sort it and general input and discussion are much
> appreciated.
> 
> I've included a short stack test program below with some run results to
> illustrate the discussion.
> 
> 
> | > Some progress to report with an 8M C stack:
> | >
> | > On a --enable-debug build using gcc 3.3.1 and friends and with
> | an 8M stack
> | > hack the Maxima ignore-errors bug did not manifest and a HEAD
> | branch random
> | > tester (from a couple of weeks ago) ran for 1000 iterations.
> | >
> | > Building without --enable-debug led to the same result on Maxima, but I
> | > foolishly  updated the HEAD random tester before running in
> | optimised mode
> | > only to discover that it no longer compiles with 2.6.1 GCL due to:
> | >
> |
> | I assume you mean here that with 3.3.1 and 8M stack, no ignore-errors
> | bug in maxima with or without optimization.
> 
> No.  With optimisation it fails, if I recall correctly, loading
> "universe.lsp" in the ANSI test.
> 

I was just talking about the ignore-errors bug.  This (alone) is gone
at all opt levels with 3.3.1/8Mb stack, yes?  Am really interested in
3.3.3/3.4.0 on the expanded stack at opt levels beneath that
triggering the universe.lsp load bug.

> |
> | One way to figure out if this is luck or real is to verify that the
> | error elimination persists for values above 8M, and is reintroduced
> | for some value between 8 and 2 M.
> |
> | On BSD like systems, there is a setrlimit/getrlimit pair of system
> | calls.  One can limit the stack and trigger a segfault on an attempted
> | function call which would push the stack pointer out of bounds.  Does
> | such exist on mingw?
> |
> | How are you increasing the stack?
> 
> See separate email; In short --stack.
> 
> |  Are you changing _stacktop and
> | _stackbottom?
> 
> Yes; in main.c set to the value fed to the linker (8x1024x1024 = 8388608):
> 
> #ifdef _WIN32
>         {
>           unsigned int dummy;
> 
>           _stackbottom = (unsigned int ) &dummy;
>           _stacktop    = _stackbottom - 8388608; // ???
> 
>         }
> #endif
> 
> 
> |  Are you adding gcc flags (i.e. --stack below)?
> 
> Yes, see above.
> 
> |  What is
> | the natural stack limit be default on mingw?
> 
> >From an earlier email:
> 
> "The ld docs at
> http://sources.redhat.com/binutils/docs-2.15/ld/Options.html#Options
> 
> say that:
> 
> "--stack reserve
> --stack reserve,commit
> Specify the amount of memory to reserve (and optionally commit) to be used
> as stack for this program. The default is 2Mb reserved, 4K committed. [This
> option is specific to the i386 PE targeted port of the linker]"
> "
> 
> | Do you have a ulimit -a
> | shell command?
> 
> Yes, under Unix emulation shells like Bash, but I doubt that it has much
> meaning on Windows as the stack size seems to be built into the executable
> and as far as I am aware there is no equivalent command in the normal
> Windows command prompt - maybe I'm wrong though.  Please note that I am very
> much feeling my way on these facts.
> 
> Also if it needs a Bash shell it ain't what we want for GCL.
> 

Agreed about bash.  Do you have setrlimit/getrlimit (I'm assuming yes
if bash can do ulimit).

> 
> | >
> | ==========================================================================
> | > gcc: --stack=8388608: linker input file unused because linking not done
> | > OPTIMIZE levels: Safety=3, Space=0, Speed=3
> | > Finished compiling ansi-aux.lsp.
> | > Loading ansi-aux.o
> | > Error in PCL::LOAD-DEFMETHOD [or a callee]: No class named:
> | RANDOM-STATE.
> | >
> | ==========================================================================
> | >
> | > However, reverting to version 1.95 of that file avoided that problem.
> | > Unfortunately the optimised compile crash in universe.lsp is
> | still there:
> | >
> | >
> | ==========================================================================
> | > OPTIMIZE levels: Safety=3, Space=0, Speed=3
> | > Finished compiling ansi-aux-macros.lsp.
> | > Loading ansi-aux-macros.o
> | > start address -T 107f0680 Finished loading ansi-aux-macros.o
> | > Loading universe.lsp
> | >
> | > Unrecoverable error: Segmentation violation..
> | >
> | > This application has requested the Runtime to terminate it in an unusual
> | > way.
> | > Please contact the application's support team for more information.
> | > make: *** [mike-test] Error 3
> | >
> | ==========================================================================
> |
> | This one should be backtraceable, no?
> 
> With difficulty possibly - I'll have to try turning on debugging information
> and also find the time to do it - very difficult at the moment.
> 
> | Have you already sent me one on
> | this, i.e. with -g -O?, where '?' is the minimum opt flag causing the
> | crash?
> 
> I think so.
> 
> 
> 
> |  This is gone with no opt on, right?
> 
> Yes, I believe it is a high optimisation bug; off the top of my head
> possibly only with the default optimisation as set by CVS configure.
> 
> >
> | > I am proceeding to try with gcc 3.4.0 to see whether the
> | optimised randowm
> | > tester compiler failure goes away.
> | >
> | > Either way with minor mods to the Windows stack via the
> | compiler options I
> | > am hoping Camm's new 2.6.2 will at least cover the main test
> | bases if built
> | > using --enable-debug.
> | >
> | > I'm wary as I don't understand how a short C stack would have caused the
> | > ignore-errors path handling failure - the test passes may just
> | be good luck.
> |
> |
> | Well, to my understanding the default marking algorithm won't traverse
> | the C stack properly, hence mark_c_stack_carefully, as in the comment:
> |
> | static void
> | mark_object(object x) {
> |
> |   long i;
> |   int j;
> |   object *p;
> |   char *cp;
> |
> |   cs_check(x);
> |  BEGIN:
> |   /* if the body of x is in the c stack, its elements
> |      are marked anyway by the c stack mark carefully, and
> |      if this x is somehow hanging around in a cons that
> |      should be dead, we dont want to mark it. -wfs
> |   */
> |
> |   if (NULL_OR_ON_C_STACK(x))
> |
> |
> | Furthermore, one absolutely needs to ensure that the heap and the
> | stack don't collide, but this should be checked for in the snippet
> | involving NULL_OR_ON_C_STACK in main.c at startup.
> 
> I spent a bit of time on that macro recently and I believe it is OK.  The
> Windows C stack is much lower and grows downwards whereas the GCL heap
> starts at 0x10100000 and grows upwards.
> 
> For example, here is a short test program to show what happens with the C
> stack:
> 
> ==========================================================================
> $ cat ./stacktest.c
> #include <stdio.h>
> void print_local_addr ( unsigned int i )
> {
>     char c;
>     fprintf ( stderr, "Invocation %d, local char at %x (%d)\n", i, &c, &c );
>     if ( i <= 10 ) {
>         print_local_addr ( i + 1 );
>     }
> }
> 
> int main ( void )
> {
>     char d;
>     fprintf ( stderr, "At start of main, local char at %x (%d)\n", &d, &d );
>     print_local_addr ( 1 );
>     exit ( 1 );
> }
> ==========================================================================
> 
> and here is what happens by default:
> 
> ==========================================================================
> $ gcc stacktest.c -o stacktest.exe
> 
> $ ./stacktest
> At start of main, local char at 22ff7f (2293631)
> Invocation 1, local char at 22ff57 (2293591)
> Invocation 2, local char at 22ff37 (2293559)
> Invocation 3, local char at 22ff17 (2293527)
> Invocation 4, local char at 22fef7 (2293495)
> Invocation 5, local char at 22fed7 (2293463)
> Invocation 6, local char at 22feb7 (2293431)
> Invocation 7, local char at 22fe97 (2293399)
> Invocation 8, local char at 22fe77 (2293367)
> Invocation 9, local char at 22fe57 (2293335)
> Invocation 10, local char at 22fe37 (2293303)
> Invocation 11, local char at 22fe17 (2293271)
> ==========================================================================
> 
> and with a bigger stack (8x1024x1024 as tested in GCL).
> 
> ==========================================================================
> $ gcc -Wl,--stack=8388608 stacktest.c -o stacktest.exe
> 
> $ ./stacktest
> At start of main, local char at c0ff7f (12648319)
> Invocation 1, local char at c0ff57 (12648279)
> Invocation 2, local char at c0ff37 (12648247)
> Invocation 3, local char at c0ff17 (12648215)
> Invocation 4, local char at c0fef7 (12648183)
> Invocation 5, local char at c0fed7 (12648151)
> Invocation 6, local char at c0feb7 (12648119)
> Invocation 7, local char at c0fe97 (12648087)
> Invocation 8, local char at c0fe77 (12648055)
> Invocation 9, local char at c0fe57 (12648023)
> Invocation 10, local char at c0fe37 (12647991)
> Invocation 11, local char at c0fe17 (12647959)
> ==========================================================================
> 
> 
> 
> | I'll try to go over your sbrk emulation just in case there could be a
> | clue in there -- does this sound possible?
> 
> Anything sounds possible right now - or - I must confess I haven't a clue
> and haven't the time to check right now.  I've been combing string.d and
> pathname.d to look for oddities in the value stack handling hence the patch
> to "string.d" mentioned in the separate email.
> 
> 
> 
> |  Also, it would be great at
> | one point to have one sbrk emulation layer shared between windows and
> | macosx.
> 
> True.
> 
> Cheers
> 
> Mike Thomas
> 
> 
> 
> 

Take care,

-- 
Camm Maguire                                            address@hidden
==========================================================================
"The earth is but one country, and mankind its citizens."  --  Baha'u'llah