Re: Branch Version_2_7_0pre

[Camm Maguire]

Greetings, and thank you again for your feedback.  We are converging.

On 3), after reading up on vfork, and examining your logs, it is indeed
deprecated.  This has been in use in 2.6 for many years -- have you been
applying a patch there as well?  If I recall, the issue with normal fork
was not so much the small performance penalty calling gcc, but oom
issues if gcl was called upon to use a substantial proportion of system
memory. vfork does not copy the caller's page tables.  I suppose it is
worthwhile migrating to posix_spawn for everyone.  I will look into
this.  In the meantime your patch working around to fork seems best.

My understanding is that macosx is just deciding not to support this
going forward given the compiler warnings.  I do not think the gcl code
is in error here.  My vfork manpage says:

       (From POSIX.1) The vfork() function has the same effect as fork(2), ex‐
       cept  that  the behavior is undefined if the process created by vfork()
       either modifies any data other than a variable of type  pid_t  used  to
       store  the  return  value from vfork(), or returns from the function in
       which vfork() was called, or calls any other function  before  success‐
       fully calling _exit(2) or one of the exec(3) family of functions.

And the GCL code in question here is (o/unixsys.c):

  if (!(pid=pvfork())) {
    errno=0;
    execvp(*p1,(void *)p1);
    _exit(128|(errno&0x7f));
  }

I'm guessing macosx is mallocing the environment. Or perhaps it is the
profile timer blocking.  You might want to try replacing pvfork above
with vfork.  If this works it is a simple fix confining the signal
handling to the parent.

On 5) if you could please do the following at the stopped part of the
build:

cd unixport
./saved_pre_gcl
(in-package :si)
(trace dlopen dlsym)
(mdlsym "memcmp")

On 1), I put AC_CHECK_INCLUDES_DEFAULT in for you which should have done
this automatically.  Do not know why it fails, but I am reverting now in
favor of your explicit stdio.h inserts.

On 2), I've created an empty gprof.ini to accompany the empty gprof.o
which hopefully should work for you.  Again your solution will break
gprof where it is available.

On 4), as I had indicated, the current bootstrap process is needlessly
slow and is there just to check all modes of operation.  saved_pre_gcl
operates from evaluated lisp source, saved_gcl0 from compiled at safety
3.  saved_gcl is fully optimized.  Until we address this at final
release, may I suggest keeping a build around, and when testing a
change, do

cd unixport
./saved_gcl
> (time (load "boot.lisp"))
> 
make saved_gcl

For me this takes 240 seconds.

Take care,

"Kirill A. Korinsky" <kirill@korins.ky> writes:

> I've switched to b66c55853870ce15554bc4d9b911410dd07d8a45
>
> 1. readline still failing, inside config.log
>
> In file included from conftest.c:89:
> In file included from /opt/local/include/readline/readline.h:36:
> /opt/local/include/readline/rltypedefs.h:71:29: error: unknown type name 
> 'FILE'
> typedef int rl_getc_func_t (FILE *);
>                             ^
> In file included from conftest.c:89:
> /opt/local/include/readline/readline.h:450:21: error: unknown type name 'FILE'
> extern int rl_getc (FILE *);
>                     ^
> /opt/local/include/readline/readline.h:591:8: error: unknown type name 'FILE'
> extern FILE *rl_instream;
>        ^
> /opt/local/include/readline/readline.h:592:8: error: unknown type name 'FILE'
> extern FILE *rl_outstream;
>        ^
> /opt/local/include/readline/readline.h:954:3: error: unknown type name 'FILE'
>   FILE *inf;
>   ^
> /opt/local/include/readline/readline.h:955:3: error: unknown type name 'FILE'
>   FILE *outf;
>   ^
> 6 errors generated.
>
> So, I've used my patch to add stdio.h
>
> 2. Unfortunately empty `gprof.c` won't work. It fails like this:
>
> ouch raw_pre_gcl_map
> ccache /usr/bin/clang -L/opt/local/lib -Wl,-headerpad_max_install_names 
> -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -arch 
> x86_64 -Wl,-no_pie -m64 -Wl,-headerpad,72 -o raw_pre_gcl   -L.   -lpre_gcl 
> `echo -lm  -lgmp -lreadline | sed -e
> 's/-lncurses/ /'` -lc -lgclp
> ld: warning: -no_pie is deprecated when targeting new OS versions
> Undefined symbols for architecture x86_64:
>   "_fSgprof_addresses_init", referenced from:
>       _NewInit in libpre_gcl.a(new_init.o)
>   "_fSmcleanup_init", referenced from:
>       _NewInit in libpre_gcl.a(new_init.o)
>   "_fSmonstartup_init", referenced from:
>       _NewInit in libpre_gcl.a(new_init.o)
>   "_fSptable_alloc_length_init", referenced from:
>       _NewInit in libpre_gcl.a(new_init.o)
>   "_fSptable_init", referenced from:
>       _NewInit in libpre_gcl.a(new_init.o)
> ld: symbol(s) not found for architecture x86_64
> clang: error: linker command failed with exit code 1 (use -v to see 
> invocation)
> make[1]: *** [raw_pre_gcl_map] Error 1
> make[1]: Leaving directory 
> `/opt/local/var/macports/build/_Users_catap_src_macports-ports_lang_gcl/gcl-devel/work/09740e74a3bc2fa09fca9b5478a0269a9aebf767/gcl/unixport'
> make: *** [unixport/saved_pre_gcl] Error 2
>
> So, the second patch is back.
>
> 3. To have a nice backtrace I must replace install_segmentation_catcher with 
> dummy function, to avoid replacing macOS handler which allows to use standard 
> toolset to get the stack trace.
>
> Build output:
>
> GCL (GNU Common Lisp)  2.7.0 Thu Oct 26 12:00:01 PM EDT 2023  CLtL1    git: 
> Version_2_7_0pre9
> Source License: LGPL(gcl,gmp), GPL(unexec,bfd,xgcl)
> Binary License:  GPL due to GPL'ed components: (READLINE UNEXEC)
> Modifications of this banner must retain notice of a compatible license
> Dedicated to the memory of W. Schelter
>
> Use (help) to get some basic information on how to use GCL.
> Temporary directory for compiler files set to 
> /opt/local/var/macports/build/_Users_catap_src_macports-ports_lang_gcl/gcl-devel/work/.tmp/
>
>>;; Loading "boot.lisp"
> ;; Compiling 
> /opt/local/var/macports/build/_Users_catap_src_macports-ports_lang_gcl/gcl-devel/work/.tmp/gazonk_52249_0.lsp.
> ;; End of Pass 1.  
> ;; End of Pass 2.  
> /bin/sh: line 1: 52248 Done                    echo '(time (load 
> "boot.lisp"))'
>      52249 Segmentation fault: 11  | ../unixport/saved_pre_gcl
> make: *** [unixport/saved_gcl0] Error 139
> rm h/mcompdefs.h
>
> System's crash report:
>
> -------------------------------------
> Translated Report (Full Report Below)
> -------------------------------------
>
> Process:               saved_pre_gcl [52249]
> Path:                  /Volumes/VOLUME/*/saved_pre_gcl
> Identifier:            saved_pre_gcl
> Version:               ???
> Code Type:             X86-64 (Native)
> Parent Process:        Exited process [52247]
> Responsible:           Terminal [641]
> User ID:               502
>
> Date/Time:             2023-12-22 10:43:47.9710 +0100
> OS Version:            macOS 12.7.1 (21G920)
> Report Version:        12
> Bridge OS Version:     8.1 (21P1069)
> Anonymous UUID:        F4C73193-A183-1E9B-1CEC-012963CBBDCA
>
> Sleep/Wake UUID:       A9712D0A-7160-471B-A09D-44186439375B
>
> Time Awake Since Boot: 710000 seconds
> Time Since Wake:       1582 seconds
>
> System Integrity Protection: enabled
>
> Crashed Thread:        0  Dispatch queue: com.apple.main-thread
>
> Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
> Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000000028
> Exception Codes:       0x0000000000000001, 0x0000000000000028
> Exception Note:        EXC_CORPSE_NOTIFY
>
> Termination Reason:    Namespace SIGNAL, Code 11 Segmentation fault: 11
> Terminating Process:   exc handler [52249]
>
> VM Region Info: 0x28 is not in any region.  Bytes before following region: 
> 140737488199640
>       REGION TYPE                    START - END         [ VSIZE] PRT/MAX 
> SHRMOD  REGION DETAIL
>       UNUSED SPACE AT START
> --->  
>       VM_ALLOCATE              7ffffffda000-7ffffffdb000 [    4K] r-x/r-x 
> SM=ALI  
>
> Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
> 0   libsystem_malloc.dylib           0x7ff817d3772c _malloc_fork_prepare + 80
> 1   libSystem.B.dylib                0x7ff822d06759 libSystem_atfork_prepare 
> + 55
> 2   libsystem_c.dylib                0x7ff817e1b8a8 vfork + 27
>
> Thread 0 crashed with X86 Thread State (64-bit):
>   rax: 0x0000000000000000  rbx: 0x0000000000000001  rcx: 0x0000000000000103  
> rdx: 0x0000000000000103
>   rdi: 0x0000000100727298  rsi: 0x0000000000000000  rbp: 0x00007ff7bfef2d20  
> rsp: 0x00007ff7bfef2d10
>    r8: 0x0000000000f02083   r9: 0x00000000000020a0  r10: 0x00007ff85963ac98  
> r11: 0x0000000000000206
>   r12: 0x00000001000cd7d5  r13: 0x000000010071e710  r14: 0x0000000000000000  
> r15: 0x00007ff7bfef2e00
>   rip: 0x00007ff817d3772c  rfl: 0x0000000000010202  cr2: 0x0000000000000028
>   
> Logical CPU:     4
> Error Code:      0x00000004 (no mapping for user data read)
> Trap Number:     14
>
> Thread 0 instruction stream:
>   4c 89 f8 48 83 c4 10 5b-41 5c 41 5e 41 5f 5d c3  L..H...[A\A^A_].
>   55 48 89 e5 41 56 53 4c-8b 35 f6 59 94 41 65 8b  UH..AVSL.5.Y.Ae.
>   0c 25 18 00 00 00 31 c0-f0 0f b1 0d 24 5a 94 41  .%....1.....$Z.A
>   74 11 48 8d 3d 1b 5a 94-41 be 00 00 05 00 e8 c1  t.H.=.Z.A.......
>   23 01 00 83 3d 02 ae 92-41 00 74 22 31 db 48 8b  #...=...A.t"1.H.
>   05 47 ac 92 41 48 8b 3c-d8 48 ff c3 48 8b 47 60  .G..AH.<.H..H.G`
>  [ff]50 28 8b 05 e3 ad 92-41 48 39 c3 72 e0 4d 85  .P(.....AH9.r.M. <==
>   f6 74 09 4c 89 f0 5b 41-5e 5d ff e0 5b 41 5e 5d  .t.L..[A^]..[A^]
>   c3 c3 90 90 90 55 48 89-e5 41 57 41 56 41 55 41  .....UH..AWAVAUA
>   54 53 50 49 89 fe 83 bf-e0 42 00 00 00 7e 7e 45  TSPI.....B...~~E
>   31 ff 45 31 e4 49 8b 86-f0 42 00 00 4b 8d 0c a4  1.E1.I...B..K...
>   48 c1 e1 09 48 8d 1c 08-4c 8d 2c 08 49 83 c5 04  H...H...L.,.I...
>
> Binary Images:
>     0x7ff817d26000 -     0x7ff817d51fff libsystem_malloc.dylib (*) 
> <2ea05464-fac2-32fe-8e03-94252869554d> /usr/lib/system/libsystem_malloc.dylib
>     0x7ff822d05000 -     0x7ff822d06fff libSystem.B.dylib (*) 
> <de1a244d-06de-372c-836c-28ee72f17d4d> /usr/lib/libSystem.B.dylib
>     0x7ff817dd7000 -     0x7ff817e5ffff libsystem_c.dylib (*) 
> <4ecd1661-9d96-3669-bb31-4c6d5c685d4c> /usr/lib/system/libsystem_c.dylib
>                0x0 - 0xffffffffffffffff ??? (*) 
> <00000000-0000-0000-0000-000000000000> ???
>
> External Modification Summary:
>   Calls made by other processes targeting this process:
>     task_for_pid: 0
>     thread_create: 0
>     thread_set_state: 0
>   Calls made by this process:
>     task_for_pid: 0
>     thread_create: 0
>     thread_set_state: 0
>   Calls made by all processes on this machine:
>     task_for_pid: 0
>     thread_create: 0
>     thread_set_state: 0
>
> VM Region Summary:
> ReadOnly portion of Libraries: Total=140.7M resident=0K(0%) 
> swapped_out_or_unallocated=140.7M(100%)
> Writable regions: Total=32.4G written=0K(0%) resident=0K(0%) 
> swapped_out=0K(0%) unallocated=32.4G(100%)
>
>                                 VIRTUAL   REGION 
> REGION TYPE                        SIZE    COUNT (non-coalesced) 
> ===========                     =======  ======= 
> Kernel Alloc Once                    8K        1 
> MALLOC                           137.2M       11 
> MALLOC guard page                   24K        5 
> MALLOC_NANO (reserved)           384.0M        1         reserved VM address 
> space (unallocated)
> STACK GUARD                          4K        1 
> Stack                             64.0M        1 
> VM_ALLOCATE                          8K        2 
> __DATA                            38.6M       42 
> __DATA_CONST                       352K       37 
> __DATA_DIRTY                        58K       22 
> __HEAP                            32.0G       24 
> __LINKEDIT                       134.3M        9 
> __OBJC_RO                         82.9M        1 
> __OBJC_RW                         3200K        2 
> __TEXT                            6624K       47 
> dyld private memory               1024K        1 
> shared memory                        8K        1 
> ===========                     =======  ======= 
> TOTAL                             32.8G      208 
> TOTAL, minus reserved VM space    32.5G      208 
>
> 4. 10 minutes isn't more than 2 hours. Something dramatical slow here. The 
> time of building of 2.6.14 hadn't created any issue as far as I recall.
>
> 5. about
>
> ERROR "There is no package with the name \"libc\"."
>
> I've attached the whole build output and config.log

-- 
Camm Maguire                                        camm@maguirefamily.org
==========================================================================
"The earth is but one country, and mankind its citizens."  --  Baha'u'llah