Re: [Maxima-discuss] Maxima works with gcl 2.7.1

[Camm Maguire]

Greetings!  Great!  This will be an interesting test case never before
encountered.

The issue is that allocating the minimum 1 page to each of the cell
types brings us to a minimum heap of 10Gb.  I need to meditate on how to
reserve memory beneath 2Gb in such a setting.

We have several options.

1) You could run using the large memory model by adding the following at
the bottom of unixport/init_raw.lsp.in:

(setq compiler::*default-large-memory-model-p* t)

Last I tested this costs 10% in performance.

2) We could ignore the system huge pagesize and max out at 2Mib by
amending configure.ac as follows:

if test -e /proc/meminfo ; then
   j=`awk '/^Hugepagesize:/ {if ($3!="kB") next;j=1;for (i=0;i<20 &&
   j!=$2;i++) j=j*2;
   if (j==$2) {if (i>11) i=11;printf("%d\n",i+10)}}' /proc/meminfo`
fi

This should automatically disable the madvise test next in configure, so
the kernel support will be gone, but you will still have a large pagesize.
I am not exactly sure how effective madvise MADV_HUGEPAGE is.

For now I favor option 2) if you would like to give it a try.

With this tremendous pagesize, of course my previous advice of export
GCL_MEM_BOUND=29 will fail, as your smallest heap is 10x this!  It is
hard for me to conceive of GCL making use of this scale, but that has
also been the case in the past and look where we are now.

This directly addresses the case with 2.7.1, but the gdb instructions I
sent earlier for master are still needed if you want to give that a try.

Take care,


David Scherfgen via Maxima-discuss
<maxima-discuss@lists.sourceforge.net> writes:

> Yes, I use 1 GiB hugepages on this machine. Commonly supported hugepage sizes 
> are 2 MiB and 1 GiB. It's not unusual to use the 1 GiB option, especially with
> virtualization.
>
> MemTotal:       131817344 kB
> MemFree:        29411484 kB
> MemAvailable:   29138728 kB
> Buffers:           18628 kB
> Cached:          1021516 kB
> SwapCached:         6620 kB
> Active:           224524 kB
> Inactive:          63028 kB
> Active(anon):      17732 kB
> Inactive(anon):     3124 kB
> Active(file):     206792 kB
> Inactive(file):    59904 kB
> Unevictable:      791724 kB
> Mlocked:           27744 kB
> SwapTotal:      16759804 kB
> SwapFree:       16563004 kB
> Zswap:                 0 kB
> Zswapped:              0 kB
> Dirty:                 0 kB
> Writeback:             0 kB
> AnonPages:         37684 kB
> Mapped:            64448 kB
> Shmem:              3108 kB
> KReclaimable:      56948 kB
> Slab:             206984 kB
> SReclaimable:      56948 kB
> SUnreclaim:       150036 kB
> KernelStack:        9464 kB
> PageTables:         4216 kB
> SecPageTables:         0 kB
> NFS_Unstable:          0 kB
> Bounce:                0 kB
> WritebackTmp:          0 kB
> CommitLimit:    32336828 kB
> Committed_AS:     549828 kB
> VmallocTotal:   34359738367 kB
> VmallocUsed:      250676 kB
> VmallocChunk:          0 kB
> Percpu:            31744 kB
> HardwareCorrupted:     0 kB
> AnonHugePages:         0 kB
> ShmemHugePages:        0 kB
> ShmemPmdMapped:        0 kB
> FileHugePages:         0 kB
> FilePmdMapped:         0 kB
> Unaccepted:            0 kB
> HugePages_Total:      96
> HugePages_Free:       96
> HugePages_Rsvd:        0
> HugePages_Surp:        0
> Hugepagesize:    1048576 kB
> Hugetlb:        100663296 kB
> DirectMap4k:      278288 kB
> DirectMap2M:     4882432 kB
> DirectMap1G:    128974848 kB
>
> Am Do., 24. Apr. 2025 um 21:31 Uhr schrieb Camm Maguire 
> <camm@maguirefamily.org>:
>
>  Following up to myself, I now think this is the problem, maybe in both
>  cases: 
>
>  page size:            1073741824 bytes
>
>  GCL will support Linux huge pages with considerable performance gain,
>  but this value when available should be around 2Mb.
>
>  I suspect a configure script failure:
>
>  j=""
>  if test -e /proc/meminfo ; then
>     j=`awk '/^Hugepagesize:/ {if ($3!="kB") next;j=1;for (i=0;i<20 && 
> j!=$2;i++) j=j*2;if (j==$2) printf("%d\n",i+10)}'  /proc/meminfo`
>  fi
>
>  Could you please send me `cat /proc/meminfo`.
>
>  Take care,
>
>  David Scherfgen via Maxima-discuss
>  <maxima-discuss@lists.sourceforge.net> writes:
>
>  > Thanks, any help is welcome!
>  > Camm, I think I already sent you the config and make logs from trying to 
> build the 2.7.1 tag, it was a few weeks ago before you went on a vacation or
>  something.
>  > But here it is again, plus new logs from trying to build the Git master:
>  >
>  > Version_2_7_1 tag: First error "ERROR: File /tmp/gazonk_625863_0.o has 
> been compiled for a restricted address space"
>  > config.log: https://pastebin.com/V4jCCAsH
>  > make.log: https://pastebin.com/1sY8UPMF
>  >
>  > master branch: First error "The assertion sigaltstack(&estack, 0)>=0 on 
> line 1285 of o/alloc.c in function gcl_init_alloc failed: Operation not 
> permitted"
>  > config.log: https://pastebin.com/8tgqpDfH
>  > make.log: https://pastebin.com/xJheBrEa
>  >
>  > I hope this helps.
>  >
>  > Best regards
>  > David
>  >
>  > Am Do., 24. Apr. 2025 um 18:19 Uhr schrieb Camm Maguire 
> <camm@maguirefamily.org>:
>  >
>  >  Greetings!
>  >
>  >  David Scherfgen via Maxima-discuss
>  >  <maxima-discuss@lists.sourceforge.net> writes:
>  >
>  >  > That's great news!
>  >  > Increasing numerical thresholds should not be a problem, the 
> differences are probably due to a different order of floating point 
> operations. By the
>  way,
>  >  some
>  >  > other Lisps also fail a few numerical tests, IIRC.
>  >  >
>  >  > I still haven't been able to build GCL on my Ubuntu 22.04.5 system. The 
> 2.7.1 source distribution (after patching as described on the website) leads 
> to
>  one
>  >  error,
>  >  > trying to build from the current Git master leads to another error. 
> Raymond, could you share your GCL build procedure?
>  >  >
>  >
>  >  I am more than happy to help with both of these if you could send me the
>  >  failed logs.
>  >
>  >  Take care,
>  >  -- 
>  >  Camm Maguire                                        camm@maguirefamily.org
>  >  ==========================================================================
>  >  "The earth is but one country, and mankind its citizens."  --  Baha'u'llah
>  >
>  > _______________________________________________
>  > Maxima-discuss mailing list
>  > Maxima-discuss@lists.sourceforge.net
>  > https://lists.sourceforge.net/lists/listinfo/maxima-discuss
>  >
>
>  -- 
>  Camm Maguire                                        camm@maguirefamily.org
>  ==========================================================================
>  "The earth is but one country, and mankind its citizens."  --  Baha'u'llah
>
> _______________________________________________
> Maxima-discuss mailing list
> Maxima-discuss@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/maxima-discuss
>

-- 
Camm Maguire                                        camm@maguirefamily.org
==========================================================================
"The earth is but one country, and mankind its citizens."  --  Baha'u'llah