Re: 64-bit indexing -- Test machine available with 128GB ram 32 CPUs 12T scuzzy on Fedora31

[Kai Torben Ohlhus]

On 2/11/20 8:12 AM, Dyer, Norman Jay wrote:
> 
> [...]  If your system supports Singularity [1,2]
> (free and open source software), you can try out my test image [3] and
> run the example from the wiki [4] in it's memory extensive form (32 GB)
> for testing:
> 
>    clear all;
>    N = 2^31;
>    ## The following lines requires about 32 GB of RAM!
>    a = ones (N, 1);
>    b = ones (N, 1);
>    c = a' * b
> 
> [...] 
> [1] https://sylabs.io/singularity/
> [2] https://github.com/sylabs/singularity
> [3] https://github.com/siko1056/GNU-Octave-64-Singularity
> [4]
> https://wiki.octave.org/Enable_large_arrays:_Build_octave_such_that_it_can_use_arrays_larger_than_2Gb.
>
>
> Yes, I have been running similar NxN arrays with random numbers
> increasing N until it breaks.
> (as predicted always breaks at 2G elements)
>
> GNU Octave, version 5.1.0
> Octave was configured for "x86_64-redhat-linux-gnu".
>
>>> clear all;
>>>
>>>    N = 2^31;
>>>
>>>    ## The following lines requires about 32 GB of RAM!
>>>
>>>    a = ones (N, 1);
> error: out of memory or dimension too large for Octave's index type
>>>
>>>    b = ones (N, 1);
> error: out of memory or dimension too large for Octave's index type
>>>
>>>    c = a' * b
>
> Obviously still stuck at 32-bit indexing.  Need a linux Octave 5.2  or
> 6.0 set up for 64-bit indexing, which evidently exists - but only
> Windows is posted at Octave.org ; awaiting distro managers to each
> release their own.
> I presume containerizing it would not alter this result; isn't that
> correct?
> kind regards,
> Norm
>

Now I am a bit confused what exactly was tested.  When using my
Singularity container [3] you should receive a ready baked Octave
compiled with all dependencies (OpenBLAS, SuiteSparse, ...) to enable
those 64-bit indices.

If you compile Octave on your own machine from scratch, with only
installing the dependent packages from your Linux distribution, it is
likely not to work.  You can check your configure logs.  If it looks
like the following, your Octave software stack cannot handle 64-bit indices:

   [...]
   checking BLAS library integer size... 4
   checking for cheev_... yes
   checking default size of Fortran INTEGER... 4
   [...]
   64-bit array dims and indexing:       yes
   64-bit BLAS array dims and indexing:  no
   [...]

HTH,
Kai