bug#63986: Julia is very slow

[Efraim Flashner]

On Thu, Jun 22, 2023 at 05:52:15PM +0200, Simon Tournier wrote:
> Hi,
> 
> On Thu, 22 Jun 2023 at 15:26, Cayetano Santos via Bug reports for GNU Guix 
> <bug-guix@gnu.org> wrote:
> >> Are we following all instructions here ?
> >>
> >>   
> >> https://docs.julialang.org/en/v1.8/devdocs/build/distributing/#Notes-on-BLAS-and-LAPACK
> 
> [...]
> 
> >   Base.USE_BLAS64
> >
> > gives "true" when running fast. Guix julia gives "false".
> 
> When I try USE_BLAS64=1, then I get:
> 
> --8<---------------cut here---------------start------------->8---
> ┌ Error: No loaded BLAS libraries were built with ILP64 support
> └ @ LinearAlgebra.BLAS 
> /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/blas.jl:155
> Quitting.
> --8<---------------cut here---------------end--------------->8---
> 
> And from the documentation above, it reads:
> 
>         [...] while on 64-bit architectures, Julia builds OpenBLAS to
>         use 64-bit integers (ILP64). It is essential that all Julia
>         functions that call BLAS and LAPACK API routines use integers of
>         the correct width.
> 
> Well using the patch attached, I get:
> 
>     6.884 ms (2 allocations: 7.63 MiB)
> 
> compared to the previous
> 
>     494.345 ms (2 allocations: 7.63 MiB)
> 
> WDYT about this patch?

(ins)efraim@3900XT ~/workspace/guix$ cat 
/gnu/store/v6z5ykkjfzbc72x1x900xflspqc5wd5r-openblas-ilp64-0.3.20/lib/pkgconfig/openblas.pc
libdir=/gnu/store/v6z5ykkjfzbc72x1x900xflspqc5wd5r-openblas-ilp64-0.3.20/lib
includedir=/gnu/store/v6z5ykkjfzbc72x1x900xflspqc5wd5r-openblas-ilp64-0.3.20/include
openblas_config= USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER=1 NO_CBLAS= 
NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= generic MAX_THREADS=128
version=0.3.20
extralib=-lm -lpthread -lgfortran -lm -lpthread -lgfortran
Name: openblas
Description: OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD 
version
Version: ${version}
URL: https://github.com/xianyi/OpenBLAS
Libs: -L${libdir} -lopenblas
Libs.private: ${extralib}
Cflags: -I${includedir}

Looks like it should be "LIBBLAS=-lopenblas"

It might need some tuning anyway, currently we have julia for i686 and
switching to solely openblas-ilp64 we'd lose 32-bit support.

I also noticed the julia expects the 64-bit openblas to be libopenblas64
(which happens to be what Debian¹ has). Would we need to adapt anything
in stdlib/OpenBLAS_jll/src/OpenBLAS_jll.jl?

Also, are we supposed to build lapack with our openblas as an input?

¹ https://sources.debian.org/src/openblas/0.3.21%2Bds-4/debian/rules/#L71

> From 024c92fac091f59dcdbd3a78eb6ea77bb15b2170 Mon Sep 17 00:00:00 2001
> Message-Id: 
> <024c92fac091f59dcdbd3a78eb6ea77bb15b2170.1687449033.git.zimon.toutoune@gmail.com>
> From: Simon Tournier <zimon.toutoune@gmail.com>
> Date: Thu, 22 Jun 2023 17:45:50 +0200
> Subject: [PATCH] gnu: julia: Use openblas with ILP64 support.
> 
> Fixes <https://bugs.gnu.org/63986>.
> Reported by Cayetano Santos <csantosb@inventati.org>.
> 
> * gnu/packages/julia.scm (julia)[arguments]<#:make-flags>: Use OpenBLAS with
> ILP64 support.
> [inputs]: Replace openblas by openblas-ilp64.
> ---
>  gnu/packages/julia.scm | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm
> index ba54175822..a034cbf543 100644
> --- a/gnu/packages/julia.scm
> +++ b/gnu/packages/julia.scm
> @@ -481,10 +481,10 @@ (define-public julia
>  
>           ,@(if (target-aarch64?)
>               `("USE_BLAS64=0")
> -             '())
> +             `("USE_BLAS64=1"))
>  
> -         "LIBBLAS=-lopenblas"
> -         "LIBBLASNAME=libopenblas"
> +         "LIBBLAS=-lopenblas_ilp64"
> +         "LIBBLASNAME=libopenblas_ilp64"
>  
>           (string-append "UTF8PROC_INC="
>                          (assoc-ref %build-inputs "utf8proc")
> @@ -513,7 +513,7 @@ (define-public julia
>         ("llvm" ,llvm-julia)
>         ("mbedtls-apache" ,mbedtls-apache)
>         ("mpfr" ,mpfr)
> -       ("openblas" ,openblas)
> +       ("openblas" ,openblas-ilp64)
>         ("openlibm" ,openlibm)
>         ("p7zip" ,p7zip)
>         ("pcre2" ,pcre2)
> 
> base-commit: 37c2e94cec6cb8b5e0e93e7b6c712c3b187ca5db
> -- 
> 2.38.1
> 

> 
> Well, I need to do more tests but I guess that’s the good direction. :-)
> 
> Cheers,
> simon


-- 
Efraim Flashner   <efraim@flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

signature.asc
Description: PGP signature