[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[bug#50261] [PATCH] gnu: node: Enable cross-compilation.
|
From: |
Pierre Langlois |
|
Subject: |
[bug#50261] [PATCH] gnu: node: Enable cross-compilation. |
|
Date: |
Sun, 29 Aug 2021 23:06:00 +0100 |
|
User-agent: |
mu4e 1.6.5; emacs 27.2 |
Hi Guix!
While working on tree-sitter I realized our node packages didn't
cross-compile [0] so I thought I'd give it a go, here's a patch!
Note that fundamentally, because of how the V8 JavaScript engine
bootstraps itself, cross-compiling only works for platforms with the
same bitness. So a 32-bit binary cannot be produced on a 64-bit
system, for example on x86_64:
--8<---------------cut here---------------start------------->8---
# Build OK.
$ guix build --target=aarch64-linux-gnu node
# Will fail...
$ guix build --target=arm-linux-gnueabihf node
# ... unless the host is 32-bit.
$ guix build --system=i686-linux --target=arm-linux-gnueabihf node
--8<---------------cut here---------------end--------------->8---
I'm not sure there's any way to express this restriction in the package
definition, it would be good to fail in a useful way, any thoughts? I
suspect a mechanism for this isn't really worth the effort, I don't know
of any other packages like this.
As a drive-by, the esbuild package didn't build on 32-bit systems so I
attached another patch for that. It was due to the build system using
the "-race" option to detect race conditions in the tests, and this
mechanism only works on 64-bit platforms.
Thanks,
Pierre
[0]: https://lists.gnu.org/archive/html/guix-patches/2021-08/msg00646.html
signature.asc
Description: PGP signature
>From 8b4cc4573d438eb108b8d1a9bcdf99f3af98b514 Mon Sep 17 00:00:00 2001
From: Pierre Langlois <pierre.langlois@gmx.com>
Date: Wed, 11 Aug 2021 16:43:45 +0100
Subject: [PATCH 1/2] gnu: node: Enable cross-compilation.
Node runs parts of itself on the host for bootstraping therefore for
cross-compiling support we need to fidle with the rpath in the build system,
as well as duplicating some of the dependencies as native-inputs and inputs.
* gnu/pakcages/node.scm (node)[arguments]: Refer to /bin/sh and /bin/env
directly instead of using (which). Add new 'set-bootstrap-host-rpath phase
to correctly set the rpath for binaries that are meant to run on the host.
Pass --cross-compiling and --dest-cpu to configure script if needed. Set the
CC_host, CXX_host, CC, CCX and PKG_CONFIG variable for cross-compilation.
Refer to the host python. Do not return #t from any phases.
[native-inputs]: Add c-ares, http-parser, icu4c, libuv, nghttp2, openssl and
zlib. Remove which.
[inputs]: Add bash and coreutils.
(llhttp-bootstrap)[arguments]: Refer to esbuild via (or native-inputs inputs).
(node-lts)[arguments]: Add new 'set-bootstrap-host-rpath phase to correctly
set the rpath for host binaries. Pass --cross-compiling and --dest-cpu to
configure script if needed. Set the CC_host, CXX_host, CCX and PKG_CONFIG
variable for cross-compilation. Refer to the host python. Do not return #t
from any phases. Refer to /bin/sh and /bin/env directly instead of
using (which). Do not return #t from any phases.
[native-inputs]: Hardcode native inputs instead of inheriting them from node.
[inputs]: Add bash and coreutils.
---
gnu/packages/node.scm | 188 ++++++++++++++++++++++++++++++++++--------
1 file changed, 153 insertions(+), 35 deletions(-)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 36c45e9c7a..2dc23b881c 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -6,7 +6,7 @@
;;; Copyright © 2017 Mike Gerwitz <mtg@gnu.org>
;;; Copyright © 2018 Tobias Geerinckx-Rice <me@tobias.gr>
;;; Copyright © 2018, 2019, 2020, 2021 Marius Bakke <marius@gnu.org>
-;;; Copyright © 2020 Pierre Langlois <pierre.langlois@gmx.com>
+;;; Copyright © 2020, 2021 Pierre Langlois <pierre.langlois@gmx.com>
;;; Copyright © 2020 Ricardo Wurmus <rekado@elephly.net>
;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
;;;
@@ -38,6 +38,7 @@
#:use-module (gnu packages)
#:use-module (gnu packages adns)
#:use-module (gnu packages base)
+ #:use-module (gnu packages bash)
#:use-module (gnu packages compression)
#:use-module (gnu packages gcc)
#:use-module (gnu packages icu4c)
@@ -48,7 +49,9 @@
#:use-module (gnu packages pkg-config)
#:use-module (gnu packages python)
#:use-module (gnu packages tls)
- #:use-module (gnu packages web))
+ #:use-module (gnu packages web)
+ #:use-module (ice-9 match)
+ #:use-module (srfi srfi-26))
(define-public node
(package
@@ -111,14 +114,14 @@
"test/parallel/test-stdio-closed.js"
"test/sequential/test-child-process-emfile.js")
(("'/bin/sh'")
- (string-append "'" (which "sh") "'")))
+ (string-append "'" (assoc-ref inputs "bash") "/bin/sh'")))
;; Fix hardcoded /usr/bin/env references.
(substitute*
'("test/parallel/test-child-process-default-options.js"
"test/parallel/test-child-process-env.js"
"test/parallel/test-child-process-exec-env.js")
(("'/usr/bin/env'")
- (string-append "'" (which "env") "'")))
+ (string-append "'" (assoc-ref inputs "coreutils")
"/bin/env'")))
;; FIXME: These tests fail in the build container, but they don't
;; seem to be indicative of real problems in practice.
@@ -155,23 +158,65 @@
;; TODO: Regenerate certs instead.
(for-each delete-file
'("test/parallel/test-tls-passphrase.js"
- "test/parallel/test-tls-server-verify.js"))
- #t))
+ "test/parallel/test-tls-server-verify.js"))))
+ (add-before 'configure 'set-bootstrap-host-rpath
+ (lambda* (#:key native-inputs inputs #:allow-other-keys)
+ (let* ((inputs (or native-inputs inputs))
+ (c-ares (assoc-ref inputs "c-ares"))
+ (http-parser (assoc-ref inputs "http-parser"))
+ (icu4c (assoc-ref inputs "icu4c"))
+ (nghttp2 (assoc-ref inputs "nghttp2"))
+ (openssl (assoc-ref inputs "openssl"))
+ (libuv (assoc-ref inputs "libuv"))
+ (zlib (assoc-ref inputs "zlib")))
+ (substitute* "deps/v8/gypfiles/v8.gyp"
+ (("'target_name': 'torque'," target)
+ (string-append target
+ "'ldflags': ['-Wl,-rpath="
+ c-ares "/lib:"
+ http-parser "/lib:"
+ icu4c "/lib:"
+ nghttp2 "/lib:"
+ openssl "/lib:"
+ libuv "/lib:"
+ zlib "/lib"
+ "'],"))))))
(replace 'configure
;; Node's configure script is actually a python script, so we can't
;; run it with bash.
- (lambda* (#:key outputs (configure-flags '()) inputs
+ (lambda* (#:key outputs (configure-flags '()) native-inputs inputs
#:allow-other-keys)
(let* ((prefix (assoc-ref outputs "out"))
- (flags (cons (string-append "--prefix=" prefix)
- configure-flags)))
+ (flags (cons*
+ (string-append "--prefix=" prefix)
+ ,@(if (%current-target-system)
+ `("--cross-compiling"
+ ,(string-append "--dest-cpu="
+ (match (%current-target-system)
+ ((? (cut string-prefix? "arm" <>))
+ "arm")
+ ((? (cut string-prefix? "aarch64" <>))
+ "arm64")
+ ((? (cut string-prefix? "i686" <>))
+ "ia32")
+ ((? (cut string-prefix? "x86_64" <>))
+ "x64")
+ ((? (cut string-prefix? "powerpc64"
<>))
+ "ppc64")
+ (_ "unsupported"))))
+ '())
+ configure-flags)))
(format #t "build directory: ~s~%" (getcwd))
(format #t "configure flags: ~s~%" flags)
;; Node's configure script expects the CC environment variable
to
;; be set.
- (setenv "CC" (string-append (assoc-ref inputs "gcc")
"/bin/gcc"))
+ (setenv "CC_host" "gcc")
+ (setenv "CXX_host" "g++")
+ (setenv "CC" ,(cc-for-target))
+ (setenv "CXX" ,(cxx-for-target))
+ (setenv "PKG_CONFIG" ,(pkg-config-for-target))
(apply invoke
- (string-append (assoc-ref inputs "python")
+ (string-append (assoc-ref (or native-inputs inputs)
"python")
"/bin/python")
"configure" flags))))
(add-after 'patch-shebangs 'patch-npm-shebang
@@ -181,29 +226,37 @@
(npm (string-append bindir "/npm"))
(target (readlink npm)))
(with-directory-excursion bindir
- (patch-shebang target (list bindir))
- #t))))
+ (patch-shebang target (list bindir))))))
(add-after 'install 'patch-node-shebang
(lambda* (#:key outputs #:allow-other-keys)
(let* ((bindir (string-append (assoc-ref outputs "out")
"/bin"))
(npx (readlink (string-append bindir "/npx"))))
(with-directory-excursion bindir
- (patch-shebang npx (list bindir))
- #t)))))))
+ (patch-shebang npx (list bindir)))))))))
(native-inputs
- `(("python" ,python-2)
+ `(;; Runtime dependencies for binaries used as a bootstrap.
+ ("c-ares" ,c-ares)
+ ("http-parser" ,http-parser)
+ ("icu4c" ,icu4c)
+ ("libuv" ,libuv)
+ ("nghttp2" ,nghttp2 "lib")
+ ("openssl" ,openssl)
+ ("zlib" ,zlib)
+ ;; Regular build-time dependencies.
("perl" ,perl)
("pkg-config" ,pkg-config)
("procps" ,procps)
- ("util-linux" ,util-linux)
- ("which" ,which)))
+ ("python" ,python-2)
+ ("util-linux" ,util-linux)))
(native-search-paths
(list (search-path-specification
(variable "NODE_PATH")
(files '("lib/node_modules")))))
(inputs
- `(("c-ares" ,c-ares)
+ `(("bash" ,bash)
+ ("coreutils" ,coreutils)
+ ("c-ares" ,c-ares)
("http-parser" ,http-parser)
("icu4c" ,icu4c)
("libuv" ,libuv)
@@ -551,9 +604,10 @@ parser definition into a C output.")
#:phases
(modify-phases %standard-phases
(replace 'configure
- (lambda* (#:key inputs #:allow-other-keys)
- (let ((esbuild (string-append (assoc-ref inputs "esbuild")
- "/bin/esbuild")))
+ (lambda* (#:key native-inputs inputs #:allow-other-keys)
+ (let ((esbuild (string-append
+ (assoc-ref (or native-inputs inputs) "esbuild")
+ "/bin/esbuild")))
(invoke esbuild
"--platform=node"
"--outfile=bin/generate.js"
@@ -625,21 +679,72 @@ source files.")
"--with-intl=system-icu"))
((#:phases phases)
`(modify-phases ,phases
+ (replace 'set-bootstrap-host-rpath
+ (lambda* (#:key native-inputs inputs #:allow-other-keys)
+ (let* ((inputs (or native-inputs inputs))
+ (c-ares (assoc-ref inputs "c-ares"))
+ (google-brotli (assoc-ref inputs "google-brotli"))
+ (icu4c (assoc-ref inputs "icu4c"))
+ (nghttp2 (assoc-ref inputs "nghttp2"))
+ (openssl (assoc-ref inputs "openssl"))
+ (libuv (assoc-ref inputs "libuv"))
+ (zlib (assoc-ref inputs "zlib"))
+ (host-binaries '("torque"
+ "bytecode_builtins_list_generator"
+ "gen-regexp-special-case"
+ "node_mksnapshot"
+ "mksnapshot")))
+ (substitute* '("node.gyp" "tools/v8_gypfiles/v8.gyp")
+ (((string-append "'target_name': '("
+ (string-join host-binaries "|")
+ ")',")
+ target)
+ (string-append target
+ "'ldflags': ['-Wl,-rpath="
+ c-ares "/lib:"
+ google-brotli "/lib:"
+ icu4c "/lib:"
+ nghttp2 "/lib:"
+ openssl "/lib:"
+ libuv "/lib:"
+ zlib "/lib"
+ "'],"))))))
(replace 'configure
;; Node's configure script is actually a python script, so we
can't
;; run it with bash.
- (lambda* (#:key outputs (configure-flags '()) inputs
+ (lambda* (#:key outputs (configure-flags '()) native-inputs inputs
#:allow-other-keys)
(let* ((prefix (assoc-ref outputs "out"))
- (flags (cons (string-append "--prefix=" prefix)
- configure-flags)))
+ (flags (cons*
+ (string-append "--prefix=" prefix)
+ ,@(if (%current-target-system)
+ `("--cross-compiling"
+ ,(string-append "--dest-cpu="
+ (match (%current-target-system)
+ ((? (cut string-prefix? "arm" <>))
+ "arm")
+ ((? (cut string-prefix? "aarch64"
<>))
+ "arm64")
+ ((? (cut string-prefix? "i686" <>))
+ "ia32")
+ ((? (cut string-prefix? "x86_64" <>))
+ "x64")
+ ((? (cut string-prefix? "powerpc64"
<>))
+ "ppc64")
+ (_ "unsupported"))))
+ '())
+ configure-flags)))
(format #t "build directory: ~s~%" (getcwd))
(format #t "configure flags: ~s~%" flags)
;; Node's configure script expects the CC environment
variable to
;; be set.
+ (setenv "CC_host" "gcc")
+ (setenv "CXX_host" "g++")
(setenv "CC" ,(cc-for-target))
+ (setenv "CXX" ,(cxx-for-target))
+ (setenv "PKG_CONFIG" ,(pkg-config-for-target))
(apply invoke
- (string-append (assoc-ref inputs "python")
+ (string-append (assoc-ref (or native-inputs inputs)
"python")
"/bin/python3")
"configure" flags))))
(replace 'patch-files
@@ -652,14 +757,14 @@ source files.")
"test/parallel/test-stdio-closed.js"
"test/sequential/test-child-process-emfile.js")
(("'/bin/sh'")
- (string-append "'" (which "sh") "'")))
+ (string-append "'" (assoc-ref inputs "bash") "/bin/sh'")))
;; Fix hardcoded /usr/bin/env references.
(substitute*
'("test/parallel/test-child-process-default-options.js"
"test/parallel/test-child-process-env.js"
"test/parallel/test-child-process-exec-env.js")
(("'/usr/bin/env'")
- (string-append "'" (which "env") "'")))
+ (string-append "'" (assoc-ref inputs "coreutils")
"/bin/env'")))
;; FIXME: These tests fail in the build container, but they
don't
;; seem to be indicative of real problems in practice.
@@ -707,20 +812,33 @@ source files.")
(copy-file (string-append llhttp "/src/http.c")
"deps/llhttp/src/http.c")
(copy-file (string-append llhttp "/include/llhttp.h")
- "deps/llhttp/include/llhttp.h"))
- #t))))))
+ "deps/llhttp/include/llhttp.h"))))))))
+ (native-inputs
+ `(;; Runtime dependencies for binaries used as a bootstrap.
+ ("c-ares" ,c-ares)
+ ("google-brotli" ,google-brotli)
+ ("icu4c" ,icu4c-67)
+ ("libuv" ,libuv-for-node)
+ ("nghttp2" ,nghttp2 "lib")
+ ("openssl" ,openssl)
+ ("zlib" ,zlib)
+ ;; Regular build-time dependencies.
+ ("perl" ,perl)
+ ("pkg-config" ,pkg-config)
+ ("procps" ,procps)
+ ("python" ,python)
+ ("util-linux" ,util-linux)))
(inputs
- `(("c-ares" ,c-ares)
+ `(("bash" ,bash)
+ ("coreutils" ,coreutils)
+ ("c-ares" ,c-ares)
("icu4c" ,icu4c-67)
("libuv" ,libuv-for-node)
("llhttp" ,llhttp-bootstrap)
("google-brotli" ,google-brotli)
("nghttp2" ,nghttp2 "lib")
("openssl" ,openssl)
- ("zlib" ,zlib)))
- (native-inputs
- (alist-replace "python" (list python-3)
- (package-native-inputs node)))))
+ ("zlib" ,zlib)))))
(define-public libnode
(package/inherit node
--
2.33.0
From 17cd8ccb2255d4bb392a94468abeb785e134ff1e Mon Sep 17 00:00:00 2001
From: Pierre Langlois <pierre.langlois@gmx.com>
Date: Mon, 30 Aug 2021 00:25:58 +0100
Subject: [PATCH 2/2] gnu: esbuild: Disable race detector on 32-bit targets.
* gnu/packages/web.scm (esbuild)[arguments]: Set the ESBUILD_RACE
variable to an empty string to remove the -race option.
---
gnu/packages/web.scm | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm
index f0ac9ccee2..5817d2dd95 100644
--- a/gnu/packages/web.scm
+++ b/gnu/packages/web.scm
@@ -1672,13 +1672,17 @@ used to validate and fix HTML data.")
#t))))
(build-system go-build-system)
(arguments
- '(#:import-path "github.com/evanw/esbuild/cmd/esbuild"
+ `(#:import-path "github.com/evanw/esbuild/cmd/esbuild"
#:unpack-path "github.com/evanw/esbuild"
#:phases
(modify-phases %standard-phases
(replace 'check
(lambda* (#:key tests? unpack-path #:allow-other-keys)
(when tests?
+ ;; The "Go Race Detector" is only supported on 64-bit
+ ;; platforms, this variable disables it.
+ (unless ,(target-64bit?)
+ (setenv "ESBUILD_RACE" ""))
(with-directory-excursion (string-append "src/" unpack-path)
(invoke "make" "test-go")))
#t)))))
--
2.33.0
- [bug#50261] [PATCH] gnu: node: Enable cross-compilation.,
Pierre Langlois <=