Re: GNUstep on Windows using Clang + MSVC ABI

[Frederik Seiffert]

>> - Linking subproject object files directly (instead of merging)
>> As David had suspected, merging the subproject object files turned out to be 
>> an issue: `ld -r` is not available on Windows, and using `ar cr` resulted in 
>> a subproject.o that didn’t contain all the symbols (I’m guessing because ar 
>> doesn't correctly handle PE/COFF files).
>> So I looked into how we could instead use the subproject object files 
>> directly when linking the project, and came up with this solution: instead 
>> of merging the object files into subproject.o, we simply write the list of 
>> object files to a subproject.txt file (in the same location), and read that 
>> to create SUBPROJECT_OBJ_FILES.
>> The change is pretty minimal and seems to work fine on all 
>> platforms/configurations in CI:
>> https://github.com/gnustep/tools-make/commit/434f957df0ad81b52a09e3f8c4a200734898b342
>> Given that this affects all platforms I’d appreciate everyone’s feedback on 
>> this, but given that issues with incremental linking with various setups has 
>> been talked about multiple times on the mailing list, I hope that this 
>> change will be an overall improvement.
> 
> ld -r is pretty flaky everywhere.  LLD wasn't going to implement it at all 
> and GNUstep was one of only two consumers so I think it's fair to say that 
> it's pretty likely that it will continue to be intermittently broken by 
> various linker versions.  I personally like the concept but it's probably 
> better to not depend on it anywhere.
> 
>> - Building
>> Building with this setup requires using a standard (non-MinGW) Clang that 
>> e.g. comes with Visual Studio or is available as pre-built binary from the 
>> LLVM website, and requires passing a host to configure like 
>> --host=x86_64-pc-windows. Invoking `clang -v` should show a target like 
>> "x86_64-pc-windows-msvc". It *might* be that using a MinGW Clang works too 
>> when invoked with --target x86_64-pc-windows-msvc, but I haven’t tried that.
> 
> The easiest way of installing clang on Windows is via Chocolatey: just run 
> `choco install llvm` once it's installed.  Choco is also the easiest way of 
> installing cmake and Ninja for building the runtime.

Thanks, good point!

>> The build is best done in an MSYS2 shell that does not have any additional 
>> *-devel packages installed that might get picked up by configure. 
>> Alternatively --disable-xxx flags can be used to prevent these dependencies 
>> to be picked up.
> 
> Have you tried with the bash.exe in C:\Windows\System32?



>> I’m currently building like this:
>> # tools-make
>> export CC=/C/LLVM/bin/clang
>> export CXX=/C/LLVM/bin/clang++
>> export OBJCXX=/C/LLVM/bin/clang++
>> ./configure --host=x86_64-pc-windows --with-library-combo=ng-gnu-gnu 
>> --with-runtime-abi=gnustep-2.0 --prefix=/c/GNUstep/x64 LDFLAGS="-fuse-ld=lld"
>> make install
> 
> Is the -fuse-ld=lld line required?  You get LLD installed when you install 
> clang, so it's not very important, but the runtime and its tests, at least, 
> also work with LINK.EXE (though you may have to disable incremental linking 
> if it's on by default).

Yeah:

>> I tried using the MS linker (link.exe), but while that seems to generally 
>> work fine linking ObjC files, it throws up at some configure tests 
>> (specifically when testing "whether objc really works"). Using LLD works 
>> though, so one must run Make configure with LDFLAGS="-fuse-ld=lld".





>> # libs-base
>> . /c/GNUstep/x64/share/GNUstep/Makefiles/GNUstep.sh
>> ./configure --host=x86_64-pc-windows --disable-iconv --disable-tls 
>> --disable-icu --disable-xml
>> make -j12 install
>> - Dependencies
>> While most dependencies should be available via NuGet, I have not been able 
>> to figure out yet how we might integrate NuGet packages into the build 
>> process. Downloading the packages results in individual folders per package 
>> with deeply nested folder structures containing the libraries for different 
>> architectures, debug/release targets, and Visual Studio versions.
>> For now I have manually copied the headers and libffi.lib import library 
>> (renamed to ffi.lib) from the libffi package, and the libffi.dll from the 
>> libffi.redist package.
>> For pthreads I have built http://www.sourceware.org/pthreads-win32/ from 
>> source by invoking "nmake clean VC" in a VS native tools command prompt, and 
>> manually copied the headers (pthread.h, sched.h, semaphore.h), 
>> pthreadVC2.lib (renamed to pthread.lib), and phtreadVC2.dll.
> 
> At some point it would be nice to lose this dependency.  I don't know how 
> much the attitude to C++ in GNUstep has changed recently, but C++11 now has a 
> nice set of threading abstractions that provide everything that NSThread, 
> NSLock and friends need.  If we use those instead of pthreads then we'd have 
> something that worked on any platform with a C++11 implementation (Linux, 
> Windows, Fuschia, Haiku, and so on).
> 
> 
>> I haven’t build with any other dependencies so far but that will be my next 
>> endeavor. If anyone has suggestions on how the NuGet packages might be used 
>> directly I’d appreciate any input. This is also the main reason I haven’t 
>> set this up on CI yet, as we’d have to replicate the currently manual 
>> process somehow.
>> I’m also thinking about writing some scripts similar to tools-android that 
>> could build all the dependencies and GNUstep itself, ideally also for the 
>> different architectures and debug/release targets.
>> - NSFileManager support
>> NSFileManager is currently non-functional with this setup due to dirent.h 
>> missing (MinGW has it, but Windows doesn’t).
>> I found this MIT-licensed header-only implementation – would that be ok to 
>> add to the project?
>> https://github.com/win32ports/dirent_h/blob/master/dirent.h
> 
> I think that looks easier than using Win32 ABIs directly.
> 
>> - Autoconf GNUC / GCC / Clang detection
>> When targeting the MSVC ABI, Clang does not define `__GNUC__`, which causes 
>> Autoconf to not define $GCC (i.e. "checking whether we are using the GNU C 
>> compiler" will be NO). I thus removed $GCC as a pre-requisite in 
>> GS_CHECK_CC_IS_CLANG(), so that Clang is still correctly detected in this 
>> setup.
> 
> I *believe* this is controlled by either a gnu dialect (e.g. -std=gnu11 for 
> C) or -fgnu-extensions.  That said, I'm not sure if you can #include Windows 
> headers with gnu extensions enabled.
> 
> 
>> - Debug/Release CRT
>> Just a heads up that on Windows the C runtime libraries (CRT) come in two 
>> versions for debug and release builds (e.g. msvcrt and msvcrtd). The 
>> libraries that are used seem to have to match between all DLLs – e.g. 
>> building libobjc2 for debug will cause all sorts of crashes when Base is 
>> build for release, e.g. when freeing memory allocated by libobjc2 in Base.
> 
> Fun.
> 
>> - Linker issues
>> I tried using the MS linker (link.exe), but while that seems to generally 
>> work fine linking ObjC files, it throws up at some configure tests 
>> (specifically when testing "whether objc really works"). Using LLD works 
>> though, so one must run Make configure with LDFLAGS="-fuse-ld=lld".
>> I also had to bypass the config checks for objc_sync_enter(), 
>> objc_setProperty(), _Block_copy(), and non-fragile-abi support and assume 
>> these functions are there, as these checks result in the following linker 
>> errors.
>>      lld-link: error: relocation against symbol in discarded section: 
>> __start_.objcrt$SEL
>>      >>> referenced by C:\msys64\tmp\conftest-78a937.o:(.objc_init)
>> This is reproducible when building a file that calls ObjC runtime functions 
>> but doesn’t actually contain any ObjC code. Not sure if that’s expected 
>> behavior or bug.
> 
> I think this is a bug, probably a clang bug (and probably my fault)
> 
>> While 64-bit builds works fine with this setup, I have not been able to get 
>> 32-bit builds to work yet due to this linker error, but I’m probably just 
>> missing some flags somewhere:
>>      lld-link: error: libcmt.lib: machine type x64 conflicts with x86
>> Also, as David noted, this setup requires dllexport/dllimport on all ObjC 
>> classes, so I annotated all our ObjC class interfaces with GS_EXPORT_CLASS 
>> (and also added missing GS_DECLARE annotations in externs.m), in order for 
>> these symbols to be correctly exported in the DLL.
> 
> Awesome work, and it makes me much happier than trying to support MinGW.  We 
> have given up supporting MinGW for snmalloc and I am considering optionally 
> supporting snmalloc for Objective-C object allocations since it is much 
> faster than the system malloc on most platforms and is particularly good for 
> fixed-size allocations as are common in Objective-C.  It would be a shame for 
> this to be an everywhere-except-Windows thing.
> 
> David
> 
>