Re: [Tinycc-devel] Why does TinyCC fail to link standard C runtime functions in 32-bit mode but works in 64-bit mode?

[Fereydoun Memarzanjany]

I had originally attempted to use `fprintf(stdout, "...");` which, due
to its use of stdout, would fail.  Not knowing the root cause of _iob,
I mistakenly generalized printf and other functions; sorry about that.

In TCC/mob, _MSVCRT_ gets cancelled by <windows.h> because it
internally includes <_mingw.h>.  For some reason, <_mingw.h> has these
two lines 41 and 42:

```
#define __MSVCRT__ 1
#undef _MSVCRT_
```

It defines __MSVCRT__ to be 1 yet undefined _MSVCRT_ (causing
<stdio.h> to continue using _imp___iob if <windows.h> is included
after _MSVCRT_)

Under both the latest mob and 0.9.27, your minimal example does
compile and print; however, the issue is that it compiles under
-subsystem=console.  For GUI apps compiled under the
-subsystem=windows switch (i.e., the default behavior), freopen() is a
necessity.  If, after allocating a console and prior to the printf()
call, freopen() is not called on stdout/stderr/stdin under a GUI
application, nothing gets printed at all.

Because freopen() depends on stdout, compilation will inevitably fail
under `-m32`, on both mob and 0.9.27; the __atribute__(dllimport) was
able to remedy that.

I dug deeper and found that those new MinGW-64 headers from w64devkit
also do the exact same fix I did (albeit they hide it under various
macro definitions).  I was surprised that you could compile my
original test case (with the _MSVCRT_ defined) under w64devkit;
because, when I look at their <stdio.h>, it appears that they only
have that __MINGW_IMP_SYMBOL macro expansion for the non-_MSVCRT_
case.  In other words, their `extern FILE _iob[]` case (which runs
whenever _MSVCRT_ is defined) was *not* decorated with
__attribute__((dllimport)), and it *should* have failed.

The difference is in these two lines of <stdio.h> (83 & 84) from new w64devkit :

```
  _CRTIMP FILE *__cdecl __iob_func(void);
#define _iob  __iob_func()
```

Whereas TCC's current mob <stdio.h> defines __iob_func based on _iob,
w64devkit does the opposite: it defines _iob based on __iob_func().
Notice how __iob_func() is decorated with _CRTIMP.  In line 30 of
crtdll.c  in w64devkit, they define _CRTIMP as follows:
#define _CRTIMP __declspec(dllimport)

Given that __declspec(dllimport) and __atribute__((dllimport)) have
the exact same behavior, this results in a successful compilation.

_CRTIMP also exists on TCC's old MinGW includes, so I suppose we could
declare _iob and _imp___iob with _CRTIMP rather than __atribute__ for
consistency's sake.

(On an unrelated note, it looks like __MINGW_IMP_SYMBOL was introduced
back in 2009; I didn't know TCC's headers are older than that.)

On Sun, Nov 24, 2024 at 4:20 PM avih <avihpit@yahoo.com> wrote:
>
> Before I'll respond to your last email, you said in your original email that 
> even printf doesn't work:
>
> > `_iob` is not the only "unresolved" symbol, either; `printf`,
> > `freopen`, `freopen_s`, and basically everything from the CRT will
> > fail to link.
>
> But in my minimal example there's only _MSVCRT_ and #include <stdio.h> and 
> printf and it does compile and link (and runs, and prints), and there's no 
> #include <windows.h> which you think cancels _MSVCRT_.
>
> Is my example incorrect somehow? or maybe it didn't work in 0.9.27 but does 
> work with recent mob?
>
> Now, to your suggested fix.
>
> Looking at a recent mingw installation (specifically w64devkit 2.0 - 
> https://github.com/skeeto/w64devkit), it looks to me pretty much the same as 
> this code in tcc stdio.h (w64devkit/i686-w64-mingw32/include/stdio.h lines 
> 101-129) with minor difference that _imp___iob is a macro expansion and not 
> explicit.
>
> Your original test case (with freopen and _MSVCRT_ after #include 
> <windows.h>) does compile with w64devkit x86, so at the very least your 
> suggestion is not how it's handled in new mingw.
>
> Can you maybe dig into that and figure out what makes it tick in w64devkit, 
> and see if a similar thing can be applied to the tcc headers?
>
>
>
> On Sunday, November 24, 2024 at 11:05:38 PM GMT+2, Fereydoun Memarzanjany 
> <thraetaona@ieee.org> wrote:
>
>
>
>
>
> After nearly 16 hours of debugging, I've found the culprit: `_iob` and
> `_imp___iob` should have been declared with either
> `__attribute__((dllimport))` or `__declspec(dllimport)`
>
> To be more specific, lines 93 to 106 of <stdio.h> in TCC's ./include
> directory are currently written as follows:
>
> ```c
> #ifndef _STDIO_DEFINED
> #ifdef _WIN64
>   _CRTIMP FILE *__cdecl __iob_func(void);
> #else
> #ifdef _MSVCRT_
> extern FILE _iob[];    /* A pointer to an array of FILE */
> #define __iob_func()    (_iob)
> #else
> extern FILE (*_imp___iob)[];    /* A pointer to an array of FILE */
> #define __iob_func()    (*_imp___iob)
> #define _iob __iob_func()
> #endif
> #endif
> #endif
> ```
>
> To fix them so that it compiles under `-m32` just as fine as it does
> under `-m64`, you need to change them to the following:
>
> ```c
> #ifndef _STDIO_DEFINED
> #ifdef _WIN64
>   _CRTIMP FILE *__cdecl __iob_func(void);
> #else
> #ifdef _MSVCRT_
> __attribute__((dllimport)) extern FILE _iob[];    /* A pointer to an
> array of FILE */
> #define __iob_func()    (_iob)
> #else
> __attribute__((dllimport)) extern FILE (*_imp___iob)[];    /* A
> pointer to an array of FILE */
> #define __iob_func()    (*_imp___iob)
> #define _iob __iob_func()
> #endif
> #endif
> #endif
> ```
>
> (NOTE: As I said earlier,  both `__attribute__((dllimport))` and
> `__declspec(dllimport)` work here.  Either of them fix it for both
> _MSVCRT_ and non-_MSVCRT_ headers.)
>
> A few additional details on why this fix is even useful in the first place:
>     - freopen() is optional in -subsystem=console (i.e., CLI) but
> absolutely required if you compile in -subsystem=windows (i.e., GUI)
> mode; if freopen() isn't called in the latter, printf and other std
> outputs don't show up on the allocated console;
>     - freopen() depends on the value of `stdout`, and `stdout` is
> defined based on either _iob or _imp___iob in stdio.h;
>     - _imp___iob is too old and no longer exists in newer msvcrt.dll
> or ucrtbase.dll, making _iob (and therefore defining _MSVCRT_) the
> go-to choice; and
>     - #define'ing _MSVCRT_  before <windows.h> is akin to not defining
> it at all; I think <windows.h> undefines it somewhere, internally.
>
> I still find it strange that such a "fix" was required, because TCC
> works just fine under `-m64` anyway.  For `-m32` builds, do you think
> we should patch the stdio.h in the mob branch ourselves?  I'm not sure
> if this is a TCC-specific patch or if it actually affects any compiler
> that uses stdio.h from MinGW.
>
> (Also, I did compile the latest mob branch using "build-tcc.bat -x -c
> cl" but this stdio.h bug affected it the same way it affected 0.9.27.)
>
>
> On Sun, Nov 24, 2024 at 12:09 PM avih <avihpit@yahoo.com> wrote:
> >
> > tcc 0.9.27 is few years old now, and hopefully 0.9.28 would be released 
> > sooner rather than later, so you should try the latest version, which is 
> > the mob (default) branch here:
> > https://repo.or.cz/tinycc.git
> >
> > You'll need to build it yourself. If you can't build it or can't find a 
> > recent pre-build binary, then I can upload a version to some pastebin (do 
> > ask, but do try to build it first if you can).
> >
> > You should also keep in mind that tcc is a not mingw gcc clone. It does use 
> > (old) mingw headers, but you should not expect 100% compatibility, and the 
> > headers set is intentionally stripped down to keep it minimal (although 
> > many programs compile out of the box with the existing headers)
> >
> > Specifically about your issue. With latest tcc, if I comment out the line 
> > with "freopen" in your sample program, or with a minimal program like this:
> >
> > #define _MSVCRT_
> > #include <stdio.h>
> >
> > int main() {
> >    printf("Hello, world\n");
> >    return 0;
> > }
> >
> > Then it compiles and works fine.
> >
> > But with your original program with freopen, tcc indeed complains about 
> > undefined symbol '_iob'.
> >
> > However, if I move the _MSVCRT_ definition above #include <windows.h>, then 
> > it does compile (I didn't try to run it, but I presume it would work).
> >
> > So I'd think largely it's OK. However, I was not aware of this _MSVCRT_ 
> > thingy, so I don't know what differences to expect and I didn't try to 
> > understand it further.
> >
> > I don't know whether this should be considered an issue that it works in 
> > tcc only if _MSVCRT_ is defined before windows.h is included (to me that 
> > feels reasonable, so do double check), but if it should be considered an 
> > issue, then maybe you could help by trying to pinpoint the cause at the 
> > headers, and report back.
> >
> > Cheers,
> > avih
> >
> >
> >  On Sunday, November 24, 2024 at 07:43:30 PM GMT+2, Fereydoun Memarzanjany 
> > via Tinycc-devel <tinycc-devel@nongnu.org> wrote:
> >
> >
> > If you use TinyCC in its 32-bit mode (`-m32`) to compile a sample
> > program that uses any CRT function/symbol from `msvcrt.dll` (such as
> > the snippet provided later in this message), you'll be met with
> > compilation failures:
> >
> > `tcc.exe -std=c11 -Wall -Werror -Wl,-subsystem=console -m32 .\main.c`
> > "tcc: error: undefined symbol '_iob', missing __declspec(dllimport)?"
> >
> > (This only happens under `-m32`, whereas `-m64` works perfectly fine.)
> >
> > `_iob` is not the only "unresolved" symbol, either; `printf`,
> > `freopen`, `freopen_s`, and basically everything from the CRT will
> > fail to link.
> >
> > Regardless of whether or not you use `-lmsvcrt`, `#pragma comment(lib,
> > "msvcrt")`, `_declspec(dllimport)`, `attribute ((dllimport))`,
> > `-static` or `-shared`, or even `-impdef` on
> > "C:\Windows\SysWow64\msvcrt.dll" (or earlier versions thereof:
> > "msvcrt40.dll"), TCC still complains.
> >
> > I've verified with `DUMPBIN.exe` that both 32- and 64-bit "msvcrt.dll"
> > do, in fact, define `_iob` and other symbols.
> >
> > By some arcane logic, the following works perfectly fine: `tcc.exe
> > -std=c11 -Wall -Werror -Wl,-subsystem=console -m64 .\main.c`
> >
> > `main.c`
> > ```c
> > //#pragma comment(lib, "msvcrt")
> > //__attribute__((dllimport)) extern __declspec(dllimport) FILE _iob[];
> >
> > #include <windows.h>
> >
> > // _MSVCRT_ being defined will cause MinGW's stdio.h to use _iob as
> > // opposed to _imp___iob; only the former is defined in msvcrt.dll.
> > // However, even though _iob is exported by both the 32- and 64-bit
> > // versions of said dll, TinyCC still fails to find _iob in the former.
> > #define _MSVCRT_
> > #include <stdio.h>
> >
> > void main() {
> >    // AllocConsole() and basically everything from kernel32.dll or
> >    // user32.dll work perfectly fine, both in -m32 and -m64; it's
> >    // only msvcrt.dll that causes issues with TinyCC.
> >    AllocConsole();
> >
> >    // Any CRT function (e.g., freopen, freopen_s, printf, etc.)
> >    // fail to get linked properly ONLY in -m32; -m64 is fine.
> >    // Even if I change the -I and -L paths to C:/Windows/SysWow64
> >    // and/or use tcc.exe -impdef to create .def files from them,
> >    // TCC still fails in finding _iob and other symbols.
> >    // Also, using #pragma comment(lib, "msvcrt") or -lmsvcrt
> >    // doesn't help at all.  Even if you do get TCC to somehow
> >    // stop complaining about missing symbols, it'd just include
> >    // a blank IAT.printf or IAT.freopen, causing segfaults.
> >    freopen("CONOUT$", "w", stdout);
> >    printf("This only compiles (and prints) under TCC in 64-bit mode.");
> > }
> > ```
> >
> > As mentioned earlier, this error in `-m32` happens regardless of other
> > switches like `-std`, `-shared`, `-static`, `-lmsvcrt`, `-subsyetem`,
> > etc.  So, at this point, I'm starting to think this might really be a
> > bug with TinyCC 0.9.27 (Win32 & Win64 builds) itself.
> >
> > _______________________________________________
> > Tinycc-devel mailing list
> > Tinycc-devel@nongnu.org
> > https://lists.nongnu.org/mailman/listinfo/tinycc-devel