Re: [Tinycc-devel] Why does TinyCC fail to link standard C runtime functions in 32-bit mode but works in 64-bit mode?

[avih]

You said originally:

> // _MSVCRT_ being defined will cause MinGW's stdio.h to use _iob as
> // opposed to _imp___iob; only the former is defined in msvcrt.dll.
> // However, even though _iob is exported by both the 32- and 64-bit
> // versions of said dll, TinyCC still fails to find _iob in the former.

Is defining _MSVCRT_ manually a documented API of mingw or otherwise? or some 
common knowledge which more than few projects apply? (link?) or is it an 
experiment which you hoped would work for you?

I also saw that in tcc win32/include/_mingw.h there's explicit #undef _MSVCRT_, 
which comes from a big commit in 2009: dc251a7d (win64: use new headers from 
mingw) by grischka.

It could be an incorrect import of the mingw headers, or it's possible that it 
was decided to not support it and strip it from the tcc headers. Hard to tell 
because there are no details on any changes that were made specifically for tcc.

Around that time (2009), I think tcc was supposed to also work with the full 
set of unmodified mingw headers, at least for the most part (if you needed more 
than the minimal subset which ships with tcc), but I don't know how much of it 
holds these days.

You could try -I /path/to/w64devkit/<whatever>/include and see if it gets 
fixed. It's a long shot (i.e. i think there would be unrelated errors because 
new mingw), but worth a try.

Do you have a suggested fix and an estimation of the scope of its effect?

Is _iob the only issue around the _MSVCRT_ definition? or are there others 
which need similar handling?

I'm hesitant to make changes with large effect scope, because honestly I'm not 
familiar enough with the headers construction and how much in them was crafted 
specifically to be embedded in tcc (for the most part they're verbatim from 
mingw, but there are exceptions, including some with big scope).

Maybe grischka can share his opinion?

avih


On Monday, November 25, 2024 at 02:06:50 AM GMT+2, Fereydoun Memarzanjany 
<thraetaona@ieee.org> wrote: 





I had originally attempted to use `fprintf(stdout, "...");` which, due
to its use of stdout, would fail.  Not knowing the root cause of _iob,
I mistakenly generalized printf and other functions; sorry about that.

In TCC/mob, _MSVCRT_ gets cancelled by <windows.h> because it
internally includes <_mingw.h>.  For some reason, <_mingw.h> has these
two lines 41 and 42:

```
#define __MSVCRT__ 1
#undef _MSVCRT_
```

It defines __MSVCRT__ to be 1 yet undefined _MSVCRT_ (causing
<stdio.h> to continue using _imp___iob if <windows.h> is included
after _MSVCRT_)

Under both the latest mob and 0.9.27, your minimal example does
compile and print; however, the issue is that it compiles under
-subsystem=console.  For GUI apps compiled under the
-subsystem=windows switch (i.e., the default behavior), freopen() is a
necessity.  If, after allocating a console and prior to the printf()
call, freopen() is not called on stdout/stderr/stdin under a GUI
application, nothing gets printed at all.

Because freopen() depends on stdout, compilation will inevitably fail
under `-m32`, on both mob and 0.9.27; the __atribute__(dllimport) was
able to remedy that.

I dug deeper and found that those new MinGW-64 headers from w64devkit
also do the exact same fix I did (albeit they hide it under various
macro definitions).  I was surprised that you could compile my
original test case (with the _MSVCRT_ defined) under w64devkit;
because, when I look at their <stdio.h>, it appears that they only
have that __MINGW_IMP_SYMBOL macro expansion for the non-_MSVCRT_
case.  In other words, their `extern FILE _iob[]` case (which runs
whenever _MSVCRT_ is defined) was *not* decorated with
__attribute__((dllimport)), and it *should* have failed.

The difference is in these two lines of <stdio.h> (83 & 84) from new w64devkit :

```
  _CRTIMP FILE *__cdecl __iob_func(void);
#define _iob  __iob_func()
```

Whereas TCC's current mob <stdio.h> defines __iob_func based on _iob,
w64devkit does the opposite: it defines _iob based on __iob_func().
Notice how __iob_func() is decorated with _CRTIMP.  In line 30 of
crtdll.c  in w64devkit, they define _CRTIMP as follows:
#define _CRTIMP __declspec(dllimport)

Given that __declspec(dllimport) and __atribute__((dllimport)) have
the exact same behavior, this results in a successful compilation.

_CRTIMP also exists on TCC's old MinGW includes, so I suppose we could
declare _iob and _imp___iob with _CRTIMP rather than __atribute__ for
consistency's sake.

(On an unrelated note, it looks like __MINGW_IMP_SYMBOL was introduced
back in 2009; I didn't know TCC's headers are older than that.)

On Sun, Nov 24, 2024 at 4:20 PM avih <avihpit@yahoo.com> wrote:
>
> Before I'll respond to your last email, you said in your original email that 
> even printf doesn't work:
>
> > `_iob` is not the only "unresolved" symbol, either; `printf`,
> > `freopen`, `freopen_s`, and basically everything from the CRT will
> > fail to link.
>
> But in my minimal example there's only _MSVCRT_ and #include <stdio.h> and 
> printf and it does compile and link (and runs, and prints), and there's no 
> #include <windows.h> which you think cancels _MSVCRT_.
>
> Is my example incorrect somehow? or maybe it didn't work in 0.9.27 but does 
> work with recent mob?
>
> Now, to your suggested fix.
>
> Looking at a recent mingw installation (specifically w64devkit 2.0 - 
> https://github.com/skeeto/w64devkit), it looks to me pretty much the same as 
> this code in tcc stdio.h (w64devkit/i686-w64-mingw32/include/stdio.h lines 
> 101-129) with minor difference that _imp___iob is a macro expansion and not 
> explicit.
>
> Your original test case (with freopen and _MSVCRT_ after #include 
> <windows.h>) does compile with w64devkit x86, so at the very least your 
> suggestion is not how it's handled in new mingw.
>
> Can you maybe dig into that and figure out what makes it tick in w64devkit, 
> and see if a similar thing can be applied to the tcc headers?
>
>
>
> On Sunday, November 24, 2024 at 11:05:38 PM GMT+2, Fereydoun Memarzanjany 
> <thraetaona@ieee.org> wrote:
>
>
>
>
>
> After nearly 16 hours of debugging, I've found the culprit: `_iob` and
> `_imp___iob` should have been declared with either
> `__attribute__((dllimport))` or `__declspec(dllimport)`
>
> To be more specific, lines 93 to 106 of <stdio.h> in TCC's ./include
> directory are currently written as follows:
>
> ```c
> #ifndef _STDIO_DEFINED
> #ifdef _WIN64
>  _CRTIMP FILE *__cdecl __iob_func(void);
> #else
> #ifdef _MSVCRT_
> extern FILE _iob[];    /* A pointer to an array of FILE */
> #define __iob_func()    (_iob)
> #else
> extern FILE (*_imp___iob)[];    /* A pointer to an array of FILE */
> #define __iob_func()    (*_imp___iob)
> #define _iob __iob_func()
> #endif
> #endif
> #endif
> ```
>
> To fix them so that it compiles under `-m32` just as fine as it does
> under `-m64`, you need to change them to the following:
>
> ```c
> #ifndef _STDIO_DEFINED
> #ifdef _WIN64
>  _CRTIMP FILE *__cdecl __iob_func(void);
> #else
> #ifdef _MSVCRT_
> __attribute__((dllimport)) extern FILE _iob[];    /* A pointer to an
> array of FILE */
> #define __iob_func()    (_iob)
> #else
> __attribute__((dllimport)) extern FILE (*_imp___iob)[];    /* A
> pointer to an array of FILE */
> #define __iob_func()    (*_imp___iob)
> #define _iob __iob_func()
> #endif
> #endif
> #endif
> ```
>
> (NOTE: As I said earlier,  both `__attribute__((dllimport))` and
> `__declspec(dllimport)` work here.  Either of them fix it for both
> _MSVCRT_ and non-_MSVCRT_ headers.)
>
> A few additional details on why this fix is even useful in the first place:
>    - freopen() is optional in -subsystem=console (i.e., CLI) but
> absolutely required if you compile in -subsystem=windows (i.e., GUI)
> mode; if freopen() isn't called in the latter, printf and other std
> outputs don't show up on the allocated console;
>    - freopen() depends on the value of `stdout`, and `stdout` is
> defined based on either _iob or _imp___iob in stdio.h;
>    - _imp___iob is too old and no longer exists in newer msvcrt.dll
> or ucrtbase.dll, making _iob (and therefore defining _MSVCRT_) the
> go-to choice; and
>    - #define'ing _MSVCRT_  before <windows.h> is akin to not defining
> it at all; I think <windows.h> undefines it somewhere, internally.
>
> I still find it strange that such a "fix" was required, because TCC
> works just fine under `-m64` anyway.  For `-m32` builds, do you think
> we should patch the stdio.h in the mob branch ourselves?  I'm not sure
> if this is a TCC-specific patch or if it actually affects any compiler
> that uses stdio.h from MinGW.
>
> (Also, I did compile the latest mob branch using "build-tcc.bat -x -c
> cl" but this stdio.h bug affected it the same way it affected 0.9.27.)
>
>
> On Sun, Nov 24, 2024 at 12:09 PM avih <avihpit@yahoo.com> wrote:
> >
> > tcc 0.9.27 is few years old now, and hopefully 0.9.28 would be released 
> > sooner rather than later, so you should try the latest version, which is 
> > the mob (default) branch here:
> > https://repo.or.cz/tinycc.git
> >
> > You'll need to build it yourself. If you can't build it or can't find a 
> > recent pre-build binary, then I can upload a version to some pastebin (do 
> > ask, but do try to build it first if you can).
> >
> > You should also keep in mind that tcc is a not mingw gcc clone. It does use 
> > (old) mingw headers, but you should not expect 100% compatibility, and the 
> > headers set is intentionally stripped down to keep it minimal (although 
> > many programs compile out of the box with the existing headers)
> >
> > Specifically about your issue. With latest tcc, if I comment out the line 
> > with "freopen" in your sample program, or with a minimal program like this:
> >
> > #define _MSVCRT_
> > #include <stdio.h>
> >
> > int main() {
> >    printf("Hello, world\n");
> >    return 0;
> > }
> >
> > Then it compiles and works fine.
> >
> > But with your original program with freopen, tcc indeed complains about 
> > undefined symbol '_iob'.
> >
> > However, if I move the _MSVCRT_ definition above #include <windows.h>, then 
> > it does compile (I didn't try to run it, but I presume it would work).
> >
> > So I'd think largely it's OK. However, I was not aware of this _MSVCRT_ 
> > thingy, so I don't know what differences to expect and I didn't try to 
> > understand it further.
> >
> > I don't know whether this should be considered an issue that it works in 
> > tcc only if _MSVCRT_ is defined before windows.h is included (to me that 
> > feels reasonable, so do double check), but if it should be considered an 
> > issue, then maybe you could help by trying to pinpoint the cause at the 
> > headers, and report back.
> >
> > Cheers,
> > avih
> >
> >
> >  On Sunday, November 24, 2024 at 07:43:30 PM GMT+2, Fereydoun Memarzanjany 
> >via Tinycc-devel <tinycc-devel@nongnu.org> wrote:
> >
> >
> > If you use TinyCC in its 32-bit mode (`-m32`) to compile a sample
> > program that uses any CRT function/symbol from `msvcrt.dll` (such as
> > the snippet provided later in this message), you'll be met with
> > compilation failures:
> >
> > `tcc.exe -std=c11 -Wall -Werror -Wl,-subsystem=console -m32 .\main.c`
> > "tcc: error: undefined symbol '_iob', missing __declspec(dllimport)?"
> >
> > (This only happens under `-m32`, whereas `-m64` works perfectly fine.)
> >
> > `_iob` is not the only "unresolved" symbol, either; `printf`,
> > `freopen`, `freopen_s`, and basically everything from the CRT will
> > fail to link.
> >
> > Regardless of whether or not you use `-lmsvcrt`, `#pragma comment(lib,
> > "msvcrt")`, `_declspec(dllimport)`, `attribute ((dllimport))`,
> > `-static` or `-shared`, or even `-impdef` on
> > "C:\Windows\SysWow64\msvcrt.dll" (or earlier versions thereof:
> > "msvcrt40.dll"), TCC still complains.
> >
> > I've verified with `DUMPBIN.exe` that both 32- and 64-bit "msvcrt.dll"
> > do, in fact, define `_iob` and other symbols.
> >
> > By some arcane logic, the following works perfectly fine: `tcc.exe
> > -std=c11 -Wall -Werror -Wl,-subsystem=console -m64 .\main.c`
> >
> > `main.c`
> > ```c
> > //#pragma comment(lib, "msvcrt")
> > //__attribute__((dllimport)) extern __declspec(dllimport) FILE _iob[];
> >
> > #include <windows.h>
> >
> > // _MSVCRT_ being defined will cause MinGW's stdio.h to use _iob as
> > // opposed to _imp___iob; only the former is defined in msvcrt.dll.
> > // However, even though _iob is exported by both the 32- and 64-bit
> > // versions of said dll, TinyCC still fails to find _iob in the former.
> > #define _MSVCRT_
> > #include <stdio.h>
> >
> > void main() {
> >    // AllocConsole() and basically everything from kernel32.dll or
> >    // user32.dll work perfectly fine, both in -m32 and -m64; it's
> >    // only msvcrt.dll that causes issues with TinyCC.
> >    AllocConsole();
> >
> >    // Any CRT function (e.g., freopen, freopen_s, printf, etc.)
> >    // fail to get linked properly ONLY in -m32; -m64 is fine.
> >    // Even if I change the -I and -L paths to C:/Windows/SysWow64
> >    // and/or use tcc.exe -impdef to create .def files from them,
> >    // TCC still fails in finding _iob and other symbols.
> >    // Also, using #pragma comment(lib, "msvcrt") or -lmsvcrt
> >    // doesn't help at all.  Even if you do get TCC to somehow
> >    // stop complaining about missing symbols, it'd just include
> >    // a blank IAT.printf or IAT.freopen, causing segfaults.
> >    freopen("CONOUT$", "w", stdout);
> >    printf("This only compiles (and prints) under TCC in 64-bit mode.");
> > }
> > ```
> >
> > As mentioned earlier, this error in `-m32` happens regardless of other
> > switches like `-std`, `-shared`, `-static`, `-lmsvcrt`, `-subsyetem`,
> > etc.  So, at this point, I'm starting to think this might really be a
> > bug with TinyCC 0.9.27 (Win32 & Win64 builds) itself.
> >
> > _______________________________________________
> > Tinycc-devel mailing list
> > Tinycc-devel@nongnu.org
> > https://lists.nongnu.org/mailman/listinfo/tinycc-devel