Re: can I tell gcc/ld in Mingw/Win32 to generate a WinNT .SYS (driver) file?

[Frantisek . Rysanek]

Just a quick update:
I took a look at the source code of binutils and BFD.
Found a reference to "Microsoft Portable Executable and Common Object
File Format Specification" and followed that (it's freely available
 for download from MS). I also found a PE structure editor called
LordPE.

I tried comparing notepad.exe and system32/drivers/serial.sys.
I found out that both have the DOS 2.0 compatible "MZ" stub at the
beginning, containing the words "This program cannot be run in DOS
mode". Yet if you rename the SYS file to an EXE, Windows refuse to run
it at all.

So I loaded it into LordPE and, apart from other differences such as
the expected base address, alignment and other sizes, there was a
difference in the file's "Subsystem" flags: a regular EXE is flagged
"Windows GUI" or "Windows Console", whereas the SYS file is flagged
"Native" (= non-executable system file).

If I rename the SYS file to an EXE and flip that Subsystem flag to
"Windows Console", voila - Windows attempt to execute the SYS file in
user space and complain that a WMIsomething.SYS "dynamically linked
library" was unavailable within the search path :-))

Which makes me wonder if I could just compile my code into an object
file, tell LD to produce an i386-pe EXE, though I wonder what it would
tell me if there was no main() in the code, rename that EXE to a .SYS
and  flip the subsystem flag to "Native". Hmm maybe I'd have to find
any referenced external symbol names in the various SYS files and pass
those to the linker...

The clean way would be to make LD produce a more appropriate header for
the .SYS file, with the subsystem flag set to "native", and maybe other
flags and settings that I'm not aware of at the moment. The trouble is
that I don't know about any command-line options to LD that could
possibly achieve that. The functionality seems present in BFD, but
probably absent in LD - which is a mere confirmation of something
mentioned in the manual: that not all features of the BFD back-end
formats are reachable via the binutils programs. Seems like I'd have to
roll my own linker? :-))

Again, any further ideas are welcome.

Frank Rysanek