Re: fseek and ftell with large files on Windows

[Bruno Haible]

Markus Mützel wrote:
> The Octave functions that are being called in the reproducer are named 
> "fseek" and "ftell". But the gnulib functions that are called internally are 
> "fseeko" and "ftello". If I understood Paul Eggert in that bug report 
> correctly, he found at least one place where Octave is using "fseek" instead 
> of "fseeko".
> But stepping through with a debugger, - at least in the code paths that 
> reproduce the issue - "fseeko" and "ftello" are used as far as I can tell.

In order to make sure that Octave does not have bugs with files > 2 GiB
(without restrictions like "as far as I can tell"), you need four things:
  1) Make sure to request the modules 'fseeko', 'ftello' from gnulib, not
     'fseek' and 'ftell'.
  2) Use "nm octave" (or "nm octave.exe" on mingw), as well as
     "nm liboctave.so", "nm liboctgui.so", "nm liboctinterp.so",
     to verify that the binaries don't use the functions 'fseek', 'ftell'.
  3) In the code around the fseeko and ftello calls, drop the use of the
     'long' type and use 'off_t' instead.
  4) Use gnulib module 'largefile' as well.

1), 2), 3) together make sure that you work with 'off_t' values for file
positions. 4) ensures that 'off_t' is 64-bit.

For testing, it is probably simplest to take a 32-bit Linux (such as a
Linux/i386 distro), possibly installed in a virtual machine, and perform
the large-files tests there.

If you have 1), 2), 3), 4) in place, and testing passes on a 32-bit Linux,
it's very likely that it will also pass on native Windows without extra effort.

> Octave is using 'off_t' as offset when calling "fseeko".

Paul's patch spotted a few 'long int' casts, no? These need to be eliminated.

> Checking the content of the generated "sys/types.h" header, I see the 
> following:
> 
> /* Override off_t if Large File Support is requested on native Windows.  */
> #if 0
> /* Same as int64_t in <stdint.h>.  */
> # if defined _MSC_VER
> #  define off_t __int64
> # else
> #  define off_t long long int
> # endif
> /* Indicator, for gnulib internal purposes.  */
> # define _GL_WINDOWS_64_BIT_OFF_T 1
> #endif
> 
> If I understand correctly, the fact that this section is skipped means that 
> 'off_t'
> has a 64-bit size here (on mingw-w64) and doesn't need to be redefined.

It means that either 'off_t' was already 64-bit, or you haven't requested the
'largefile' module.

Still, you need to do step 3).

> With the proposed changes, 'fseeko' would be defined to '_fseeki64' for the 
> function body. (And similar changes for "ftello.c".) Isn't that needed for 
> large files on Windows?

It doesn't need to be done in Gnulib if mingw already does it.

Please come back with proposed patches for Gnulib after you have done 1), 2),
3), 4), and problems persist.

Bruno