Re: Is gawk cloud ready?

[arnold]

Hello.

Thanks for the note.

The problem is the cifs filesystem (I guess on Linux).  When the remote disk
fills up, the appropriate error is not returned (fwrite returning zero
with errno set to ENOSPC).

In this case, gawk is only as good as the underlying filesystem support,
there isn't anything else I can do.

I note that in the cifs mount options there is the word "nounix".
I don't know what the semantics of "unix" vs. "nounix" are; that might
be an area for further investigation.

Gawk 4.2.1 is probably a decade old, or more, but I doubt that
a newer version would behave any differently.

Best wishes,

Arnold

Simon Liesenfeld <surgeonde@yahoo.de> wrote:

> From: dadalus
> To: bug-gawk@gnu.org
> Subject: [Is gawk cloud ready?]
>
> Configuration Information [Automatically generated, do not change]:
> Machine: x86_64
> OS: linux-gnu
> Compiler: gcc
> Compilation CFLAGS: -g -O2 -DNDEBUG
> uname output: Linux DADAEL 6.7.0-060700rc5-generic #202312102332 SMP 
> PREEMPT_DYNAMIC Sun Dec 10 23:41:31 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
> Machine Type: x86_64-pc-linux-gnu
>
> Gawk Version: 5.2.0b
>
> Attestation 1:
>        I have read 
> https://www.gnu.org/software/gawk/manual/html_node/Bugs.html.
>        Yes / No  [ Choose one. If "No", then why haven't you? ]
>
> Attestation 2:
>        I have not modified the sources before building gawk.
>        True / False
>        [ Choose one. If "False", then please explain what you did and why. ]
>
> Description:
>        [
> Imagine the following:
> Since centuries a one liner gawk call in a script checks 
> whether any other subsequent program with fluffy disk problem handling
> can be successful in writing apprx. 2Gb to file system.
> And this all shall be in the cloud soon.
>
> awk 'BEGIN{for(i=1;i<= 2  * 1000;i++) { print sprintf("%-999997s", 
> sprintf("%09d %s",i,um)); }   }' > /fillup/fillup.dat ; echo $?
>
>
> First the local legacy local/NFS filesystem:
>
> ---------------------------------
> ubuntu ext3 /fillup which just has setup in erroneously with 1GB only )
> Filesystem     1M-blocks  Used Available Use% Mounted on
> /dev/sdb             975              1            924   1% /fillup
>
> awk: cmd. line:1: fatal: print to "standard output" failed: No space left on 
> device
> 2
> This is great news, gawk detected the problem and compensated the fluffiness
> of the subsequent program.
> I get one message and a proper return code 2
>
> Now the Azure 1Gb Cloud drive, erroneously with 1GB only too:
>
> ----------------------------------
>
> Filesystem                              1M-blocks  Used Available Use% 
> Mounted on
> %df -m
> //dev.file.core.windows.net/fillupaz      1024  1024         1 100% /fillupaz
>
> %mount
> //dev.file.core.windows.net/fillupaz on /fillupaz type cifs 
> (rw,relatime,vers=3.0,cache=strict,username=fillapaz,uid=1005,
> noforceuid,gid=2074562182,noforcegid,addr=xx.xx.xx.xx,file_mode=0770,dir_mode=0770,
> soft,persistenthandles,nounix,serverino,mapposix,mfsymlinks,rsize=1048576,wsize=1048576,
> bsize=1048576,echo_interval=60,actimeo=1)
>
> awk --- as above --- > /fillupaz/fillup.dat ; echo $?
>
> <No error message >
> 0 (return code, sucess)
>
> This is bad news, gawk did not detect the problem,
>  On the Azure machine there is just
> % gawk --version
> GNU Awk 4.2.1, API: 2.0 (GNU MPFR 3.1.6-p2, GNU MP 6.1.2)
> May somebody can try with a newer version of gawk.
>
> ____________________________________________________
>
> In both cases only a fraction of the file is left on the device,
>
> that does not surprise.
>
> Amazingly the cloud drive almost fulfills the requirements during the run,
>
> file size and last record approaches 2GB and 2 Million lines,I can see by 
> %wc /fillupaz/fillup.dat
>
> but at the end, the file truncated. This is not bad, but good cloudiness.
> Given the drive is monitored --at an increased cloud tariff off course--, 
> an operator can request an extension of the drive, 
> thanks to the behavior of the cloud drive the unwritten content is preserved 
> meanwhile.
>
> Given not, we do not have notification about the problem, 
> and that's even worse than the problem as such.
>
> ____________________________________________________
>
>
>
>
> ]
>
> Repeat-By:
>        [Describe the sequence of events that causes the problem to occur.]
>
> Fix:
>        [Description of how to fix the problem.  If you don't know a
>        fix for the problem, don't include this section.]
>