Re: read command sometimes misses newline on timeout

[alex xmb sw ratchev]

plz write to bash list

about ur case , i dont much get it
async read ?

just small question .. why do u ' read again , to get full line '

and a small note
there is -t 0 , that returns when data'd be available
u maybe can test this approach

On Monday, October 7, 2024, Thomas Oettli <thomas.oettli@sfs.com> wrote:

> Sorry, English is not my mother tongue, so let me try to clear this.
>
> I have a script that asynchronously reads from a pipe by using read with
> timeout (basically the "reader" function in the example). The separator
> could be any character, I just assume that it is "\n" in this case. I do
> not control the writer to the pipe, lags may occur at any moment.
>
> Now this happens:
>
>
>    1. the script calls "read" on pipe
>    2. "read" hits timeout just after it has read "\n" from the pipe
>    3. "read" returns the full line, but exits with rc > 128
>    4. the script assumes a partial line, appends received data to a
>    buffer and calls "read" again, but without timeout to get the rest of the
>    line
>    5. because "\n" was already read at step 2, "read" returns the next
>    line
>    6. script appends received data to the buffer to complete the line
>    (due to wrong rc from step 3)
>
> The result is two lines in the buffer (without "\n" in between). Do you
> understand now what I mean?
> ------------------------------
> *Von:* alex xmb sw ratchev <fxmbsw7@gmail.com>
> *Gesendet:* Montag, 7. Oktober 2024 20:02
> *An:* Thomas Oettli <thomas.oettli@sfs.com>
> *Cc:* bug-bash@gnu.org <bug-bash@gnu.org>; chet.ramey@case.edu <
> chet.ramey@case.edu>
> *Betreff:* [EXT] Re: read command sometimes misses newline on timeout
>
> CAUTION: This email originated from outside the SFS organization. Do not
> follow guidance, click links or open attachments unless you recognize the
> sender and know the content is safe.
> MAIL FROM:  <@fxmbsw7@gmail.com> Report as Spam / Check Mail
> <https://mail1.sfs.biz/pyquarantine.php?quarantine=orig&action=report_or_check&mailfrom=fxmbsw7%40gmail.com&id=20241007200237_2002029C296CC>
>
> to know if read returned anything , even if its exit code is not 0 , check
> using [[ $reply ]]
>
> i really didnt get ur english or issue
>
> On Monday, October 7, 2024, Thomas Oettli <thomas.oettli@sfs.com> wrote:
>
>
> I know that it works in that case.
> This is just an example that tries to force read into the same
> situatiation that I hit in a real world example in which I don't control
> the input.
> ------------------------------
> *Von:* alex xmb sw ratchev <fxmbsw7@gmail.com>
> *Gesendet:* Montag, 7. Oktober 2024 19:17
> *An:* Thomas Oettli <thomas.oettli@sfs.com>
> *Cc:* bug-bash@gnu.org <bug-bash@gnu.org>; chet.ramey@case.edu <
> chet.ramey@case.edu>
> *Betreff:* [EXT] Re: read command sometimes misses newline on timeout
>
> CAUTION: This email originated from outside the SFS organization. Do not
> follow guidance, click links or open attachments unless you recognize the
> sender and know the content is safe.
> MAIL FROM:  <@fxmbsw7@gmail.com> Report as Spam / Check Mail
> <https://mail2.sfs.biz/pyquarantine.php?quarantine=orig&action=report_or_check&mailfrom=fxmbsw7%40gmail.com&id=20241007191742_006EB2A9EAD22>
>
> well try remove the -n in echo
>
> On Monday, October 7, 2024, Thomas Oettli <thomas.oettli@sfs.com> wrote:
>
> Sorry Alex, I don't understand exactly what you mean.
> Here is the Test-Script again with some comments:
>
>     function reader() {
>       local buf line
>       while :; do
>         read -t .01 buf                  # try to read line to $buf with 
> timeout
>         rc=$?
>         if (( rc == 0 )); then           # got a full line or the rest of a 
> partial line, append $buf to $line
>           line+=$buf
>         elif (( rc > 128 )); then        # ran into timeout
>           line+=$buf                     # maybe received partial line, 
> append $buf to $line and continue reading
>           continue
>         fi
>
>         # at this point, the content of $line should always be "TEST"
>
>         [[ $line != TEST ]] && echo Invalid line: $line && exit
>         echo OK
>         line=""                          # prepare to read next line, set 
> $line to empty string
>       done
>     }
>     reader < <(
>       while :; do
>         echo -n TEST
>         sleep .00$(($RANDOM%10))
>         echo
>       done
>     )
>
>
> Could you please explain to me, where exactly the mistake is? I don't know
> if you ran the script by yourself, but on all of my machines it exits after
> just a few seconds
> with the following output:
>
> Invalid line: TESTTEST
> ------------------------------
> *Von:* alex xmb sw ratchev <fxmbsw7@gmail.com>
> *Gesendet:* Montag, 7. Oktober 2024 16:41
> *An:* Thomas Oettli <thomas.oettli@sfs.com>
> *Cc:* bug-bash@gnu.org <bug-bash@gnu.org>; chet.ramey@case.edu <
> chet.ramey@case.edu>
> *Betreff:* [EXT] Re: read command sometimes misses newline on timeout
>
> CAUTION: This email originated from outside the SFS organization. Do not
> follow guidance, click links or open attachments unless you recognize the
> sender and know the content is safe.
> MAIL FROM:  <@fxmbsw7@gmail.com> Report as Spam / Check Mail
> <https://mail1.sfs.biz/pyquarantine.php?quarantine=orig&action=report_or_check&mailfrom=fxmbsw7%40gmail.com&id=20241007164140_2F94A29C1A462>
>
> there is a case , u [[ $readreply ]] after read
>
> On Monday, October 7, 2024, Thomas Oettli via Bug reports for the GNU
> Bourne Again SHell <bug-bash@gnu.org> wrote:
>
> I agree with you, but it should never happen that read returns timeout,
> also returns the full line and has already read the newline character.
> If that happens, there is no way for the script to decide what to do.
> Please see the provided test script, it showcases the error.
>
> If I did a mistake there, I would gladly change it. But I currently don't
> see any way how to handle this properly from the script side.
>
> Please also see the answer from Martin D Kealey, I think he is on to
> something:
> https://lists.gnu.org/archive/html/bug-bash/2024-10/msg00007.html
>
>
>
>
> On 10/4/24 8:18 AM, Thomas Oettli via Bug reports for the GNU Bourne Again
> SHell wrote:
>
> > Bash Version: 5.2
> > Patch Level: 26
> > Release Status: release
> >
> > Description:
> > I have tried to write a bash script that asynchronously reads from a
> pipe (line by line) with the help of "read -t".
> > If the timeout occurs in just the right moment, read returns the full
> line, but the return code says timeout (rc > 128).
>
> If the read command times out, it always returns > 128, so if you have a
> return code in that range, you can assume read timed out and react
> accordingly.
>
> > Therefor it is not possible to know if a full line was returned or not.
>
> When read times out, it always returns what it read before the timeout in
> the buffer, so you don't lose any data. Whether or not that's a `full line'
> is up to timing, and it's up to the script to decide how to cope with it.
>
> --
> ``The lyf so short, the craft so long to lerne.'' - Chaucer
> ``Ars longa, vita brevis'' - Hippocrates
> Chet Ramey, UTech, CWRU chet@case.edu http://tiswww.cwru.edu/~chet/
>
>