[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
puzzling problem with read seemingly losing data
From: |
Hendrik Fritsch |
Subject: |
puzzling problem with read seemingly losing data |
Date: |
Tue, 29 Dec 2020 02:03:23 +0100 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 |
Hello,
I am writing with a rather puzzling problem.
It seems that 'read' loses some data?
I want to process some JSON data with the following:
> videodata="$(jq -r -c .data[] "$t/videos.json" | tee "$t/videodata")"
> while read -r video; do
> echo ----
> echo "video $video"
>
> videoid="$(echo "$video" | jq -r .id -)"
> echo "videoid $videoid"
>
> # ... other stuff
> ffmpeg -n -i "$hlsurl" -c copy "$outname"
>
> done <<<"$videodata"
(videos.json is just one big json object, and videodata will contain one
object per line (verified with tee).)
The first few iterations work just fine and 'jq' can parse '$video' to
get '$videoid'.
Until ffmpeg ran once. (It does not run, when an output file exists
already.)
The problem is the first iteration AFTER FFMPEG HAS RUN ONCE. Then I get
the following output:
> ----
> video
> 00Z"}],"clearkeyEnabled":true,"contentDescriptors":[],"contentRatings":[],"customAttributes":{},"description":"<snip>
There is a substantial amount of data missing at the beginning of the line.
In '$t/videodata' I can lookup the actual content of the line:
> {"attributes":{"airDate":"","alternateId":"","availabilityWindows":[{"package":"","playableEnd":"","playableStart":"<snip>00Z"}],"clearkeyEnabled":true,"contentDescriptors":[],"contentRatings":[],"customAttributes":{},"description":"<snip>
I also tried this syntax: ( while...; done < <(jq ...) ) but the problem
is the same.
(It might be relevant here - ffmpeg outputs enormous amounts of log data.
I don't know why, they seem to come from the matroska muxer(?) and look
like debug output.
'-loglevel warning' does not get rid of them.)
Debian GNU/Linux 9.13 (stretch)
GNU bash, version 4.4.12(1)-release (x86_64-pc-linux-gnu)
Package version 4.4-5
Thanks,
Hendrik
- puzzling problem with read seemingly losing data,
Hendrik Fritsch <=