Re: Can't find file

[Greg Wooledge]

On Fri, Nov 15, 2019 at 07:04:32PM -0500, David Niklas wrote:
> I'm still a bit curious why printing a quoted filename from awk will not
> work -- even when it has no spaces.
> ls -- $( awk -v z="EXAMPLE" 'BEGIN{ print "\"" z "\"" }' )

wooledg:~$ args $( awk -v z="EXAMPLE" 'BEGIN{ print "\"" z "\"" }' )
1 args: <"EXAMPLE">

> Oh, wait a second. The file names quotes are being treated as part of the
> filename and not being stripped.

Correct.

> OTOH: If I don't include quotes
> filenames with spaces cease to work correctly. So why is only partial
> parameter expansion occurring?

wooledg:~$ awk 'BEGIN {print "\"foo bar\" \"baz quux\""}'
"foo bar" "baz quux"

OK?  Now, if you wrap a command substitution around that, what happens?

wooledg:~$ args $(awk 'BEGIN {print "\"foo bar\" \"baz quux\""}')
4 args: <"foo> <bar"> <"baz> <quux">

This is covered in <https://mywiki.wooledge.org/BashPitfalls#pf1> as
well as many other places.

What you APPEAR to be ATTEMPTING to do is generating a sequence of
shell-quoted words that a shell could parse as arguments.  If that's
the goal, then you would want to use eval at some point.  However,
the WAY you're going about this assumed goal is all wrong.  You are
not escaping the filename's contents correctly.  So, eval on this
stream would NOT be safe.

Simply putting double quotes around a filename is NOT sufficient.  The
filename can contain backticks and $( ) and so on.  All of these
syntactic elements would be treated as live code by eval.

The correct way to shell-quote a string SAFELY is to replace every
single-quote character with the 4 characters '\'' and then put
single quotes around the string.

This is massively painful in awk.  It's only severely painful in bash,
so let's do it in bash.

wooledg:~$ f="don't stop believing.mp3"
wooledg:~$ q=\'
wooledg:~$ printf "'%s'\\n" "${f//$q/$q\\$q$q}"
'don'\''t stop believing.mp3'

There's our shell-quoted result for a single filename.  Let's give it
a few.

wooledg:~$ files=("don't stop believing.mp3" "'ere's lookin' at you")
wooledg:~$ printf "'%s'\\n" "${files[@]//$q/$q\\$q$q}"
'don'\''t stop believing.mp3'
''\''ere'\''s lookin'\'' at you'

Now, we wrap a command substitution around it, but we QUOTE the result,
because we do not want the shell to perform any premature expansions.

Then we stuff that quoted shell-quoted command output into an array
assignment.  This gives us a shell command that could be evaluated.

Then we eval it.

wooledg:~$ eval array=("$(printf "'%s'\\n" "${files[@]//$q/$q\\$q$q}")")
wooledg:~$ declare -p array
declare -a array=([0]="don't stop believing.mp3" [1]="'ere's lookin' at you")

So, what we've done is take an array of filenames, successfully SERIALIZE
it into an eval-able format, CAPTURE that serialized stream in a separate
shell process, and EVALUATE (deserialize) it in the capturing process,
in order to recreate the original array of filenames.

Doing the serializing in awk will be WORSE, because you've got an extra
layer of quotes around the awk script.

This is the absolute last resort, and is not a thing you should be
doing lightly.  You should definitely not be doing it at all if you
don't understand every single step, WHY that step is needed, and HOW to
make sure it's executed correctly.  A single mistake, and your eval'ed
serialized stream becomes a code injection bug.