Re: [Help-bash] Performing quote removal on data

[Chet Ramey]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 5/28/14, 4:05 PM, Maarten Billemont wrote:

> I'd rather not rely on eval at all.  The behaviour I want is essentially the 
> default bash pathname completion:

It's important to note, before getting into a discussion of how bash and
readline work together to do completion, that readline performs filename
completion.  The `bash default' completion adds things on top of that --
things like command word completion, globbing, shell variable expansion,
looking into shell constructs like command substitution, and so on.  The
first two examples in your list below are actually done internally by
readline.

This is the difference between the `-o default' and `-o bashdefault'
options to complete and compgen.

> mkdir "dir 1"
> 
> Now:
> - "dir" completes to "dir\ 1"
> - "dir\ " completes to "dir\ 1"
> - "di*" completes to "dir\ 1"
> - "$(:)di" does not complete
> 
> man bash does not describe exactly what bash's default pathname completion 
> does, but it looks like we're seeing quote removal, word splitting and then 
> pathname expansion.

Bash doesn't do default pathname completion; it relies on readline for
that.  Here's how completion works, in a nutshell.

1.  The user enters a key sequence bound to `complete'. Readline identifies
    the word to be completed using its set of boundary characters, records
    some state, and calls its internal completion function.

2.  The first thing the internal completion does is check whether or not
    the calling application has registered an attempted-completion hook.
    If it has, that gets first shot.  This is how bash makes programmable
    completion and its own shell-specific completions work.  This function
    returns a list of matches.  If there are no matches, readline uses its
    own default filename completion.  This hook can also tell readline to
    not attempt its own completion at all.

3.  Readline takes the matches, does or does not add its own possible
    completions using its filename completion, and sorts and displays them.

In general, compgen dequotes its argument as necessary, depending on
whether or not readline detects quote characters in the word to be
completed, produces `raw' possible completions, and readline takes them
and quotes as necessary before inserting them into the line.

It's not that easy to use compgen to test completion standalone, since it
doesn't have all of the readline context, but it's ok.  It's better in
bash-4.3 than previous versions.  The rules about when it's appropriate to
dequote the word supplied as an argument are still up in the air, though,
and work much better when called via readline than from the prompt.


> Since the goal is essentially to write a reliable programmable completion 
> function, and that function needs to populate a COMPREPLY array based on the 
> word that is being completed, it might be possible to avoid converting the 
> word to be completed into a literal:
> 
> 1. If we have a list of possible completions to filter with the word 
> currently being completed, we can use this instead:
> 
> partial=${COMP_WORDS[COMP_CWORD]} # This contains the readline word currently 
> being completed.
> completions=( house ball printer school "school bus" ) # A list of eligible 
> words for completion.
> IFS=$'\n' read -r -d '' COMPREPLY < <(compgen -W "$(printf '%q ' 
> "address@hidden")" "$partial") # holy crap, it really shouldn't have to be 
> this hard to do it right!
> 
> 2. If we do NOT have a list of possible completions but rather need the 
> current word being completed to populate that list, we're in trouble.

So in what contexts do you need to dequote the argument?  I mean, pathname
completion is one, but you aren't taking advantage of the options complete
already has.

> 2a. One such case is doing pathname completion, but here we can fall back to 
> compgen -o bashdefault:
> 
> IFS=$'\n' read -r -d '' COMPREPLY < <(compgen -o bashdefault "$partial*") # 
> notice the trailing *
> 
> Sadly though, while it completes "dir\ " into "dir 1" just fine, it does NOT 
> complete "dir' " or "dir' '" into "dir 1".  Consistency?  Why would you want 
> that?

If you had asked for readline's default filename completion with
`-o default', you might have gotten farther.  Also, readline has its own
ideas about quoted strings, and so the open-quoted "dir' " is treated
differently.  That's not going to do what you want no matter how good your
completion function is.

> Notice also that compgen -o dirnames "$partial" can be used to complete "dir' 
> '" into "dir 1" nicely, but that in turn does not support pathname expansion 
> and also only works on dirs.  The similarly named -o filenames does not do 
> the same for filenames (I couldn't actually reproduce any effect of -o 
> filenames).

Since -o filenames tells readline what to do with the completions, it's
not surprising that you didn't see any effects when running compgen at the
shell prompt.

> So on that note, how do I, in programmable completion, perform a simple, 
> regular, default bash pathname completion without re-implementing quote 
> removal in bash code?

Maybe identify the cases where compgen doesn't perform the right quote
removal as a start.

Chet
- -- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRU    address@hidden    http://cnswww.cns.cwru.edu/~chet/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (Darwin)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlOI5H8ACgkQu1hp8GTqdKsR+QCeKVmsuykC1aLEiId9WfaNZFXI
y14AoJvajSFA1zf1n4od4eykURWoheRw
=twdF
-----END PGP SIGNATURE-----