[Help-bash] Behavior of 'complete -o filenames' unclear

[Michael Siegel]

Hello,

I'm out of ideas on how to solve a problem in an auto-completion script
I've written. The problem is with the behavior of 'complete -o
filenames'.

Here's the script:


# dbm_completion_bash
#
# Auto-completion for dbm in Bash

_dbm_complete_bash() {

local IFS=$'\n'
local dbm_dir="$HOME/.dbm"

# Only perform bookmark name completion on the first argument to dbm,
# except when that first argument is '-d' (the delete switch). In that
# case, perform bookmark name completion on any given argument (in
# effect: on any argument supplied to '-d').
if [ "${#COMP_WORDS[@]}" -lt 3 ] || [ "${COMP_WORDS[1]}" = '-d' ]
then
# Complete the current word, based on file names in $dbm_dir, and
# cut the directory prefix.
COMPREPLY=($(compgen -f -- "$dbm_dir/${COMP_WORDS[COMP_CWORD]}" | \
cut -d '/' -f 5))
fi
}

complete -o filenames -F _dbm_complete_bash dbm
# -o filenames: Perform escaping of shell special characters.


To give some background: dbm is a simple directory bookmarking system
for the shell, implemented as a set of shell functions that are sourced
in ~/.bashrc. The completion script given above is sourced in the file
containing the dbm function set. Bookmarks are simply symbolic links
inside ~/.dbm.

Running dbm without arguments will simply display the existing bookmarks
and their destinations in the file system:

~$ dbm
dbm -> /home/msi/devel/shell/dbm
msiism -> /home/msi/devel/www/msiism
sand box -> /home/msi/devel/sandbox

To jump to a bookmarked location, you'd invoke dbm with the respective
bookmark name as an argument. Having that argument completed when you
hit <Tab> generally works, except for one particular situation. And that
is when there is a directory of the same name as the completed bookmark
name in the current working directory.

In that case, the shell will add a slash to the name, as if it were
completing directory names within $PWD:

~/devel/sandbox/probe$ ls
a_file msiism/ tests/
~/devel/sandbox/probe$ dbm msi<Tab> # Results in 'dbm msiism/'
~/devel/sandbox/probe$

Now, the Bash Reference Manual does say that '-o filename' will "perform
any filename-specific processing"[1], "like adding a slash to directory
names"[1] and such. What I don't understand is why it is still adding
the slash when completion is based on file names in $dbm_dir instead of
$PWD.

Any ideas?


Thanks,

msi


[1]
https://www.gnu.org/software/bash/manual/html_node/Programmable-Completion-Builtins.html

[Chet Ramey]

Post by Michael Siegel
In that case, the shell will add a slash to the name, as if it were
~/devel/sandbox/probe$ ls
a_file msiism/ tests/
~/devel/sandbox/probe$ dbm msi<Tab> # Results in 'dbm msiism/'
~/devel/sandbox/probe$
Now, the Bash Reference Manual does say that '-o filename' will "perform
any filename-specific processing"[1], "like adding a slash to directory
names"[1] and such. What I don't understand is why it is still adding
the slash when completion is based on file names in $dbm_dir instead of
$PWD.

How would readline (because readline is adding the slash) know that the
possible completion should be interpreted relative to some other directory
and not $PWD? It's not an absolute pathname, the current working directory
hasn't changed, and there's not currently a way to direct readline
otherwise.

Chet

[Michael Siegel]

Post by Chet Ramey

Post by Michael Siegel
In that case, the shell will add a slash to the name, as if it were
~/devel/sandbox/probe$ ls
a_file msiism/ tests/
~/devel/sandbox/probe$ dbm msi<Tab> # Results in 'dbm msiism/'
~/devel/sandbox/probe$
Now, the Bash Reference Manual does say that '-o filename' will "perform
any filename-specific processing"[1], "like adding a slash to directory
names"[1] and such. What I don't understand is why it is still adding
the slash when completion is based on file names in $dbm_dir instead of
$PWD.

How would readline (because readline is adding the slash) know that the
possible completion should be interpreted relative to some other directory
and not $PWD? It's not an absolute pathname, the current working directory
hasn't changed, and there's not currently a way to direct readline
otherwise.

My understanding was that specifying a list of possible completions
through 'compgen' would make the shell ignore $PWD for completions,
unless you use 'complete' with the 'default' option later.

The thing that appears odd to me here is: I'm telling the shell to use
only the file names that exist in $dbm_dir as possible completions for
arguments to 'dbm', but it's still looking for matches in $PWD.

Seems like I'm missing something about how these things work together.

[Chet Ramey]

Post by Michael Siegel

Post by Chet Ramey

Post by Michael Siegel
In that case, the shell will add a slash to the name, as if it were
~/devel/sandbox/probe$ ls
a_file msiism/ tests/
~/devel/sandbox/probe$ dbm msi<Tab> # Results in 'dbm msiism/'
~/devel/sandbox/probe$
Now, the Bash Reference Manual does say that '-o filename' will "perform
any filename-specific processing"[1], "like adding a slash to directory
names"[1] and such. What I don't understand is why it is still adding
the slash when completion is based on file names in $dbm_dir instead of
$PWD.

How would readline (because readline is adding the slash) know that the
possible completion should be interpreted relative to some other directory
and not $PWD? It's not an absolute pathname, the current working directory
hasn't changed, and there's not currently a way to direct readline
otherwise.

My understanding was that specifying a list of possible completions
through 'compgen' would make the shell ignore $PWD for completions,
unless you use 'complete' with the 'default' option later.

OK, that's an assumption. It just can't be verified using documentation or
evidence.

Post by Michael Siegel
The thing that appears odd to me here is: I'm telling the shell to use
only the file names that exist in $dbm_dir as possible completions for
arguments to 'dbm', but it's still looking for matches in $PWD.

How is the shell supposed to know that? You call compgen in a command
substitution subshell, which would isolate any directory change you
did (if you did one), and you take great care to cut off the path to
$dbm_dir in the returned words by piping the words through `cut'.

Post by Michael Siegel
Seems like I'm missing something about how these things work together.

Look at it this way. You return a list of words from your completion
function and tell readline to treat them as filenames. These filenames
are handled in the standard way: if they're not absolute, they are
relative to the current directory. These words are all relative pathnames,
since your completion function cuts $dbm_dir off the front, so when these
relative pathnames are inspected using stat(), the attributes returned are
going to depend on whether or not there is a file with that name in the
current directory.

Chet

[Michael Siegel]

Post by Chet Ramey

Post by Michael Siegel
Seems like I'm missing something about how these things work together.

Look at it this way. You return a list of words from your completion
function and tell readline to treat them as filenames. These filenames
are handled in the standard way: if they're not absolute, they are
relative to the current directory. These words are all relative pathnames,
since your completion function cuts $dbm_dir off the front, so when these
relative pathnames are inspected using stat(), the attributes returned are
going to depend on whether or not there is a file with that name in the
current directory.

Ok, thanks for clarifying that. So, I cannot rely on '-o filenames' to
do the escaping of shell special characters in the file names I put into
COMPREPLY, but will have to implement that myself. What would be the
best way to do this? I was thinking of piping the output of 'compgen' to
'sed' (after 'cut').

Apropos putting file names into COMPREPLY: After some further discussion
in #bash on Freenode, I have now sanitized the way this is done:

while IFS=$'\n' read -r line
do
COMPREPLY+=("$line")
done < <(compgen -f -- "$dbm_dir/${COMP_WORDS[COMP_CWORD]}" | \
cut -d '/' -f 5)

The reason I didn't use 'mapfile' is that I wanted to keep the script
compatible with Bash 3.x.


msi

[Chet Ramey]

Post by Michael Siegel

Post by Chet Ramey

Post by Michael Siegel
Seems like I'm missing something about how these things work together.

Look at it this way. You return a list of words from your completion
function and tell readline to treat them as filenames. These filenames
are handled in the standard way: if they're not absolute, they are
relative to the current directory. These words are all relative pathnames,
since your completion function cuts $dbm_dir off the front, so when these
relative pathnames are inspected using stat(), the attributes returned are
going to depend on whether or not there is a file with that name in the
current directory.

Ok, thanks for clarifying that. So, I cannot rely on '-o filenames' to
do the escaping of shell special characters in the file names I put into
COMPREPLY, but will have to implement that myself. What would be the
best way to do this? I was thinking of piping the output of 'compgen' to
'sed' (after 'cut').

If you want to prevent readline from quoting filenames that appear in
the current directory by removing `-o filenames', you can use one of
the shell facilities that quotes arguments to add the quotes back: printf's

Post by Michael Siegel
Apropos putting file names into COMPREPLY: After some further discussion
while IFS=$'\n' read -r line
do
COMPREPLY+=("$line")
done < <(compgen -f -- "$dbm_dir/${COMP_WORDS[COMP_CWORD]}" | \
cut -d '/' -f 5)

You could avoid the pipe by using something like
"${COMPREPLY[@]#$dbm_dir/}" to remove the leading directory name.

[Michael Siegel]

Post by Chet Ramey
If you want to prevent readline from quoting filenames that appear in
the current directory by removing `-o filenames', you can use one of
the shell facilities that quotes arguments to add the quotes back: printf's

Post by Michael Siegel
Apropos putting file names into COMPREPLY: After some further discussion
while IFS=$'\n' read -r line
do
COMPREPLY+=("$line")
done < <(compgen -f -- "$dbm_dir/${COMP_WORDS[COMP_CWORD]}" | \
cut -d '/' -f 5)

You could avoid the pipe by using something like

Following both of your suggestions, I've now created a version that
should be good to go. I've had it tested by shellcheck, which didn't
find any reason to complain.

Here's the script:

_dbm_complete_bash() {

local dbm_dir="$HOME/.config/dbm"

# Only perform bookmark name completion on the first argument to dbm,
# except when that first argument is -d (the delete switch). In that
# case, perform bookmark name completion on any given argument.
if [ "${#COMP_WORDS[@]}" -lt 3 ] || [ "${COMP_WORDS[1]}" = '-d' ]
then
# Put the names of all files found in $dbm_dir into the COMPREPLY
# array and omit their directory prefix.
# Use IFS=$'\n' for read, as compgen cannot generate null-delimited
# output.
# Use printf '%q' to have shell special characters escaped properly.
while IFS=$'\n' read -r line
do
COMPREPLY+=("$(printf '%q' "${line#"$dbm_dir/"}")")
done < <(compgen -f -- "$dbm_dir/${COMP_WORDS[COMP_CWORD]}")
fi
}

complete -F _dbm_complete_bash dbm