[Help-bash] Evaluations of backticks in if statements

[Ryan Marples]

Hi,

I’ve seen bash statements like the following: if `cmd` …

This appears to run the body of the if when cmd exits successfully. But I don’t understand the mechanics of why. My understanding is that `cmd` (assuming in this case cmd prints nothing to either stdout nor stderr but exits successfully) should evaluate to an empty string, and then you’d be saying if “” … which should be false.

Please enlighten be as to why `cmd` in the context of if `cmd` means “the exit code of cmd” rat

[Greg Wooledge]

I???ve seen bash statements like the following: if `cmd` ???

Probably just crappy code. Most of the bash and sh code you will find
in this world is crap.

Also note that `...` is deprecated in favor of $(...) if you actually
do want a command substitution.

This appears to run the body of the if when cmd exits successfully. But I
don???t understand the mechanics of why. My understanding is that `cmd`
(assuming in this case cmd prints nothing to either stdout nor stderr but
exits successfully) should evaluate to an empty string, and then you???d be
saying if ?????? ??? which should be false.

It would help if we knew what "cmd" really was, and what its output is.

In the simplest case, you have essentially this:

imadev:~$ if $(echo true); then echo yeah; else echo nope; fi
yeah

The command inside $(...) is executed, and then its standard output is
"captured", and this becomes ANOTHER command, and THAT command is
executed, and the exit status of THAT command is what drives the "if".

In short, this code you've found is almost certainly wrong.

Compare:

imadev:~$ if $(echo false); then echo yeah; else echo nope; fi
nope

As you can see here, it's the exit status of the GENERATED command
(in this case, "false") that drives the "if". The exit status of
the substituted command ("echo false") does not matter.

[Andy Chu]

Post by Ryan Marples
Hi,
I’ve seen bash statements like the following: if `cmd` …
This appears to run the body of the if when cmd exits successfully. But I
don’t understand the mechanics of why. My understanding is that `cmd`
(assuming in this case cmd prints nothing to either stdout nor stderr but
exits successfully) should evaluate to an empty string, and then you’d be
saying if “” … which should be false.

Shell doesn't work this way. The syntax is "if command", and

if `command`

is a particular case of that which is not special in any way (and as you
mention it's also not useful). There's no such thing as

if ""

where "" is a string/variable value -- the "" is interpreted a command name.

If you want to test if a value is empty, you can use

if test -n ""

or as a shortcut

if test ""

Or any of these:

[ -n "" ]
[ "" ]
[[ -n "" ]]
[[ "" ]]


is an alias for test and [[ is a special shell construct that behaves
somewhat like test / [.

Andy

[Seth David Schoen]

Post by Andy Chu
Shell doesn't work this way. The syntax is "if command", and
if `command`
is a particular case of that which is not special in any way (and as you
mention it's also not useful). There's no such thing as
if ""
where "" is a string/variable value -- the "" is interpreted a command name.

I agree with this interpretation, but I see behavior that I don't know
how to account for. Presumably the shell should be running the output
of the substituted command and then checking the return code of _that_,
which accounts for

$ if `echo true`; then echo foo; fi
foo
$ if `echo false`; then echo foo; fi
$

And it also accounts for

$ if `echo wow`; then echo foo; fi
wow: command not found

But since neither true(1) nor false(1) outputs anything, there shouldn't
be a difference between

$ if `true`; then echo foo; fi
foo
$ if `false`; then echo foo; fi
$

and indeed both of them should presumably complain about the unknown
command with an empty name -- but neither does, and they exhibit
different behavior from one another. So is there a special case after
all? I can't find a justification for this in the man page's
documentation of "if" or command substitution.

[Greg Wooledge]

Post by Seth David Schoen
$ if `true`; then echo foo; fi
foo
$ if `false`; then echo foo; fi
$

First of all, and I cannot possibly stress this enough, this is shitty
code and you should never EVER write code like this.

Right, so. Apparently bash treats a command substitution that returns an
empty string in some special-snowflake way, rather than as a syntax error.

imadev:~$ `false`
imadev:~$ echo $?
1

imadev:~$ if ; then echo "yes"; else echo "no"; fi
bash: syntax error near unexpected token `;'

imadev:~$ if ""; then echo "yes"; else echo "no"; fi
bash: : command not found
no

imadev:~$ foo=; if $foo; then echo "yes"; else echo "no"; fi
yes


So yes, somehow apparently a command substitution (or unquoted variable
substitution) that returns an empty string is treated differently from
*nothing* (syntax error), or an actual attempt to execute an empty string
(command not found).

I assume there is some POSIX justification for this, blah blah blah.

Point is, it's crap. Stop doing it. Chet has to worry about making
bash handle all of this crap in exactly the right way that POSIX
dictates. You, as a script writer, do not need to worry about this.
Just don't do it. Problem solved.

[Seth David Schoen]

Post by Greg Wooledge
Point is, it's crap. Stop doing it. Chet has to worry about making
bash handle all of this crap in exactly the right way that POSIX
dictates. You, as a script writer, do not need to worry about this.
Just don't do it. Problem solved.

I've never written code like this and don't expect to; I'm just
curious about what the behavior is.

[Chet Ramey]

Post by Greg Wooledge

Post by Seth David Schoen
$ if `true`; then echo foo; fi
foo
$ if `false`; then echo foo; fi
$

First of all, and I cannot possibly stress this enough, this is shitty
code and you should never EVER write code like this.
Right, so. Apparently bash treats a command substitution that returns an
empty string in some special-snowflake way, rather than as a syntax error.

Please. Don't overthink this.

First, why should bash treat a command substitution that returns no output
as anything different from a variable expansion that has no value?

Consider `something`. That's a simple command, with a single word, that
undergoes word expansion. It can either expand to some text, in which
case that text undergoes word splitting and globbing and is executed as a
command, or nothing.

If it returns nothing, it falls into this Posix-standardized and historical
shell behavior category:

"If there is no command name, but the command contained a command
substitution, the command shall complete with the exit status of the last
command substitution performed."

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_01

Virtually the same language appears in the bash manual.

[Eric Blake]

Post by Greg Wooledge
Right, so. Apparently bash treats a command substitution that returns an
empty string in some special-snowflake way, rather than as a syntax error.
imadev:~$ foo=; if $foo; then echo "yes"; else echo "no"; fi
yes

and:

$ set -; if "$@"; then echo yes; fi
yes

Post by Greg Wooledge
So yes, somehow apparently a command substitution (or unquoted variable
substitution) that returns an empty string is treated differently from
*nothing* (syntax error), or an actual attempt to execute an empty string
(command not found).
I assume there is some POSIX justification for this, blah blah blah.

Yes, POSIX requires that if 0 words result after expansion (whether
unquoted variable expansion, quoted "$@" expansion, or command
substitution), then $? is set to 0 as the successful "execution" of that
empty command. For the same reason that:

$ eval ''; echo $?
0
$ . /dev/null; echo $?
0

Post by Greg Wooledge
Point is, it's crap. Stop doing it. Chet has to worry about making
bash handle all of this crap in exactly the right way that POSIX
dictates. You, as a script writer, do not need to worry about this.
Just don't do it. Problem solved.

Agreed :)

[Andy Chu]

Post by Seth David Schoen

Post by Andy Chu
Shell doesn't work this way. The syntax is "if command", and
if `command`
is a particular case of that which is not special in any way (and as you
mention it's also not useful). There's no such thing as
if ""
where "" is a string/variable value -- the "" is interpreted a command

name.
I agree with this interpretation, but I see behavior that I don't know
how to account for. Presumably the shell should be running the output
of the substituted command and then checking the return code of _that_,
which accounts for
$ if `echo true`; then echo foo; fi
foo
$ if `echo false`; then echo foo; fi
$
And it also accounts for
$ if `echo wow`; then echo foo; fi
wow: command not found
But since neither true(1) nor false(1) outputs anything, there shouldn't
be a difference between
$ if `true`; then echo foo; fi
foo
$ if `false`; then echo foo; fi
$
and indeed both of them should presumably complain about the unknown
command with an empty name -- but neither does, and they exhibit
different behavior from one another. So is there a special case after
all? I can't find a justification for this in the man page's
documentation of "if" or command substitution.

Yes, this is surprising to me:

$ if ''; then echo TRUE; else echo FALSE; fi
: command not found
FALSE

$ if `true`; then echo TRUE; else echo FALSE; fi
TRUE

$ if `false`; then echo TRUE; else echo FALSE; fi
FALSE

So basically the exit code of the subshell command IS checked, and it's not
just treated as a word. There is some special interaction between if and
subshell.

Moreover I tested bash, dash, zsh, and mksh, and they all exhibit this
behavior! But I don't see it in POSIX.

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html

Digging through the source should probably reveal this as a special case.

I have a test framework running against all shells and mine is the only one
that's "wrong" here!

Andy

[Andy Chu]

Hm I guess it's not so special. I will have to think about this a bit:

$ '' && echo YES
: command not found

$ `true` && echo YES
YES


$ `false` && echo YES
<no output, doesn't try to execute empty command>

-------------

$ ''; echo $?
: command not found
127

$ `true`; echo $?
0

$ `false`; echo $?
1

Post by Andy Chu

Post by Seth David Schoen

Post by Andy Chu
Shell doesn't work this way. The syntax is "if command", and
if `command`
is a particular case of that which is not special in any way (and as you
mention it's also not useful). There's no such thing as
if ""
where "" is a string/variable value -- the "" is interpreted a command

name.
I agree with this interpretation, but I see behavior that I don't know
how to account for. Presumably the shell should be running the output
of the substituted command and then checking the return code of _that_,
which accounts for
$ if `echo true`; then echo foo; fi
foo
$ if `echo false`; then echo foo; fi
$
And it also accounts for
$ if `echo wow`; then echo foo; fi
wow: command not found
But since neither true(1) nor false(1) outputs anything, there shouldn't
be a difference between
$ if `true`; then echo foo; fi
foo
$ if `false`; then echo foo; fi
$
and indeed both of them should presumably complain about the unknown
command with an empty name -- but neither does, and they exhibit
different behavior from one another. So is there a special case after
all? I can't find a justification for this in the man page's
documentation of "if" or command substitution.

$ if ''; then echo TRUE; else echo FALSE; fi
: command not found
FALSE
$ if `true`; then echo TRUE; else echo FALSE; fi
TRUE
$ if `false`; then echo TRUE; else echo FALSE; fi
FALSE
So basically the exit code of the subshell command IS checked, and it's
not just treated as a word. There is some special interaction between if
and subshell.
Moreover I tested bash, dash, zsh, and mksh, and they all exhibit this
behavior! But I don't see it in POSIX.
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html
Digging through the source should probably reveal this as a special case.
I have a test framework running against all shells and mine is the only
one that's "wrong" here!
Andy

[Andy Chu]

OK, there is no special case. This is because of what I think of as "empty
unquoted word elision" (not sure if there is a better name).

`true` is not quoted so it doesn't result in an empty string in argv, it
results in NOTHING in argv (empty argv).

Compare:

$ if `true`; then echo TRUE; else echo FALSE; fi
TRUE

$ if "`true`"; then echo TRUE; else echo FALSE; fi
: command not found
FALSE

# passing argument -- ZZZ is the command name, not empty string
$ if `true` ZZZ; then echo TRUE; else echo FALSE; fi
ZZZ: command not found
FALSE

# now it has output, but this isn't a special case either
$ if `sh -c 'echo YYY; true'`; then echo TRUE; else echo FALSE; fi
YYY: command not found
FALSE

This is just like how variables are elided if empty and quoted:

$ empty=; python -c 'import sys; print sys.argv[1:]' a $empty b
['a', 'b']

$ empty=; python -c 'import sys; print sys.argv[1:]' a "$empty" b
['a', '', 'b']


Andy

[Andy Chu]

Sorry for so many messages, but I remembered that this relates to a curious
idiom I found while parsing bash scripts "in the wild" (example:
http://www.oilshell.org/blog/2016/11/09.html ):

In this script:

https://github.com/sandstorm-io/sandstorm/blob/master/install.sh

The `# comment` hack takes advantage of the fact that empty unquoted words
are elided. If you quoted them then your argv array would have '' all over
the place and the command wouldn't work.

And comments aren't a lexical construct; they can appear INSIDE a
subshell. Unlike in other languages, a comment is more naturally handled
at the parsing level rather than the lexing level, because say

$ echo foo#bar
foo#bar

doesn't require quotes.



# Generate key for client certificate. OpenSSL will read from
# /dev/urandom by default, so this won't block. We abuse the ``
# operator so we can have inline comments in a multi-line command.
openssl \
req `# Invoke OpenSSL's PKCS#10 X.509 bits.` \
-new `# Create a new certificate/request.` \
-newkey rsa:4096 `# Create a new RSA key of length 4096 bits.` \
-days 3650 `# Make the self-signed cert valid for 10 years.` \
-nodes `# no DES -- that is, do not encrypt the key at rest.` \
-x509 `# Output a certificate, rather than a signing request.` \
`# Sandcats ignores the subject in the certificate; use` \
`# OpenSSL defaults.` \
-subj "/C=AU/ST=Some-State/O=Internet Widgits Pty Ltd" \
-keyout var/sandcats/id_rsa `# Store the resulting RSA private key in id_rsa
` \
-out var/sandcats/id_rsa.pub `# Store the resulting certificate in
id_rsa.pub` \
2>/dev/null `# Silence the progress output.`

Post by Andy Chu
OK, there is no special case. This is because of what I think of as
"empty unquoted word elision" (not sure if there is a better name).
`true` is not quoted so it doesn't result in an empty string in argv, it
results in NOTHING in argv (empty argv).
$ if `true`; then echo TRUE; else echo FALSE; fi
TRUE
$ if "`true`"; then echo TRUE; else echo FALSE; fi
: command not found
FALSE
# passing argument -- ZZZ is the command name, not empty string
$ if `true` ZZZ; then echo TRUE; else echo FALSE; fi
ZZZ: command not found
FALSE
# now it has output, but this isn't a special case either
$ if `sh -c 'echo YYY; true'`; then echo TRUE; else echo FALSE; fi
YYY: command not found
FALSE
$ empty=; python -c 'import sys; print sys.argv[1:]' a $empty b
['a', 'b']
$ empty=; python -c 'import sys; print sys.argv[1:]' a "$empty" b
['a', '', 'b']
Andy

[Andy Chu]

Sorry for the long delay, was pulled off onto other things. Andy – I’m not
100% clear what your conclusion was. Taking Seth’s example earlier in the

Post by Seth David Schoen
$ if `true`; then echo foo; fi
foo
$ if `false`; then echo foo; fi
$

Can you walk me through the explanation of the result of these two command
variations please?

Basically `true` and `false` evaluate to an argv of [] (length 0), in
contrast to argv of [""] (length 1).

It's impossible exec a command of zero arguments, so it just skips it
and uses the last value of $?, which happens to be the one from inside
the subshell. I'm not sure if this is really a special case or not --
it seems to fall out of the fact that $? is a variable mutated at
every command step in the interpreter loop.

But Chet Ramey pointed out the explicit rule in POSIX:

"If there is no command name, but the command contained a command
substitution, the command shall complete with the exit status of the last
command substitution performed."


If you are confused why `true` and `false` evaluate to [] and not
[""], define a function like this:

argv() { python -c 'import sys; print sys.argv[1:]' "$@"; }

And then note all these differences. Empty words are elided from the
argv array unless quoted.

$ argv a $empty b
['a', 'b']

$ argv a `true` b
['a', 'b']

$ argv a $(true) b
['a', 'b']


$ argv a "$empty" b
['a', '', 'b']

$ argv a "`true`" b
['a', '', 'b']

$ argv a "$(true)" b
['a', '', 'b']


Andy