[Help-bash] Undocumented caret logical operator?

[G. Branden Robinson]

Can someone tell me what is going on here? This operator is not
documented in SUSv4 nor in the Bash man page, at least not per my
full-text search. The experiments show that it is not acting as an
"undocumented synonym for |" (bashref re: SVR4.2's Bourne shell), and
the behavior does not change from a script or with set +o history, so
it's not a null version of the ^foo^bar history operator. We can also
rule out arithmetic expansion (bitwise xor) and glob expansion
(character class inversion), as we're in the wrong lexical context.

dash and ksh93 behave identically. What am I missing? What use cases
does it serve?

echo -n expect TRUE:
true ^ true && echo TRUE || echo FALSE
echo -n expect TRUE:
true ^ false && echo TRUE || echo FALSE
echo -n expect FALSE:
false ^ true && echo TRUE || echo FALSE
echo -n expect FALSE:
false ^ false && echo TRUE || echo FALSE

echo -n expect no output and no diagnostic:
false ^ echo OUTPUT
echo

echo -n expect OUTPUT and no diagnostic:
false ^ nonexistent-command | echo OUTPUT

[Eric Blake]

Post by G. Branden Robinson
Can someone tell me what is going on here? This operator is not
documented in SUSv4 nor in the Bash man page, at least not per my
full-text search. The experiments show that it is not acting as an
"undocumented synonym for |" (bashref re: SVR4.2's Bourne shell),

Correct, because it is NOT an operator any more. (only super-old sh,
not POSIX compatible, treated it as an operator)

Post by G. Branden Robinson
and
the behavior does not change from a script or with set +o history, so
it's not a null version of the ^foo^bar history operator. We can also
rule out arithmetic expansion (bitwise xor) and glob expansion
(character class inversion), as we're in the wrong lexical context.
dash and ksh93 behave identically. What am I missing? What use cases
does it serve?
true ^ true && echo TRUE || echo FALSE

This executes the command 'true ^ true'; then, if that command had
status 0, it executes 'echo TRUE'; then, if that && group had status 1,
it executes 'echo FALSE'.

The command 'true ^ true' has status 0 ('true' always ignores all of its
arguments), the same way that 'true gibberish' has status 0.

If you want to see a bit more into what's going on:

$ (set -x; true ^ true && echo TRUE || echo FALSE)
+ true '^' true
+ echo TRUE
TRUE

Post by G. Branden Robinson
false ^ nonexistent-command | echo OUTPUT

And this says to run the command 'false ^ nonexistent-command' (as with
true, false ignores argv[1]=="^" and argv[2]=="nonexistent-command" to
produce no output), and pipes the result to the input of 'echo OUTPUT'
(which ignores input and produces its output unconditionally).

It does not attempt to run nonexistent-command, because ^ is not an
operator.

[G. Branden Robinson]

Post by G. Branden Robinson
Can someone tell me what is going on here? This operator is not
documented in SUSv4 nor in the Bash man page, at least not per my
full-text search. The experiments show that it is not acting as an
"undocumented synonym for |" (bashref re: SVR4.2's Bourne shell),

Correct, because it is NOT an operator any more. (only super-old sh, not
POSIX compatible, treated it as an operator)

[...]

Hah! Of course. If I override my instincts and parse it as an ordinary
uninterpreted argument, everything makes sense. Replace "^" with "ARG"
in all the examples and the behavior is, understandably, identical.

Here's why I asked. There's a piece of OpenSSH that seems to believe ^
is, or might be, an operator. It's down in the weeds of some
shell-detection madness, so I'm not sure if they're on crack or not.

https://pastebin.com/df7jYEib

Thanks, Eric!

[Greg Wooledge]

Post by G. Branden Robinson
Here's why I asked. There's a piece of OpenSSH that seems to believe ^
is, or might be, an operator. It's down in the weeds of some
shell-detection madness, so I'm not sure if they're on crack or not.
https://pastebin.com/df7jYEib

If they're accounting for the possibility that the shell is a Bourne
shell, then yes, ^ could be a pipe operator. It is completely possible
(although growing increasingly unlikely over time) that someone could
ssh into an account on a system where the login shell is a Bourne shell.

[G. Branden Robinson]

Post by Greg Wooledge

Post by G. Branden Robinson
Here's why I asked. There's a piece of OpenSSH that seems to believe ^
is, or might be, an operator. It's down in the weeds of some
shell-detection madness, so I'm not sure if they're on crack or not.
https://pastebin.com/df7jYEib

If they're accounting for the possibility that the shell is a Bourne
shell, then yes, ^ could be a pipe operator. It is completely possible
(although growing increasingly unlikely over time) that someone could
ssh into an account on a system where the login shell is a Bourne shell.

Looking at it again, "false | printf" is (probably, without having an
SVR4.2 shell handy to test on) a good test for this functionality.

But the friend who brought this to my attention and I agree that this
whole goose chase could have been avoided if the OpenSSH guys had
written comments and diagnostics that were actually meaningful, instead
of slouchy editorializing.

[Eric Blake]

Post by G. Branden Robinson
Here's why I asked. There's a piece of OpenSSH that seems to believe ^
is, or might be, an operator. It's down in the weeds of some
shell-detection madness, so I'm not sure if they're on crack or not.
https://pastebin.com/df7jYEib

(Pastebins expire, but the mailing list archives do not; copying some of
that example here)

# check that we have something mildly sane as our shell, or try to find
something better
if false ^ printf "%s: WARNING: ancient shell, hunting for a more modern
one... " "$0"
then

Yes, on all modern systems (and any POSIXy shell), this executes the
single command 'false' with ignored arguments, and skips the 'then'
clause. Meanwhile, on old Solaris systems, where /bin/sh is Bourne
shell rather than POSIX sh and has a ^ operator, this is parsed the same
as 'false | printf ...', which executes the printf command (which may or
may not have been a builtin in that old shell, and may or may not be on
your PATH); it is HOPING that the exit status of printf will be 0, and
that the WARNING message was printed, and that the 'then' clause then
gets a chance to try re-execing under a saner shell.

But since printf is a relatively modern shell invention, I'm seriously
wondering if a system old enough to have /bin/sh that understands ^ as a
pipe operator would also lack printf, and thus break this snippet.

[Greg Wooledge]

Post by Eric Blake
# check that we have something mildly sane as our shell, or try to find
something better
if false ^ printf "%s: WARNING: ancient shell, hunting for a more modern
one... " "$0"
then
Yes, on all modern systems (and any POSIXy shell), this executes the single
command 'false' with ignored arguments, and skips the 'then' clause.
Meanwhile, on old Solaris systems, where /bin/sh is Bourne shell rather than
POSIX sh and has a ^ operator, this is parsed the same as 'false | printf
...', which executes the printf command (which may or may not have been a
builtin in that old shell, and may or may not be on your PATH); it is HOPING
that the exit status of printf will be 0, and that the WARNING message was
printed, and that the 'then' clause then gets a chance to try re-execing
under a saner shell.
But since printf is a relatively modern shell invention, I'm seriously
wondering if a system old enough to have /bin/sh that understands ^ as a
pipe operator would also lack printf, and thus break this snippet.

It also breaks if printf can't write to stdout for any reason, e.g. if
some fool closed file descriptor 1. (I've seen people do this, because
they think >&- is a "shorter" or "more elegant" form of >/dev/null --
it may be shorter, but it's not equivalent!) Or if stdout has been
redirected to a file on a file system that has run out of space, etc.

[Bob Proulx]

Post by Eric Blake
But since printf is a relatively modern shell invention, I'm seriously
wondering if a system old enough to have /bin/sh that understands ^ as a
pipe operator would also lack printf, and thus break this snippet.

True and agreed. However when I administered HP-UX I always installed
printf on the older systems which were ones that also happened to have
a Bourne shell. Therefore while non-standard it isn't inconceivable
that a /bin/sh Bourne shell system might have /usr/local/bin/printf
available.

The test feels like it is *almost* a good test for it if the printf
were replaced with an appropriate echo command.

Bob

[Greg Wooledge]

Post by G. Branden Robinson
The experiments show that it is not acting as an
"undocumented synonym for |" (bashref re: SVR4.2's Bourne shell)
true ^ false && echo TRUE || echo FALSE

Outside of legacy Bourne shells, the ^ is just a character with no
special meaning, like x or % or 8. Here (above), you're passing two
extra arguments to the "true" command, which simply ignores them.

Here (below), you're passing two extra arguments to "false", which
also ignores them.

Post by G. Branden Robinson
false ^ true && echo TRUE || echo FALSE
false ^ false && echo TRUE || echo FALSE

It's also a really bad idea to chain && and || in the same command
like this. It is NOT a synonym for "if ... then ... else ... fi",
as I explain at <https://mywiki.wooledge.org/BashPitfalls#pf22>.

[G. Branden Robinson]

Post by Greg Wooledge

Post by G. Branden Robinson
false ^ true && echo TRUE || echo FALSE
false ^ false && echo TRUE || echo FALSE

It's also a really bad idea to chain && and || in the same command
like this. It is NOT a synonym for "if ... then ... else ... fi",
as I explain at <https://mywiki.wooledge.org/BashPitfalls#pf22>.

Yes, that was sloppy of me. I was lashing together the test cases in a
hurry. Meaning it's an even worse habit, as it was an unthinking one!

[Eric Blake]

Post by Greg Wooledge

Post by G. Branden Robinson
The experiments show that it is not acting as an
"undocumented synonym for |" (bashref re: SVR4.2's Bourne shell)
true ^ false && echo TRUE || echo FALSE

Outside of legacy Bourne shells, the ^ is just a character with no
special meaning, like x or % or 8. Here (above), you're passing two
extra arguments to the "true" command, which simply ignores them.

And for more historical trivia, the reason POSIX standardized:

case a in [!b]) echo yes;; esac

and NOT

case a in [^b]) echo yes;; esac

(although the latter is permitted as an extension, and bash implements
that extension), is BECAUSE of the legacy shells where ^ was an operator
and had to be quoted, such that quoting made it harder to use in negated
group globs (when compared to the usual spelling of negated groups in
regular expressions), while ! did not need quoting.

On a distant tangent, Microsoft's cmd still uses ^-newline as a line
continuation escape (although cmd and sh are so radically different that
it's unusual to even attempt to write a file that would perform in a
sane manner under both shells).