[Help-bash] Where the complete behavior of & is documented?

[Peng Yu]

Hi,

I have the following testing script.

$ cat main1.sh
#!/usr/bin/env bash
# vim: set noexpandtab tabstop=2:

set -v
echo "$BASHPID"
awk -e 'BEGIN { for(i=0;;i++) { system("sleep 5"); print i } }' &
pid=$!
echo "$pid"
wait

When I run it, I see
$ ./main1.sh
echo "$BASHPID"
49839
awk -e 'BEGIN { for(i=0;;i++) { system("sleep 5"); print i } }' &
pid=$!
echo "$pid"
49840
wait
0
1
2
...

When I check the processes, I see

$ ps -f | grep 49839 |head -n 2
504 49839 48607 0 2:50PM ttys021 0:00.04 bash ./main1.sh
504 49840 49839 0 2:50PM ttys021 0:00.02 awk -e BEGIN {
for(i=0;;i++) { system("sleep 5"); print i } }

Now, once I type ^C at the terminal running ./main1.sh, I don't find
the process 49839 anymore.

Running `jobs` in this terminal, show no jobs running in the
background despite 49840 is still running. Note that the pid of the
parent of 49840 becomes 1.

$ ps -f | grep 49840 | head -n 2
504 49840 1 0 2:50PM ttys021 0:00.03 awk -e BEGIN {
for(i=0;;i++) { system("sleep 5"); print i } }
504 50005 49840 0 2:52PM ttys021 0:00.01 sleep 5

Then, when I type `exit` in the terminal used to run ./main1.sh, the
awk process 49840 will be terminated as well.

This means that somehow, bash understand the difference between the
processing running main1.sh and the terminal process from which
main1.sh is called.

But how does bash know to kill awk upon exit, since the parent pid of
the awk process is 1. It sounds like bash shouldn't know it. So this
is confusing to me.

I checked the section of "Lists" and "JOB CONTROL". I don't where this
is clearly document.

Does anybody know where this is documented in bash manpage?

[John McKown]

I'm not an expert, but I think what you are seeing is because of the
"process group" concept in UNIX.

http://www.informit.com/articles/article.aspx?p=397655&seqNum=6

Post by Peng Yu
Hi,
I have the following testing script.
$ cat main1.sh
#!/usr/bin/env bash
set -v
echo "$BASHPID"
awk -e 'BEGIN { for(i=0;;i++) { system("sleep 5"); print i } }' &
pid=$!
echo "$pid"
wait
When I run it, I see
$ ./main1.sh
echo "$BASHPID"
49839
awk -e 'BEGIN { for(i=0;;i++) { system("sleep 5"); print i } }' &
pid=$!
echo "$pid"
49840
wait
0
1
2
...
When I check the processes, I see
$ ps -f | grep 49839 |head -n 2
504 49839 48607 0 2:50PM ttys021 0:00.04 bash ./main1.sh
504 49840 49839 0 2:50PM ttys021 0:00.02 awk -e BEGIN {
for(i=0;;i++) { system("sleep 5"); print i } }
Now, once I type ^C at the terminal running ./main1.sh, I don't find
the process 49839 anymore.
Running `jobs` in this terminal, show no jobs running in the
background despite 49840 is still running. Note that the pid of the
parent of 49840 becomes 1.
$ ps -f | grep 49840 | head -n 2
504 49840 1 0 2:50PM ttys021 0:00.03 awk -e BEGIN {
for(i=0;;i++) { system("sleep 5"); print i } }
504 50005 49840 0 2:52PM ttys021 0:00.01 sleep 5
Then, when I type `exit` in the terminal used to run ./main1.sh, the
awk process 49840 will be terminated as well.
This means that somehow, bash understand the difference between the
processing running main1.sh and the terminal process from which
main1.sh is called.
But how does bash know to kill awk upon exit, since the parent pid of
the awk process is 1. It sounds like bash shouldn't know it. So this
is confusing to me.
I checked the section of "Lists" and "JOB CONTROL". I don't where this
is clearly document.
Does anybody know where this is documented in bash manpage?
--
Regards,
Peng

[Chet Ramey]

Post by Peng Yu
When I check the processes, I see
$ ps -f | grep 49839 |head -n 2
504 49839 48607 0 2:50PM ttys021 0:00.04 bash ./main1.sh
504 49840 49839 0 2:50PM ttys021 0:00.02 awk -e BEGIN {
for(i=0;;i++) { system("sleep 5"); print i } }
Now, once I type ^C at the terminal running ./main1.sh, I don't find
the process 49839 anymore.
Running `jobs` in this terminal, show no jobs running in the
background despite 49840 is still running. Note that the pid of the
parent of 49840 becomes 1.
$ ps -f | grep 49840 | head -n 2
504 49840 1 0 2:50PM ttys021 0:00.03 awk -e BEGIN {
for(i=0;;i++) { system("sleep 5"); print i } }
504 50005 49840 0 2:52PM ttys021 0:00.01 sleep 5
Then, when I type `exit` in the terminal used to run ./main1.sh, the
awk process 49840 will be terminated as well.
This means that somehow, bash understand the difference between the
processing running main1.sh and the terminal process from which
main1.sh is called.
But how does bash know to kill awk upon exit, since the parent pid of
the awk process is 1. It sounds like bash shouldn't know it. So this
is confusing to me.

This is the premise that results in your faulty conclusion.

When you ^C the script, that kills the script. But since the awk process
was started in the background when job control was not active, it ignores
SIGINT, as documented.

The script was started from an interactive shell with job control enabled,
and the script does not have job control enabled, so all of the processes
started from the script, including the awk command, are in the same process
group, and the process interpreting the script is the process group leader.

When that process gets killed, not only does the awk command get orphaned
and reparented to init (process 1), it becomes the only member of the
process group. The process still has a parent in another process group that
can conceivably control it, so the process group itself is not orphaned,
though the awk process is. Orphaned process groups are the key.

All processes that share the same controlling terminal are members of the
same `session'. In your case, the controlling terminal is the terminal
emulator (xterm or whatever) you ran the script from -- it's what you talk
to when you open /dev/tty. The interactive shell (or the xterm process
itself, it doesn't make a practical difference) is called the `controlling
process' or `session leader'.

So the awk process is orphaned. But the process group?

The session leader (the interactive shell) exits, and the terminal emulator
process exits in turn. When the session leader exits, the awk process
becomes a member of an orphaned process group: it has lost its controlling
terminal, its parent is not in the same session, and there's no living
process that can control it (read the Posix standard for the definition of
an orphaned process group).

The kernel cleans up the members of the orphaned process group by sending
them SIGHUP. That's how the awk process gets killed.

This is all fundamental to the Posix process model. It's not appropriate to
spell it all out in the bash manual.

[Peng Yu]

Hi Chet,

Post by Chet Ramey
This is all fundamental to the Posix process model. It's not appropriate to
spell it all out in the bash manual.

I am trying to learn linux process manangement in more detail. Do you
have any good references that you would recommend?

What about this one? It is eight years old. Is there anything better
and more current than it?

https://www.win.tue.nl/~aeb/linux/lk/lk-10.html

[Andy Chu]

Try APUE by Stevens. It has a lot of sample code and talks about platform
differences.

Linux is basically a superset of POSIX -- it's better to learn POSIX
first. (bash of course runs on many non-Linux kernels.)

https://www.amazon.com/Advanced-Programming-UNIX-Environment-3rd/dp/0321637739

Andy

Post by Peng Yu
Hi Chet,

Post by Chet Ramey
This is all fundamental to the Posix process model. It's not appropriate

to

Post by Chet Ramey
spell it all out in the bash manual.

I am trying to learn linux process manangement in more detail. Do you
have any good references that you would recommend?
What about this one? It is eight years old. Is there anything better
and more current than it?
https://www.win.tue.nl/~aeb/linux/lk/lk-10.html
--
Regards,
Peng

[Bob Proulx]

Post by Andy Chu
Try APUE by Stevens. It has a lot of sample code and talks about platform
differences.
Linux is basically a superset of POSIX -- it's better to learn POSIX
first. (bash of course runs on many non-Linux kernels.)

I wholeheartedly second this recommendation! That book is truly an
excellent reference. Arguably the single most comprehensive reference
of "everything" that is available.

My interaction with that book is with the older editions as Richard
Stevens left us way too young. Recent editions have been updated by
Stephen Rago and I dearly need to update my battered copy with a the
current edition. I assume however that the book continues its very
high quality level.

Bob

[Chet Ramey]

Post by Peng Yu
Hi Chet,

Post by Chet Ramey
This is all fundamental to the Posix process model. It's not appropriate to
spell it all out in the bash manual.

I am trying to learn linux process manangement in more detail. Do you
have any good references that you would recommend?

When Dave Lennert was at HP, he wrote a really good paper about how they
had reconciled the process handling and process group differences between
4.2 BSD and System V. This eventually became the basis of the POSIX
process model.

I'm not going to send it to the mailing list, but you can find a copy at
http://web.physics.ucsb.edu/~pcs/apps/bash/job.control.ps.

[Bob Proulx]

Post by Chet Ramey
When Dave Lennert was at HP, he wrote a really good paper about how they
had reconciled the process handling and process group differences between
4.2 BSD and System V. This eventually became the basis of the POSIX
process model.
I'm not going to send it to the mailing list, but you can find a copy at
http://web.physics.ucsb.edu/~pcs/apps/bash/job.control.ps.

Thank you Chet for this pointer!

It was a work that I was personally not previously aware. I am
familiar with his other work such as his seminal paper, How to Write a
UNIX Daemon by Dave Lennert but this one you reference, A System V
Compatible Implementation of 4.2BSD Job Control by David C. Lennert,
had escaped my notice thus far. I have only been able to skim it
briefly but it appears to be typical of the high quality of Dave's
works. I have already learned several things from it.

"A good pilot is always learning." And the same is true of
engineers and programmers too! :-)

Bob