[Help-bash] Fwd: Proper use of nohup within here-document?

[Mun]

Moving to help-bash ; my apologies for sending the following to bash-bug.

Hi,

I'm using Bash version 4.1.2 on RedHat EL 6.8 (I realize these are old
releases).

In one of my bash scripts I have a here-document and at the end of
it's execution it will pop-up an gxmessage. My problem is that when
the here-document exits, the gxmessage closes.

Within the here-document I'm essentially calling gxmessage thusly:
$ nohup gxmessage -timeout 0 -file abc.txt > /dev/null 2>&1 &

Any ideas on how I can allow the here-document to exit but allow the
gxmessage process to live on?

Thanks,

--
Mun

[Greg Wooledge]

Post by Mun
$ nohup gxmessage -timeout 0 -file abc.txt > /dev/null 2>&1 &
Any ideas on how I can allow the here-document to exit but allow the
gxmessage process to live on?

I think you will need to show us this alleged "here-document".
Here-documents don't normally execute commands, and certainly not using
that syntax.

This is a here-document:


cat <<'xxx'
usage: myscript [-abqz] weight beauty

Specify weight in decapounds, and beauty in microhelens.
xxx


The only way you can normally execute a command during the reading of
a here-document would be with a command substitution, using an unquoted
here-document sentinel word.

[Mun]

Hi Greg,

My here-document looks something like this:

========================================
cat >myScript <<@EOF
#! /bin/bash

date > showMe.txt
nohup gxmessage -timeout 0 -file showMe.txt > /dev/null 2>&1 &

@EOF

chmod 777 myScript
qsub -b yes -sync n ./myScript
========================================

Note the last line with the qsub command. The whole purpose of the
here-document is to package up a set of commands into a script form
that will be dispatched to another server for execution using a "grid"
tool (aka load sharing facility).

The commands run fine except I can't figure out a way for gxmessage to
not die when myScript exits.

Thanks,
--
Mun

Post by Greg Wooledge

Post by Mun
$ nohup gxmessage -timeout 0 -file abc.txt > /dev/null 2>&1 &
Any ideas on how I can allow the here-document to exit but allow the
gxmessage process to live on?

I think you will need to show us this alleged "here-document".
Here-documents don't normally execute commands, and certainly not using
that syntax.
cat <<'xxx'
usage: myscript [-abqz] weight beauty
Specify weight in decapounds, and beauty in microhelens.
xxx
The only way you can normally execute a command during the reading of
a here-document would be with a command substitution, using an unquoted
here-document sentinel word.

[Greg Wooledge]

Post by Mun
========================================
#! /bin/bash
date > showMe.txt
nohup gxmessage -timeout 0 -file showMe.txt > /dev/null 2>&1 &
@EOF
chmod 777 myScript
qsub -b yes -sync n ./myScript
========================================

So it's a script that writes a second script, and then executes the
second script. There is no execution of background commands during the
here-document processing at all. The background process is launched
by the second script.

P.S. 777 permissions are just wrong. 755 would be much more reasonable.

Post by Mun
Note the last line with the qsub command. The whole purpose of the
here-document is to package up a set of commands into a script form
that will be dispatched to another server for execution using a "grid"
tool (aka load sharing facility).
The commands run fine except I can't figure out a way for gxmessage to
not die when myScript exits.

So the REAL question is how to run a long-running background command on
a remote system using this "grid" thing, which I'm going to assume is
something like ssh.

It looks like you've already redirected stdout and stderr, which is
the normal problem that people encounter when trying to run long-lived
background processes over ssh.

So, since I don't know anything about "gxmessage" or "grid", I don't know
that there's anything more I can do for you. Sorry. Maybe someone else
can take a stab at it.

Random guesswork:

(One possibility is that "grid" doesn't simply run a program, but
instead acts in coordination with some kind of remote overseer/supervisor
process, and this supervisor kills everything when "grid" disconnects,
in which case you are actively trying to subvert the design of the
system you're using.)

(Another is that gxmessage requires a terminal, and dies instantly
when executed by "grid" which I am assuming does not allocate a
pseudoterminal by default.)

(I'm sure there are a billion other random things that I just can't
think of at the moment.)

(Is there a support mailing list for "grid" and/or "gxmessage" that
might actually know such things?)

[Mun]

Hi Greg,

Post by Greg Wooledge

Post by Mun
========================================
#! /bin/bash
date > showMe.txt
nohup gxmessage -timeout 0 -file showMe.txt > /dev/null 2>&1 &
@EOF
chmod 777 myScript
qsub -b yes -sync n ./myScript
========================================

So it's a script that writes a second script, and then executes the
second script. There is no execution of background commands during the
here-document processing at all. The background process is launched
by the second script.

Correct. Sorry for my fuzzy description.

Post by Greg Wooledge
P.S. 777 permissions are just wrong. 755 would be much more reasonable.

I agree. In fact, I'm typically a 750 person. I had used 777 during
alpha testing and will change it before releasing the script.

Post by Greg Wooledge

Post by Mun
Note the last line with the qsub command. The whole purpose of the
here-document is to package up a set of commands into a script form
that will be dispatched to another server for execution using a "grid"
tool (aka load sharing facility).
The commands run fine except I can't figure out a way for gxmessage to
not die when myScript exits.

So the REAL question is how to run a long-running background command on
a remote system using this "grid" thing, which I'm going to assume is
something like ssh.
It looks like you've already redirected stdout and stderr, which is
the normal problem that people encounter when trying to run long-lived
background processes over ssh.
So, since I don't know anything about "gxmessage" or "grid", I don't know
that there's anything more I can do for you. Sorry. Maybe someone else
can take a stab at it.
(One possibility is that "grid" doesn't simply run a program, but
instead acts in coordination with some kind of remote overseer/supervisor
process, and this supervisor kills everything when "grid" disconnects,
in which case you are actively trying to subvert the design of the
system you're using.)

Yes, that's possible. I will look into that.

Post by Greg Wooledge
(Another is that gxmessage requires a terminal, and dies instantly
when executed by "grid" which I am assuming does not allocate a
pseudoterminal by default.)
(I'm sure there are a billion other random things that I just can't
think of at the moment.)
(Is there a support mailing list for "grid" and/or "gxmessage" that
might actually know such things?)

Yes, grid has a mailing list as well and I'll make use of it shortly.

Thanks for your ideas and suggestions!

Regards,

[Assaf Gordon]

Hello,

Post by Mun
========================================
#! /bin/bash
date > showMe.txt
nohup gxmessage -timeout 0 -file showMe.txt > /dev/null 2>&1 &
@EOF
chmod 777 myScript
qsub -b yes -sync n ./myScript
========================================

There seem to be several strange things here:

1. qsub submits a job to the grid engine (and I assume it is SGE = Sun Grid Engine
based on the paramaters). The job (your script) will run on a different machine
(one of the SGE nodes).


2. Using "-b yes" means that you are executing a binary file (e.g. compiled C program)
as opposed to a shell script. But you *are* queueing a shell script.
So why use "-b yes" ?


3. If "gxmessage" is GTK-xmessage [1] then it is a graphical program
that needs X11 display. But you will certainly not have X11 connection
on the SGE node that actually runs the job (i.e. the "$DISPLAY" variable
will be empty, and gxmessage will fail like so:

$ DISPLAY= gxmessage
Unable to init server: Could not connect: Connection refused
gxmessage: unable to initialize GTK

So it is not entirely clear what you are aiming to do here.

[1] http://manpages.ubuntu.com/manpages/trusty/man1/gxmessage.1.html


4. A typical SGE job is scheduled (queued) to be run later (when an SGE
node is available, which might be immediately). The job will be active
until the main script is done.
So why execute a program in the background with nohup?
Do you want to try and leave a background program running on the SGE node
and keep consuming resources after the job is officially over?


5. It is recommended not to redirect STDOUT and STDERR to /dev/null with SGE.
Typically, 'qsub' will print a job ID (e.g. "Job 12345 submitted"),
and once the job is completed (on a different node), STDOUT and STDERR of the job
will be saved in your $HOME directory as "myScript.o12345" and "myScript.e12345".
(o=out, e=err). This will quickly show you what errors are encounterd
(e.g. "gxmessage" will complain about missing $DISPLAY).


HTH,
- assaf

[Mun]

Hi Assaf,

Firstly, I've changed the structure of my scripts to avoid this
altogether; but, it would have nice had it worked.
See my comments below.

Post by Assaf Gordon
Hello,

Post by Mun
========================================
#! /bin/bash
date > showMe.txt
nohup gxmessage -timeout 0 -file showMe.txt > /dev/null 2>&1 &
@EOF
chmod 777 myScript
qsub -b yes -sync n ./myScript
========================================

1. qsub submits a job to the grid engine (and I assume it is SGE = Sun Grid Engine
based on the paramaters). The job (your script) will run on a different machine
(one of the SGE nodes).

Correct: I'm using SGE.

Post by Assaf Gordon
2. Using "-b yes" means that you are executing a binary file (e.g. compiled C program)
as opposed to a shell script. But you *are* queueing a shell script.
So why use "-b yes" ?

I have found that any executable--even a script--requires use of "-b yes".

Post by Assaf Gordon
3. If "gxmessage" is GTK-xmessage [1] then it is a graphical program
that needs X11 display. But you will certainly not have X11 connection
on the SGE node that actually runs the job (i.e. the "$DISPLAY" variable
$ DISPLAY= gxmessage
Unable to init server: Could not connect: Connection refused
gxmessage: unable to initialize GTK
So it is not entirely clear what you are aiming to do here.
[1] http://manpages.ubuntu.com/manpages/trusty/man1/gxmessage.1.html

We do have X11 on our qhosts and they are able to push an X11 window
back to the user's display.

Post by Assaf Gordon
4. A typical SGE job is scheduled (queued) to be run later (when an SGE
node is available, which might be immediately). The job will be active
until the main script is done.
So why execute a program in the background with nohup?
Do you want to try and leave a background program running on the SGE node
and keep consuming resources after the job is officially over?

Pretty much, yes. It's not really consuming any significant resources
though; it's just the gxmessage window showing the status. And it
will only be up for as long it takes the user to acknowledge that s/he
has seen it--which I admit may be several hours in some cases.

Post by Assaf Gordon
5. It is recommended not to redirect STDOUT and STDERR to /dev/null with SGE.
Typically, 'qsub' will print a job ID (e.g. "Job 12345 submitted"),
and once the job is completed (on a different node), STDOUT and STDERR of the job
will be saved in your $HOME directory as "myScript.o12345" and "myScript.e12345".
(o=out, e=err). This will quickly show you what errors are encounterd
(e.g. "gxmessage" will complain about missing $DISPLAY).

I wasn't redirecting the file descriptors for the qsub; only for the gxmessage.

Again, I have a workaround so this thread can be closed. I would like
to thank everyone for their comments and suggestions!

Best regards,