[Help-bash] How to create a real copy of file descriptors stdout / stderr?

[Patrick Schleizer]

Hi,

my goal is to duplicate, redirect all output (stdout and stderr) of an
application (apt-get) to a file while retaining the usual behavior of
apt-get, stdout and stderr.

How to create a real copy of file descriptor stdout and stderr? I would
like to keep stdout and stderr as "natural" as possible. I.e. if it was
running for example apt-get, I would wish to retain colors and progress
information.

By using `tee`, colors and progress information [1] are lost. Therefore
I experimented with using `unbuffer`, however I would appreciate if both
`unbuffer` and `tee` would not required. [2]

The following example is partially functional. (The wrapper does more
stuff, just kept it simple.)

#####

#!/bin/bash

temp_dir="$(mktemp --directory)"
logfile="$temp_dir/log"

exec 3>&1
exec 1> >(tee -a "$logfile")

#exec 4>&2
#exec 2> >(tee -a "$logfile")

unbuffer apt-get "$@"

#####

Broken: stderr redirection. Once I enable...

exec 4>&2
exec 2> >(tee -a "$logfile")

...nothing gets written to stdout anymore. Could you advice please on
how to fix this?

Cheers,
Patrick

[1]
22% [8 Packages 3,449 kB/7,098 kB 49%]
174
kB/s 4min 57s
[2] To keep the wrapper simpler and dependent on less external binaries.
Otherwise this could cause conflicts with mandatory access control and
so forth.

[João Eiras]

Why not just...

$ unbuffer apt-get "$@" > 1> >(tee -a ) 2> >(tee -a "$logfile" >&2)

however, you should be careful having two tee instances writting to
the same file. Surely you want separate log files for each file
descriptor ?

[Patrick Schleizer]

Post by João Eiras
Why not just...

Generally good idea. Did not work for me, but the following does...

Post by João Eiras
however, you should be careful having two tee instances writting to
the same file. Surely you want separate log files for each file
descriptor ?

In this case I like to have stdout and stderr in the same log file so
it's looking the same as if manually run.

[João Eiras]

On 27 February 2017 at 15:03, Patrick Schleizer

Post by Patrick Schleizer

Post by João Eiras
Why not just...

Generally good idea. Did not work for me, but the following does...

Yeah, two errors in my example. Just typed it on my webmail :)

Post by Patrick Schleizer
In this case I like to have stdout and stderr in the same log file so
it's looking the same as if manually run.

Then you can redirect stderr to stdout
$ unbuffer apt-get "$@" 1> >(tee ...) 2>&1

Or do you want to keep the file descriptors separate ? I've done that
but it's more tricky.

[Patrick Schleizer]

Post by João Eiras
On 27 February 2017 at 15:03, Patrick Schleizer

Post by Patrick Schleizer
In this case I like to have stdout and stderr in the same log file so
it's looking the same as if manually run.

Then you can redirect stderr to stdout
Or do you want to keep the file descriptors separate ? I've done that
but it's more tricky.

It's alright in this case.

Is unbuffer / script unavoidable?

[João Eiras]

Post by Patrick Schleizer
Is unbuffer / script unavoidable?

I think so. Not an expert on the subject, but this is something
handled by the system. Either the stdout of the program is directed to
a pty device (a terminal) or another process. The system will do line
buffering for the first, and block buffering for the second (like 4k).

[Patrick Schleizer]

The following worked for me.

echo "int isatty(int fd) { return 1; }" | gcc -O2 -fpic -shared -ldl -o
"$isatty_so_file" -xc -

LD_PRELOAD+=" $isatty_so_file

LD_PRELOAD="$LD_PRELOAD" apt-get "$@" 2>&1 | tee -a "$logfile"

gcc compilation on the fly seems crazy but was a worthwhile test.

Is there some pre-build command line tool isatty or shared object isatty.so?

For now, ending up using the following.

python -c 'import pty, sys; pty.spawn(sys.argv[1:])' \
| apt-get "$@" 2>&1 \
| tee -a "$logfile"

[Branden Robinson]

[I am not subscribed to whonix-devel.]

On Tue, Feb 28, 2017 at 2:28 PM, Patrick Schleizer

Post by Patrick Schleizer
Is there some pre-build command line tool isatty or shared object isatty.so?

Thomas Dickey has an isatty command-line tool.

http://invisible-island.net/misc_tools/index.html

Regards,
Branden

[Greg Wooledge]

Post by Branden Robinson
On Tue, Feb 28, 2017 at 2:28 PM, Patrick Schleizer

Post by Patrick Schleizer
Is there some pre-build command line tool isatty or shared object isatty.so?

Thomas Dickey has an isatty command-line tool.
http://invisible-island.net/misc_tools/index.html

If all you want is a shell command, you can use test -t 1 (or 0, or 2).

[Patrick Schleizer]

Post by Patrick Schleizer
The following worked for me.
echo "int isatty(int fd) { return 1; }" | gcc -O2 -fpic -shared -ldl -o
"$isatty_so_file" -xc -
LD_PRELOAD+=" $isatty_so_file
gcc compilation on the fly seems crazy but was a worthwhile test.
Is there some pre-build command line tool isatty or shared object isatty.so?

Need to specify what I meant. Sorry for the confusion.

What the shared object for LD_PRELOAD with "int isatty(int fd) { return
1; }" apparently does not not showing if a tool is running inside a tty
or not. It's fooling the application asking for "am I running in a tty"
with "yes". It overwrites the C function "am I running in a tty" with
"always return yes". That's my understanding from testing it.

Is there a pre-build command line tool for that?

(I am asking, because then there is a chance I can install it using the
package manager of my distribution, which would be a much cleaner solution.)

Post by Patrick Schleizer
For now, ending up using the following.
python -c 'import pty, sys; pty.spawn(sys.argv[1:])' \
| tee -a "$logfile"

[Patrick Schleizer]

Could we even use
&>
?

[Greg Wooledge]

Post by Patrick Schleizer
Could we even use
&>
?

In a previous email, he said it was *noninteractive*. So why not just
do:

unbuffer apt-get blah 2>&1 | tee -a logfile

Since both stdout and stderr are going to the same logfile, keep it
simple.

I don't even know whether apt-get needs unbuffer in the first place.
You could try leaving it out.

[Patrick Schleizer]

Post by Greg Wooledge

Post by Patrick Schleizer
Could we even use
&>
?

In a previous email, he said it was *noninteractive*. So why not just
unbuffer apt-get blah 2>&1 | tee -a logfile
Since both stdout and stderr are going to the same logfile, keep it
simple.

Simpler and works for me.

Post by Greg Wooledge
I don't even know whether apt-get needs unbuffer in the first place.
You could try leaving it out.

Without unbuffer, colors and progress information are lost from console
output, which I wanted to prevent.

[Greg Wooledge]

Post by Patrick Schleizer
my goal is to duplicate, redirect all output (stdout and stderr) of an
application (apt-get) to a file while retaining the usual behavior of
apt-get, stdout and stderr.

I strongly recommend script(1) instead of some redirection kludge.

[Patrick Schleizer]

Post by Greg Wooledge

Post by Patrick Schleizer
my goal is to duplicate, redirect all output (stdout and stderr) of an
application (apt-get) to a file while retaining the usual behavior of
apt-get, stdout and stderr.

I strongly recommend script(1) instead of some redirection kludge.

script(1) looks interesting. Almost like the tool I am missing. However,
it's man page discourages use in non-interactive scripts?