emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
* Redirecting stderr to stdout with babel/shell
@ 2015-01-01 16:27 Karl Voit
  2015-01-01 17:10 ` Michael Brand
                   ` (2 more replies)
  0 siblings, 3 replies; 11+ messages in thread
From: Karl Voit @ 2015-01-01 16:27 UTC (permalink / raw)
  To: emacs-orgmode

Hi!

I enjoyed [1] where John is describing the issue of vanishing
stderr-messages through babel.

This could be fixed by "(setq org-babel-python-command "python -i -c
\"import sys; sys.stderr = sys.stdout\"")".

Is there a similar method to re-direct the stderr of a shell script
to stdout as well?

The issue is demonstrated below:

#+BEGIN_SRC sh :results output
echo "testing stdout" >&1
echo "testing stderr" >&2
echo "testing stderr with manual redirect" 2>&1 >&2
#+END_SRC

#+RESULTS:
: testing stdout
: testing stderr with manual redirect

I want to skip the manual redirect after each command.

Thanks for any idea!


[1] http://kitchingroup.cheme.cmu.edu/blog/2014/12/21/Capturing-stderr-from-Python-in-org-mode-take-2/

-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Redirecting stderr to stdout with babel/shell
  2015-01-01 16:27 Redirecting stderr to stdout with babel/shell Karl Voit
@ 2015-01-01 17:10 ` Michael Brand
  2015-01-02 21:43 ` Samuel Wales
  2015-01-11 11:43 ` Achim Gratz
  2 siblings, 0 replies; 11+ messages in thread
From: Michael Brand @ 2015-01-01 17:10 UTC (permalink / raw)
  To: Karl Voit; +Cc: Org Mode

Hi Karl

On Thu, Jan 1, 2015 at 5:27 PM, Karl Voit <devnull@karl-voit.at> wrote:
> Is there a similar method to re-direct the stderr of a shell script
> to stdout as well?

Not that I know of an other way than what you already demonstrated.

Once I wrote a lengthy post with my opinion about stdout, stderr and
the exit status in babel/shell. It contains a complete set of the
different cases as I see them:
http://thread.gmane.org/gmane.emacs.orgmode/45828/focus=46415

Michael

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Redirecting stderr to stdout with babel/shell
  2015-01-01 16:27 Redirecting stderr to stdout with babel/shell Karl Voit
  2015-01-01 17:10 ` Michael Brand
@ 2015-01-02 21:43 ` Samuel Wales
  2015-01-04 14:07   ` John Kitchin
  2015-01-11 11:43 ` Achim Gratz
  2 siblings, 1 reply; 11+ messages in thread
From: Samuel Wales @ 2015-01-02 21:43 UTC (permalink / raw)
  To: Karl Voit; +Cc: emacs-orgmode

hi karl,

i always wrap as follows and it works for me.

{
  your code
} 2>&1
:

the : eliminates fancy error handling.  i find that much less confusing.


samuel

-- 
The Kafka Pandemic: http://thekafkapandemic.blogspot.com

The disease DOES progress.  MANY people have died from it.  And
ANYBODY can get it.

Denmark: free Karina Hansen NOW.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Redirecting stderr to stdout with babel/shell
  2015-01-02 21:43 ` Samuel Wales
@ 2015-01-04 14:07   ` John Kitchin
  2015-01-11 10:54     ` Karl Voit
  0 siblings, 1 reply; 11+ messages in thread
From: John Kitchin @ 2015-01-04 14:07 UTC (permalink / raw)
  To: Samuel Wales; +Cc: Karl Voit, emacs-orgmode

Check out this solution:

http://kitchingroup.cheme.cmu.edu/blog/2015/01/04/Redirecting-stderr-in-org-mode-shell-blocks/

I think it does what you want, and uses the idea below.

Samuel Wales <samologist@gmail.com> writes:

> hi karl,
>
> i always wrap as follows and it works for me.
>
> {
>   your code
> } 2>&1
> :
>
> the : eliminates fancy error handling.  i find that much less confusing.
>
>
> samuel

-- 
-----------------------------------
John Kitchin
Professor
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
http://kitchingroup.cheme.cmu.edu

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Redirecting stderr to stdout with babel/shell
  2015-01-04 14:07   ` John Kitchin
@ 2015-01-11 10:54     ` Karl Voit
  2015-01-11 16:44       ` John Kitchin
  0 siblings, 1 reply; 11+ messages in thread
From: Karl Voit @ 2015-01-11 10:54 UTC (permalink / raw)
  To: emacs-orgmode

* John Kitchin <johnrkitchin@gmail.com> wrote:
> Check out this solution:
>
> http://kitchingroup.cheme.cmu.edu/blog/2015/01/04/Redirecting-stderr-in-org-mode-shell-blocks/

Hm. This does not work on my machine: Debian Wheezy GNU/Linux

Testing the current satus (again):

#+BEGIN_SRC sh :results output
echo "testing stdout" >&1
echo "testing stderr" >&2
date -g
#+END_SRC

#+RESULTS:

... no stderr on stdout and date error message in second buffer


Creating a wrapper-script similar as described in web-page above:

#+BEGIN_SRC sh
echo '#!/usr/bin/zsh
{
/usr/bin/zsh $1
} 2>&1' > ~/src/misc/zsh_stderr_redirected_to_stdout.sh
chmod +x ~/src/misc/zsh_stderr_redirected_to_stdout.sh
cd ~/bin
ln -s ../src/misc/zsh_stderr_redirected_to_stdout.sh .
#+END_SRC

#+RESULTS:

Setting the sh-command to this wrapper-script:

#+BEGIN_SRC elisp
(setq org-babel-sh-command "~/bin/zsh_stderr_redirected_to_stdout.sh")
#+END_SRC

#+RESULTS:
: ~/bin/zsh_stderr_redirected_to_stdout.sh

Re-testing status:

#+BEGIN_SRC sh :results output
echo "testing stdout" >&1
echo "testing stderr" >&2
date -g
#+END_SRC

#+RESULTS:

... no change except the second buffer for the date error message is
empty. So it's actually worse than before.


Trying with bash (as web-page did):

#+BEGIN_SRC sh
echo '#!/bin/bash
{
/bin/bash $1
} 2>&1' > ~/src/misc/bash_stderr_redirected_to_stdout.sh
chmod +x ~/src/misc/bash_stderr_redirected_to_stdout.sh
cd ~/bin
ln -s ../src/misc/bash_stderr_redirected_to_stdout.sh .
#+END_SRC

#+RESULTS:

Setting the sh-command to this wrapper-script:

#+BEGIN_SRC elisp
(setq org-babel-sh-command "~/bin/bash_stderr_redirected_to_stdout.sh")
#+END_SRC

#+RESULTS:
: ~/bin/bash_stderr_redirected_to_stdout.sh

Same result as with zsh :-(


Re-setting to standard settings to revoke tests from above:

#+BEGIN_SRC elisp
(setq org-babel-sh-command "sh")
#+END_SRC

#+RESULTS:
: sh

Re-testing status:

#+BEGIN_SRC sh :results output
echo "testing stdout" >&1
echo "testing stderr" >&2
date -g
#+END_SRC

#+RESULTS:


-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Redirecting stderr to stdout with babel/shell
  2015-01-01 16:27 Redirecting stderr to stdout with babel/shell Karl Voit
  2015-01-01 17:10 ` Michael Brand
  2015-01-02 21:43 ` Samuel Wales
@ 2015-01-11 11:43 ` Achim Gratz
  2015-01-11 15:02   ` Karl Voit
  2 siblings, 1 reply; 11+ messages in thread
From: Achim Gratz @ 2015-01-11 11:43 UTC (permalink / raw)
  To: emacs-orgmode

Karl Voit writes:
> echo "testing stderr with manual redirect" 2>&1 >&2

The last redirection ">&2" is nonsense, it only works because STDERR is
already reopened on STDOUT and redirection to the same file descriptor
is ignored.

And to solve your original problem:

#+BEGIN_SRC sh :results output
exec 2>&1
echo "testing stdout" >&1
echo "testing stderr" >&2
date -g
:
#+END_SRC


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Redirecting stderr to stdout with babel/shell
  2015-01-11 11:43 ` Achim Gratz
@ 2015-01-11 15:02   ` Karl Voit
  0 siblings, 0 replies; 11+ messages in thread
From: Karl Voit @ 2015-01-11 15:02 UTC (permalink / raw)
  To: emacs-orgmode

* Achim Gratz <Stromeko@nexgo.de> wrote:
> Karl Voit writes:
>> echo "testing stderr with manual redirect" 2>&1 >&2
>
> The last redirection ">&2" is nonsense, it only works because STDERR is
> already reopened on STDOUT and redirection to the same file descriptor
> is ignored.

Absolutely right. Must have been a non-deleted artefact after I
tested something different in that line.

> And to solve your original problem:
>
> #+BEGIN_SRC sh :results output
> exec 2>&1
> echo "testing stdout" >&1
> echo "testing stderr" >&2
> date -g
> :
> #+END_SRC

Together with the '{ ... } 2>&1' trick, this is a valid workaround.
What I wanted to achieve is a re-direct of any (sh) babel script
without this exec-command I do have to add.

However, I could imagine an additional babel-parameter like ":stderr
redirect" (or something more meaningful) because in most cases I
have to add the ":results output" line as well.

-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Redirecting stderr to stdout with babel/shell
  2015-01-11 10:54     ` Karl Voit
@ 2015-01-11 16:44       ` John Kitchin
  2015-01-11 18:31         ` Karl Voit
  0 siblings, 1 reply; 11+ messages in thread
From: John Kitchin @ 2015-01-11 16:44 UTC (permalink / raw)
  To: Karl Voit; +Cc: Karl Voit, emacs-orgmode

Karl Voit <devnull@Karl-Voit.at> writes:

Weird. It works for me on MacOSX with bash. Out of curiosity, did you
try the source: http://kitchingroup.cheme.cmu.edu/org/2015/01/04/Redirecting-stderr-in-org-mode-shell-blocks.org?

I updated the web page with Achim's  simpler solution.

> * John Kitchin <johnrkitchin@gmail.com> wrote:
>> Check out this solution:
>>
>> http://kitchingroup.cheme.cmu.edu/blog/2015/01/04/Redirecting-stderr-in-org-mode-shell-blocks/
>
> Hm. This does not work on my machine: Debian Wheezy GNU/Linux
>
> Testing the current satus (again):
>
> #+BEGIN_SRC sh :results output
> echo "testing stdout" >&1
> echo "testing stderr" >&2
> date -g
> #+END_SRC
>
> #+RESULTS:
>
> ... no stderr on stdout and date error message in second buffer
>
>
> Creating a wrapper-script similar as described in web-page above:
>
> #+BEGIN_SRC sh
> echo '#!/usr/bin/zsh
> {
> /usr/bin/zsh $1
> } 2>&1' > ~/src/misc/zsh_stderr_redirected_to_stdout.sh
> chmod +x ~/src/misc/zsh_stderr_redirected_to_stdout.sh
> cd ~/bin
> ln -s ../src/misc/zsh_stderr_redirected_to_stdout.sh .
> #+END_SRC
>
> #+RESULTS:
>
> Setting the sh-command to this wrapper-script:
>
> #+BEGIN_SRC elisp
> (setq org-babel-sh-command "~/bin/zsh_stderr_redirected_to_stdout.sh")
> #+END_SRC
>
> #+RESULTS:
> : ~/bin/zsh_stderr_redirected_to_stdout.sh
>
> Re-testing status:
>
> #+BEGIN_SRC sh :results output
> echo "testing stdout" >&1
> echo "testing stderr" >&2
> date -g
> #+END_SRC
>
> #+RESULTS:
>
> ... no change except the second buffer for the date error message is
> empty. So it's actually worse than before.
>
>
> Trying with bash (as web-page did):
>
> #+BEGIN_SRC sh
> echo '#!/bin/bash
> {
> /bin/bash $1
> } 2>&1' > ~/src/misc/bash_stderr_redirected_to_stdout.sh
> chmod +x ~/src/misc/bash_stderr_redirected_to_stdout.sh
> cd ~/bin
> ln -s ../src/misc/bash_stderr_redirected_to_stdout.sh .
> #+END_SRC
>
> #+RESULTS:
>
> Setting the sh-command to this wrapper-script:
>
> #+BEGIN_SRC elisp
> (setq org-babel-sh-command "~/bin/bash_stderr_redirected_to_stdout.sh")
> #+END_SRC
>
> #+RESULTS:
> : ~/bin/bash_stderr_redirected_to_stdout.sh
>
> Same result as with zsh :-(
>
>
> Re-setting to standard settings to revoke tests from above:
>
> #+BEGIN_SRC elisp
> (setq org-babel-sh-command "sh")
> #+END_SRC
>
> #+RESULTS:
> : sh
>
> Re-testing status:
>
> #+BEGIN_SRC sh :results output
> echo "testing stdout" >&1
> echo "testing stderr" >&2
> date -g
> #+END_SRC
>
> #+RESULTS:

--
-----------------------------------
John Kitchin
Professor
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
http://kitchingroup.cheme.cmu.edu

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Redirecting stderr to stdout with babel/shell
  2015-01-11 16:44       ` John Kitchin
@ 2015-01-11 18:31         ` Karl Voit
  2015-01-11 18:58           ` Achim Gratz
  0 siblings, 1 reply; 11+ messages in thread
From: Karl Voit @ 2015-01-11 18:31 UTC (permalink / raw)
  To: emacs-orgmode

* John Kitchin <johnrkitchin@gmail.com> wrote:
> Karl Voit <devnull@Karl-Voit.at> writes:
>
> Weird. It works for me on MacOSX with bash. Out of curiosity, did you
> try the source: http://kitchingroup.cheme.cmu.edu/org/2015/01/04/Redirecting-stderr-in-org-mode-shell-blocks.org?

I did it now.

What I found out:

#+BEGIN_SRC sh
echo '#!/usr/bin/zsh
{
/usr/bin/zsh $1
} 2>&1
#end' > ~/src/misc/zsh_stderr_redirected_to_stdout.sh
chmod +x ~/src/misc/zsh_stderr_redirected_to_stdout.sh
cd ~/bin
ln -s ../src/misc/zsh_stderr_redirected_to_stdout.sh .
#+END_SRC

#+BEGIN_SRC elisp
(setq org-babel-sh-command "~/bin/zsh_stderr_redirected_to_stdout.sh")
#+END_SRC

... my old script from my previous posting:

#+BEGIN_SRC sh :results output
echo "testing stdout" >&1
echo "testing stderr" >&2
date -g
#+END_SRC

#+RESULTS:

... with empty additional buffer window. So not happy.

However with an additional "echo" at the end:

#+BEGIN_SRC sh :results output
echo "testing stdout" >&1
echo "testing stderr" >&2
date -g
echo
#+END_SRC

#+RESULTS:
: testing stdout
: testing stderr
: date: invalid option -- 'g'
: Try `date --help' for more information.
: 

... it works. :-O

Is there an issue with flushing stdout or something?

What is the explanation and the general rule?


Btw, the trick with tangling for writing the script file did not work on my
machine.

-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Redirecting stderr to stdout with babel/shell
  2015-01-11 18:31         ` Karl Voit
@ 2015-01-11 18:58           ` Achim Gratz
  2015-01-21 16:52             ` Karl Voit
  0 siblings, 1 reply; 11+ messages in thread
From: Achim Gratz @ 2015-01-11 18:58 UTC (permalink / raw)
  To: emacs-orgmode

Karl Voit writes:
> However with an additional "echo" at the end:

You need to understand what you're doing or at least copy the code
exactly.  The last line in my example is a colon ":" so that the shell
exit code is always zero.  If not, Babel will ignore the output since it
interprets any non-zero exit code as failure.  Since you've redirected
STDERR to STDOUT the error buffer that would normally let you know what
happened stays empty.  The echo has the same effect, but produces a
blank line you don't want.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Waldorf MIDI Implementation & additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: Redirecting stderr to stdout with babel/shell
  2015-01-11 18:58           ` Achim Gratz
@ 2015-01-21 16:52             ` Karl Voit
  0 siblings, 0 replies; 11+ messages in thread
From: Karl Voit @ 2015-01-21 16:52 UTC (permalink / raw)
  To: emacs-orgmode

* Achim Gratz <Stromeko@nexgo.de> wrote:
> Karl Voit writes:
>> However with an additional "echo" at the end:
>
> You need to understand what you're doing or at least copy the code
> exactly.  The last line in my example is a colon ":" so that the shell
> exit code is always zero.  

You're right. The only thing I did not know was the meaning of the
colon.

http://stackoverflow.com/a/3224910 did fix my ignorance :-)

> If not, Babel will ignore the output since it interprets any
> non-zero exit code as failure.  Since you've redirected STDERR to
> STDOUT the error buffer that would normally let you know what
> happened stays empty.  The echo has the same effect, but produces
> a blank line you don't want.

Yes, now everything makes sense to me. Thanks for your pointer!

-- 
mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML to Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <

https://github.com/novoid/extract_pdf_annotations_to_orgmode + more on github

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2015-01-21 16:55 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-01-01 16:27 Redirecting stderr to stdout with babel/shell Karl Voit
2015-01-01 17:10 ` Michael Brand
2015-01-02 21:43 ` Samuel Wales
2015-01-04 14:07   ` John Kitchin
2015-01-11 10:54     ` Karl Voit
2015-01-11 16:44       ` John Kitchin
2015-01-11 18:31         ` Karl Voit
2015-01-11 18:58           ` Achim Gratz
2015-01-21 16:52             ` Karl Voit
2015-01-11 11:43 ` Achim Gratz
2015-01-11 15:02   ` Karl Voit

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).