Re: Bug: babel: results switch (output vs. value) has no or wrong effect for sh source block [7.7 (release_7.7.107.g7a82)]

[Michael Brand <michael.ch.brand@gmail.com>]

Hi Eric

On Tue, Aug 23, 2011 at 19:13, Eric Schulte <schulte.eric@gmail.com> wrote:
> Note that during interactive evaluation if the exit status is non-0 then
> STDERR will be dumped into a babel error buffer which will be poped up,
> so this information will not be silently discarded.

Thanks for emphasizing this. I didn't use code block evaluation much
for ob-sh and haven't yet noticed this rule with the last exit status
that seems very important to know.

Two things I would like to comment referring to the examples 1) to 5)
below:
a) _completeness_ of the process output:
   - When the last exit status is 0, babel silently discards possible
     important information from stderr, see mainly 1), also 2) to 3).
   - When the last exit status is >0, babel pops up the "Org-Babel
     Error Output" and silently discards possible important
     information from stdout, see 4).
b) _decision_ whether the process succeeded or failed:
   - This is made by discriminating the last exit status 0 or not. But
     there are even standard processes with an exit status and stderr
     not simple, see 4) to 5) ("simple": 0 => success and output only
     on stdout, >0 => failure and output only on stderr).

My suggestions for changes (I have understood that this would not be
easy to do, babel grew differently):
a) _completeness_ of the process output:
   - When the last exit status is 0: Include stderr in "#+results:" by
     default. In the minority of use cases where the behavior changes
     I think this is most of the time an improvement.
   - When the last exit status is >0 and babel pops up the "Org-Babel
     Error Output": Include stdout there by default. In the minority
     of use cases where the behavior changes I think this is always an
     improvement.
b) _decision_ whether the process succeeded or failed:
   - My point of view is that babel code block evaluation is not
     always able to make such a decision for two reasons, at least in
     the case of the languages awk, sh and shell: Firstly not
     universally because it would require to consider which process it
     runs, e. g. the exit status 1 means success for diff or grep but
     >0 means failure for most other processes. Secondly not reliably
     because it would require to consider for which purpose this
     process is used during this evaluation. When one wants to
     document the output of a failure then "Org-Babel Error Output"
     does not come in handy.
   - An option to move this decision to the user would help. For this
     I wish to have an additional and somehow more manual operation
     mode that bypasses the interpretation of the last exit status and
     omits "Org-Babel Error Output". Achim already phrased it in this
     thread as:
     - "I think that it would be generally useful (not just for shell
       blocks) to be able to capture stderr, either together with
       stdout or separately into a result target block and have the
       return status available as a variable."
     - I would only add: It would be nice to have both variants of
       "either or" available to choose from. The first for better
       human readability and the second to read stdout without stderr
       into the next chained code block again.

----------------------------------------------------------------

Comments and questions for the examples and workarounds:

Examples 4) to 5) end up with an empty "Org-Babel Error Output" and
"#+results:" is emptied.
- Since they are both empty it is not necessarily clear what happened.
- The Messages buffer says "Babel evaluation exited with code 1" and
  "Code block produced no output.". The second statement is not
  necessarily clear:
  - Is this an _info_ or was output expected and this is an _error_
    that even stopped babel from doing more work?
    - Is the message "Code block evaluation complete." from other
      cases an indication whether babel finished all its work? If this
      is an indication, isn't this message missing in the following
      case?
        #+begin_src sh :results output
          true
        #+end_src

        #+results:

  - Which "output" was meant to be missing?
    - Output missing on stdout (example 5) or was stdout not
      considered due to last exit status >0 (example 4)?
    - Output missing on stderr?

At the end are the workarounds I will use for now for the examples 1)
to 5). Shown is the complete variant when I need to know the last exit
status.
- 2>&1 :: collect stderr
- $?   :: last exit status
- true :: omit any "Org-Babel Error Output"

----------------------------------------------------------------

Examples:

1) command(s) before the last command have error
   - non-existent file myfile, output mixed on stdout and stderr
     - in a terminal (not babel code block evaluation):
       - shell command:
         #+begin_example
           ls myfile
           echo b
         #+end_example

       - output (stderr: ls, stdout: echo, exit status ls: 2, exit
         status echo: 0):
         #+begin_example
           ls: myfile: No such file or directory
           b
         #+end_example

     - with babel:
       #+begin_src sh :results output
         ls myfile
         echo b
       #+end_src

       #+results:
       : b

2) strace (Linux)
   - output on stderr
     - in a terminal (not babel code block evaluation):
       - shell command:
         #+begin_example
           strace true
         #+end_example

       - output (stdout: empty, stderr: syscalls, exit status: 0):
         #+begin_example
           execve("/bin/true", ["true"], [/* 54 vars */]) = 0
           [...]
           exit_group(0)                           = ?
         #+end_example

     - with babel:
       #+begin_src sh :results output
         strace true
       #+end_src

       #+results:

3) truss (Solaris)
   - output on stderr
     - in a terminal (not babel code block evaluation):
       - shell command:
         #+begin_example
           truss true
         #+end_example

       - output (stdout: empty, stderr: syscalls, exit status: 0):
         #+begin_example
           execve("/usr/bin/true", 0xFFBFF0FC, 0xFFBFF104)  argc = 1
           [...]
           _exit(0)
         #+end_example

     - with babel:
       #+begin_src sh :results output
         truss true
       #+end_src

       #+results:

4) diff
   - the exit status as standardized by POSIX/SUS
     http://pubs.opengroup.org/onlinepubs/009695399/utilities/diff.html
     - 0  :: No differences were found.
     - 1  :: Differences were found.
     - >1 :: An error occurred.

   - example with two differing files
     #+begin_src sh :results silent
       echo a > /tmp/a
       echo b > /tmp/b
     #+end_src

     - in a terminal (not babel code block evaluation):
       - shell command:
         #+begin_example
           diff /tmp/a /tmp/b
         #+end_example

       - output (stdout: differences, stderr: empty, exit status: 1):
         #+begin_example
           1c1
           < a
           ---
           > b
         #+end_example

     - with babel:
       #+begin_src sh :results output
         diff /tmp/a /tmp/b
       #+end_src

       #+results:

5) grep
   - the exit status as standardized by POSIX/SUS
     http://pubs.opengroup.org/onlinepubs/009695399/utilities/grep.html
     - 0  :: One or more lines were selected.
     - 1  :: No lines were selected.
     - >1 :: An error occurred.

   - example:
     - in a terminal (not babel code block evaluation):
       - shell command:
         #+begin_example
           echo a | grep b
         #+end_example

       - output (stdout: empty, stderr: empty, exit status: 1):
         #+begin_example
         #+end_example

     - with babel:
       #+begin_src sh :results output
         echo a | grep b
       #+end_src

       #+results:

----------------------------------------------------------------

Workarounds:

1) command(s) before the last command have error
   #+begin_src sh :results output
     ls myfile 2>&1
     echo b 2>&1
     echo "last exit status: $?" 2>&1
     true 2>&1
   #+end_src

   #+results:
   : ls: myfile: No such file or directory
   : b
   : last exit status: 0

2) strace (Linux)
   #+begin_src sh :results output
     strace true b 2>&1
     echo "last exit status: $?" 2>&1
     true 2>&1
   #+end_src

   #+results:
   : execve("/bin/true", ["true", "b"], [/* 104 vars */]) = 0
   : [...]
   : exit_group(0)                           = ?
   : last exit status: 0

3) truss (Solaris)
   #+begin_src sh :results output
     truss true b 2>&1
     echo "last exit status: $?" 2>&1
     true 2>&1
   #+end_src

   #+results:
   : execve("/usr/bin/true", 0xFFBFF0FC, 0xFFBFF104)  argc = 1
   : [...]
   : _exit(0)
   : last exit status: 0

4) diff
   #+begin_src sh :results output
     diff /tmp/a /tmp/b 2>&1
     echo "last exit status: $?" 2>&1
     true 2>&1
   #+end_src

   #+results:
   : 1c1
   : < a
   : ---
   : > b
   : last exit status: 1

5) grep
   #+begin_src sh :results output
     echo a | grep b 2>&1
     echo "last exit status: $?" 2>&1
     true 2>&1
   #+end_src

   #+results:
   : last exit status: 1

----------------------------------------------------------------

Michael