emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
* bug: Error handling in source blocks.
@ 2021-08-10  4:13 James Powell
  2021-08-10 19:42 ` Berry, Charles via General discussions about Org-mode.
  2021-08-10 20:30 ` Tim Cross
  0 siblings, 2 replies; 5+ messages in thread
From: James Powell @ 2021-08-10  4:13 UTC (permalink / raw)
  To: emacs-orgmode

   Error handling is important and hard to get right.  Me, I prefer to
   treat every warning as an error (-Werror in gcc, "options(warn=2)" in
   R, etc).  I want the system to grind to a halt at the least sign of
   trouble.

   When I write some nonsense into a code block as in this example:

   ,----
   | : This next code block is intended to trigger an error, because that
   | : variable "fffff838293483" with that attribute/member "x8483848"
   | : probably does not exist.
   | :
   | : #+begin_src R :exports both
   | : x <- fffff838293483$x8483848
   | : #+end_src
   | :
   | : #+RESULTS:
   | :
   | : #+begin_src R :exports both :results table :colnames yes
   | : require(tidyverse)
   | : tribble(~a, ~b, 1, 2)
   | : #+end_src
   | :
   | : #+RESULTS:
   | : | a | b |
   | : |---+---|
   | : | 1 | 2 |
   | : #+end_src
   `----


   which does trigger an error in R
   ,----
   | Error: object 'fffff838293483' not found
   | [traceback follows]
   `----


   and I do org-to-PDF export, I would like the PDF to be not built and
   loud alarm bells to ring bringing the error to my attention.

   What happens instead: the document builds from org into PDF with no
   indication that anything went wrong at all, except perhaps minor bits
   of missing or incorrect text in the PDF file.  The error can easily be
   buried in the output as more and later source blocks from the same
   document are evaluated.

   Likewise, in 'sh',
   ,----
   | #+begin_src sh :exports results
   | exit 1
   | #+end_src
   `----


   Sh exits 1 to indicate an error, so I would like this to ring alarm
   bells and fail to produce a PDF on org-to-PDF export.

   We might add examples in Java, Python, C++, C along the same lines.
   All of these should be in a unit test (because error handling is 
important).
   There do not seem to be any unit tests that exercise error handling in
   org-9.4.4.

   Exceptions that are thrown in the block should also at least
   optionally always and reliably abort the org-to-PDX export and ring
   alarm bells.

   I would like to see a section in the org manual called "Error handling
   in source blocks" that discusses the issue.

   I searched for "error handling" at <https://lists.gnu.org/archive>,
   with no results that are relevant.

   thank you for your time,
   - JP

-- 
James E. Powell, MS
Pronouns: he/him/his
Applied Physics PhD Candidate
Department of Physics
Portland State University
Home page: http://web.pdx.edu/~powellj
Office: SRTC 409B Phone: +1-503-725-8515



^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: bug: Error handling in source blocks.
  2021-08-10  4:13 bug: Error handling in source blocks James Powell
@ 2021-08-10 19:42 ` Berry, Charles via General discussions about Org-mode.
  2021-08-10 20:30 ` Tim Cross
  1 sibling, 0 replies; 5+ messages in thread
From: Berry, Charles via General discussions about Org-mode. @ 2021-08-10 19:42 UTC (permalink / raw)
  To: James Powell; +Cc: emacs-orgmode



> On Aug 9, 2021, at 9:13 PM, James Powell <powellj@pdx.edu> wrote:
> 
>  Error handling is important and hard to get right.  Me, I prefer to
>   treat every warning as an error (-Werror in gcc, "options(warn=2)" in
>   R, etc).  I want the system to grind to a halt at the least sign of
>   trouble.

If the effect of an error is to return no result (as in your example), you can use a :post header-arg to check for a nil value in `*this*' and issue a user-error. 

Example:

#+name: check-res
#+begin_src emacs-lisp
(or *this* (error "nil result")) 
#+end_src


#+begin_src R :exports both :post check-res()
  x <- fffff838293483$x8483848
#+end_src


See (info "(org) Results of Evaluation") and scroll to `post-processing' for details on the :post header.

The user-error halts export.

HTH,
Chuck





^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: bug: Error handling in source blocks.
  2021-08-10  4:13 bug: Error handling in source blocks James Powell
  2021-08-10 19:42 ` Berry, Charles via General discussions about Org-mode.
@ 2021-08-10 20:30 ` Tim Cross
  2021-08-10 23:14   ` Tom Gillespie
  2021-08-11  6:23   ` tomas
  1 sibling, 2 replies; 5+ messages in thread
From: Tim Cross @ 2021-08-10 20:30 UTC (permalink / raw)
  To: emacs-orgmode


James Powell <powellj@pdx.edu> writes:

>   Error handling is important and hard to get right.  Me, I prefer to
>   treat every warning as an error (-Werror in gcc, "options(warn=2)" in
>   R, etc).  I want the system to grind to a halt at the least sign of
>   trouble.
>
>   When I write some nonsense into a code block as in this example:
>
>   ,----
>   | : This next code block is intended to trigger an error, because that
>   | : variable "fffff838293483" with that attribute/member "x8483848"
>   | : probably does not exist.
>   | :
>   | : #+begin_src R :exports both
>   | : x <- fffff838293483$x8483848
>   | : #+end_src
>   | :
>   | : #+RESULTS:
>   | :
>   | : #+begin_src R :exports both :results table :colnames yes
>   | : require(tidyverse)
>   | : tribble(~a, ~b, 1, 2)
>   | : #+end_src
>   | :
>   | : #+RESULTS:
>   | : | a | b |
>   | : |---+---|
>   | : | 1 | 2 |
>   | : #+end_src
>   `----
>
>
>   which does trigger an error in R
>   ,----
>   | Error: object 'fffff838293483' not found
>   | [traceback follows]
>   `----
>
>
>   and I do org-to-PDF export, I would like the PDF to be not built and
>   loud alarm bells to ring bringing the error to my attention.
>
>   What happens instead: the document builds from org into PDF with no
>   indication that anything went wrong at all, except perhaps minor bits
>   of missing or incorrect text in the PDF file.  The error can easily be
>   buried in the output as more and later source blocks from the same
>   document are evaluated.
>
>   Likewise, in 'sh',
>   ,----
>   | #+begin_src sh :exports results
>   | exit 1
>   | #+end_src
>   `----
>
>
>   Sh exits 1 to indicate an error, so I would like this to ring alarm
>   bells and fail to produce a PDF on org-to-PDF export.
>
>   We might add examples in Java, Python, C++, C along the same lines.
>   All of these should be in a unit test (because error handling is important).
>   There do not seem to be any unit tests that exercise error handling in
>   org-9.4.4.
>
>   Exceptions that are thrown in the block should also at least
>   optionally always and reliably abort the org-to-PDX export and ring
>   alarm bells.
>
>   I would like to see a section in the org manual called "Error handling
>   in source blocks" that discusses the issue.
>
>   I searched for "error handling" at <https://lists.gnu.org/archive>,
>   with no results that are relevant.
>
>   thank you for your time,
>   - JP

Hi James,

I wanted to reply to your message mainly to let you know it hasn't been
ignored. However, I think you have only uncovered a tip of a very large
iceberg and dealing with it will be non-trivial. 

I pretty much agree with most of what you wrote. However, I'm not sure
I'd call this a bug. Rather, I would argue this is a request for a
feature enhancement. For me, a bug is something which does not behave or
work as intended. I'm not sure anyone has really considered a consistent
approach to error handling. It is something which I think would be very
beneficial, but I also think it is something which will be difficult to
achieve consistently across all languages. For example, in an
interpreted language, you could have errors due to problems with the
interpreter, you could have errors in the code or you could have a code
block which legitimately returns an error. Do you always want everything
to halt/stop on all errors? What about a code block which is used by
another code block which returns an error which the calling code block
actually uses to control it's behaviour? Then we have warnings. You want
warnings to act like errors, but I don't. For me, warnings and errors
have different semantics. I see a warning as something which alerts me
to something which might be a problem, but has not caused an error.
Provided you understand the warning, you should be free to ignore it. 

To me, babel is something which has evolved inside org mode rather than
something which has been designed and implemented. Based on a fairly
simple idea initially, it has grown to be powerful, but with some
confusing semantics, complex options, language inconsistencies and
missing features. This is not a criticism of the work done by
maintainers and contributors. Rather, it is simply the consequence of a
system which has evolved to meet user requirements.

I think a consistent approach to error handling in source blocks would
be an excellent addition. However, I think we also need to clarify the
syntax and semantics around code blocks. I still find the whole
semantics surrounding return values from code blocks somewhat confusing
- is what is written to stdout the return value or is it what the block
of code returns? What is the precise relationship between stdout and
stderr and results? What about differences between languages like a bash
script where both the return value and the output are important compared to a
language like clojure or common lisp where the return value is really
what matters or languages which don't return anything unless you
explicitly tell it to.

There was some work done with respect to return values only a few months
back and it has improved things. However, how this would all relate to
error handling is unclear. How will the different result options
interact with error handling? Is it sufficient to just halt on errors or
will this cause problems for some users? Do we need to distinguish
between different types of errors? How complex does an error handling
approach need to be? Can we really sustain yet more header arguments?
Are we yet in a position to seriously look at error handling or do we
need to improve consistency across languages and code blocks?

I apologise this is more question than answer. To be honest, while I
agree with your sentiments, I really am not sure what the right
answer/approach is. I do think it is an important question and I hope
others in the community will be prepared to assist, especially those
with more familiarity with the whole bable infrastructure. I think a lot
more discussion on this topic is required and hope you will be able to
participate if/when it occurs. 

regards,

Tim


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: bug: Error handling in source blocks.
  2021-08-10 20:30 ` Tim Cross
@ 2021-08-10 23:14   ` Tom Gillespie
  2021-08-11  6:23   ` tomas
  1 sibling, 0 replies; 5+ messages in thread
From: Tom Gillespie @ 2021-08-10 23:14 UTC (permalink / raw)
  To: Tim Cross; +Cc: emacs-orgmode

I will also chime in here to say that managing output streams and
errors for babel is a major new feature that I am interested in. The
issue, as Tim points out, is that there is a lot of complexity lurking
here due to the fact that certain languages have fundamentally
different capabilities and ways of handling or not handling errors,
and of running code (on arbitrary hosts) in the first place.

What works for one will almost certainly not work for another. Take
for example ob-lisp where there is already built in error handling in
emacs itself. Compare that with python where someone would likely need
to implement a special PYTHONBREAKPOINT entrypoint or something like
that, if it were possible at all.

I have had a draft of a document on what I called "babel
regularization" for well over a year now, but it is not in a state
that would be productive to share due to the sheer number of ob-langs
and systems affected and the need to be able to clearly catalog and
articulate the diversity of existing behaviors.

If you dig through old conversations on this list you will find a
discussion of the default behavior for ob-shell :returns values vs
output as the default, we were barely able to agree on which
principles should be followed to make the decision. In that case we
were lucky that there was already a way for users to set their desired
behavior in their init file or in a setup file or in the file itself.
How to handle errors will be much more complex, in part because it
will touch on what ob-lang implementations are able to overwrite
and/or must provide in order to actually function. At the moment there
are practically no constraints.

Lots of work to do here, so grateful for a report on the variability
in the behavior of the existing system.

Best!
Tom


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: bug: Error handling in source blocks.
  2021-08-10 20:30 ` Tim Cross
  2021-08-10 23:14   ` Tom Gillespie
@ 2021-08-11  6:23   ` tomas
  1 sibling, 0 replies; 5+ messages in thread
From: tomas @ 2021-08-11  6:23 UTC (permalink / raw)
  To: emacs-orgmode

[-- Attachment #1: Type: text/plain, Size: 984 bytes --]

On Wed, Aug 11, 2021 at 06:30:26AM +1000, Tim Cross wrote:

>                                      [...] For example, in an
> interpreted language, you could have errors due to problems with the
> interpreter, you could have errors in the code or you could have a code
> block which legitimately returns an error [...]

Yes, your error might be my nugget. For one illustrative example, I
once cobbled up a small shell tutorial. Babel seemed like the right
thing -- but I needed script snippets erroring out to show off.

Needless to say, Babel freaked out, not only on non-zero exit codes,
but also on any hint of activity on stderr.

I cheated myself out with preambles and postambles, which redirected
stderr to stdout and lied about exit codes.

This is, of course, horribly ugly, because it's language-specific
(you'd have to do it for C or Scheme or Perl or...), although the
concepts of exit code and stderr are OS wide. We could do better :)

Just one data point.

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2021-08-11  6:24 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-10  4:13 bug: Error handling in source blocks James Powell
2021-08-10 19:42 ` Berry, Charles via General discussions about Org-mode.
2021-08-10 20:30 ` Tim Cross
2021-08-10 23:14   ` Tom Gillespie
2021-08-11  6:23   ` tomas

Code repositories for project(s) associated with this inbox:

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).