From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id AOumBB7xEmE/PwEAgWs5BA (envelope-from ) for ; Tue, 10 Aug 2021 23:35:26 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id iMk1AB7xEmE/HQAAB5/wlQ (envelope-from ) for ; Tue, 10 Aug 2021 21:35:26 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 3541D83A2 for ; Tue, 10 Aug 2021 23:35:25 +0200 (CEST) Received: from localhost ([::1]:41552 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mDZP8-0003Yh-KK for larch@yhetil.org; Tue, 10 Aug 2021 17:35:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55698) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mDZOZ-0003YZ-E1 for emacs-orgmode@gnu.org; Tue, 10 Aug 2021 17:34:47 -0400 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]:39729) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mDZOX-0001Dk-Lo for emacs-orgmode@gnu.org; Tue, 10 Aug 2021 17:34:47 -0400 Received: by mail-pj1-x102a.google.com with SMTP id u21-20020a17090a8915b02901782c36f543so6247306pjn.4 for ; Tue, 10 Aug 2021 14:34:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=references:user-agent:from:to:subject:date:in-reply-to:message-id :mime-version:content-transfer-encoding; bh=6VIoeYfyjEigsZzKyFWg0X0KD8uw3Gzgp/biPHtp/DU=; b=Nk39tvUvNy8exSxZGocEn7EAXBJCR3Yln51PHvmW5YOctE9UgHLLA/6IiS3vzHvKaP DQLnwLeMC41T1R+yJBnJv6NxQUwM5KFhRtneBLGDf1UyosmyRcKnb3b8J9NXpmWKYVaG lrjK8czfwT7lLXXqWf60Fz2eQi2LDVSR9aJYJSfiFMOxZj8XocEfvTLTEfRliQVuHpvK ogcvLTq44801KBSzZVRQLNF6mxDYJitJ/wAkx49iCvAzPWVmtgUIBmmkJJDw8htERx06 PrjXMnSPkVQXG4BZL9Wmj934JgeIS4C35iY4K8gTdH6Wy8g+D7uGnHKbP8dj/ZC13nKw 32bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:references:user-agent:from:to:subject:date :in-reply-to:message-id:mime-version:content-transfer-encoding; bh=6VIoeYfyjEigsZzKyFWg0X0KD8uw3Gzgp/biPHtp/DU=; b=tRsFFlOmx64iMvVQjxj0Ebbmzuz5QE9vxXvy/0pti2GRK7GS4sE2a989Ts/C2ARYYN GOYn/qEoW6mpMn3Kd/+mE9SlqH6118knxtEjKjUGt1eWlwZFLAtLF1+ke/uK8vjvs5jz 6hM93VKqeuKs4SKPEYF87oWlTSlgUnzaOY+ooZf3ciBNsWhocH3wqX/RvxR+z5kWT61c 7TToB4qqY0CpwHuPfChXvrnUMrFj38wA+Mmnykh2yJzYGZhOzxqF7XAIZnCF3vqAdTni Zw4CBv8VN0ietqoJxE/qbinqrPLTCdojjQ5CdWlN5WfjpprDgW8p4Edj1MLXPJn2pBkv H6pw== X-Gm-Message-State: AOAM531PtgZDhr5/Hd/kX1UxHEBSjlqBGKUokGnDX+6pIYYuexT33iSd kXm5Hxe6QgyeSmq1SpGKgU4Reuxxxrc= X-Google-Smtp-Source: ABdhPJyX06uSHbTlrElhh6koU1/9xTy/1XxcChj8F5bIidFqbOPHFuKj0UMxZbrxAVu9AJT7LffV4A== X-Received: by 2002:a63:ec45:: with SMTP id r5mr78378pgj.440.1628631283836; Tue, 10 Aug 2021 14:34:43 -0700 (PDT) Received: from tim-desktop (106-69-100-43.dyn.iinet.net.au. [106.69.100.43]) by smtp.gmail.com with ESMTPSA id i14sm28732229pgh.79.2021.08.10.14.34.42 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Aug 2021 14:34:43 -0700 (PDT) References: <3ac06f2a-c14b-593f-a6e2-8e1c85c19436@pdx.edu> User-agent: mu4e 1.6.2; emacs 28.0.50 From: Tim Cross To: emacs-orgmode@gnu.org Subject: Re: bug: Error handling in source blocks. Date: Wed, 11 Aug 2021 06:30:26 +1000 In-reply-to: <3ac06f2a-c14b-593f-a6e2-8e1c85c19436@pdx.edu> Message-ID: <87czqlf0gg.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2607:f8b0:4864:20::102a; envelope-from=theophilusx@gmail.com; helo=mail-pj1-x102a.google.com X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, LOTS_OF_MONEY=0.001, MONEY_NOHTML=0.826, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1628631325; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=6VIoeYfyjEigsZzKyFWg0X0KD8uw3Gzgp/biPHtp/DU=; b=sTzsPGV3JbaETe4GL/36A4mK/nuXhX0UIiwZznFYdBnlNmvosci0E2fc/A1GnlVsOF4+h7 zWrM9B3voqIjSnoy1JyxKRUf8AmmIm1wSislVlqPWo8g+PFcBsvi/KjwHFsvivo1NDEGBh 8ArMAZBgrZg3DjPfgzVkgASsLHkWSG0auFdJxV4gQ7aSyWtfszEMxrNnR7WVYRKxzwFwML t6GaW+uSeYDL/urQccVb1ymvZCw6QhlEGodjJFlfn1mX4En5Fccsu6tPZp22Pav1UsEFqs yZgbKFHqW+fQY5VBscJk8I+pfemXM/gnhgXeyt/9pxlBCuv/12hJruUGs62NGg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1628631325; a=rsa-sha256; cv=none; b=XogslSv0HwcAkUeikkDzzRRVcYWgz4FlBWGYmkyi+0FV8neHJySPUJ6rfLGNIbEp6ekqzb dpS+3MVJDFyHz3rVjlVYtwcfGn7ioHUicIfo79IB2lHP6Gp9WLmRVXTALEqdEgiPqOWbDQ LGxrSl2EYbxyZoJmgUCi5KeOE6kcqTAtROtmIrNsRdcXk0Zl4QXRGh7GUnggYSWe9JKmCd ouQLxgFQ7UIkzr/h8Psa5uQc4aTqEfj0daaw9qqx2FzdwUys+lJ61kiTSWmp7MrUCfnDWr gzJe1ajGYWzMsFQXwfP1O8DLCsKGJV0Z5MulMePEM/qNgjdr9DcXY5584E5f9Q== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=Nk39tvUv; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Spam-Score: -3.11 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=Nk39tvUv; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Queue-Id: 3541D83A2 X-Spam-Score: -3.11 X-Migadu-Scanner: scn1.migadu.com X-TUID: HGpd9KDERzD+ James Powell writes: > =C2=A0 Error handling is important and hard to get right.=C2=A0 Me, I pre= fer to > =C2=A0 treat every warning as an error (-Werror in gcc, "options(warn=3D2= )" in > =C2=A0 R, etc).=C2=A0 I want the system to grind to a halt at the least s= ign of > =C2=A0 trouble. > > =C2=A0 When I write some nonsense into a code block as in this example: > > =C2=A0 ,---- > =C2=A0 | : This next code block is intended to trigger an error, because = that > =C2=A0 | : variable "fffff838293483" with that attribute/member "x8483848" > =C2=A0 | : probably does not exist. > =C2=A0 | : > =C2=A0 | : #+begin_src R :exports both > =C2=A0 | : x <- fffff838293483$x8483848 > =C2=A0 | : #+end_src > =C2=A0 | : > =C2=A0 | : #+RESULTS: > =C2=A0 | : > =C2=A0 | : #+begin_src R :exports both :results table :colnames yes > =C2=A0 | : require(tidyverse) > =C2=A0 | : tribble(~a, ~b, 1, 2) > =C2=A0 | : #+end_src > =C2=A0 | : > =C2=A0 | : #+RESULTS: > =C2=A0 | : | a | b | > =C2=A0 | : |---+---| > =C2=A0 | : | 1 | 2 | > =C2=A0 | : #+end_src > =C2=A0 `---- > > > =C2=A0 which does trigger an error in R > =C2=A0 ,---- > =C2=A0 | Error: object 'fffff838293483' not found > =C2=A0 | [traceback follows] > =C2=A0 `---- > > > =C2=A0 and I do org-to-PDF export, I would like the PDF to be not built a= nd > =C2=A0 loud alarm bells to ring bringing the error to my attention. > > =C2=A0 What happens instead: the document builds from org into PDF with no > =C2=A0 indication that anything went wrong at all, except perhaps minor b= its > =C2=A0 of missing or incorrect text in the PDF file.=C2=A0 The error can = easily be > =C2=A0 buried in the output as more and later source blocks from the same > =C2=A0 document are evaluated. > > =C2=A0 Likewise, in 'sh', > =C2=A0 ,---- > =C2=A0 | #+begin_src sh :exports results > =C2=A0 | exit 1 > =C2=A0 | #+end_src > =C2=A0 `---- > > > =C2=A0 Sh exits 1 to indicate an error, so I would like this to ring alarm > =C2=A0 bells and fail to produce a PDF on org-to-PDF export. > > =C2=A0 We might add examples in Java, Python, C++, C along the same lines. > =C2=A0 All of these should be in a unit test (because error handling is i= mportant). > =C2=A0 There do not seem to be any unit tests that exercise error handlin= g in > =C2=A0 org-9.4.4. > > =C2=A0 Exceptions that are thrown in the block should also at least > =C2=A0 optionally always and reliably abort the org-to-PDX export and ring > =C2=A0 alarm bells. > > =C2=A0 I would like to see a section in the org manual called "Error hand= ling > =C2=A0 in source blocks" that discusses the issue. > > =C2=A0 I searched for "error handling" at , > =C2=A0 with no results that are relevant. > > =C2=A0 thank you for your time, > =C2=A0 - JP Hi James, I wanted to reply to your message mainly to let you know it hasn't been ignored. However, I think you have only uncovered a tip of a very large iceberg and dealing with it will be non-trivial.=20 I pretty much agree with most of what you wrote. However, I'm not sure I'd call this a bug. Rather, I would argue this is a request for a feature enhancement. For me, a bug is something which does not behave or work as intended. I'm not sure anyone has really considered a consistent approach to error handling. It is something which I think would be very beneficial, but I also think it is something which will be difficult to achieve consistently across all languages. For example, in an interpreted language, you could have errors due to problems with the interpreter, you could have errors in the code or you could have a code block which legitimately returns an error. Do you always want everything to halt/stop on all errors? What about a code block which is used by another code block which returns an error which the calling code block actually uses to control it's behaviour? Then we have warnings. You want warnings to act like errors, but I don't. For me, warnings and errors have different semantics. I see a warning as something which alerts me to something which might be a problem, but has not caused an error. Provided you understand the warning, you should be free to ignore it.=20 To me, babel is something which has evolved inside org mode rather than something which has been designed and implemented. Based on a fairly simple idea initially, it has grown to be powerful, but with some confusing semantics, complex options, language inconsistencies and missing features. This is not a criticism of the work done by maintainers and contributors. Rather, it is simply the consequence of a system which has evolved to meet user requirements. I think a consistent approach to error handling in source blocks would be an excellent addition. However, I think we also need to clarify the syntax and semantics around code blocks. I still find the whole semantics surrounding return values from code blocks somewhat confusing - is what is written to stdout the return value or is it what the block of code returns? What is the precise relationship between stdout and stderr and results? What about differences between languages like a bash script where both the return value and the output are important compared to= a language like clojure or common lisp where the return value is really what matters or languages which don't return anything unless you explicitly tell it to. There was some work done with respect to return values only a few months back and it has improved things. However, how this would all relate to error handling is unclear. How will the different result options interact with error handling? Is it sufficient to just halt on errors or will this cause problems for some users? Do we need to distinguish between different types of errors? How complex does an error handling approach need to be? Can we really sustain yet more header arguments? Are we yet in a position to seriously look at error handling or do we need to improve consistency across languages and code blocks? I apologise this is more question than answer. To be honest, while I agree with your sentiments, I really am not sure what the right answer/approach is. I do think it is an important question and I hope others in the community will be prepared to assist, especially those with more familiarity with the whole bable infrastructure. I think a lot more discussion on this topic is required and hope you will be able to participate if/when it occurs.=20 regards, Tim