From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 8NuCCPHYo2A4VgAAgWs5BA (envelope-from ) for ; Tue, 18 May 2021 17:10:41 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id WJaVA/HYo2BSWAAA1q6Kng (envelope-from ) for ; Tue, 18 May 2021 15:10:41 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 0D09714A8A for ; Tue, 18 May 2021 17:10:40 +0200 (CEST) Received: from localhost ([::1]:43024 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lj1Mk-0000Yk-11 for larch@yhetil.org; Tue, 18 May 2021 11:10:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58886) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lj1GC-0001L4-6f for emacs-orgmode@gnu.org; Tue, 18 May 2021 11:03:52 -0400 Received: from mail-wr1-x429.google.com ([2a00:1450:4864:20::429]:41651) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lj1G9-00061C-6d for emacs-orgmode@gnu.org; Tue, 18 May 2021 11:03:51 -0400 Received: by mail-wr1-x429.google.com with SMTP id d11so10594350wrw.8 for ; Tue, 18 May 2021 08:03:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=andrew-cmu-edu.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=+1/wfmSSd53KAt+YCdcT1ce6VukoILqr95d3l0IALnE=; b=X0EYmX6hDsua8Mxf45HYraF/2qrdXuDmp9n/WdzmJSZmvRjyQc+I6sWYiODpxRDURH 2gk0T7l11NpU5qzpVugw1PZ19wlOyZprj16l82XjYysps7LpjDDfBw/ao8JWsUfaa3dw 3gRiAn9dANRCqwoO1W3jbiyVCrfiALSqS6fuFOmMPoEDIpYu2iDxitUXLUOzHsOc5gb+ WHAbo00yp1/jV5BxbQkT80ahxsWDaquIqjK4GW2+UjSZBJwil2aqTZCt2V/BhHBCKPJi ilaxtm/AHym9IdrnImeuvFjgPbWVuIEfaRVvD4Zq88tLahxdy4OxKuFZQ0bWqTiaW4t5 q1Kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=+1/wfmSSd53KAt+YCdcT1ce6VukoILqr95d3l0IALnE=; b=kRHx391ufk3Iznk73rRcRvC8KCH8Q7NBDVQ5oWCrxlCXuv+2KW25CbwnOnedxzPJyr +lkdcetx5JMDP7gZ9YTNr4salQbDhcNJrjiA+rsjuIw05fx8thXqaN0YbNxdDlarJZ6I JOe+uFG5+5JBGpA7BjlQvZGxTla6XkFNalNwrL4ov8dVoDiDkya7BZICiAHJrmS6lAPS GGypwFAortg8QviqTVoS6RTTZ8A0jLbM5u2xtlR/g9qEQL2vrnT/WSUbt9Ma3G4tiXHB gxc0wJ/sdXEowRqSXop5BzRfRhaJCMHLFIxaO5MSHEorZbAEGHyMBgoCyoLrWCV90Xoo C8gQ== X-Gm-Message-State: AOAM532OBSX7SJpT57pfAqDX7l5tlDhfPgjheGhQWN5mICl7NQ1v3unk DLLY3+9cyhdxSsow7aOaVLWDYtxEsIdPjJLxoXA= X-Google-Smtp-Source: ABdhPJz/6WjCtL4IMw1GmD6vxpUGxs8a8k27KAilVcTgyfrhvDjnwKDmD1eCXbVynwldHGfllr+Y68WaPOjvjfpt7yI= X-Received: by 2002:adf:f3cb:: with SMTP id g11mr7577284wrp.75.1621350226792; Tue, 18 May 2021 08:03:46 -0700 (PDT) MIME-Version: 1.0 References: <56b14034-f6c4-0528-9235-e0fb9f3f1f28@karssen.org> In-Reply-To: <56b14034-f6c4-0528-9235-e0fb9f3f1f28@karssen.org> From: John Kitchin Date: Tue, 18 May 2021 11:03:35 -0400 Message-ID: Subject: Re: Global variables in Org mode document with source blocks To: "Lennart C. Karssen" Content-Type: multipart/alternative; boundary="00000000000053127b05c29c047b" Received-SPF: pass client-ip=2a00:1450:4864:20::429; envelope-from=johnrkitchin@gmail.com; helo=mail-wr1-x429.google.com X-Spam_score_int: -13 X-Spam_score: -1.4 X-Spam_bar: - X-Spam_report: (-1.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.25, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: org-mode-email Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1621350640; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=+1/wfmSSd53KAt+YCdcT1ce6VukoILqr95d3l0IALnE=; b=MrNztQ5hYRpivNZLwupnVHzXUzxztIngVZCW3JRiIZWen1CJBYy8OqPDNhZb8MFsyy09Jo jD0C2v3JNgidwKYH2NdR18LAg/c9pLqDSBAiIKZjZZnOggwL4cKtiLjjv8nVnraTqxdMOn UlCqie4mbRu56zT93+XjIf1fzwlNVRCWvTjv37E7DgSGefF56xz3J7ZNsvuhYgHyLqlT3H LcHnNCWosHlUetLvm66oskg8C3R4zk1VFzs7rOUhc3ceONCjCjMtrVpLyVIAyPFKdMdHjJ vHGFfPbneS53L+kaPyfUimH8xOoWu4csp1BHYg1DxKQfI7GhK1ziyJLO1f2ysA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1621350640; a=rsa-sha256; cv=none; b=bUwDZQ4wWk0nBlzIAB0SeUEJesxpvG+i8+DG4SUiG6g27fcjapb+JjRfpbF9MHuJb13jOl MGxlcu98iWZC75lOMZO0wmzg0gljJlUiaR0Mw1gfQ3sp3Q8ZLhKsHuifgJw1xW9Arxc3o+ DqO92OTPDvKjjTXDzmA7Rg8TpNXQAEcFUQMSkwd90SFx7uXde/KESwnrUKnOk9BGDQclwv uvobPfapn4YlZvt17PQKKO5RYNBUa8jNyrn8LPswQP/Ei3DOIWbIqQWkhnUu1mjgf9wDXK PwAfCinjwD3rGJmTxz4H7ZWA1r4rsyhMZs/fm4IEzbVFNrRwKt/sjR52oO28eA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=andrew-cmu-edu.20150623.gappssmtp.com header.s=20150623 header.b=X0EYmX6h; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Spam-Score: -3.19 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=andrew-cmu-edu.20150623.gappssmtp.com header.s=20150623 header.b=X0EYmX6h; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=andrew.cmu.edu (policy=none); spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Queue-Id: 0D09714A8A X-Spam-Score: -3.19 X-Migadu-Scanner: scn0.migadu.com X-TUID: V+MrLOCjq6U9 --00000000000053127b05c29c047b Content-Type: text/plain; charset="UTF-8" Given all the different languages involved, I don't think there is a way to use a common variable. One way might be to have each block output some kind of string if it fails, and then in the last block you could search for the buffer for that string. Something like this: * Section 1 #+BEGIN_SRC sh false || echo "failed"-`date +'%s'` #+END_SRC #+RESULTS: : failed-1621348872 #+BEGIN_SRC python import time if not False: print(f'failed-{time.time()}') #+END_SRC #+RESULTS: : failed-1621348926.125608 * Final section #+BEGIN_SRC emacs-lisp (format "There were %s failed blocks" (count-matches "failed-[0-9]" (point-min) (point-max))) #+END_SRC #+RESULTS: : There were 2 failed blocks Some alternatives include writing/appending to a file on error, and then counting the number of lines. Another route is to use a :post header and search for the string there. #+BEGIN_SRC emacs-lisp (setq n-failures 0) #+END_SRC #+RESULTS: : 0 #+name: fail-capture #+BEGIN_SRC emacs-lisp :var data="" (when (string-match "failed-[0-9]" data) (incf n-failures) (message "captured a failure in %s. n-failures=%s" data n-failures)) #+END_SRC #+BEGIN_SRC sh :post fail-capture(*this*) false || echo "failed"-`date +'%s'` #+END_SRC #+RESULTS: : captured a failure in failed-1621349398. n-failures=1 #+BEGIN_SRC python :post fail-capture(*this*) print('This did not fail') #+END_SRC #+RESULTS: : nil #+BEGIN_SRC python :post fail-capture(*this*) print('This failed-9') #+END_SRC #+RESULTS: : captured a failure in This failed-9 : . n-failures=2 All these approaches need you to handle and catch the errors. I think other kinds of errors would stop the export process. e.g. if a block errors out from division by zero or something. I don't know how easy it would be to check if a src block succeeded or not. If it was easy, you might use the org-babel-after-execute-hook variable to update something when a failure is detected. Some blocks create an *Org-Babel Error Output buffer somewhere, but i don't know if this is 100% reliable across all blocks. John ----------------------------------- Professor John Kitchin (he/him/his) Doherty Hall A207F Department of Chemical Engineering Carnegie Mellon University Pittsburgh, PA 15213 412-268-7803 @johnkitchin http://kitchingroup.cheme.cmu.edu On Tue, May 18, 2021 at 9:58 AM Lennart C. Karssen wrote: > Dear list, > > I am working on a dynamic report in Org mode, where I use source blocks > in various languages to process data. Several blocks produce text or > tables that become part of the PDF on export. > > The final chapter should state whether all checks passed, or whether one > or more failed (it is not necessary to know which step failed). > > In a single language environment, I would use a variable (called e.g. > nrChecksFailed) that would be incremented for each failing check. In a > single language Org document this could probably be done with a > :session, but given that I mix Awk, Bash, Emacs lisp and R that doesn't > look like the way to go. Do Org documents/source blocks have some > concept of a (global) variable that I can pass to my SRC blocks and > increment inside them? > > E.g. after somehow initialising nrChecksFailed = 0, I would like to do: > > #+header :var nFailed=nrChecksFailed :var someData=someData > #+begin_src R :results raw > do_some_check_here_on_someData > > if (check_results_OK) { > cat("check A passed\n") > } else { > cat("check A *failed*\n") > nFailed <- nFailed + 1 > } > #+end_src > > So that in my conclusion chapter I can do for example: > > #+header: :var nFailed=nrChecksFailed > #+begin_src bash :results raw > if [[ nFailed -eq 0 ]]; then > echo "All checks passed > else > echo "One or more checks *failed!*" > fi > #+end_src > > > Best regards, > > Lennart. > > -- > *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-* > L.C. Karssen > The Netherlands > > lennart@karssen.org > http://blog.karssen.org > GPG key ID: A88F554A > -*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- > > --00000000000053127b05c29c047b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Given all the different languages involved, I don't th= ink there is a way to use a common variable.=C2=A0

One w= ay might be to have each block output some kind of string if it fails, and = then in the last block you could search for the buffer for that string. Som= ething=C2=A0like this:


* Section 1

#+BE= GIN_SRC sh
false || echo "failed"-`date +'%s'`
#+EN= D_SRC

#+RESULTS:
: failed-1621348872


#+BEGIN_SRC pytho= n
import time

if not False:
=C2=A0 =C2=A0 print(f'failed-{= time.time()}')
#+END_SRC

#+RESULTS:
: failed-1621348926.12= 5608


* Final section

#+BEGIN_SRC emacs-lisp
(format &q= uot;There were %s failed blocks" (count-matches "failed-[0-9]&quo= t; (point-min) (point-max)))
#+END_SRC

#+RESULTS:
: There were= 2 failed blocks

Some alternatives include writing/a= ppending to a file on error, and then counting the number of lines.

Another route is to use a :post header and search for the= string there.

#+BEGIN_SRC emacs-lisp
(setq n-f= ailures 0)
#+END_SRC

#+RESULTS:
: 0

#+name: fail-captur= e
#+BEGIN_SRC emacs-lisp :var data=3D""
(when (string-match= "failed-[0-9]" data)
=C2=A0 (incf n-failures)
=C2=A0 (mess= age "captured a failure in %s. n-failures=3D%s" data n-failures))=
#+END_SRC

#+BEGIN_SRC sh :post fail-capture(*this*)
false || = echo "failed"-`date +'%s'`
#+END_SRC

#+RESULTS:=
: captured a failure in failed-1621349398. n-failures=3D1


#+= BEGIN_SRC python :post fail-capture(*this*)
print('This did not fail= ')
#+END_SRC

#+RESULTS:
: nil

#+BEGIN_SRC python :p= ost fail-capture(*this*)
print('This failed-9')
#+END_SRC
=
#+RESULTS:
: captured a failure in This failed-9
: . n-failures= =3D2

All these approaches need you to handle a= nd catch the errors. I think other kinds of errors would stop the export pr= ocess. e.g. if a block errors out from division by zero or something.
=

I don't know how easy it would be to check if a src= block succeeded or not. If it was easy, you might use the=C2=A0org-babel-a= fter-execute-hook variable to update something when a failure is detected. = Some blocks create an *Org-Babel Error Output buffer somewhere, but i don&#= 39;t know if this is 100% reliable across all blocks.

John

-------= ----------------------------
Professor John Kitchin (he/him/his)
Dohe= rty Hall A207F
Department of Chemical Engineering
Carnegie Mellon Uni= versity
Pittsburgh, PA 15213
412-268-7803
=

On Tue, May 18, 2021 at 9:58 AM Lennart C. Karssen <lennart@karssen.org> wrote:
Dear list,

I am working on a dynamic report in Org mode, where I use source blocks
in various languages to process data. Several blocks produce text or
tables that become part of the PDF on export.

The final chapter should state whether all checks passed, or whether one or more failed (it is not necessary to know which step failed).

In a single language environment, I would use a variable (called e.g.
nrChecksFailed) that would be incremented for each failing check. In a
single language Org document this could probably be done with a
:session, but given that I mix Awk, Bash, Emacs lisp and R that doesn't=
look like the way to go. Do Org documents/source blocks have some
concept of a (global) variable that I can pass to my SRC blocks and
increment inside them?

E.g. after somehow initialising nrChecksFailed =3D 0, I would like to do:
#+header :var nFailed=3DnrChecksFailed :var someData=3DsomeData
#+begin_src R :results raw
do_some_check_here_on_someData

if (check_results_OK) {
=C2=A0 cat("check A passed\n")
} else {
=C2=A0 cat("check A *failed*\n")
=C2=A0 nFailed <- nFailed + 1
}
#+end_src

So that in my conclusion chapter I can do for example:

#+header: :var nFailed=3DnrChecksFailed
#+begin_src bash=C2=A0 :results raw
if [[ nFailed -eq 0 ]]; then
=C2=A0 echo "All checks passed
else
=C2=A0echo "One or more checks *failed!*"
fi
#+end_src


Best regards,

Lennart.

--
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
L.C. Karssen
The Netherlands

lennart@karssen.or= g
ht= tp://blog.karssen.org
GPG key ID: A88F554A
-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

--00000000000053127b05c29c047b--