From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:403:4789::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id EMtVEPLaN2VfPQEAauVa8A:P1 (envelope-from ) for ; Tue, 24 Oct 2023 16:55:46 +0200 Received: from aspmx1.migadu.com ([2001:41d0:403:4789::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id EMtVEPLaN2VfPQEAauVa8A (envelope-from ) for ; Tue, 24 Oct 2023 16:55:46 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id DFEEF6278C for ; Tue, 24 Oct 2023 16:55:44 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1698159346; a=rsa-sha256; cv=none; b=JD8Brm2QY+oQF8g1N79h7aRf1EDyQiXzWRcKsaSldJTqN0IwKauXCG7jczNnsNV2LwLRJG eE9S3r6VwOZXi+DDdSuSPWcSPi2L5hNnxlmflzCNbMY5EQh9fyFOoGZBBMyA2yAmlMcUlc /7nuOM30MXYhsSccFeZ/H2/9oVQQsuCSJa/yLWwitRsSo/pb4gBj67UpcavKTAAqESJsCU 3aolTMeHxmvkdzkqj3Wd/F3O9nT2oP0tD7FdsqkEHK7w0na1S+pRqfLxfrqSDIG1Zg24Cb 6+aox0HZoYen2qzyXeH3cggNslC4m+koFTy9s913za7Xr8+LnvVqXH0YGEc9+A== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1698159346; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=Ua4ScHi/Hph22MJ8FebmUV5waNxyulw1RAQRi/f1fMk=; b=j4kH9Vg+0PnEs4zQZKwWBwyDBPvFBKxvSDA1A+nIcsLesuegznapHDpgQHEcSUcew5GioC jFc8MXULmDkUQHz8yNI3tzF3caI709diVhFglSAo2+vLwalEvpvb6RYFvZXr+Q6iZqgeXU w0SP2Kaavhz0teFy3/YXvLNDiS5cUsWncNJ/daYNLP3t9TVhLuaPbiBolHPsAlsdjlhFlv NDf4ZYpKfG2Y4ZS54PLUXJfqdl9YEt8ne1zuBg5Ec+e//bdDDkIbk08HJFH/rKQK6JAs+S VkD/kaJ9vpcQHNl6sD9HACl1xuD2h/KXSta8RBGqO3LRNzbrboJdZAaPSG2l7w== Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qvIo5-0003Oj-TL; Tue, 24 Oct 2023 10:54:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qvIo3-0003OV-2X for emacs-orgmode@gnu.org; Tue, 24 Oct 2023 10:54:55 -0400 Received: from gavdos.tim-landscheidt.de ([2a01:4f8:1c0c:4bd6::1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qvIo0-00061N-L5 for emacs-orgmode@gnu.org; Tue, 24 Oct 2023 10:54:54 -0400 Received: from port-62-145-29-194.static.as20676.net ([62.145.29.194]:51790 helo=vagabond) by gavdos.tim-landscheidt.de with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1qvInx-003Iuk-31 for emacs-orgmode@gnu.org; Tue, 24 Oct 2023 14:54:50 +0000 From: Tim Landscheidt To: emacs-orgmode@gnu.org Subject: Best practice for writing and debugging shell source blocks? Organization: https://www.tim-landscheidt.de/ Date: Tue, 24 Oct 2023 14:54:48 +0000 Message-ID: <87pm14qddz.fsf@vagabond.tim-landscheidt.de> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2a01:4f8:1c0c:4bd6::1; envelope-from=tim@tim-landscheidt.de; helo=gavdos.tim-landscheidt.de X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Scanner: mx1.migadu.com X-Migadu-Spam-Score: -4.77 X-Spam-Score: -4.77 X-Migadu-Queue-Id: DFEEF6278C X-TUID: o6JzxJNnSQlK Hi, inspired by "Literate DevOps" (https://howardism.org/Technical/Emacs/literate-devops.html), I want to document some "stuff" in an Org file with the goal to be able to replay the steps done in the future so that I can either recreate the same "product" or variations there- of. My actual topic involves (inter alia) containers, but for simplicity, let us assume that I want to document the se- quence of shell commands: | $ mktemp | /tmp/tmp.gSffc4XtQ0 | $ echo a >> /tmp/tmp.gSffc4XtQ0 | $ echo b >> /tmp/tmp.gSffc4XtQ0 | $ fgrep -x a /tmp/tmp.gSffc4XtQ0 | a | $ In other words: An "entity" is created and its identifier shown, this entity is then referenced by this identifier, changed and queried. If I put the shell commands in one source block each: | #+NAME: create-temporary-file | #+BEGIN_SRC sh | mktemp | #+END_SRC | #+BEGIN_SRC sh :var tmpfile=3Dcreate-temporary-file | echo a >> $tmpfile | #+END_SRC | #+BEGIN_SRC sh :var tmpfile=3Dcreate-temporary-file | echo b >> $tmpfile | #+END_SRC | #+BEGIN_SRC sh :var tmpfile=3Dcreate-temporary-file | fgrep -x a $tmpfile | #+END_SRC and then evaluate the last source block, it calls the source block create-temporary-file, then searches the (empty) file and fails, i. e. the second and third source blocks are not evaluated at all. (Dealing with failing shell commands is another headache=E2=80=94thankfully, the trick at https://emacs.stackexchange.com/questions/59875/org-src-block-does-not-retu= rn-any-output (provide header arguments :prologue "exec 2>&1" and :epilogue ":") works reasonably well.) Adding more to the confusion, if I manually evaluate the first source block (create-temporary-file), it produces for example: | #+RESULTS: create-temporary-file | : /tmp/tmp.lOKZtyJ124 If I then evaluate the last source block (again), the first code block gets called, but its (different) result gets only passed to the last source block and used there, while the "#+RESULTS" section stays the same. The same thing happens if I add ":session my-test" to all source blocks, except that I get an additional buffer "my-test" with the content: | mktemp | sh-5.2$ echo 'org_babel_sh_eoe' | /tmp/tmp.e19GfJsH7w | sh-5.2$ org_babel_sh_eoe | sh-5.2$ tmpfile=3D'/tmp/tmp.e19GfJsH7w' | sh-5.2$ fgrep -x a $tmpfile | sh-5.2$ echo 'org_babel_sh_eoe' | org_babel_sh_eoe | sh-5.2$ where the "mixed" output of "mktemp" and "echo 'org_babel_sh_eoe'" does not necessarily instill confidence to run commands that are destructive in nature. So what is The Right Way=E2=84=A2 to document sequences of shell commands so that they are evaluated in sequence, being able to refer to the output of previous commands? Are there any obvious collections of examples that I missed? TIA, Tim