Re: ob-shell intentions and paperwork (was Bash results broken?)

[Matt <matt@excalamus.com>]

[-- Attachment #1: Type: text/plain, Size: 1082 bytes --]


 ---- On Sat, 31 Dec 2022 09:31:16 -0500  Ihor Radchenko  wrote --- 
 > Matt matt@excalamus.com> writes:
 > 
 > > I've backed out the `require' change and adjusted everything else based on your feedback.  There is a separate patch for each refactor that created a new test.   The remaining refactors are in a single patch.  I was also able to resolve the issue I had with inserting the test name for the session (the "yes" string in `test-ob-shell/session').
 > 
 > Thanks!
 > 
 > I can see that you still prefer to use `string-join' for Org body.
 > Why not a bare string?

Because I often program in Python where double quoted strings typically don't span lines.  One of the original tests had the string "#+BEGIN_SRC sh :results list\necho 1\necho 2\necho 3\n#+END_SRC" in it (that is, used newline characters instead of inserting new lines), so my brain assumed strings Emacs Lisp behaved similarly to Python.  Silly brain :)  Thanks for continuing to follow up on that.

I've updated the patches to use bare strings.

If this set of patches look good, I can push them to main.

[-- Attachment #2: 0001-test-ob-shell.el-Split-test-ob-shell-dont-error-on-e.patch --]
[-- Type: application/octet-stream, Size: 1899 bytes --]

From e204c3a6ccdaf1770db5e931f1116235597a8c0b Mon Sep 17 00:00:00 2001
From: Matt Trzcinski <matt@excalamus.com>
Date: Fri, 30 Dec 2022 11:18:15 -0500
Subject: [PATCH 1/7] test-ob-shell.el: Split
 test-ob-shell/dont-error-on-empty-results

* testing/lisp/test-ob-shell.el
(test-ob-shell/dont-error-on-empty-results): Explicitly test handling
of empty results.
(test-ob-shell/dont-error-on-babel-error): Explicitly test handling of
Babel errors.  On pass, remove buffers created during test.

Original test conflated empty results (a valid Babel return) and Babel
errors.
---
 testing/lisp/test-ob-shell.el | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el
index b0d9beff4..fc3ecd3c8 100644
--- a/testing/lisp/test-ob-shell.el
+++ b/testing/lisp/test-ob-shell.el
@@ -34,12 +34,16 @@ unless the body of the tangled block does."
   (should-not (string-match "^[\n\r][\t ]*[\n\r]"
 			    (org-babel-expand-body:generic "echo 2" '())))
   (should (string-match "^[\n\r][\t ]*[\n\r]"
-			(org-babel-expand-body:generic "\n\necho 2" '()))))
+                        (org-babel-expand-body:generic "\n\necho 2" '()))))
 
 (ert-deftest test-ob-shell/dont-error-on-empty-results ()
-  "Was throwing an elisp error when shell blocks threw errors and
-returned empty results."
-  (should (null (org-babel-execute:sh "ls NoSuchFileOrDirectory.txt" nil))))
+  "Empty results should not cause a Lisp error."
+  (should (null (org-babel-execute:sh "" nil))))
+
+(ert-deftest test-ob-shell/dont-error-on-babel-error ()
+  "Errors within Babel execution should not cause Lisp errors."
+  (if (should (null (org-babel-execute:sh "ls NoSuchFileOrDirectory.txt" nil)))
+      (kill-buffer "*Org-Babel Error Output*")))
 
 (ert-deftest test-ob-shell/session ()
   "This also tests `org-babel-comint-with-output' in
-- 
2.38.1


[-- Attachment #3: 0002-test-ob-shell.el-Refactor-test-ob-shell-session.patch --]
[-- Type: application/octet-stream, Size: 2607 bytes --]

From 6e0012867d8594913cf4783d7afdc109b8307de3 Mon Sep 17 00:00:00 2001
From: Matt Trzcinski <matt@excalamus.com>
Date: Fri, 30 Dec 2022 11:35:10 -0500
Subject: [PATCH 2/7] test-ob-shell.el: Refactor test-ob-shell/session

* testing/lisp/test-ob-shell.el (ob-shell/session): Split
`ob-shell/session' into
`test-ob-shell/session-single-return-returns-string' and
`test-ob-shell/session-multiple-returns-returns-list'.
(ob-shell/session): Rename comint from to test name. On
pass, kill process and remove process buffer.
---
 testing/lisp/test-ob-shell.el | 32 +++++++++++++++++++++-----------
 1 file changed, 21 insertions(+), 11 deletions(-)

diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el
index fc3ecd3c8..b08bf8413 100644
--- a/testing/lisp/test-ob-shell.el
+++ b/testing/lisp/test-ob-shell.el
@@ -45,18 +45,28 @@ unless the body of the tangled block does."
   (if (should (null (org-babel-execute:sh "ls NoSuchFileOrDirectory.txt" nil)))
       (kill-buffer "*Org-Babel Error Output*")))
 
-(ert-deftest test-ob-shell/session ()
-  "This also tests `org-babel-comint-with-output' in
-ob-comint.el, which was not previously tested."
-  (let ((res (org-babel-execute:sh "echo 1; echo 2" '((:session . "yes")))))
-    (should res)
-    (should (listp res)))
-  ;; Test multi-line input.
-  (let ((result (org-babel-execute:sh
-		 "if true \n then \n echo yes \n fi"
-		 '((:session . "yes")))))
+(ert-deftest test-ob-shell/session-single-return-returns-string ()
+  "Sessions with a single result should return a string."
+  (let* ((session-name "test-ob-shell/session-evaluation-single-return-returns-string")
+         (kill-buffer-query-functions nil)
+         (result (org-babel-execute:sh
+                  (format "echo %s" session-name)
+                  `((:session . ,session-name)))))
     (should result)
-    (should (string= "yes" result))))
+    (if (should (string= session-name result))
+        (kill-buffer session-name))))
+
+(ert-deftest test-ob-shell/session-multiple-returns-returns-list ()
+  "Sessions with multiple results should return a list."
+  (let* ((session-name "test-ob-shell/session-multiple-returns-returns-list")
+         (kill-buffer-query-functions nil)
+         (result (org-babel-execute:sh
+                  "echo 1; echo 2"
+                  `((:session . ,session-name)))))
+    (should result)
+    (should (listp result))
+    (if (should (equal '((1) (2)) result))
+        (kill-buffer session-name))))
 
 ; A list of tests using the samples in ob-shell-test.org
 (ert-deftest ob-shell/generic-uses-no-arrays ()
-- 
2.38.1


[-- Attachment #4: 0003-test-ob-shell.el-Refactor-ob-shell-generic-uses-no-a.patch --]
[-- Type: application/octet-stream, Size: 3342 bytes --]

From 728e7fca81201993a1a411980b4ec8210d607af9 Mon Sep 17 00:00:00 2001
From: Matt Trzcinski <matt@excalamus.com>
Date: Fri, 30 Dec 2022 11:39:35 -0500
Subject: [PATCH 3/7] test-ob-shell.el: Refactor
 ob-shell/generic-uses-no-assoc-arrays

* test-ob-shell.el (ob-shell/generic-uses-no-assoc-arrays): Split test
based on simple versus complex mapping cases,
`test-ob-shell/generic-uses-no-assoc-arrays-simple-map' and
`test-ob-shell/generic-uses-no-assoc-arrays-3-columns'.  Rename prefix
from "ob-shell" to "test-ob-shell".
(ob-shell/generic-uses-no-assoc-arrays): Move
headlines and commentary from `testing/examples/ob-shell-test.org' to
docstring.
(ob-shell/generic-uses-no-assoc-arrays): Remove dependency on
testing/examples/ob-shell-test.org.
---
 testing/lisp/test-ob-shell.el | 52 +++++++++++++++++++++++++----------
 1 file changed, 37 insertions(+), 15 deletions(-)

diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el
index b08bf8413..a39c99d9f 100644
--- a/testing/lisp/test-ob-shell.el
+++ b/testing/lisp/test-ob-shell.el
@@ -68,7 +68,6 @@ unless the body of the tangled block does."
     (if (should (equal '((1) (2)) result))
         (kill-buffer session-name))))
 
-; A list of tests using the samples in ob-shell-test.org
 (ert-deftest ob-shell/generic-uses-no-arrays ()
   "No arrays for generic"
   (org-test-at-id "0ba56632-8dc1-405c-a083-c204bae477cf"
@@ -81,20 +80,43 @@ unless the body of the tangled block does."
     (org-babel-next-src-block 2)
     (should (equal "one" (org-trim (org-babel-execute-src-block))))))
 
-(ert-deftest ob-shell/generic-uses-no-assoc-arrays ()
-  "No associative arrays for generic"
-  (should
-   (equal "first one second two third three"
-	  (org-test-at-id
-	   "bec1a5b0-4619-4450-a8c0-2a746b44bf8d"
-	   (org-babel-next-src-block)
-	   (org-trim (org-babel-execute-src-block)))))
-  (should
-   (equal "bread 2 kg spaghetti 20 cm milk 50 dl"
-	  (org-test-at-id
-	   "82320a48-3409-49d7-85c9-5de1c6d3ff87"
-	   (org-babel-next-src-block)
-	   (org-trim (org-babel-execute-src-block))))))
+(ert-deftest test-ob-shell/generic-uses-no-assoc-arrays-simple-map ()
+  "Generic shell: no special handing for key-value mapping table
+
+No associative arrays for generic.  The shell will see all values
+as a single string."
+  (org-test-with-temp-text
+      "#+NAME: sample_mapping_table
+| first  | one   |
+| second | two   |
+| third  | three |
+
+#+begin_src sh :exports results :results output :var table=sample_mapping_table
+echo ${table}
+<point>
+#+end_src"
+    (should
+     (equal "first one second two third three"
+            (org-trim (org-babel-execute-src-block))))))
+
+(ert-deftest test-ob-shell/generic-uses-no-assoc-arrays-3-columns ()
+  "Associative array tests (more than 2 columns)
+
+No associative arrays for generic.  The shell will see all values
+as a single string."
+  (org-test-with-temp-text
+      "#+NAME: sample_big_table
+| bread     |  2 | kg |
+| spaghetti | 20 | cm |
+| milk      | 50 | dl |
+
+#+begin_src sh :exports results :results output :var table=sample_big_table
+echo ${table}
+<point>
+#+end_src"
+    (should
+     (equal "bread 2 kg spaghetti 20 cm milk 50 dl"
+            (org-trim (org-babel-execute-src-block))))))
 
 (ert-deftest ob-shell/bash-uses-assoc-arrays ()
   "Bash associative arrays"
-- 
2.38.1


[-- Attachment #5: 0004-Refactor-ob-shell-bash-uses-assoc-arrays.patch --]
[-- Type: application/octet-stream, Size: 2885 bytes --]

From 2a0f5a5153546599bd673f7724250a921896c4ca Mon Sep 17 00:00:00 2001
From: Matt Trzcinski <matt@excalamus.com>
Date: Sat, 31 Dec 2022 12:56:51 -0500
Subject: [PATCH 4/7] Refactor ob-shell/bash-uses-assoc-arrays

* testing/lisp/test-ob-shell.el (ob-shell/bash-uses-assoc-arrays):
Split test cases into separate tests,
`test-ob-shell/bash-uses-assoc-arrays' and
`test-ob-shell/bash-uses-assoc-arrays-with-lists'.  Rename prefix from
"ob-shell" to "test-ob-shell".
(ob-shell/bash-uses-assoc-arrays): Move comments from
testing/examples/ob-shell-test.org to docstrings.
(ob-shell/bash-uses-assoc-arrays): Remove dependency on
testing/examples/ob-shell-test.org.
---
 testing/lisp/test-ob-shell.el | 53 +++++++++++++++++++++++++----------
 1 file changed, 38 insertions(+), 15 deletions(-)

diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el
index a39c99d9f..78267880b 100644
--- a/testing/lisp/test-ob-shell.el
+++ b/testing/lisp/test-ob-shell.el
@@ -118,21 +118,44 @@ echo ${table}
      (equal "bread 2 kg spaghetti 20 cm milk 50 dl"
             (org-trim (org-babel-execute-src-block))))))
 
-(ert-deftest ob-shell/bash-uses-assoc-arrays ()
-  "Bash associative arrays"
-  (should
-   (equal "two"
-	  (org-test-at-id
-	   "bec1a5b0-4619-4450-a8c0-2a746b44bf8d"
-	   (org-babel-next-src-block 2)
-	   (org-trim (org-babel-execute-src-block)))))
-  ;; Bash associative arrays as strings for the row.
-  (should
-   (equal "20 cm"
-	  (org-test-at-id
-	   "82320a48-3409-49d7-85c9-5de1c6d3ff87"
-	   (org-babel-next-src-block 2)
-	   (org-trim (org-babel-execute-src-block))))))
+(ert-deftest test-ob-shell/bash-uses-assoc-arrays ()
+  "Bash shell: support for associative arrays
+
+Bash will see a table that contains the first column as the
+'index' of the associative array, and the second column as the
+value. "
+  (org-test-with-temp-text
+      "#+NAME: sample_mapping_table
+| first  | one   |
+| second | two   |
+| third  | three |
+
+#+begin_src bash :exports :results output results :var table=sample_mapping_table
+echo ${table[second]}
+<point>
+#+end_src "
+    (should
+     (equal "two"
+            (org-trim (org-babel-execute-src-block))))))
+
+(ert-deftest test-ob-shell/bash-uses-assoc-arrays-with-lists ()
+  "Bash shell: support for associative arrays with lists
+
+Bash will see an associative array that contains each row as a single
+string. Bash cannot handle lists in associative arrays."
+  (org-test-with-temp-text
+      "#+NAME: sample_big_table
+| bread     |  2 | kg |
+| spaghetti | 20 | cm |
+| milk      | 50 | dl |
+
+#+begin_src bash :exports results :results output :var table=sample_big_table
+echo ${table[spaghetti]}
+<point>
+#+end_src"
+    (should
+     (equal "20 cm"
+            (org-trim (org-babel-execute-src-block))))))
 
 (ert-deftest ob-shell/simple-list ()
   "Test list variables in shell."
-- 
2.38.1


[-- Attachment #6: 0005-test-ob-shell.el-Refactor-test-names-and-kill-test-b.patch --]
[-- Type: application/octet-stream, Size: 14542 bytes --]

From 3dbc7849a366740fc507063abd048dbe04e73630 Mon Sep 17 00:00:00 2001
From: Matt Trzcinski <matt@excalamus.com>
Date: Fri, 30 Dec 2022 12:39:47 -0500
Subject: [PATCH 5/7] test-ob-shell.el: Refactor test names and kill test
 buffers

* testing/lisp/test-ob-shell.el:
(ob-shell/generic-uses-no-arrays): Rename test from
`ob-shell/generic-uses-no-arrays' to
`test-ob-shell/generic-uses-no-arrays'.
(ob-shell/generic-uses-no-arrays): Move comments from
testing/examples/ob-shell-test.org to docstring.
(ob-shell/generic-uses-no-arrays): Remove dependency on
testing/examples/ob-shell-test.org.
(ob-shell/bash-uses-arrays): Rename `ob-shell/bash-uses-arrays' to
`test-ob-shell/bash-uses-arrays'.
(ob-shell/bash-uses-arrays): Move comments from
testing/examples/ob-shell-test.org to docstring.
(ob-shell/bash-uses-arrays): Remove dependency on
testing/examples/ob-shell-test.org.
(ob-shell/simple-list): Change test name from `ob-shell/simple-list'
to `test-ob-shell/simple-list'.
(ob-shell/remote-with-stdin-or-cmdline): Change test name from
`ob-shell/remote-with-stdin-or-cmdline' to
`test-ob-shell/remote-with-stdin-or-cmdline'.
(ob-shell/remote-with-stdin-or-cmdline): On pass, kill buffer created
by test.
(ob-shell/results-table): Rename `ob-shell/results-table' to
`test-ob-shell/results-table'.
(ob-shell/results-list): Rename `ob-shell/results-list' to
`test-ob-shell/results-list'.
 (ob-shell/standard-output-after-success): Rename
`ob-shell/standard-output-after-success' to
`test-ob-shell/standard-output-after-success'.
(ob-shell/standard-output-after-failure): Rename
`ob-shell/standard-output-after-failure' to
`test-ob-shell/standard-output-after-failure'.
(ob-shell/standard-output-after-failure): On pass, kill buffer created
during test.
(ob-shell/error-output-after-success): Rename
`ob-shell/error-output-after-success' to
`test-ob-shell/error-output-after-success'.
(ob-shell/error-output-after-success): On pass, kill buffer created by
test.
(ob-shell/error-output-after-failure): Rename
`ob-shell/error-output-after-failure' to
`test-ob-shell/error-output-after-failure'.
(ob-shell/error-output-after-failure): On pass, kill buffer created by
test.
(ob-shell/error-output-after-failure-multiple): Rename
`ob-shell/error-output-after-failure-multiple' to
`test-ob-shell/error-output-after-failure-multiple'.
(ob-shell/error-output-after-failure-multiple): On pass, kill buffer
created by test.
(ob-shell/exit-code): Rename `ob-shell/exit-code' to
`test-ob-shell/exit-code'.
(ob-shell/exit-code): On pass, kill buffer created by test.
(ob-shell/exit-code-multiple): Rename `ob-shell/exit-code-multiple' to
`test-ob-shell/exit-code-multiple'.
(ob-shell/exit-code-multiple): On pass, kill buffer created by test.
---
 testing/lisp/test-ob-shell.el | 220 ++++++++++++++++++++--------------
 1 file changed, 133 insertions(+), 87 deletions(-)

diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el
index 78267880b..3ee81d6b8 100644
--- a/testing/lisp/test-ob-shell.el
+++ b/testing/lisp/test-ob-shell.el
@@ -18,21 +18,23 @@
 ;; You should have received a copy of the GNU General Public License
 ;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
 
-;;; Comment:
 
-;; Template test file for Org tests
+;;; Comment:
 
-;;; Code:
+;;; Requirements:
 (org-test-for-executable "sh")
+
 (require 'ob-core)
+
 (unless (featurep 'ob-shell)
   (signal 'missing-test-dependency "Support for Shell code blocks"))
 
+;;; Code:
 (ert-deftest test-ob-shell/dont-insert-spaces-on-expanded-bodies ()
-  "Expanded shell bodies should not start with a blank line
-unless the body of the tangled block does."
+  "Expanded shell bodies should not start with a blank line unless
+the body of the tangled block does."
   (should-not (string-match "^[\n\r][\t ]*[\n\r]"
-			    (org-babel-expand-body:generic "echo 2" '())))
+                            (org-babel-expand-body:generic "echo 2" '())))
   (should (string-match "^[\n\r][\t ]*[\n\r]"
                         (org-babel-expand-body:generic "\n\necho 2" '()))))
 
@@ -68,16 +70,36 @@ unless the body of the tangled block does."
     (if (should (equal '((1) (2)) result))
         (kill-buffer session-name))))
 
-(ert-deftest ob-shell/generic-uses-no-arrays ()
-  "No arrays for generic"
-  (org-test-at-id "0ba56632-8dc1-405c-a083-c204bae477cf"
-    (org-babel-next-src-block)
+(ert-deftest test-ob-shell/generic-uses-no-arrays ()
+  "Test generic serialization of array into a single string."
+  (org-test-with-temp-text
+      " #+NAME: sample_array
+| one   |
+| two   |
+| three |
+
+#+begin_src sh :exports results :results output :var array=sample_array
+echo ${array}
+<point>
+#+end_src"
     (should (equal "one two three" (org-trim (org-babel-execute-src-block))))))
 
-(ert-deftest ob-shell/bash-uses-arrays ()
-  "Bash arrays"
-  (org-test-at-id "0ba56632-8dc1-405c-a083-c204bae477cf"
-    (org-babel-next-src-block 2)
+(ert-deftest test-ob-shell/bash-uses-arrays ()
+  "Bash sees named array as a simple indexed array.
+
+In this test, we check that the returned value is indeed only the
+first item of the array, as opposed to the generic serialiation
+that will return all elements of the array as a single string."
+  (org-test-with-temp-text
+      "#+NAME: sample_array
+| one   |
+| two   |
+| three |
+
+#+begin_src bash :exports results :results output :var array=sample_array
+echo ${array}
+<point>
+#+end_src"
     (should (equal "one" (org-trim (org-babel-execute-src-block))))))
 
 (ert-deftest test-ob-shell/generic-uses-no-assoc-arrays-simple-map ()
@@ -157,22 +179,27 @@ echo ${table[spaghetti]}
      (equal "20 cm"
             (org-trim (org-babel-execute-src-block))))))
 
-(ert-deftest ob-shell/simple-list ()
-  "Test list variables in shell."
-  ;; With bash, a list is turned into an array.
+(ert-deftest test-ob-shell/simple-list ()
+  "Test list variables."
+  ;; bash: a list is turned into an array
   (should
    (equal "2"
-	  (org-test-with-temp-text
-	   "#+BEGIN_SRC bash :results output :var l='(1 2)\necho ${l[1]}\n#+END_SRC"
-	   (org-trim (org-babel-execute-src-block)))))
-  ;; On sh, it is a string containing all values.
+          (org-test-with-temp-text
+              "#+BEGIN_SRC bash :results output :var l='(1 2)
+               echo ${l[1]}
+               #+END_SRC"
+            (org-trim (org-babel-execute-src-block)))))
+
+  ;; sh: a list is a string containing all values
   (should
    (equal "1 2"
-	  (org-test-with-temp-text
-	   "#+BEGIN_SRC sh :results output :var l='(1 2)\necho ${l}\n#+END_SRC"
-	   (org-trim (org-babel-execute-src-block))))))
+          (org-test-with-temp-text
+              "#+BEGIN_SRC sh :results output :var l='(1 2)
+               echo ${l}
+               #+END_SRC"
+            (org-trim (org-babel-execute-src-block))))))
 
-(ert-deftest ob-shell/remote-with-stdin-or-cmdline ()
+(ert-deftest test-ob-shell/remote-with-stdin-or-cmdline ()
   "Test :stdin and :cmdline with a remote directory."
   ;; We assume `default-directory' is a local directory.
   (skip-unless (not (memq system-type '(ms-dos windows-nt))))
@@ -219,20 +246,27 @@ echo ${table[spaghetti]}
                            (org-trim (org-babel-execute-src-block))))
                  (expected (concat "ARGS: --verbose 23 71"
                                    "\nhello tramp from " (file-local-name default-directory))))
-            (should (equal result expected)))))))
+            (if (should (equal result expected))
+                (kill-matching-buffers (format "\\*tramp/mock\\s-%s\\*" system-name) t t)))))))
 
-(ert-deftest ob-shell/results-table ()
+(ert-deftest test-ob-shell/results-table ()
   "Test :results table."
   (should
    (equal '(("I \"want\" it all"))
-	  (org-test-with-temp-text
-	      "#+BEGIN_SRC sh :results table\necho 'I \"want\" it all'\n#+END_SRC"
-	    (org-babel-execute-src-block)))))
+          (org-test-with-temp-text
+              "#+BEGIN_SRC sh :results table
+               echo 'I \"want\" it all'
+               #+END_SRC"
+            (org-babel-execute-src-block)))))
 
-(ert-deftest ob-shell/results-list ()
+(ert-deftest test-ob-shell/results-list ()
   "Test :results list."
   (org-test-with-temp-text
-      "#+BEGIN_SRC sh :results list\necho 1\necho 2\necho 3\n#+END_SRC"
+      "#+BEGIN_SRC sh :results list
+echo 1
+echo 2
+echo 3
+#+END_SRC"
     (should
      (equal '((1) (2) (3))
             (org-babel-execute-src-block)))
@@ -245,86 +279,98 @@ echo ${table[spaghetti]}
 
 ;;; Standard output
 
-(ert-deftest ob-shell/standard-output-after-success ()
+(ert-deftest test-ob-shell/standard-output-after-success ()
   "Test standard output after exiting with a zero code."
   (should (= 1
              (org-babel-execute:sh
               "echo 1" nil))))
 
-(ert-deftest ob-shell/standard-output-after-failure ()
+(ert-deftest test-ob-shell/standard-output-after-failure ()
   "Test standard output after exiting with a non-zero code."
-  (should (= 1
-             (org-babel-execute:sh
-              "echo 1; exit 2" nil))))
+  (if
+      (should (= 1
+                 (org-babel-execute:sh
+                  "echo 1; exit 2" nil)))
+      (kill-buffer "*Org-Babel Error Output*")))
 
 ;;; Standard error
 
-(ert-deftest ob-shell/error-output-after-success ()
-  "Test that standard error shows in the error buffer, alongside the
-exit code, after exiting with a zero code."
-  (should
-   (string= "1
+(ert-deftest test-ob-shell/error-output-after-success ()
+  "Test that standard error shows in the error buffer, alongside
+the exit code, after exiting with a zero code."
+  (if
+      (should
+       (string= "1
 [ Babel evaluation exited with code 0 ]"
-            (progn (org-babel-eval-wipe-error-buffer)
-                   (org-babel-execute:sh
-                    "echo 1 >&2" nil)
-                   (with-current-buffer org-babel-error-buffer-name
-                     (buffer-string))))))
-
-(ert-deftest ob-shell/error-output-after-failure ()
-  "Test that standard error shows in the error buffer, alongside the
-exit code, after exiting with a non-zero code."
-  (should
-   (string= "1
+                (progn (org-babel-eval-wipe-error-buffer)
+                       (org-babel-execute:sh
+                        "echo 1 >&2" nil)
+                       (with-current-buffer org-babel-error-buffer-name
+                         (buffer-string)))))
+      (kill-buffer "*Org-Babel Error Output*")))
+
+(ert-deftest test-ob-shell/error-output-after-failure ()
+  "Test that standard error shows in the error buffer, alongside
+the exit code, after exiting with a non-zero code."
+  (if
+      (should
+       (string= "1
 [ Babel evaluation exited with code 2 ]"
-            (progn (org-babel-eval-wipe-error-buffer)
-                   (org-babel-execute:sh
-                    "echo 1 >&2; exit 2" nil)
-                   (with-current-buffer org-babel-error-buffer-name
-                     (buffer-string))))))
+                (progn (org-babel-eval-wipe-error-buffer)
+                       (org-babel-execute:sh
+                        "echo 1 >&2; exit 2" nil)
+                       (with-current-buffer org-babel-error-buffer-name
+                         (buffer-string)))))
+      (kill-buffer "*Org-Babel Error Output*")))
 
-(ert-deftest ob-shell/error-output-after-failure-multiple ()
+(ert-deftest test-ob-shell/error-output-after-failure-multiple ()
   "Test that multiple standard error strings show in the error
 buffer, alongside multiple exit codes."
-  (should
-   (string= "1
+  (if
+      (should
+       (string= "1
 [ Babel evaluation exited with code 2 ]
 3
 [ Babel evaluation exited with code 4 ]"
-            (progn (org-babel-eval-wipe-error-buffer)
-                   (org-babel-execute:sh
-                    "echo 1 >&2; exit 2" nil)
-                   (org-babel-execute:sh
-                    "echo 3 >&2; exit 4" nil)
-                   (with-current-buffer org-babel-error-buffer-name
-                     (buffer-string))))))
+                (progn (org-babel-eval-wipe-error-buffer)
+                       (org-babel-execute:sh
+                        "echo 1 >&2; exit 2" nil)
+                       (org-babel-execute:sh
+                        "echo 3 >&2; exit 4" nil)
+                       (with-current-buffer org-babel-error-buffer-name
+                         (buffer-string)))))
+      (kill-buffer "*Org-Babel Error Output*")))
 
 ;;; Exit codes
 
-(ert-deftest ob-shell/exit-code ()
+(ert-deftest test-ob-shell/exit-code ()
   "Test that the exit code shows in the error buffer after exiting
 with a non-zero return code."
-  (should
-   (string= "[ Babel evaluation exited with code 1 ]"
-            (progn (org-babel-eval-wipe-error-buffer)
-                   (org-babel-execute:sh
-                    "exit 1" nil)
-                   (with-current-buffer org-babel-error-buffer-name
-                     (buffer-string))))))
-
-(ert-deftest ob-shell/exit-code-multiple ()
+  (if
+      (should
+       (string= "[ Babel evaluation exited with code 1 ]"
+                (progn (org-babel-eval-wipe-error-buffer)
+                       (org-babel-execute:sh
+                        "exit 1" nil)
+                       (with-current-buffer org-babel-error-buffer-name
+                         (buffer-string)))))
+      (kill-buffer "*Org-Babel Error Output*")))
+
+(ert-deftest test-ob-shell/exit-code-multiple ()
   "Test that multiple exit codes show in the error buffer after
 exiting with a non-zero return code multiple times."
-  (should
-   (string= "[ Babel evaluation exited with code 1 ]
+  (if
+      (should
+       (string= "[ Babel evaluation exited with code 1 ]
 [ Babel evaluation exited with code 2 ]"
-            (progn (org-babel-eval-wipe-error-buffer)
-                   (org-babel-execute:sh
-                    "exit 1" nil)
-                   (org-babel-execute:sh
-                    "exit 2" nil)
-                   (with-current-buffer org-babel-error-buffer-name
-                     (buffer-string))))))
+                (progn (org-babel-eval-wipe-error-buffer)
+                       (org-babel-execute:sh
+                        "exit 1" nil)
+                       (org-babel-execute:sh
+                        "exit 2" nil)
+                       (with-current-buffer org-babel-error-buffer-name
+                         (buffer-string)))))
+      (kill-buffer "*Org-Babel Error Output*")))
 
 (provide 'test-ob-shell)
 
-- 
2.38.1


[-- Attachment #7: 0006-ob-shell-test.org-Remove-ob-shell-test.org.patch --]
[-- Type: application/octet-stream, Size: 3136 bytes --]

From 9485450222cda5c3f09e02f6055aefea128ea215 Mon Sep 17 00:00:00 2001
From: Matt Trzcinski <matt@excalamus.com>
Date: Sat, 24 Dec 2022 11:29:06 -0500
Subject: [PATCH 6/7] ob-shell-test.org: Remove ob-shell-test.org

* testing/examples/ob-shell-test.org: Delete file

File was used by test-ob-shell.el to test ob-shell.el.  Tests have
been updated to use temporary buffers, rendering ob-shell-test.org
unnecessary.
---
 testing/examples/ob-shell-test.org | 88 ------------------------------
 1 file changed, 88 deletions(-)
 delete mode 100644 testing/examples/ob-shell-test.org

diff --git a/testing/examples/ob-shell-test.org b/testing/examples/ob-shell-test.org
deleted file mode 100644
index 2510f4f96..000000000
--- a/testing/examples/ob-shell-test.org
+++ /dev/null
@@ -1,88 +0,0 @@
-#+Title: a collection of examples for ob-shell tests
-#+OPTIONS: ^:nil
-
-* Sample data structures
-#+NAME: sample_array
-| one   |
-| two   |
-| three |
-
-#+NAME: sample_mapping_table
-| first  | one   |
-| second | two   |
-| third  | three |
-
-#+NAME: sample_big_table
-| bread     |  2 | kg |
-| spaghetti | 20 | cm |
-| milk      | 50 | dl |
-
-* Array tests
-  :PROPERTIES:
-  :ID:       0ba56632-8dc1-405c-a083-c204bae477cf
-  :END:
-** Generic shell: no arrays
-#+begin_src sh :exports results :results output :var array=sample_array
-echo ${array}
-#+end_src
-
-#+RESULTS:
-: one two three
-
-** Bash shell: support for arrays
-Bash will see a simple indexed array. In this test, we check that the
-returned value is indeed only the first item of the array, as opposed to
-the generic serialiation that will return all elements of the array as 
-a single string.
-#+begin_src bash :exports results :results output :var array=sample_array
-echo ${array}
-#+end_src
-
-#+RESULTS:
-: one
-
-* Associative array tests (simple map)
-  :PROPERTIES:
-  :ID:       bec1a5b0-4619-4450-a8c0-2a746b44bf8d
-  :END:
-** Generic shell: no special handing
-The shell will see all values as a single string.
-#+begin_src sh :exports results :results output :var table=sample_mapping_table
-echo ${table}
-#+end_src
-
-#+RESULTS:
-: first one second two third three
-
-** Bash shell: support for associative arrays
-Bash will see a table that contains the first column as the 'index'
-of the associative array, and the second column as the value.
-#+begin_src bash :exports :results output results :var table=sample_mapping_table
-echo ${table[second]}
-#+end_src
-
-#+RESULTS:
-: two
-
-* Associative array tests (more than 2 columns)
-  :PROPERTIES:
-  :ID:       82320a48-3409-49d7-85c9-5de1c6d3ff87
-  :END:
-** Generic shell: no special handing
-#+begin_src sh :exports results :results output :var table=sample_big_table
-echo ${table}
-#+end_src
-
-#+RESULTS:
-: bread 2 kg spaghetti 20 cm milk 50 dl
-   
-** Bash shell: support for associative arrays with lists
-Bash will see an associative array that contains each row as a single
-string. Bash cannot handle lists in associative arrays.
-#+begin_src bash :exports results :results output :var table=sample_big_table
-echo ${table[spaghetti]}
-#+end_src
-
-#+RESULTS:
-: 20 cm
-
-- 
2.38.1


[-- Attachment #8: 0007-test-ob-shell.el-Organize-tests.patch --]
[-- Type: application/octet-stream, Size: 1918 bytes --]

From 9ed2821053cf7b0c2f02fe941a54e1d3a9837aeb Mon Sep 17 00:00:00 2001
From: Matt Trzcinski <matt@excalamus.com>
Date: Fri, 30 Dec 2022 13:03:59 -0500
Subject: [PATCH 7/7] test-ob-shell.el: Organize tests

* testing/lisp/test-ob-shell.el:
  - Give instructions on how to run tests
  - Require `org-test' explicitly
  - Group tests into sections
  - Insert `page-delimiter's between sections
---
 testing/lisp/test-ob-shell.el | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el
index 3ee81d6b8..5318425e6 100644
--- a/testing/lisp/test-ob-shell.el
+++ b/testing/lisp/test-ob-shell.el
@@ -18,17 +18,22 @@
 ;; You should have received a copy of the GNU General Public License
 ;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
 
-
+\f
 ;;; Comment:
 
+;; See testing/README for how to run tests.
+
+\f
 ;;; Requirements:
-(org-test-for-executable "sh")
 
 (require 'ob-core)
 
 (unless (featurep 'ob-shell)
   (signal 'missing-test-dependency "Support for Shell code blocks"))
 
+(org-test-for-executable "sh")
+
+\f
 ;;; Code:
 (ert-deftest test-ob-shell/dont-insert-spaces-on-expanded-bodies ()
   "Expanded shell bodies should not start with a blank line unless
@@ -277,6 +282,7 @@ echo 3
       "- 1\n- 2\n- 3\n"
       (buffer-substring-no-properties (point) (point-max))))))
 
+\f
 ;;; Standard output
 
 (ert-deftest test-ob-shell/standard-output-after-success ()
@@ -293,6 +299,7 @@ echo 3
                   "echo 1; exit 2" nil)))
       (kill-buffer "*Org-Babel Error Output*")))
 
+\f
 ;;; Standard error
 
 (ert-deftest test-ob-shell/error-output-after-success ()
@@ -341,6 +348,7 @@ buffer, alongside multiple exit codes."
                          (buffer-string)))))
       (kill-buffer "*Org-Babel Error Output*")))
 
+\f
 ;;; Exit codes
 
 (ert-deftest test-ob-shell/exit-code ()
-- 
2.38.1