Re: ob-shell intentions and paperwork (was Bash results broken?)

[Matt <matt@excalamus.com>]

[-- Attachment #1: Type: text/plain, Size: 1025 bytes --]


 ---- On Fri, 30 Dec 2022 00:34:38 -0500  Matt  wrote --- 
 > 
 >  ---- On Thu, 29 Dec 2022 06:08:59 -0500  Ihor Radchenko  wrote --- 

 >  > > From: Matt Trzcinski matt@excalamus.com>
 >  > > +(require 'org-test (expand-file-name "../org-test.el"))
 >  > 
 >  > I am unsure here. What will happen if you run this file from
 >  > default-directory not the same with file location?

I was mistaken.  Loading a new Emacs instance and running `(load "/path/to/test-ob-shell.el")' fails with file-missing error.  I'm content to leave that alone for now.

I've backed out the `require' change and adjusted everything else based on your feedback.  There is a separate patch for each refactor that created a new test.   The remaining refactors are in a single patch.  I was also able to resolve the issue I had with inserting the test name for the session (the "yes" string in `test-ob-shell/session').  The issue with `org-babel-shell-initialize' still stands but is outside the scope of these patches and isn't dealt with here.


[-- Attachment #2: 0001-test-ob-shell.el-Split-test-ob-shell-dont-error-on-e.patch --]
[-- Type: application/octet-stream, Size: 1899 bytes --]

From e204c3a6ccdaf1770db5e931f1116235597a8c0b Mon Sep 17 00:00:00 2001
From: Matt Trzcinski <matt@excalamus.com>
Date: Fri, 30 Dec 2022 11:18:15 -0500
Subject: [PATCH 1/7] test-ob-shell.el: Split
 test-ob-shell/dont-error-on-empty-results

* testing/lisp/test-ob-shell.el
(test-ob-shell/dont-error-on-empty-results): Explicitly test handling
of empty results.
(test-ob-shell/dont-error-on-babel-error): Explicitly test handling of
Babel errors.  On pass, remove buffers created during test.

Original test conflated empty results (a valid Babel return) and Babel
errors.
---
 testing/lisp/test-ob-shell.el | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el
index b0d9beff4..fc3ecd3c8 100644
--- a/testing/lisp/test-ob-shell.el
+++ b/testing/lisp/test-ob-shell.el
@@ -34,12 +34,16 @@ unless the body of the tangled block does."
   (should-not (string-match "^[\n\r][\t ]*[\n\r]"
 			    (org-babel-expand-body:generic "echo 2" '())))
   (should (string-match "^[\n\r][\t ]*[\n\r]"
-			(org-babel-expand-body:generic "\n\necho 2" '()))))
+                        (org-babel-expand-body:generic "\n\necho 2" '()))))
 
 (ert-deftest test-ob-shell/dont-error-on-empty-results ()
-  "Was throwing an elisp error when shell blocks threw errors and
-returned empty results."
-  (should (null (org-babel-execute:sh "ls NoSuchFileOrDirectory.txt" nil))))
+  "Empty results should not cause a Lisp error."
+  (should (null (org-babel-execute:sh "" nil))))
+
+(ert-deftest test-ob-shell/dont-error-on-babel-error ()
+  "Errors within Babel execution should not cause Lisp errors."
+  (if (should (null (org-babel-execute:sh "ls NoSuchFileOrDirectory.txt" nil)))
+      (kill-buffer "*Org-Babel Error Output*")))
 
 (ert-deftest test-ob-shell/session ()
   "This also tests `org-babel-comint-with-output' in
-- 
2.38.1


[-- Attachment #3: 0002-test-ob-shell.el-Refactor-test-ob-shell-session.patch --]
[-- Type: application/octet-stream, Size: 2607 bytes --]

From 6e0012867d8594913cf4783d7afdc109b8307de3 Mon Sep 17 00:00:00 2001
From: Matt Trzcinski <matt@excalamus.com>
Date: Fri, 30 Dec 2022 11:35:10 -0500
Subject: [PATCH 2/7] test-ob-shell.el: Refactor test-ob-shell/session

* testing/lisp/test-ob-shell.el (ob-shell/session): Split
`ob-shell/session' into
`test-ob-shell/session-single-return-returns-string' and
`test-ob-shell/session-multiple-returns-returns-list'.
(ob-shell/session): Rename comint from to test name. On
pass, kill process and remove process buffer.
---
 testing/lisp/test-ob-shell.el | 32 +++++++++++++++++++++-----------
 1 file changed, 21 insertions(+), 11 deletions(-)

diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el
index fc3ecd3c8..b08bf8413 100644
--- a/testing/lisp/test-ob-shell.el
+++ b/testing/lisp/test-ob-shell.el
@@ -45,18 +45,28 @@ unless the body of the tangled block does."
   (if (should (null (org-babel-execute:sh "ls NoSuchFileOrDirectory.txt" nil)))
       (kill-buffer "*Org-Babel Error Output*")))
 
-(ert-deftest test-ob-shell/session ()
-  "This also tests `org-babel-comint-with-output' in
-ob-comint.el, which was not previously tested."
-  (let ((res (org-babel-execute:sh "echo 1; echo 2" '((:session . "yes")))))
-    (should res)
-    (should (listp res)))
-  ;; Test multi-line input.
-  (let ((result (org-babel-execute:sh
-		 "if true \n then \n echo yes \n fi"
-		 '((:session . "yes")))))
+(ert-deftest test-ob-shell/session-single-return-returns-string ()
+  "Sessions with a single result should return a string."
+  (let* ((session-name "test-ob-shell/session-evaluation-single-return-returns-string")
+         (kill-buffer-query-functions nil)
+         (result (org-babel-execute:sh
+                  (format "echo %s" session-name)
+                  `((:session . ,session-name)))))
     (should result)
-    (should (string= "yes" result))))
+    (if (should (string= session-name result))
+        (kill-buffer session-name))))
+
+(ert-deftest test-ob-shell/session-multiple-returns-returns-list ()
+  "Sessions with multiple results should return a list."
+  (let* ((session-name "test-ob-shell/session-multiple-returns-returns-list")
+         (kill-buffer-query-functions nil)
+         (result (org-babel-execute:sh
+                  "echo 1; echo 2"
+                  `((:session . ,session-name)))))
+    (should result)
+    (should (listp result))
+    (if (should (equal '((1) (2)) result))
+        (kill-buffer session-name))))
 
 ; A list of tests using the samples in ob-shell-test.org
 (ert-deftest ob-shell/generic-uses-no-arrays ()
-- 
2.38.1


[-- Attachment #4: 0003-test-ob-shell.el-Refactor-ob-shell-generic-uses-no-a.patch --]
[-- Type: application/octet-stream, Size: 3522 bytes --]

From d44204d940b81c284102981291433180ca9a438f Mon Sep 17 00:00:00 2001
From: Matt Trzcinski <matt@excalamus.com>
Date: Fri, 30 Dec 2022 11:39:35 -0500
Subject: [PATCH 3/7] test-ob-shell.el: Refactor
 ob-shell/generic-uses-no-assoc-arrays

* test-ob-shell.el (ob-shell/generic-uses-no-assoc-arrays): Split test
based on simple versus complex mapping cases,
`test-ob-shell/generic-uses-no-assoc-arrays-simple-map' and
`test-ob-shell/generic-uses-no-assoc-arrays-3-columns'.  Rename prefix
from "ob-shell" to "test-ob-shell".
(ob-shell/generic-uses-no-assoc-arrays): Move
headlines and commentary from `testing/examples/ob-shell-test.org' to
docstring.
(ob-shell/generic-uses-no-assoc-arrays): Remove dependency on
testing/examples/ob-shell-test.org.
---
 testing/lisp/test-ob-shell.el | 56 +++++++++++++++++++++++++----------
 1 file changed, 41 insertions(+), 15 deletions(-)

diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el
index b08bf8413..57347229e 100644
--- a/testing/lisp/test-ob-shell.el
+++ b/testing/lisp/test-ob-shell.el
@@ -68,7 +68,6 @@ unless the body of the tangled block does."
     (if (should (equal '((1) (2)) result))
         (kill-buffer session-name))))
 
-; A list of tests using the samples in ob-shell-test.org
 (ert-deftest ob-shell/generic-uses-no-arrays ()
   "No arrays for generic"
   (org-test-at-id "0ba56632-8dc1-405c-a083-c204bae477cf"
@@ -81,20 +80,47 @@ unless the body of the tangled block does."
     (org-babel-next-src-block 2)
     (should (equal "one" (org-trim (org-babel-execute-src-block))))))
 
-(ert-deftest ob-shell/generic-uses-no-assoc-arrays ()
-  "No associative arrays for generic"
-  (should
-   (equal "first one second two third three"
-	  (org-test-at-id
-	   "bec1a5b0-4619-4450-a8c0-2a746b44bf8d"
-	   (org-babel-next-src-block)
-	   (org-trim (org-babel-execute-src-block)))))
-  (should
-   (equal "bread 2 kg spaghetti 20 cm milk 50 dl"
-	  (org-test-at-id
-	   "82320a48-3409-49d7-85c9-5de1c6d3ff87"
-	   (org-babel-next-src-block)
-	   (org-trim (org-babel-execute-src-block))))))
+(ert-deftest test-ob-shell/generic-uses-no-assoc-arrays-simple-map ()
+  "Generic shell: no special handing for key-value mapping table
+
+No associative arrays for generic.  The shell will see all values
+as a single string."
+  (org-test-with-temp-text
+   (string-join
+    '("#+NAME: sample_mapping_table"
+      "| first  | one   |"
+      "| second | two   |"
+      "| third  | three |"
+      ""
+      "#+begin_src sh :exports results :results output :var table=sample_mapping_table"
+      "echo ${table}"
+      "<point>"
+      "#+end_src")
+    "\n")
+   (should
+    (equal "first one second two third three"
+           (org-trim (org-babel-execute-src-block))))))
+
+(ert-deftest test-ob-shell/generic-uses-no-assoc-arrays-3-columns ()
+  "Associative array tests (more than 2 columns)
+
+No associative arrays for generic.  The shell will see all values
+as a single string."
+  (org-test-with-temp-text
+   (string-join
+    '("#+NAME: sample_big_table"
+      "| bread     |  2 | kg |"
+      "| spaghetti | 20 | cm |"
+      "| milk      | 50 | dl |"
+      ""
+      "#+begin_src sh :exports results :results output :var table=sample_big_table"
+      "echo ${table}"
+      "<point>"
+      "#+end_src")
+    "\n")
+   (should
+    (equal "bread 2 kg spaghetti 20 cm milk 50 dl"
+           (org-trim (org-babel-execute-src-block))))))
 
 (ert-deftest ob-shell/bash-uses-assoc-arrays ()
   "Bash associative arrays"
-- 
2.38.1


[-- Attachment #5: 0004-Refactor-ob-shell-bash-uses-assoc-arrays.patch --]
[-- Type: application/octet-stream, Size: 3295 bytes --]

From 8a9f5b29ec6bf09f63bc41dab09781b3456ff2aa Mon Sep 17 00:00:00 2001
From: Matt Trzcinski <matt@excalamus.com>
Date: Fri, 30 Dec 2022 12:36:41 -0500
Subject: [PATCH 4/7] Refactor ob-shell/bash-uses-assoc-arrays

* testing/lisp/test-ob-shell.el (ob-shell/bash-uses-assoc-arrays):
Split test cases into separate tests,
`test-ob-shell/bash-uses-assoc-arrays' and
`test-ob-shell/bash-uses-assoc-arrays-with-lists'.  Rename prefix from
"ob-shell" to "test-ob-shell".
(ob-shell/bash-uses-assoc-arrays): Move comments from
testing/examples/ob-shell-test.org to docstrings.
(ob-shell/bash-uses-assoc-arrays): Remove dependency on
testing/examples/ob-shell-test.org.
---
 testing/lisp/test-ob-shell.el | 61 +++++++++++++++++++++++++----------
 1 file changed, 44 insertions(+), 17 deletions(-)

diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el
index 57347229e..1e4bd6729 100644
--- a/testing/lisp/test-ob-shell.el
+++ b/testing/lisp/test-ob-shell.el
@@ -119,24 +119,51 @@ as a single string."
       "#+end_src")
     "\n")
    (should
-    (equal "bread 2 kg spaghetti 20 cm milk 50 dl"
-           (org-trim (org-babel-execute-src-block))))))
+     (equal "bread 2 kg spaghetti 20 cm milk 50 dl"
+            (org-trim (org-babel-execute-src-block))))))
 
-(ert-deftest ob-shell/bash-uses-assoc-arrays ()
-  "Bash associative arrays"
-  (should
-   (equal "two"
-	  (org-test-at-id
-	   "bec1a5b0-4619-4450-a8c0-2a746b44bf8d"
-	   (org-babel-next-src-block 2)
-	   (org-trim (org-babel-execute-src-block)))))
-  ;; Bash associative arrays as strings for the row.
-  (should
-   (equal "20 cm"
-	  (org-test-at-id
-	   "82320a48-3409-49d7-85c9-5de1c6d3ff87"
-	   (org-babel-next-src-block 2)
-	   (org-trim (org-babel-execute-src-block))))))
+(ert-deftest test-ob-shell/bash-uses-assoc-arrays ()
+  "Bash shell: support for associative arrays
+
+Bash will see a table that contains the first column as the
+'index' of the associative array, and the second column as the
+value. "
+  (org-test-with-temp-text
+      (string-join
+       '("#+NAME: sample_mapping_table"
+         "| first  | one   |"
+         "| second | two   |"
+         "| third  | three |"
+         ""
+         "#+begin_src bash :exports :results output results :var table=sample_mapping_table"
+         "echo ${table[second]}"
+         "<point>"
+         "#+end_src")
+       "\n")
+    (should
+     (equal "two"
+            (org-trim (org-babel-execute-src-block))))))
+
+(ert-deftest test-ob-shell/bash-uses-assoc-arrays-with-lists ()
+  "Bash shell: support for associative arrays with lists
+
+Bash will see an associative array that contains each row as a single
+string. Bash cannot handle lists in associative arrays."
+  (org-test-with-temp-text
+      (string-join
+       '("#+NAME: sample_big_table"
+         "| bread     |  2 | kg |"
+         "| spaghetti | 20 | cm |"
+         "| milk      | 50 | dl |"
+         ""
+         "#+begin_src bash :exports results :results output :var table=sample_big_table"
+         "echo ${table[spaghetti]}"
+         "<point>"
+         "#+end_src")
+       "\n")
+    (should
+     (equal "20 cm"
+            (org-trim (org-babel-execute-src-block))))))
 
 (ert-deftest ob-shell/simple-list ()
   "Test list variables in shell."
-- 
2.38.1


[-- Attachment #6: 0005-test-ob-shell.el-Refactor-test-names-and-kill-test-b.patch --]
[-- Type: application/octet-stream, Size: 15159 bytes --]

From fbdb368efdbedc5b8f5ed6b27f74be060a4c4cb8 Mon Sep 17 00:00:00 2001
From: Matt Trzcinski <matt@excalamus.com>
Date: Fri, 30 Dec 2022 12:39:47 -0500
Subject: [PATCH 5/7] test-ob-shell.el: Refactor test names and kill test
 buffers

* testing/lisp/test-ob-shell.el:
(ob-shell/generic-uses-no-arrays): Rename test from
`ob-shell/generic-uses-no-arrays' to
`test-ob-shell/generic-uses-no-arrays'.
(ob-shell/generic-uses-no-arrays): Move comments from
testing/examples/ob-shell-test.org to docstring.
(ob-shell/generic-uses-no-arrays): Remove dependency on
testing/examples/ob-shell-test.org.
(ob-shell/bash-uses-arrays): Rename `ob-shell/bash-uses-arrays' to
`test-ob-shell/bash-uses-arrays'.
(ob-shell/bash-uses-arrays): Move comments from
testing/examples/ob-shell-test.org to docstring.
(ob-shell/bash-uses-arrays): Remove dependency on
testing/examples/ob-shell-test.org.
(ob-shell/simple-list): Change test name from `ob-shell/simple-list'
to `test-ob-shell/simple-list'.
(ob-shell/remote-with-stdin-or-cmdline): Change test name from
`ob-shell/remote-with-stdin-or-cmdline' to
`test-ob-shell/remote-with-stdin-or-cmdline'.
(ob-shell/remote-with-stdin-or-cmdline): On pass, kill buffer created
by test.
(ob-shell/results-table): Rename `ob-shell/results-table' to
`test-ob-shell/results-table'.
(ob-shell/results-list): Rename `ob-shell/results-list' to
`test-ob-shell/results-list'.
 (ob-shell/standard-output-after-success): Rename
`ob-shell/standard-output-after-success' to
`test-ob-shell/standard-output-after-success'.
(ob-shell/standard-output-after-failure): Rename
`ob-shell/standard-output-after-failure' to
`test-ob-shell/standard-output-after-failure'.
(ob-shell/standard-output-after-failure): On pass, kill buffer created
during test.
(ob-shell/error-output-after-success): Rename
`ob-shell/error-output-after-success' to
`test-ob-shell/error-output-after-success'.
(ob-shell/error-output-after-success): On pass, kill buffer created by
test.
(ob-shell/error-output-after-failure): Rename
`ob-shell/error-output-after-failure' to
`test-ob-shell/error-output-after-failure'.
(ob-shell/error-output-after-failure): On pass, kill buffer created by
test.
(ob-shell/error-output-after-failure-multiple): Rename
`ob-shell/error-output-after-failure-multiple' to
`test-ob-shell/error-output-after-failure-multiple'.
(ob-shell/error-output-after-failure-multiple): On pass, kill buffer
created by test.
(ob-shell/exit-code): Rename `ob-shell/exit-code' to
`test-ob-shell/exit-code'.
(ob-shell/exit-code): On pass, kill buffer created by test.
(ob-shell/exit-code-multiple): Rename `ob-shell/exit-code-multiple' to
`test-ob-shell/exit-code-multiple'.
(ob-shell/exit-code-multiple): On pass, kill buffer created by test.
---
 testing/lisp/test-ob-shell.el | 234 +++++++++++++++++++++-------------
 1 file changed, 146 insertions(+), 88 deletions(-)

diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el
index 1e4bd6729..2c4491ffa 100644
--- a/testing/lisp/test-ob-shell.el
+++ b/testing/lisp/test-ob-shell.el
@@ -18,21 +18,23 @@
 ;; You should have received a copy of the GNU General Public License
 ;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
 
-;;; Comment:
 
-;; Template test file for Org tests
+;;; Comment:
 
-;;; Code:
+;;; Requirements:
 (org-test-for-executable "sh")
+
 (require 'ob-core)
+
 (unless (featurep 'ob-shell)
   (signal 'missing-test-dependency "Support for Shell code blocks"))
 
+;;; Code:
 (ert-deftest test-ob-shell/dont-insert-spaces-on-expanded-bodies ()
-  "Expanded shell bodies should not start with a blank line
-unless the body of the tangled block does."
+  "Expanded shell bodies should not start with a blank line unless
+the body of the tangled block does."
   (should-not (string-match "^[\n\r][\t ]*[\n\r]"
-			    (org-babel-expand-body:generic "echo 2" '())))
+                            (org-babel-expand-body:generic "echo 2" '())))
   (should (string-match "^[\n\r][\t ]*[\n\r]"
                         (org-babel-expand-body:generic "\n\necho 2" '()))))
 
@@ -68,16 +70,40 @@ unless the body of the tangled block does."
     (if (should (equal '((1) (2)) result))
         (kill-buffer session-name))))
 
-(ert-deftest ob-shell/generic-uses-no-arrays ()
-  "No arrays for generic"
-  (org-test-at-id "0ba56632-8dc1-405c-a083-c204bae477cf"
-    (org-babel-next-src-block)
+(ert-deftest test-ob-shell/generic-uses-no-arrays ()
+  "Test generic serialization of array into a single string."
+  (org-test-with-temp-text
+      (string-join
+       '("#+NAME: sample_array"
+         "| one   |"
+         "| two   |"
+         "| three |"
+         ""
+         "#+begin_src sh :exports results :results output :var array=sample_array"
+         "echo ${array}"
+         "<point>"
+         "#+end_src")
+       "\n")
     (should (equal "one two three" (org-trim (org-babel-execute-src-block))))))
 
-(ert-deftest ob-shell/bash-uses-arrays ()
-  "Bash arrays"
-  (org-test-at-id "0ba56632-8dc1-405c-a083-c204bae477cf"
-    (org-babel-next-src-block 2)
+(ert-deftest test-ob-shell/bash-uses-arrays ()
+  "Bash sees named array as a simple indexed array.
+
+In this test, we check that the returned value is indeed only the
+first item of the array, as opposed to the generic serialiation
+that will return all elements of the array as a single string."
+  (org-test-with-temp-text
+      (string-join
+       '("#+NAME: sample_array"
+         "| one   |"
+         "| two   |"
+         "| three |"
+         ""
+         "#+begin_src bash :exports results :results output :var array=sample_array"
+         "echo ${array}"
+         "<point>"
+         "#+end_src")
+       "\n")
     (should (equal "one" (org-trim (org-babel-execute-src-block))))))
 
 (ert-deftest test-ob-shell/generic-uses-no-assoc-arrays-simple-map ()
@@ -165,22 +191,31 @@ string. Bash cannot handle lists in associative arrays."
      (equal "20 cm"
             (org-trim (org-babel-execute-src-block))))))
 
-(ert-deftest ob-shell/simple-list ()
-  "Test list variables in shell."
-  ;; With bash, a list is turned into an array.
+(ert-deftest test-ob-shell/simple-list ()
+  "Test list variables."
+  ;; bash: a list is turned into an array
   (should
    (equal "2"
-	  (org-test-with-temp-text
-	   "#+BEGIN_SRC bash :results output :var l='(1 2)\necho ${l[1]}\n#+END_SRC"
-	   (org-trim (org-babel-execute-src-block)))))
-  ;; On sh, it is a string containing all values.
+          (org-test-with-temp-text
+              (string-join
+               '("#+BEGIN_SRC bash :results output :var l='(1 2)"
+                 "echo ${l[1]}"
+                 "#+END_SRC")
+               "\n")
+            (org-trim (org-babel-execute-src-block)))))
+
+  ;; sh: a list is a string containing all values
   (should
    (equal "1 2"
-	  (org-test-with-temp-text
-	   "#+BEGIN_SRC sh :results output :var l='(1 2)\necho ${l}\n#+END_SRC"
-	   (org-trim (org-babel-execute-src-block))))))
+          (org-test-with-temp-text
+              (string-join
+               '("#+BEGIN_SRC sh :results output :var l='(1 2)"
+                 "echo ${l}"
+                 "#+END_SRC")
+               "\n")
+            (org-trim (org-babel-execute-src-block))))))
 
-(ert-deftest ob-shell/remote-with-stdin-or-cmdline ()
+(ert-deftest test-ob-shell/remote-with-stdin-or-cmdline ()
   "Test :stdin and :cmdline with a remote directory."
   ;; We assume `default-directory' is a local directory.
   (skip-unless (not (memq system-type '(ms-dos windows-nt))))
@@ -227,20 +262,31 @@ string. Bash cannot handle lists in associative arrays."
                            (org-trim (org-babel-execute-src-block))))
                  (expected (concat "ARGS: --verbose 23 71"
                                    "\nhello tramp from " (file-local-name default-directory))))
-            (should (equal result expected)))))))
+            (if (should (equal result expected))
+                (kill-matching-buffers (format "\\*tramp/mock\\s-%s\\*" system-name) t t)))))))
 
-(ert-deftest ob-shell/results-table ()
+(ert-deftest test-ob-shell/results-table ()
   "Test :results table."
   (should
    (equal '(("I \"want\" it all"))
-	  (org-test-with-temp-text
-	      "#+BEGIN_SRC sh :results table\necho 'I \"want\" it all'\n#+END_SRC"
-	    (org-babel-execute-src-block)))))
-
-(ert-deftest ob-shell/results-list ()
+          (org-test-with-temp-text
+              (string-join
+               '("#+BEGIN_SRC sh :results table"
+                 "echo 'I \"want\" it all'"
+                 "#+END_SRC")
+               "\n")
+            (org-babel-execute-src-block)))))
+
+(ert-deftest test-ob-shell/results-list ()
   "Test :results list."
   (org-test-with-temp-text
-      "#+BEGIN_SRC sh :results list\necho 1\necho 2\necho 3\n#+END_SRC"
+      (string-join
+       '("#+BEGIN_SRC sh :results list"
+         "echo 1"
+         "echo 2"
+         "echo 3"
+         "#+END_SRC")
+       "\n")
     (should
      (equal '((1) (2) (3))
             (org-babel-execute-src-block)))
@@ -253,86 +299,98 @@ string. Bash cannot handle lists in associative arrays."
 
 ;;; Standard output
 
-(ert-deftest ob-shell/standard-output-after-success ()
+(ert-deftest test-ob-shell/standard-output-after-success ()
   "Test standard output after exiting with a zero code."
   (should (= 1
              (org-babel-execute:sh
               "echo 1" nil))))
 
-(ert-deftest ob-shell/standard-output-after-failure ()
+(ert-deftest test-ob-shell/standard-output-after-failure ()
   "Test standard output after exiting with a non-zero code."
-  (should (= 1
-             (org-babel-execute:sh
-              "echo 1; exit 2" nil))))
+  (if
+      (should (= 1
+                 (org-babel-execute:sh
+                  "echo 1; exit 2" nil)))
+      (kill-buffer "*Org-Babel Error Output*")))
 
 ;;; Standard error
 
-(ert-deftest ob-shell/error-output-after-success ()
-  "Test that standard error shows in the error buffer, alongside the
-exit code, after exiting with a zero code."
-  (should
-   (string= "1
+(ert-deftest test-ob-shell/error-output-after-success ()
+  "Test that standard error shows in the error buffer, alongside
+the exit code, after exiting with a zero code."
+  (if
+      (should
+       (string= "1
 [ Babel evaluation exited with code 0 ]"
-            (progn (org-babel-eval-wipe-error-buffer)
-                   (org-babel-execute:sh
-                    "echo 1 >&2" nil)
-                   (with-current-buffer org-babel-error-buffer-name
-                     (buffer-string))))))
-
-(ert-deftest ob-shell/error-output-after-failure ()
-  "Test that standard error shows in the error buffer, alongside the
-exit code, after exiting with a non-zero code."
-  (should
-   (string= "1
+                (progn (org-babel-eval-wipe-error-buffer)
+                       (org-babel-execute:sh
+                        "echo 1 >&2" nil)
+                       (with-current-buffer org-babel-error-buffer-name
+                         (buffer-string)))))
+      (kill-buffer "*Org-Babel Error Output*")))
+
+(ert-deftest test-ob-shell/error-output-after-failure ()
+  "Test that standard error shows in the error buffer, alongside
+the exit code, after exiting with a non-zero code."
+  (if
+      (should
+       (string= "1
 [ Babel evaluation exited with code 2 ]"
-            (progn (org-babel-eval-wipe-error-buffer)
-                   (org-babel-execute:sh
-                    "echo 1 >&2; exit 2" nil)
-                   (with-current-buffer org-babel-error-buffer-name
-                     (buffer-string))))))
+                (progn (org-babel-eval-wipe-error-buffer)
+                       (org-babel-execute:sh
+                        "echo 1 >&2; exit 2" nil)
+                       (with-current-buffer org-babel-error-buffer-name
+                         (buffer-string)))))
+      (kill-buffer "*Org-Babel Error Output*")))
 
-(ert-deftest ob-shell/error-output-after-failure-multiple ()
+(ert-deftest test-ob-shell/error-output-after-failure-multiple ()
   "Test that multiple standard error strings show in the error
 buffer, alongside multiple exit codes."
-  (should
-   (string= "1
+  (if
+      (should
+       (string= "1
 [ Babel evaluation exited with code 2 ]
 3
 [ Babel evaluation exited with code 4 ]"
-            (progn (org-babel-eval-wipe-error-buffer)
-                   (org-babel-execute:sh
-                    "echo 1 >&2; exit 2" nil)
-                   (org-babel-execute:sh
-                    "echo 3 >&2; exit 4" nil)
-                   (with-current-buffer org-babel-error-buffer-name
-                     (buffer-string))))))
+                (progn (org-babel-eval-wipe-error-buffer)
+                       (org-babel-execute:sh
+                        "echo 1 >&2; exit 2" nil)
+                       (org-babel-execute:sh
+                        "echo 3 >&2; exit 4" nil)
+                       (with-current-buffer org-babel-error-buffer-name
+                         (buffer-string)))))
+      (kill-buffer "*Org-Babel Error Output*")))
 
 ;;; Exit codes
 
-(ert-deftest ob-shell/exit-code ()
+(ert-deftest test-ob-shell/exit-code ()
   "Test that the exit code shows in the error buffer after exiting
 with a non-zero return code."
-  (should
-   (string= "[ Babel evaluation exited with code 1 ]"
-            (progn (org-babel-eval-wipe-error-buffer)
-                   (org-babel-execute:sh
-                    "exit 1" nil)
-                   (with-current-buffer org-babel-error-buffer-name
-                     (buffer-string))))))
-
-(ert-deftest ob-shell/exit-code-multiple ()
+  (if
+      (should
+       (string= "[ Babel evaluation exited with code 1 ]"
+                (progn (org-babel-eval-wipe-error-buffer)
+                       (org-babel-execute:sh
+                        "exit 1" nil)
+                       (with-current-buffer org-babel-error-buffer-name
+                         (buffer-string)))))
+      (kill-buffer "*Org-Babel Error Output*")))
+
+(ert-deftest test-ob-shell/exit-code-multiple ()
   "Test that multiple exit codes show in the error buffer after
 exiting with a non-zero return code multiple times."
-  (should
-   (string= "[ Babel evaluation exited with code 1 ]
+  (if
+      (should
+       (string= "[ Babel evaluation exited with code 1 ]
 [ Babel evaluation exited with code 2 ]"
-            (progn (org-babel-eval-wipe-error-buffer)
-                   (org-babel-execute:sh
-                    "exit 1" nil)
-                   (org-babel-execute:sh
-                    "exit 2" nil)
-                   (with-current-buffer org-babel-error-buffer-name
-                     (buffer-string))))))
+                (progn (org-babel-eval-wipe-error-buffer)
+                       (org-babel-execute:sh
+                        "exit 1" nil)
+                       (org-babel-execute:sh
+                        "exit 2" nil)
+                       (with-current-buffer org-babel-error-buffer-name
+                         (buffer-string)))))
+      (kill-buffer "*Org-Babel Error Output*")))
 
 (provide 'test-ob-shell)
 
-- 
2.38.1


[-- Attachment #7: 0006-ob-shell-test.org-Remove-ob-shell-test.org.patch --]
[-- Type: application/octet-stream, Size: 3136 bytes --]

From ec1653eed46254fba0596d494da9142fcea6da13 Mon Sep 17 00:00:00 2001
From: Matt Trzcinski <matt@excalamus.com>
Date: Sat, 24 Dec 2022 11:29:06 -0500
Subject: [PATCH 6/7] ob-shell-test.org: Remove ob-shell-test.org

* testing/examples/ob-shell-test.org: Delete file

File was used by test-ob-shell.el to test ob-shell.el.  Tests have
been updated to use temporary buffers, rendering ob-shell-test.org
unnecessary.
---
 testing/examples/ob-shell-test.org | 88 ------------------------------
 1 file changed, 88 deletions(-)
 delete mode 100644 testing/examples/ob-shell-test.org

diff --git a/testing/examples/ob-shell-test.org b/testing/examples/ob-shell-test.org
deleted file mode 100644
index 2510f4f96..000000000
--- a/testing/examples/ob-shell-test.org
+++ /dev/null
@@ -1,88 +0,0 @@
-#+Title: a collection of examples for ob-shell tests
-#+OPTIONS: ^:nil
-
-* Sample data structures
-#+NAME: sample_array
-| one   |
-| two   |
-| three |
-
-#+NAME: sample_mapping_table
-| first  | one   |
-| second | two   |
-| third  | three |
-
-#+NAME: sample_big_table
-| bread     |  2 | kg |
-| spaghetti | 20 | cm |
-| milk      | 50 | dl |
-
-* Array tests
-  :PROPERTIES:
-  :ID:       0ba56632-8dc1-405c-a083-c204bae477cf
-  :END:
-** Generic shell: no arrays
-#+begin_src sh :exports results :results output :var array=sample_array
-echo ${array}
-#+end_src
-
-#+RESULTS:
-: one two three
-
-** Bash shell: support for arrays
-Bash will see a simple indexed array. In this test, we check that the
-returned value is indeed only the first item of the array, as opposed to
-the generic serialiation that will return all elements of the array as 
-a single string.
-#+begin_src bash :exports results :results output :var array=sample_array
-echo ${array}
-#+end_src
-
-#+RESULTS:
-: one
-
-* Associative array tests (simple map)
-  :PROPERTIES:
-  :ID:       bec1a5b0-4619-4450-a8c0-2a746b44bf8d
-  :END:
-** Generic shell: no special handing
-The shell will see all values as a single string.
-#+begin_src sh :exports results :results output :var table=sample_mapping_table
-echo ${table}
-#+end_src
-
-#+RESULTS:
-: first one second two third three
-
-** Bash shell: support for associative arrays
-Bash will see a table that contains the first column as the 'index'
-of the associative array, and the second column as the value.
-#+begin_src bash :exports :results output results :var table=sample_mapping_table
-echo ${table[second]}
-#+end_src
-
-#+RESULTS:
-: two
-
-* Associative array tests (more than 2 columns)
-  :PROPERTIES:
-  :ID:       82320a48-3409-49d7-85c9-5de1c6d3ff87
-  :END:
-** Generic shell: no special handing
-#+begin_src sh :exports results :results output :var table=sample_big_table
-echo ${table}
-#+end_src
-
-#+RESULTS:
-: bread 2 kg spaghetti 20 cm milk 50 dl
-   
-** Bash shell: support for associative arrays with lists
-Bash will see an associative array that contains each row as a single
-string. Bash cannot handle lists in associative arrays.
-#+begin_src bash :exports results :results output :var table=sample_big_table
-echo ${table[spaghetti]}
-#+end_src
-
-#+RESULTS:
-: 20 cm
-
-- 
2.38.1


[-- Attachment #8: 0007-test-ob-shell.el-Organize-tests.patch --]
[-- Type: application/octet-stream, Size: 2018 bytes --]

From d50f940799fb5d6f2da5712a6423e498db3c530d Mon Sep 17 00:00:00 2001
From: Matt Trzcinski <matt@excalamus.com>
Date: Fri, 30 Dec 2022 13:03:59 -0500
Subject: [PATCH 7/7] test-ob-shell.el: Organize tests

* testing/lisp/test-ob-shell.el:
  - Give instructions on how to run tests
  - Require `org-test' explicitly
  - Group tests into sections
  - Insert `page-delimiter's between sections
---
 testing/lisp/test-ob-shell.el | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el
index 2c4491ffa..d1244c9e4 100644
--- a/testing/lisp/test-ob-shell.el
+++ b/testing/lisp/test-ob-shell.el
@@ -18,17 +18,22 @@
 ;; You should have received a copy of the GNU General Public License
 ;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
 
-
+\f
 ;;; Comment:
 
+;; See testing/README for how to run tests.
+
+\f
 ;;; Requirements:
-(org-test-for-executable "sh")
 
 (require 'ob-core)
 
 (unless (featurep 'ob-shell)
   (signal 'missing-test-dependency "Support for Shell code blocks"))
 
+(org-test-for-executable "sh")
+
+\f
 ;;; Code:
 (ert-deftest test-ob-shell/dont-insert-spaces-on-expanded-bodies ()
   "Expanded shell bodies should not start with a blank line unless
@@ -297,6 +302,7 @@ string. Bash cannot handle lists in associative arrays."
       "- 1\n- 2\n- 3\n"
       (buffer-substring-no-properties (point) (point-max))))))
 
+\f
 ;;; Standard output
 
 (ert-deftest test-ob-shell/standard-output-after-success ()
@@ -313,6 +319,7 @@ string. Bash cannot handle lists in associative arrays."
                   "echo 1; exit 2" nil)))
       (kill-buffer "*Org-Babel Error Output*")))
 
+\f
 ;;; Standard error
 
 (ert-deftest test-ob-shell/error-output-after-success ()
@@ -361,6 +368,7 @@ buffer, alongside multiple exit codes."
                          (buffer-string)))))
       (kill-buffer "*Org-Babel Error Output*")))
 
+\f
 ;;; Exit codes
 
 (ert-deftest test-ob-shell/exit-code ()
-- 
2.38.1