* Org 9.6-pre and Bash sessions @ 2022-10-13 15:14 Rudolf Adamkovič 2022-10-13 17:07 ` Michael Welle 2022-10-14 3:44 ` Ihor Radchenko 0 siblings, 2 replies; 37+ messages in thread From: Rudolf Adamkovič @ 2022-10-13 15:14 UTC (permalink / raw) To: emacs-orgmode Hello there! I noticed today that Bash source blocks with ':session" does not seem to work with Org 9.6-pre (3e8648775). Does anyone else have the problem? CONFIGURATION FILE: ------------------ > (setq-default org-use-extra-keys t) > (with-eval-after-load 'org > (add-to-list 'org-babel-load-languages '(shell . t)) > ;; The weirdest API ever. > (org-babel-do-load-languages > 'org-babel-load-languages > org-babel-load-languages)) TEST FILE: --------- > * Something > > 1. Something > > #+begin_src bash :session test :results output > if true > then > echo "hello" > fi > #+end_src REPRODUCTION STEPS: ------------------ Execute the source block with C-c C-c. Expected: Block executes with the result "hello". Actual: Emacs says "Executing Bash unknown at position 22..." and freezes. (C-g unfreezes Emacs.) NOTES: ----- When I remove the leading whitespace ... > * Something > > 1. Something > > #+begin_src bash :session test :results output > if true > then > echo "hello" > fi > #+end_src ... then Emacs still freezes, but at least the message makes sense, saying "Bash code block" instead of "Bash unknown". Rudy -- "Simplicity is complexity resolved." -- Constantin Brâncuși, 1876-1957 Rudolf Adamkovič <salutis@me.com> [he/him] Studenohorská 25 84103 Bratislava Slovakia ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Org 9.6-pre and Bash sessions 2022-10-13 15:14 Org 9.6-pre and Bash sessions Rudolf Adamkovič @ 2022-10-13 17:07 ` Michael Welle 2022-10-14 3:44 ` Ihor Radchenko 1 sibling, 0 replies; 37+ messages in thread From: Michael Welle @ 2022-10-13 17:07 UTC (permalink / raw) To: emacs-orgmode Hello, Rudolf Adamkovič <salutis@me.com> writes: > Hello there! > > I noticed today that Bash source blocks with ':session" does not seem to > work with Org 9.6-pre (3e8648775). Does anyone else have the problem? funnily I observed something similar while generating semester projects for my students. Example: #+name: seprj_conf_table | xxxzwei | 234 2bar | | xxxeins | 2342bar | | xxxdrei | aaaa${eins}bbbb | #+begin_src sh :session foo :var conf=seprj_conf_table :separator = IFS=$'^\n\t' for val in "$conf" ; do eval "export '$val'" done env |grep xxx #+end_src With main this hangs until C-g. I have an other machine with a little bit older Org: commit 9b62bcc9c60302467d1d45b6c240057e80fce656 (HEAD) Author: Lars Ingebrigtsen <larsi@gnus.org> Date: Sat Sep 24 15:17:23 2022 +0200 Backport commit 8dacd8cd9 from Emacs That works as expected. So I guess the regression was introduced within the last week or two. Regards hmw ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Org 9.6-pre and Bash sessions 2022-10-13 15:14 Org 9.6-pre and Bash sessions Rudolf Adamkovič 2022-10-13 17:07 ` Michael Welle @ 2022-10-14 3:44 ` Ihor Radchenko 2022-10-15 20:56 ` Rudolf Adamkovič 1 sibling, 1 reply; 37+ messages in thread From: Ihor Radchenko @ 2022-10-14 3:44 UTC (permalink / raw) To: Rudolf Adamkovič; +Cc: emacs-orgmode Rudolf Adamkovič <salutis@me.com> writes: > I noticed today that Bash source blocks with ':session" does not seem to > work with Org 9.6-pre (3e8648775). Does anyone else have the problem? Confirmed. The cause is a35d16368 Author: Ihor Radchenko <yantar92@gmail.com> ob-shell: Fix output containing strings matching `comint-prompt-regexp' * lisp/ob-comint.el (org-babel-comint-wait-for-output): Do not rely on `face-at-point' returning non-nil. * lisp/ob-shell.el (org-babel-shell-set-prompt-commands): New constant holding shell-specific commands to change prompt. (org-babel-prompt-command): New variable holding command to be user to set distinguishable prompt. (org-babel-shell-initialize): Set `org-babel-prompt-command' according to shell name. (org-babel-sh-prompt): New variable holding default shell prompt. (org-babel-sh-initiate-session): Change the default prompt to `org-babel-sh-prompt' and alter `comint-prompt-regexp' to match it tightly. Fixes https://list.orgmode.org/CKK9TULBP2BG.2UITT31YJV03J@laptop/T/#mc8e3ca2f5f1b9a94040a68b4c6201234b209041c > CONFIGURATION FILE: > ------------------ > >> (setq-default org-use-extra-keys t) >> (with-eval-after-load 'org >> (add-to-list 'org-babel-load-languages '(shell . t)) >> ;; The weirdest API ever. >> (org-babel-do-load-languages >> 'org-babel-load-languages >> org-babel-load-languages)) > > TEST FILE: > --------- > >> * Something >> >> 1. Something >> >> #+begin_src bash :session test :results output >> if true >> then >> echo "hello" >> fi >> #+end_src > > REPRODUCTION STEPS: > ------------------ > > Execute the source block with C-c C-c. > > Expected: > > Block executes with the result "hello". Is it what you actually saw in the past? I just tried with older Org versions on my system and the output is : [00m > > > hello -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Org 9.6-pre and Bash sessions 2022-10-14 3:44 ` Ihor Radchenko @ 2022-10-15 20:56 ` Rudolf Adamkovič 2022-10-17 8:34 ` Ihor Radchenko 0 siblings, 1 reply; 37+ messages in thread From: Rudolf Adamkovič @ 2022-10-15 20:56 UTC (permalink / raw) To: Ihor Radchenko; +Cc: emacs-orgmode Ihor Radchenko <yantar92@gmail.com> writes: > Is it what you actually saw in the past? I just tried with older Org > versions on my system and the output is > > : [00m > > > hello No. By "expected", I mean what I expect as a user (and what I would assert on in a regression test as a developer). For the record, with Org in Emacs 29, I get: : : > > > hello on the first run and then : : > > hello on the subsequent runs. Better than the new version but still wrong. :) Rudy -- "Genius is 1% inspiration and 99% perspiration." -- Thomas Alva Edison, 1932 Rudolf Adamkovič <salutis@me.com> [he/him] Studenohorská 25 84103 Bratislava Slovakia ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Org 9.6-pre and Bash sessions 2022-10-15 20:56 ` Rudolf Adamkovič @ 2022-10-17 8:34 ` Ihor Radchenko 2022-10-21 5:29 ` Ihor Radchenko 0 siblings, 1 reply; 37+ messages in thread From: Ihor Radchenko @ 2022-10-17 8:34 UTC (permalink / raw) To: Rudolf Adamkovič; +Cc: Ihor Radchenko, emacs-orgmode Rudolf Adamkovič <salutis@me.com> writes: > : > : > > hello > > on the subsequent runs. > > Better than the new version but still wrong. :) And this is what drove me crazy during debugging. I do not understand the logic of all that ob-comint code. I have identified that the hang happens because Org does not change PS2 prompt. Just PS1. But fixing this would yield to : org_babel_sh_prompt> org_babel_sh_prompt> hello Then, I tried to see how the original code works. And it does not :facepalm: I asked emacs-devel https://yhetil.org/emacs-devel/87y1tgqhmc.fsf@localhost/T/#u No clue how to read the script output programatically in comint buffer. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Org 9.6-pre and Bash sessions 2022-10-17 8:34 ` Ihor Radchenko @ 2022-10-21 5:29 ` Ihor Radchenko 2022-10-21 13:38 ` Rudolf Adamkovič 0 siblings, 1 reply; 37+ messages in thread From: Ihor Radchenko @ 2022-10-21 5:29 UTC (permalink / raw) To: Rudolf Adamkovič; +Cc: Ihor Radchenko, emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 811 bytes --] Ihor Radchenko <yantar92@posteo.net> writes: > Rudolf Adamkovič <salutis@me.com> writes: > >> : >> : > > hello >> >> on the subsequent runs. >> >> Better than the new version but still wrong. :) > > And this is what drove me crazy during debugging. I do not understand > the logic of all that ob-comint code. > > I have identified that the hang happens because Org does not change PS2 > prompt. Just PS1. But fixing this would yield to > > : org_babel_sh_prompt> org_babel_sh_prompt> hello > > Then, I tried to see how the original code works. And it does not > :facepalm: > > I asked emacs-devel > https://yhetil.org/emacs-devel/87y1tgqhmc.fsf@localhost/T/#u See the attached tentative patch. I'd appreciate some testing. Hopefully, I did not break anything. Comint is tricky. [-- Attachment #2: 0001-ob-shell-Fix-multi-line-scripts-in-sessions.patch --] [-- Type: text/x-patch, Size: 4766 bytes --] From 4c6eead351cbdce1b9210a738c65b3a139d1cc0c Mon Sep 17 00:00:00 2001 Message-Id: <4c6eead351cbdce1b9210a738c65b3a139d1cc0c.1666330028.git.yantar92@posteo.net> From: Ihor Radchenko <yantar92@posteo.net> Date: Fri, 21 Oct 2022 13:21:57 +0800 Subject: [PATCH] ob-shell: Fix multi-line scripts in sessions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * lisp/ob-comint.el (org-babel-comint-with-output): Cleanup empty output. Such output is spitted unnecessarily for multi-line scripts. * lisp/ob-shell.el (org-babel-shell-set-prompt-commands): Disable PS2 and equivalent prompts. Make sure that PROMPT_COMMAND does not interfere with PS1 setting in POSIX shells. (org-babel-sh-evaluate): Do not send input line-by-line. Instead, let `org-babel-coming-with-output' handle waiting for the output as well as recording it. Update to the new `org-babel-coming-with-output' behavior of cleaning empty outputs. * testing/lisp/test-ob-shell.el (test-ob-shell/session): Add test. Reported-by: Rudolf Adamkovič <salutis@me.com> Link: https://list.orgmode.org/orgmode/m2r0zboix1.fsf@me.com/ --- lisp/ob-comint.el | 2 +- lisp/ob-shell.el | 29 ++++++++++++++--------------- testing/lisp/test-ob-shell.el | 6 +++++- 3 files changed, 20 insertions(+), 17 deletions(-) diff --git a/lisp/ob-comint.el b/lisp/ob-comint.el index 36a55d36c..ec7d3f6c9 100644 --- a/lisp/ob-comint.el +++ b/lisp/ob-comint.el @@ -109,7 +109,7 @@ (defmacro org-babel-comint-with-output (meta &rest body) "\n" "[\r\n]+" (regexp-quote (or ,full-body ""))) string-buffer)) (setq string-buffer (substring string-buffer (match-end 0)))) - (split-string string-buffer comint-prompt-regexp))))) + (delete "" (split-string string-buffer comint-prompt-regexp)))))) (defun org-babel-comint-input-command (buffer cmd) "Pass CMD to BUFFER. diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el index 4d579ae87..d38d2d335 100644 --- a/lisp/ob-shell.el +++ b/lisp/ob-shell.el @@ -47,10 +47,15 @@ (defvar org-babel-default-header-args:shell '()) (defvar org-babel-shell-names) (defconst org-babel-shell-set-prompt-commands - '(("fish" . "function fish_prompt\n\techo \"%s\"\nend") - ("csh" . "set prompt=\"%s\"") + '(;; Fish has no PS2 equivalent. + ("fish" . "function fish_prompt\n\techo \"%s\"\nend") + ;; prompt2 is like PS2 in POSIX shells. + ("csh" . "set prompt=\"%s\"\nset prompt2=\"\"") + ;; PowerShell, similar to fish, does not have PS2 equivalent. ("posh" . "function prompt { \"%s\" }") - (t . "PS1=\"%s\"")) + ;; PROMPT_COMMAND can override PS1 settings. Disable it. + ;; Disable PS2 to avoid garbage in multi-line inputs. + (t . "PROMPT_COMMAND=;PS1=\"%s\";PS2=")) "Alist assigning shells with their prompt setting command. Each element of the alist associates a shell type from @@ -299,20 +304,14 @@ (defun org-babel-sh-evaluate (session body &optional params stdin cmdline) #'org-babel-sh-strip-weird-long-prompt (mapcar #'org-trim - (butlast + (butlast ; Remove eoe indicator (org-babel-comint-with-output (session org-babel-sh-eoe-output t body) - (dolist (line (append (split-string (org-trim body) "\n") - (list org-babel-sh-eoe-indicator))) - (insert line) - (comint-send-input nil t) - (while (save-excursion - (goto-char comint-last-input-end) - (not (re-search-forward - comint-prompt-regexp nil t))) - (accept-process-output - (get-buffer-process (current-buffer)))))) - 2)) + (insert (org-trim body) "\n" + org-babel-sh-eoe-indicator) + (comint-send-input nil t)) + ;; Remove `org-babel-sh-eoe-indicator' output line. + 1)) "\n")) ;; External shell script, with or without a predefined ;; shebang. diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el index a0d5a8d22..05c369174 100644 --- a/testing/lisp/test-ob-shell.el +++ b/testing/lisp/test-ob-shell.el @@ -46,7 +46,11 @@ (ert-deftest test-ob-shell/session () ob-comint.el, which was not previously tested." (let ((res (org-babel-execute:sh "echo 1; echo 2" '((:session . "yes"))))) (should res) - (should (listp res)))) + (should (listp res))) + ;; Test multi-line input. + (let ((res (org-babel-execute:sh "if true; then\necho \"yes\"\nfi" '((:session . "yes"))))) + (should res) + (should (string= "yes" res)))) ; A list of tests using the samples in ob-shell-test.org (ert-deftest ob-shell/generic-uses-no-arrays () -- 2.35.1 [-- Attachment #3: Type: text/plain, Size: 225 bytes --] -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply related [flat|nested] 37+ messages in thread
* Re: Org 9.6-pre and Bash sessions 2022-10-21 5:29 ` Ihor Radchenko @ 2022-10-21 13:38 ` Rudolf Adamkovič 2022-10-22 4:22 ` Ihor Radchenko 0 siblings, 1 reply; 37+ messages in thread From: Rudolf Adamkovič @ 2022-10-21 13:38 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Ihor Radchenko, emacs-orgmode Ihor Radchenko <yantar92@posteo.net> writes: > See the attached tentative patch. > I'd appreciate some testing. Hopefully, I did not break anything. I did some testing and found no issues. Great work! Below, I include some suggestions regarding the patch. > * lisp/ob-comint.el (org-babel-comint-with-output): Cleanup empty > output. Such output is spitted unnecessarily for multi-line scripts. - 'cleanup' (noun) -> 'clean up' (verb) - 'spitted' -> ??? (did you mean 'split', perhaps?) > * testing/lisp/test-ob-shell.el (test-ob-shell/session): Add test. - 'test' -> 'a test' > + (let ((res (org-babel-execute:sh "if true; then\necho \"yes\"\nfi" '((:session . "yes"))))) This line has 93 columns, which makes it needlessly hard to read. Consider, (let ((result (org-babel-execute:sh "if true \n then \n echo yes \n fi" '((:session . "yes"))))) (should result) (should (string= "yes" result))) I replaced the remaining semicolons with newlines, given the test focuses on "multi-line scripts". Further, I removed the needless (escaped) quotes around "yes". Lastly, I renamed 'res' to 'result', improving readability and consistency, given the file already uses 'result' in another 'let'. Rudy -- "Programming reliably -- must be an activity of an undeniably mathematical nature […] You see, mathematics is about thinking, and doing mathematics is always trying to think as well as possible." -- Edsger W. Dijkstra, 1981 Rudolf Adamkovič <salutis@me.com> [he/him] Studenohorská 25 84103 Bratislava Slovakia ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Org 9.6-pre and Bash sessions 2022-10-21 13:38 ` Rudolf Adamkovič @ 2022-10-22 4:22 ` Ihor Radchenko 2022-10-22 9:44 ` Rudolf Adamkovič 0 siblings, 1 reply; 37+ messages in thread From: Ihor Radchenko @ 2022-10-22 4:22 UTC (permalink / raw) To: Rudolf Adamkovič; +Cc: Ihor Radchenko, emacs-orgmode Rudolf Adamkovič <salutis@me.com> writes: > Ihor Radchenko <yantar92@posteo.net> writes: > >> See the attached tentative patch. >> I'd appreciate some testing. Hopefully, I did not break anything. > > I did some testing and found no issues. > > Great work! > > Below, I include some suggestions regarding the patch. Thanks! Applied onto main incorporating your suggestions. https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=801c93638ab4b6616567035c6438ad229bc14c1d -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Org 9.6-pre and Bash sessions 2022-10-22 4:22 ` Ihor Radchenko @ 2022-10-22 9:44 ` Rudolf Adamkovič 2022-10-22 10:59 ` Ihor Radchenko 0 siblings, 1 reply; 37+ messages in thread From: Rudolf Adamkovič @ 2022-10-22 9:44 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Ihor Radchenko, emacs-orgmode Ihor Radchenko <yantar92@posteo.net> writes: >> I did some testing and found no issues. Update: Today I needed to run some Bash, and it seems broken. PROBLEM 1: WITHOUT A SESSION ============================ CONFIGURATION: Emacs 29, Org from Git (2f5e7103e59f06631e985d3dd39af21b5b7464ea) REPRODUCTION STEPS: Run the following code block: #+begin_src bash :results output echo one > one.txt echo two > two.txt diff one.txt two.txt #+end_src ACTUAL RESULTS: #+RESULTS: EXPECTED RESULTS: #+RESULTS: : 1c1 : < one : --- : > two PROBLEM 2: WITH A SESSION ========================= CONFIGURATION: Emacs 29, Org from Git (2f5e7103e59f06631e985d3dd39af21b5b7464ea) REPRODUCTION STEPS: #+begin_src bash :session bash :results output echo one > one.txt echo two > two.txt diff one.txt two.txt #+end_src ACTUAL: #+RESULTS: : org_babel_sh_prompt> 1c1 : < one : --- : > two EXPECTED: #+RESULTS: : 1c1 : < one : --- : > two ADDENDUM ======== A friend of mine, who uses Org that comes with Emacs 28.1 on Linux, tried the PROBLEM 1 source block shown above, the one *without* a session, and he got no results either. (We might need better tests.) Rudy -- "One can begin to reason only when a clear picture has been formed in the imagination." -- Walter Warwick Sawyer, Mathematician's Delight, 1943 Rudolf Adamkovič <salutis@me.com> [he/him] Studenohorská 25 84103 Bratislava Slovakia ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Org 9.6-pre and Bash sessions 2022-10-22 9:44 ` Rudolf Adamkovič @ 2022-10-22 10:59 ` Ihor Radchenko 2022-10-23 4:27 ` Ihor Radchenko 0 siblings, 1 reply; 37+ messages in thread From: Ihor Radchenko @ 2022-10-22 10:59 UTC (permalink / raw) To: Rudolf Adamkovič; +Cc: Ihor Radchenko, emacs-orgmode Rudolf Adamkovič <salutis@me.com> writes: > Ihor Radchenko <yantar92@posteo.net> writes: > >>> I did some testing and found no issues. > > Update: Today I needed to run some Bash, and it seems broken. > > PROBLEM 1: WITHOUT A SESSION > ============================ > > CONFIGURATION: > > Emacs 29, Org from Git (2f5e7103e59f06631e985d3dd39af21b5b7464ea) > > REPRODUCTION STEPS: > > Run the following code block: > > #+begin_src bash :results output > echo one > one.txt > echo two > two.txt > diff one.txt two.txt > #+end_src > > ACTUAL RESULTS: > > #+RESULTS: > > EXPECTED RESULTS: > > #+RESULTS: > : 1c1 > : < one > : --- > : > two This is actually expected. Without session, ob-shell discards results of src blocks that fail and display *Error* window (empty in this case). If you try #+begin_src bash :results output echo one > one.txt echo two > two.txt diff one.txt two.txt || true #+end_src things will work as expected. I guess we can display a bit more info in `org-babel-eval'. > PROBLEM 2: WITH A SESSION > ========================= > > CONFIGURATION: > > Emacs 29, Org from Git (2f5e7103e59f06631e985d3dd39af21b5b7464ea) > > REPRODUCTION STEPS: > > #+begin_src bash :session bash :results output > echo one > one.txt > echo two > two.txt > diff one.txt two.txt > #+end_src > > ACTUAL: > > #+RESULTS: > : org_babel_sh_prompt> 1c1 > : < one > : --- > : > two > > EXPECTED: > > #+RESULTS: > : 1c1 > : < one > : --- > : > two Confirmed. I know what is the problem. It is comint sometimes throwing output without newlines, which leads to comint-prompt-regexp not matching all the prompts after concatenating. Will take a look tomorrow. > ADDENDUM > ======== > > A friend of mine, who uses Org that comes with Emacs 28.1 on Linux, > tried the PROBLEM 1 source block shown above, the one *without* a > session, and he got no results either. > > (We might need better tests.) Better tests are always needed :) https://orgmode.org/list/874k81bsvz.fsf@localhost -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Org 9.6-pre and Bash sessions 2022-10-22 10:59 ` Ihor Radchenko @ 2022-10-23 4:27 ` Ihor Radchenko 2022-10-26 11:56 ` Rudolf Adamkovič 0 siblings, 1 reply; 37+ messages in thread From: Ihor Radchenko @ 2022-10-23 4:27 UTC (permalink / raw) To: Rudolf Adamkovič; +Cc: Ihor Radchenko, emacs-orgmode Ihor Radchenko <yantar92@posteo.net> writes: > This is actually expected. Without session, ob-shell discards results of > src blocks that fail and display *Error* window (empty in this case). > > If you try > > #+begin_src bash :results output > echo one > one.txt > echo two > two.txt > diff one.txt two.txt || true > #+end_src > > things will work as expected. > > I guess we can display a bit more info in `org-babel-eval'. I improved the error buffer a bit and also made babel return the available output with non-0 exit code. https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=ab7eff9d9c0968392bd9783d72c56a4023bd10be https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=f8a9cd23087799519bd709ea5c208ed29cb6587f >> #+begin_src bash :session bash :results output >> echo one > one.txt >> echo two > two.txt >> diff one.txt two.txt >> #+end_src >> >> ACTUAL: >> >> #+RESULTS: >> : org_babel_sh_prompt> 1c1 >> : < one >> : --- >> : > two >> >> EXPECTED: >> >> #+RESULTS: >> : 1c1 >> : < one >> : --- >> : > two > > Confirmed. > I know what is the problem. It is comint sometimes throwing output > without newlines, which leads to comint-prompt-regexp not matching all > the prompts after concatenating. Will take a look tomorrow. Fixed. https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=1ef420b19d2e310d9ef67e090ef84cdb6da4744d -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Org 9.6-pre and Bash sessions 2022-10-23 4:27 ` Ihor Radchenko @ 2022-10-26 11:56 ` Rudolf Adamkovič 2022-10-26 13:21 ` Rudolf Adamkovič 2022-10-27 3:53 ` Ihor Radchenko 0 siblings, 2 replies; 37+ messages in thread From: Rudolf Adamkovič @ 2022-10-26 11:56 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Ihor Radchenko, emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 949 bytes --] Ihor, Thank you for investigating, explaining, and also fixing the problem! I pulled the latest 'main' and everything works a bit, it seems. Then, to avoid walking in circles, I decided to write some tests, for I think shell blocks should never, never, never break in such basic ways, let alone in production Emacs. Please, see the WIP patch attached to this message. I noticed that the following tests do not pass: 1. ob-shell/error-output-after-success We seem to trash error output, such as warnings, on success. I think we should not do this. Now, on the execution of "echo X &>2", Org says "Code block produced no output." But that does hold true. The block did produce output, just on the other output stream, namely error output. 2. ob-shell/error-output-after-failure We seem not to show the exit code in this case. Why? 3. ob-shell/exit-codes Should we add a newline after the exit code message? Rudy [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-WIP-test-ob-shell-Add-tests-for-some-recent-changes.patch --] [-- Type: text/x-patch, Size: 3021 bytes --] From 90432170552a6e922d48d51138b2062aa11e5575 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Rudolf=20Adamkovi=C4=8D?= <salutis@me.com> Date: Wed, 26 Oct 2022 13:35:26 +0200 Subject: [PATCH] WIP test-ob-shell: Add tests for some recent changes --- testing/lisp/test-ob-shell.el | 60 +++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el index 4c00faa49..8cd967343 100644 --- a/testing/lisp/test-ob-shell.el +++ b/testing/lisp/test-ob-shell.el @@ -170,6 +170,66 @@ ob-comint.el, which was not previously tested." "#+BEGIN_SRC sh :results table\necho 'I \"want\" it all'\n#+END_SRC" (org-babel-execute-src-block))))) +;;; Standard output + +(ert-deftest ob-shell/standard-output-after-failure () + "Test standard output after exiting with a non-zero code." + (should (= 1 + (org-babel-execute:sh + "echo 1; exit 2" nil)))) + +;;; Error output + +(ert-deftest ob-shell/error-output-after-success () + "Test that error output shows in the error buffer after exiting +with a zero code." + (should + (string= "1 +[ Babel evaluation exited with code 2 ]" + (progn (org-babel-eval-wipe-error-buffer) + (org-babel-execute:sh + "echo 1 >&2" nil) + (with-current-buffer org-babel-error-buffer-name + (buffer-string)))))) + +(ert-deftest ob-shell/error-output-after-failure () + "Test that error output shows in the error buffer, alongside the +exit code, after exiting with a non-zero code." + (should + (string= "1 +[ Babel evaluation exited with code 2 ]" + (progn (org-babel-eval-wipe-error-buffer) + (org-babel-execute:sh + "echo 1 >&2; exit 2" nil) + (with-current-buffer org-babel-error-buffer-name + (buffer-string)))))) + +;;; Exit codes + +(ert-deftest ob-shell/exit-code () + "Test that the exit code shows in the error buffer after exiting +with a non-zero return code." + (should + (string= "[ Babel evaluation exited with code 1 ]" + (progn (org-babel-eval-wipe-error-buffer) + (org-babel-execute:sh + "exit 1" nil) + (with-current-buffer org-babel-error-buffer-name + (buffer-string)))))) + +(ert-deftest ob-shell/exit-codes () + "Test that multiple exit codes show in the error buffer after +exiting with a non-zero return code more than once." + (should + (string= "[ Babel evaluation exited with code 1 ] +[ Babel evaluation exited with code 2 ]" + (progn (org-babel-eval-wipe-error-buffer) + (org-babel-execute:sh + "exit 1" nil) + (org-babel-execute:sh + "exit 2" nil) + (with-current-buffer org-babel-error-buffer-name + (buffer-string)))))) (provide 'test-ob-shell) -- 2.38.1 [-- Attachment #3: Type: text/plain, Size: 246 bytes --] -- "One can begin to reason only when a clear picture has been formed in the imagination." -- Walter Warwick Sawyer, Mathematician's Delight, 1943 Rudolf Adamkovič <salutis@me.com> [he/him] Studenohorská 25 84103 Bratislava Slovakia ^ permalink raw reply related [flat|nested] 37+ messages in thread
* Re: Org 9.6-pre and Bash sessions 2022-10-26 11:56 ` Rudolf Adamkovič @ 2022-10-26 13:21 ` Rudolf Adamkovič 2022-10-27 3:53 ` Ihor Radchenko 1 sibling, 0 replies; 37+ messages in thread From: Rudolf Adamkovič @ 2022-10-26 13:21 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Ihor Radchenko, emacs-orgmode Rudolf Adamkovič <salutis@me.com> writes: > I pulled the latest 'main' and everything works a bit, it seems. I meant to say "a bit better" not "a bit". :) Rudy -- "It is no paradox to say that in our most theoretical moods we may be nearest to our most practical applications." -- Alfred North Whitehead, 1861-1947 Rudolf Adamkovič <salutis@me.com> [he/him] Studenohorská 25 84103 Bratislava Slovakia ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Org 9.6-pre and Bash sessions 2022-10-26 11:56 ` Rudolf Adamkovič 2022-10-26 13:21 ` Rudolf Adamkovič @ 2022-10-27 3:53 ` Ihor Radchenko 2022-10-28 13:12 ` Rudolf Adamkovič 1 sibling, 1 reply; 37+ messages in thread From: Ihor Radchenko @ 2022-10-27 3:53 UTC (permalink / raw) To: Rudolf Adamkovič; +Cc: Ihor Radchenko, emacs-orgmode Rudolf Adamkovič <salutis@me.com> writes: > Thank you for investigating, explaining, and also fixing the problem! I > pulled the latest 'main' and everything works a bit, it seems. > > Then, to avoid walking in circles, I decided to write some tests, for I > think shell blocks should never, never, never break in such basic ways, > let alone in production Emacs. Thanks! I will not look much into details of the patch for now. Let's discuss the general expectations from shell blocks first. You seem to be missing several things described in the documentation. > I noticed that the following tests do not pass: > > 1. ob-shell/error-output-after-success > > We seem to trash error output, such as warnings, on success. I think > we should not do this. Now, on the execution of "echo X &>2", Org > says "Code block produced no output." But that does hold true. The > block did produce output, just on the other output stream, namely > error output. Trashing error output is expected by default. I suggest you to read through https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-shell.html The default :results argument is output, which implies: output returns stdout, default As expected, stdout is empty in your example. ob-shell follows the spec here. Another question is if users may want to obtain strerr in some cases. A new feature like :results stderr might be implemented. Though users can already do the usual 2>&1 to merge stderr into stdout. > 2. ob-shell/error-output-after-failure > > We seem not to show the exit code in this case. Why? Exit code was not shown in the past until my commit https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=ab7eff9d9c0968392bd9783d72c56a4023bd10be I guess that showing the error code unconditionally (when exit code is non-0) could be an option. Now done in https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=d87a496b54b502d98e49f8bd6596e0562be9d105 > 3. ob-shell/exit-codes > > Should we add a newline after the exit code message? Done. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Org 9.6-pre and Bash sessions 2022-10-27 3:53 ` Ihor Radchenko @ 2022-10-28 13:12 ` Rudolf Adamkovič 2022-10-28 13:29 ` Ihor Radchenko 0 siblings, 1 reply; 37+ messages in thread From: Rudolf Adamkovič @ 2022-10-28 13:12 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Ihor Radchenko, emacs-orgmode Ihor Radchenko <yantar92@posteo.net> writes: >> 1. ob-shell/error-output-after-success >> >> We seem to trash error output, such as warnings, on success. I >> think we should not do this. Now, on the execution of "echo X >> &>2", Org says "Code block produced no output." But that does >> hold true. The block did produce output, just on the other output >> stream, namely error output. > Trashing error output is expected by default. [...] I suggest you to > read through [...] ob-shell follows the spec here. I get that ':results output' makes Org capture standard output, but not error output, for its #+RESULTS. However, if you look at the test in question, you will see that it expects error output in the *Org-Babel Error Output* buffer, not in the #+RESULTS. We should not trash error output; it belongs in the *Org-Babel Error Output* buffer. >> 2. ob-shell/error-output-after-failure > Now done in [...] Thanks! This works *correctly* now. >> 3. ob-shell/exit-codes >> >> Should we add a newline after the exit code message? > Done. I updated the tests and now every string needs a final newline, which does not seem correct. If we let the tests guide our design, all becomes clear. Buffers that use the 'compilation-mode' never append data on re-execution. Instead, they clear the buffer. Thus, we should 1. undo this change and instead 2. call 'org-babel-eval-wipe-error-buffer' before execution. Rudy -- "The whole science is nothing more than a refinement of everyday thinking." -- Albert Einstein, 1879-1955 Rudolf Adamkovič <salutis@me.com> [he/him] Studenohorská 25 84103 Bratislava Slovakia ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Org 9.6-pre and Bash sessions 2022-10-28 13:12 ` Rudolf Adamkovič @ 2022-10-28 13:29 ` Ihor Radchenko 2022-10-28 21:52 ` Rudolf Adamkovič 0 siblings, 1 reply; 37+ messages in thread From: Ihor Radchenko @ 2022-10-28 13:29 UTC (permalink / raw) To: Rudolf Adamkovič; +Cc: Ihor Radchenko, emacs-orgmode Rudolf Adamkovič <salutis@me.com> writes: >>> 1. ob-shell/error-output-after-success >>> >>> We seem to trash error output, such as warnings, on success. I >>> think we should not do this. Now, on the execution of "echo X >>> &>2", Org says "Code block produced no output." But that does >>> hold true. The block did produce output, just on the other output >>> stream, namely error output. > >> Trashing error output is expected by default. [...] I suggest you to >> read through [...] ob-shell follows the spec here. > > I get that ':results output' makes Org capture standard output, but not > error output, for its #+RESULTS. However, if you look at the test in > question, you will see that it expects error output in the *Org-Babel > Error Output* buffer, not in the #+RESULTS. We should not trash error > output; it belongs in the *Org-Babel Error Output* buffer. I do not think that it make sense to display that buffer when the code finishes successfully. I can see this kind of behaviour breaking/spamming automated scripts or export---code working in the past may throw error output into unsuspecting users. >>> 3. ob-shell/exit-codes >>> >>> Should we add a newline after the exit code message? > >> Done. > > I updated the tests and now every string needs a final newline, which > does not seem correct. If we let the tests guide our design, all > becomes clear. Buffers that use the 'compilation-mode' never append > data on re-execution. Instead, they clear the buffer. > > Thus, we should > > 1. undo this change and instead > 2. call 'org-babel-eval-wipe-error-buffer' before execution. I do not think that it is a good idea. Code block execution may involve a whole chain of blocks when expanding references. If we wipe the error buffer and multiple blocks are failing, some errors may go unnoticed by the user. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Org 9.6-pre and Bash sessions 2022-10-28 13:29 ` Ihor Radchenko @ 2022-10-28 21:52 ` Rudolf Adamkovič 2022-10-29 4:05 ` [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions) Ihor Radchenko 2022-10-29 4:05 ` Org 9.6-pre and Bash sessions Ihor Radchenko 0 siblings, 2 replies; 37+ messages in thread From: Rudolf Adamkovič @ 2022-10-28 21:52 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Ihor Radchenko, emacs-orgmode Ihor Radchenko <yantar92@posteo.net> writes: > I do not think that it make sense to display that buffer when the code > finishes successfully. I can see this kind of behaviour > breaking/spamming automated scripts or export---code working in the > past may throw error output into unsuspecting users. But the exit code has nothing to do with the standard error. Unix programs can, and often do, halt with non-zero exit codes while producing error output containing important information, such as deprecation warnings. Further, many programs use error output as the alternative "anything but the result" stream. Preserving user data, instead of trashing it, data does not count as "spamming ... unsuspected users". On the contrary! For example, I use a program for work that uploads data to a certain 3rd-party server. It exits with a zero code but also shows extremely important notices on error output. As an "unsuspecting user", if I used Babel to run the program, I would end up in a trouble. So, we should never implicitly trash user-generated data, let alone based on a "completely made up" belief that a non-zero exit code somehow implies "no important error output". It does not. (I speak only about Unix-like systems here. Perhaps on other operating systems, things work differently.) > I do not think that it is a good idea. Code block execution may > involve a whole chain of blocks when expanding references. If we wipe > the error buffer and multiple blocks are failing, some errors may go > unnoticed by the user. In that case, can we prepend a newline instead of appending it? That way, the tests would look "more normal" and the buffer content would have the content one would expect, with no "hanging whitespace". Rudy -- "Genius is 1% inspiration and 99% perspiration." p-- Thomas Alva Edison, 1932 Rudolf Adamkovič <salutis@me.com> [he/him] Studenohorská 25 84103 Bratislava Slovakia ^ permalink raw reply [flat|nested] 37+ messages in thread
* [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions) 2022-10-28 21:52 ` Rudolf Adamkovič @ 2022-10-29 4:05 ` Ihor Radchenko 2022-10-29 6:14 ` tomas ` (2 more replies) 2022-10-29 4:05 ` Org 9.6-pre and Bash sessions Ihor Radchenko 1 sibling, 3 replies; 37+ messages in thread From: Ihor Radchenko @ 2022-10-29 4:05 UTC (permalink / raw) To: Rudolf Adamkovič; +Cc: Ihor Radchenko, emacs-orgmode Rudolf Adamkovič <salutis@me.com> writes: > Ihor Radchenko <yantar92@posteo.net> writes: > >> I do not think that it make sense to display that buffer when the code >> finishes successfully. I can see this kind of behaviour >> breaking/spamming automated scripts or export---code working in the >> past may throw error output into unsuspecting users. > > But the exit code has nothing to do with the standard error. > > Unix programs can, and often do, halt with non-zero exit codes while > producing error output containing important information, such as > deprecation warnings. Further, many programs use error output as the > alternative "anything but the result" stream. > > Preserving user data, instead of trashing it, data does not count as > "spamming ... unsuspected users". On the contrary! > > For example, I use a program for work that uploads data to a certain > 3rd-party server. It exits with a zero code but also shows extremely > important notices on error output. As an "unsuspecting user", if I used > Babel to run the program, I would end up in a trouble. > > So, we should never implicitly trash user-generated data, let alone > based on a "completely made up" belief that a non-zero exit code somehow > implies "no important error output". It does not. > > (I speak only about Unix-like systems here. Perhaps on other operating > systems, things work differently.) Dear All, As explained in the above quote, it may be reasonable to display stderr in the shell (and possibly other) src blocks upon execution. + Stderr may contain important information even if the code block succeeds - Displaying stderr will raise *Error* buffer, which may or may not be expected or desired. What do you think? -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions) 2022-10-29 4:05 ` [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions) Ihor Radchenko @ 2022-10-29 6:14 ` tomas 2022-10-29 6:43 ` Samuel Wales 2022-10-29 11:58 ` Max Nikulin 2022-10-30 20:28 ` Tim Cross 2 siblings, 1 reply; 37+ messages in thread From: tomas @ 2022-10-29 6:14 UTC (permalink / raw) To: emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 2360 bytes --] On Sat, Oct 29, 2022 at 04:05:19AM +0000, Ihor Radchenko wrote: > Rudolf Adamkovič <salutis@me.com> writes: > > > Ihor Radchenko <yantar92@posteo.net> writes: > > > >> I do not think that it make sense to display that buffer when the code > >> finishes successfully. I can see this kind of behaviour > >> breaking/spamming automated scripts or export---code working in the > >> past may throw error output into unsuspecting users. > > > > But the exit code has nothing to do with the standard error. [...] > > For example, I use a program for work that uploads data to a certain > > 3rd-party server. It exits with a zero code but also shows extremely > > important notices on error output. As an "unsuspecting user", if I used > > Babel to run the program, I would end up in a trouble. [...] > Dear All, > > As explained in the above quote, it may be reasonable to display stderr > in the shell (and possibly other) src blocks upon execution. > > + Stderr may contain important information even if the code block > succeeds > - Displaying stderr will raise *Error* buffer, which may or may not be > expected or desired. > > What do you think? My take as an Org user is that this makes a lot of sense. I don't know whether there is an elegant way to accomodate all the use cases in an elegant way, but to provide a concrete example where I'd have found it handy... While preparing a handout for an introduction to shell programming (at a very basic level), I wanted to embed little examples with their results. Org rocks at this kind of task. But in this case it's important to show everything the students are going to see. One could argue that the error part is even the most important. So what I needed was not only the stderr (optionally somehow separated from stdout -- optionally as someone would see it in some terminal session), but also all the above even when the exit code was nonzero. Ideally, a display of that exit code, too. I ended up massaging prologue and epilogue, which worked nicely, but sadly is language dependent for a set of concepts which are, one could argue, independent of the language [1]. So yes, I would be thrilled by such a possibility. Cheers [1] These are OS conventions. So also a language of sorts, but at another level. -- tomás [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions) 2022-10-29 6:14 ` tomas @ 2022-10-29 6:43 ` Samuel Wales 2022-10-29 9:00 ` tomas 0 siblings, 1 reply; 37+ messages in thread From: Samuel Wales @ 2022-10-29 6:43 UTC (permalink / raw) To: tomas; +Cc: emacs-orgmode long ago i made the contents of my shell blocks look like this: { code } 2>&1 : On 10/28/22, tomas@tuxteam.de <tomas@tuxteam.de> wrote: > On Sat, Oct 29, 2022 at 04:05:19AM +0000, Ihor Radchenko wrote: >> Rudolf Adamkovič <salutis@me.com> writes: >> >> > Ihor Radchenko <yantar92@posteo.net> writes: >> > >> >> I do not think that it make sense to display that buffer when the code >> >> finishes successfully. I can see this kind of behaviour >> >> breaking/spamming automated scripts or export---code working in the >> >> past may throw error output into unsuspecting users. >> > >> > But the exit code has nothing to do with the standard error. > > [...] > >> > For example, I use a program for work that uploads data to a certain >> > 3rd-party server. It exits with a zero code but also shows extremely >> > important notices on error output. As an "unsuspecting user", if I >> > used >> > Babel to run the program, I would end up in a trouble. > > [...] > >> Dear All, >> >> As explained in the above quote, it may be reasonable to display stderr >> in the shell (and possibly other) src blocks upon execution. >> >> + Stderr may contain important information even if the code block >> succeeds >> - Displaying stderr will raise *Error* buffer, which may or may not be >> expected or desired. >> >> What do you think? > > My take as an Org user is that this makes a lot of sense. I don't know > whether there is an elegant way to accomodate all the use cases in > an elegant way, but to provide a concrete example where I'd have found > it handy... > > While preparing a handout for an introduction to shell programming > (at a very basic level), I wanted to embed little examples with > their results. Org rocks at this kind of task. > > But in this case it's important to show everything the students are > going to see. One could argue that the error part is even the most > important. > > So what I needed was not only the stderr (optionally somehow separated > from stdout -- optionally as someone would see it in some terminal > session), but also all the above even when the exit code was nonzero. > > Ideally, a display of that exit code, too. > > I ended up massaging prologue and epilogue, which worked nicely, > but sadly is language dependent for a set of concepts which are, > one could argue, independent of the language [1]. > > So yes, I would be thrilled by such a possibility. > > Cheers > > [1] These are OS conventions. So also a language of sorts, but at > another level. > > -- > tomás > -- The Kafka Pandemic A blog about science, health, human rights, and misopathy: https://thekafkapandemic.blogspot.com ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions) 2022-10-29 6:43 ` Samuel Wales @ 2022-10-29 9:00 ` tomas 2022-10-29 9:09 ` Ihor Radchenko 0 siblings, 1 reply; 37+ messages in thread From: tomas @ 2022-10-29 9:00 UTC (permalink / raw) To: Samuel Wales; +Cc: emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 471 bytes --] On Fri, Oct 28, 2022 at 11:43:39PM -0700, Samuel Wales wrote: > long ago i made the contents of my shell blocks look like this: > > { > code > } 2>&1 > : As I hinted at, I also wanted to have the exit code documented. So my setup was a bit more involved than this. Plus, I didn't want all that code scaffolding to show up in the typeset results (my audience had enough to digest as it was), so I hid all that in the prologue/ epilogue. Cheers -- t [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions) 2022-10-29 9:00 ` tomas @ 2022-10-29 9:09 ` Ihor Radchenko 2022-10-29 9:18 ` tomas 0 siblings, 1 reply; 37+ messages in thread From: Ihor Radchenko @ 2022-10-29 9:09 UTC (permalink / raw) To: tomas; +Cc: Samuel Wales, emacs-orgmode tomas@tuxteam.de writes: > As I hinted at, I also wanted to have the exit code documented. So > my setup was a bit more involved than this. Plus, I didn't want all > that code scaffolding to show up in the typeset results (my audience > had enough to digest as it was), so I hid all that in the prologue/ > epilogue. Note that what we are discussing here is different from your description. A popup window accumulating stderr is displayed upon code evaluation. stderr does not go into results. What you are describing would better fit into a new :results option. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions) 2022-10-29 9:09 ` Ihor Radchenko @ 2022-10-29 9:18 ` tomas 2022-10-30 3:31 ` Ihor Radchenko 0 siblings, 1 reply; 37+ messages in thread From: tomas @ 2022-10-29 9:18 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Samuel Wales, emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 636 bytes --] On Sat, Oct 29, 2022 at 09:09:04AM +0000, Ihor Radchenko wrote: > tomas@tuxteam.de writes: > > > As I hinted at, I also wanted to have the exit code documented [...] > Note that what we are discussing here is different from your > description. A popup window accumulating stderr is displayed upon code > evaluation. stderr does not go into results. > > What you are describing would better fit into a new :results option. Oh, I missed the popup part. Sounds useful for yet another profile, yes. Thanks and sorry for the noise. And thanks for your incredible work on Org. Do you ever sleep? In awe -- tomás [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions) 2022-10-29 9:18 ` tomas @ 2022-10-30 3:31 ` Ihor Radchenko 2022-10-30 6:11 ` tomas 0 siblings, 1 reply; 37+ messages in thread From: Ihor Radchenko @ 2022-10-30 3:31 UTC (permalink / raw) To: tomas; +Cc: Samuel Wales, emacs-orgmode tomas@tuxteam.de writes: >> What you are describing would better fit into a new :results option. > > Oh, I missed the popup part. Sounds useful for yet another profile, > yes. If you need it, feel free to open a separate feature request for extra :results options in ob-shell. Possibly providing more details how you envision this header argument to work. > And thanks for your incredible work on Org. Do you ever sleep? Remember that we may have radically different time zones. What appears to be nightly emails may not necessarily be sent at night. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions) 2022-10-30 3:31 ` Ihor Radchenko @ 2022-10-30 6:11 ` tomas 2022-10-30 7:09 ` Ihor Radchenko 0 siblings, 1 reply; 37+ messages in thread From: tomas @ 2022-10-30 6:11 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Samuel Wales, emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 837 bytes --] On Sun, Oct 30, 2022 at 03:31:32AM +0000, Ihor Radchenko wrote: > tomas@tuxteam.de writes: > > >> What you are describing would better fit into a new :results option. > > > > Oh, I missed the popup part. Sounds useful for yet another profile, > > yes. > > If you need it, feel free to open a separate feature request for extra > :results options in ob-shell. Possibly providing more details how you > envision this header argument to work. I'll mull over it. Perhaps I come up with some (patch) proposal. > > And thanks for your incredible work on Org. Do you ever sleep? > > Remember that we may have radically different time zones. What appears > to be nightly emails may not necessarily be sent at night. I'm aware of that. My awe is more directed at your bandwidth than at your timings. Cheers -- t [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions) 2022-10-30 6:11 ` tomas @ 2022-10-30 7:09 ` Ihor Radchenko 2022-10-30 8:18 ` tomas 0 siblings, 1 reply; 37+ messages in thread From: Ihor Radchenko @ 2022-10-30 7:09 UTC (permalink / raw) To: tomas; +Cc: Samuel Wales, emacs-orgmode tomas@tuxteam.de writes: >> If you need it, feel free to open a separate feature request for extra >> :results options in ob-shell. Possibly providing more details how you >> envision this header argument to work. > > I'll mull over it. Perhaps I come up with some (patch) proposal. Thinking more about it in the context of the current feature request, displaying the stderr buffer unconditionally may be undesired when the script already does 2>&1 or, potentially, uses imaginary :results stderr header argument. >> > And thanks for your incredible work on Org. Do you ever sleep? >> >> Remember that we may have radically different time zones. What appears >> to be nightly emails may not necessarily be sent at night. > > I'm aware of that. My awe is more directed at your bandwidth than at > your timings. I use Org mode and Org agenda to manage the maintenance 😀 -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions) 2022-10-30 7:09 ` Ihor Radchenko @ 2022-10-30 8:18 ` tomas 0 siblings, 0 replies; 37+ messages in thread From: tomas @ 2022-10-30 8:18 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Samuel Wales, emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 851 bytes --] On Sun, Oct 30, 2022 at 07:09:30AM +0000, Ihor Radchenko wrote: > tomas@tuxteam.de writes: > > >> If you need it, feel free to open a separate feature request for extra > >> :results options in ob-shell. Possibly providing more details how you > >> envision this header argument to work. > > > > I'll mull over it. Perhaps I come up with some (patch) proposal. > > Thinking more about it in the context of the current feature request, > displaying the stderr buffer unconditionally may be undesired when the > script already does 2>&1 or, potentially, uses imaginary :results stderr > header argument. Definitely. It's not for everyone. > >> > And thanks for your incredible work on Org. Do you ever sleep? [...] > I use Org mode and Org agenda to manage the maintenance 😀 Still genuinely impressed. Cheers -- t [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions) 2022-10-29 4:05 ` [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions) Ihor Radchenko 2022-10-29 6:14 ` tomas @ 2022-10-29 11:58 ` Max Nikulin 2022-10-30 3:37 ` Ihor Radchenko 2022-10-30 20:28 ` Tim Cross 2 siblings, 1 reply; 37+ messages in thread From: Max Nikulin @ 2022-10-29 11:58 UTC (permalink / raw) To: emacs-orgmode On 29/10/2022 11:05, Ihor Radchenko wrote: > > As explained in the above quote, it may be reasonable to display stderr > in the shell (and possibly other) src blocks upon execution. > > + Stderr may contain important information even if the code block > succeeds > - Displaying stderr will raise *Error* buffer, which may or may not be > expected or desired. > > What do you think? I agree that stderr may be important. I would not mind to have a buffer that combines both stderr and stdout (maybe fontified). Despite order of messages in separate streams is not strictly defined, it still may shed some light on the reason what went wrong. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions) 2022-10-29 11:58 ` Max Nikulin @ 2022-10-30 3:37 ` Ihor Radchenko 0 siblings, 0 replies; 37+ messages in thread From: Ihor Radchenko @ 2022-10-30 3:37 UTC (permalink / raw) To: Max Nikulin; +Cc: emacs-orgmode Max Nikulin <manikulin@gmail.com> writes: > I agree that stderr may be important. I would not mind to have a buffer > that combines both stderr and stdout (maybe fontified). Despite order of > messages in separate streams is not strictly defined, it still may shed > some light on the reason what went wrong. I am not sure about the right approach of mixing streams. What we currently do is directing stderr to a separate file. See `org-babel--shell-command-on-region' and `org-babel-eval'. Also, note that our error display code is not only used by shell scripts. Also, by other interpreted languages like python. And we use compilation-mode, which provides reasonable fontification in many cases. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions) 2022-10-29 4:05 ` [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions) Ihor Radchenko 2022-10-29 6:14 ` tomas 2022-10-29 11:58 ` Max Nikulin @ 2022-10-30 20:28 ` Tim Cross 2022-10-31 1:13 ` Org babel API (was: [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions)) Ihor Radchenko 2 siblings, 1 reply; 37+ messages in thread From: Tim Cross @ 2022-10-30 20:28 UTC (permalink / raw) To: emacs-orgmode Ihor Radchenko <yantar92@posteo.net> writes: > Rudolf Adamkovič <salutis@me.com> writes: > >> Ihor Radchenko <yantar92@posteo.net> writes: >> >>> I do not think that it make sense to display that buffer when the code >>> finishes successfully. I can see this kind of behaviour >>> breaking/spamming automated scripts or export---code working in the >>> past may throw error output into unsuspecting users. >> >> But the exit code has nothing to do with the standard error. >> >> Unix programs can, and often do, halt with non-zero exit codes while >> producing error output containing important information, such as >> deprecation warnings. Further, many programs use error output as the >> alternative "anything but the result" stream. >> >> Preserving user data, instead of trashing it, data does not count as >> "spamming ... unsuspected users". On the contrary! >> >> For example, I use a program for work that uploads data to a certain >> 3rd-party server. It exits with a zero code but also shows extremely >> important notices on error output. As an "unsuspecting user", if I used >> Babel to run the program, I would end up in a trouble. >> >> So, we should never implicitly trash user-generated data, let alone >> based on a "completely made up" belief that a non-zero exit code somehow >> implies "no important error output". It does not. >> >> (I speak only about Unix-like systems here. Perhaps on other operating >> systems, things work differently.) > > Dear All, > > As explained in the above quote, it may be reasonable to display stderr > in the shell (and possibly other) src blocks upon execution. > > + Stderr may contain important information even if the code block > succeeds > - Displaying stderr will raise *Error* buffer, which may or may not be > expected or desired. > > What do you think? I think this is perhaps the tip of a more complex iceberg. Not sure if we can address bits individually. Suspect we are better off clarifying the basic babel model and then look at specific language exceptions/limits and adjusting the model or documenting where back ends need to diverge from the basic model. Part of the challenge here is that the relevance/importance of stderr is somewhat dependent on the language. Same holds true for handling of error/return codes and to 'results' output. For me, this is another symptom of our need to define a clearer model for babel processes so that we can get some consistency across the board. Such a model would likely also make it easier for people to develop new babel back ends. We may even need 2 models, one for interactive/repl based back ends and one for non-interactive/scripted back ends. The 2 big questions I see are "What should be defaults be?" and "How do we handle the options without adding lots of new switches or suffering an option permutation blow out?". I do agree with the other posts regarding the importance of stderr. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Org babel API (was: [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions)) 2022-10-30 20:28 ` Tim Cross @ 2022-10-31 1:13 ` Ihor Radchenko 2022-10-31 2:03 ` Tim Cross 0 siblings, 1 reply; 37+ messages in thread From: Ihor Radchenko @ 2022-10-31 1:13 UTC (permalink / raw) To: Tim Cross; +Cc: emacs-orgmode Tim Cross <theophilusx@gmail.com> writes: > For me, this is another symptom of our need to define a clearer model > for babel processes so that we can get some consistency across the > board. Such a model would likely also make it easier for people to > develop new babel back ends. We may even need 2 models, one for > interactive/repl based back ends and one for non-interactive/scripted > back ends. We do have two models already. We have some small bits documented in https://orgmode.org/worg/org-contrib/babel/languages/index.html Generally, the defaults are implemented in ob-core.el. Session API is in ob-comint.el. Shell command API is in ob-eval.el. > The 2 big questions I see are "What should be defaults be?" and "How do > we handle the options without adding lots of new switches or suffering > an option permutation blow out?". I am sorry, but I do not fully understand what you are talking about. Which defaults are you referring to? -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Org babel API (was: [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions)) 2022-10-31 1:13 ` Org babel API (was: [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions)) Ihor Radchenko @ 2022-10-31 2:03 ` Tim Cross 2022-10-31 3:12 ` Ihor Radchenko 0 siblings, 1 reply; 37+ messages in thread From: Tim Cross @ 2022-10-31 2:03 UTC (permalink / raw) To: Ihor Radchenko; +Cc: emacs-orgmode Ihor Radchenko <yantar92@posteo.net> writes: > Tim Cross <theophilusx@gmail.com> writes: > >> For me, this is another symptom of our need to define a clearer model >> for babel processes so that we can get some consistency across the >> board. Such a model would likely also make it easier for people to >> develop new babel back ends. We may even need 2 models, one for >> interactive/repl based back ends and one for non-interactive/scripted >> back ends. > > We do have two models already. > We have some small bits documented in > https://orgmode.org/worg/org-contrib/babel/languages/index.html > > Generally, the defaults are implemented in ob-core.el. > Session API is in ob-comint.el. Shell command API is in ob-eval.el. > >> The 2 big questions I see are "What should be defaults be?" and "How do >> we handle the options without adding lots of new switches or suffering >> an option permutation blow out?". > > I am sorry, but I do not fully understand what you are talking about. > Which defaults are you referring to? In general, all the defaults for babel blocks, but morfe specifically here, deffault handling for stderr. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Org babel API (was: [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions)) 2022-10-31 2:03 ` Tim Cross @ 2022-10-31 3:12 ` Ihor Radchenko 0 siblings, 0 replies; 37+ messages in thread From: Ihor Radchenko @ 2022-10-31 3:12 UTC (permalink / raw) To: Tim Cross; +Cc: emacs-orgmode Tim Cross <theophilusx@gmail.com> writes: >>> The 2 big questions I see are "What should be defaults be?" and "How do >>> we handle the options without adding lots of new switches or suffering >>> an option permutation blow out?". >> >> I am sorry, but I do not fully understand what you are talking about. >> Which defaults are you referring to? > > In general, all the defaults for babel blocks, but morfe specifically > here, deffault handling for stderr. stdout is the current default. And I'd say that it is expected default. Sorry, I still do not understand the concrete problem you are trying to highlight. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Org 9.6-pre and Bash sessions 2022-10-28 21:52 ` Rudolf Adamkovič 2022-10-29 4:05 ` [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions) Ihor Radchenko @ 2022-10-29 4:05 ` Ihor Radchenko 2022-11-03 16:30 ` Rudolf Adamkovič 1 sibling, 1 reply; 37+ messages in thread From: Ihor Radchenko @ 2022-10-29 4:05 UTC (permalink / raw) To: Rudolf Adamkovič; +Cc: Ihor Radchenko, emacs-orgmode Rudolf Adamkovič <salutis@me.com> writes: >> I do not think that it is a good idea. Code block execution may >> involve a whole chain of blocks when expanding references. If we wipe >> the error buffer and multiple blocks are failing, some errors may go >> unnoticed by the user. > > In that case, can we prepend a newline instead of appending it? That > way, the tests would look "more normal" and the buffer content would > have the content one would expect, with no "hanging whitespace". Feel free to change it in the patch together with tests. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Org 9.6-pre and Bash sessions 2022-10-29 4:05 ` Org 9.6-pre and Bash sessions Ihor Radchenko @ 2022-11-03 16:30 ` Rudolf Adamkovič 2022-11-04 2:52 ` Ihor Radchenko 0 siblings, 1 reply; 37+ messages in thread From: Rudolf Adamkovič @ 2022-11-03 16:30 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Ihor Radchenko, emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 294 bytes --] Ihor Radchenko <yantar92@posteo.net> writes: > Feel free to change it in the patch together with tests. Thank you for driving the discussion and for giving me the opportunity to solve the problem (and not just temporarily, but with tests). What do you think about the attached patch? Rudy [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-ob-shell-Never-throw-away-standard-error.patch --] [-- Type: text/x-patch, Size: 7362 bytes --] From 7001ed80e5fd146f6acf586b3cf3ef20bbba04e7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Rudolf=20Adamkovi=C4=8D?= <salutis@me.com> Date: Wed, 26 Oct 2022 13:35:26 +0200 Subject: [PATCH] ob-shell: Never throw away standard error * lisp/ob-eval.el (org-babel-eval-error-notify): Do not insert superfluous whitespace. * lisp/ob-eval.el (org-babel-eval): Show standard error even if the command exits with a zero code. * testing/lisp/test-ob-shell.el( ob-shell/standard-output-after-success, ob-shell/standard-output-after-failure, ob-shell/error-output-after-success, ob-shell/error-output-after-failure, ob-shell/error-output-after-failure-multiple, ob-shell/exit-code, ob-shell/exit-code-multiple ): Add tests to avoid regressions. --- lisp/ob-eval.el | 47 +++++++++++--------- testing/lisp/test-ob-shell.el | 82 +++++++++++++++++++++++++++++++++++ 2 files changed, 108 insertions(+), 21 deletions(-) diff --git a/lisp/ob-eval.el b/lisp/ob-eval.el index e1278bbad..af9283c0a 100644 --- a/lisp/ob-eval.el +++ b/lisp/ob-eval.el @@ -40,39 +40,44 @@ (with-current-buffer buf (goto-char (point-max)) (save-excursion - (insert stderr) (unless (bolp) (insert "\n")) - (insert (format "[ Babel evaluation exited with code %S ]\n" exit-code)))) + (insert stderr) + (insert (format "[ Babel evaluation exited with code %S ]" exit-code)))) (display-buffer buf)) (message "Babel evaluation exited with code %S" exit-code)) (defun org-babel-eval (command query) "Run COMMAND on QUERY. +Return standard output produced by COMMAND. If COMMAND exits +with a non-zero code or produces error output, show it with +`org-babel-eval-error-notify'. + Writes QUERY into a temp-buffer that is processed with -`org-babel--shell-command-on-region'. If COMMAND succeeds then return -its results, otherwise display STDERR with -`org-babel-eval-error-notify'." +`org-babel--shell-command-on-region'." (let ((error-buffer (get-buffer-create " *Org-Babel Error*")) exit-code) (with-current-buffer error-buffer (erase-buffer)) (with-temp-buffer (insert query) (setq exit-code - (org-babel--shell-command-on-region - command error-buffer)) - (if (or (not (numberp exit-code)) (> exit-code 0)) - (progn - (with-current-buffer error-buffer - (org-babel-eval-error-notify exit-code (buffer-string))) - (save-excursion - (when (get-buffer org-babel-error-buffer-name) - (with-current-buffer org-babel-error-buffer-name - (unless (derived-mode-p 'compilation-mode) - (compilation-mode)) - ;; Compilation-mode enforces read-only, but Babel expects the buffer modifiable. - (setq buffer-read-only nil)))) - ;; Return output, if any. - (buffer-string)) - (buffer-string))))) + (org-babel--shell-command-on-region + command error-buffer)) + (let ((stderr (with-current-buffer error-buffer (buffer-string)))) + (if (or (not (numberp exit-code)) + (> exit-code 0) + (not (string-empty-p stderr))) + (progn + (org-babel-eval-error-notify exit-code stderr) + (save-excursion + (when (get-buffer org-babel-error-buffer-name) + (with-current-buffer org-babel-error-buffer-name + (unless (derived-mode-p 'compilation-mode) + (compilation-mode)) + ;; Compilation-mode enforces read-only, but + ;; Babel expects the buffer modifiable. + (setq buffer-read-only nil)))) + ;; Return output, if any. + (buffer-string)) + (buffer-string)))))) (defun org-babel-eval-read-file (file) "Return the contents of FILE as a string." diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el index 4c00faa49..7aa45cc8a 100644 --- a/testing/lisp/test-ob-shell.el +++ b/testing/lisp/test-ob-shell.el @@ -170,6 +170,88 @@ ob-comint.el, which was not previously tested." "#+BEGIN_SRC sh :results table\necho 'I \"want\" it all'\n#+END_SRC" (org-babel-execute-src-block))))) +;;; Standard output + +(ert-deftest ob-shell/standard-output-after-success () + "Test standard output after exiting with a zero code." + (should (= 1 + (org-babel-execute:sh + "echo 1" nil)))) + +(ert-deftest ob-shell/standard-output-after-failure () + "Test standard output after exiting with a non-zero code." + (should (= 1 + (org-babel-execute:sh + "echo 1; exit 2" nil)))) + +;;; Standard error + +(ert-deftest ob-shell/error-output-after-success () + "Test that standard error shows in the error buffer, alongside the +exit code, after exiting with a zero code." + (should + (string= "1 +[ Babel evaluation exited with code 0 ]" + (progn (org-babel-eval-wipe-error-buffer) + (org-babel-execute:sh + "echo 1 >&2" nil) + (with-current-buffer org-babel-error-buffer-name + (buffer-string)))))) + +(ert-deftest ob-shell/error-output-after-failure () + "Test that standard error shows in the error buffer, alongside the +exit code, after exiting with a non-zero code." + (should + (string= "1 +[ Babel evaluation exited with code 2 ]" + (progn (org-babel-eval-wipe-error-buffer) + (org-babel-execute:sh + "echo 1 >&2; exit 2" nil) + (with-current-buffer org-babel-error-buffer-name + (buffer-string)))))) + +(ert-deftest ob-shell/error-output-after-failure-multiple () + "Test that multiple standard error strings show in the error +buffer, alongside multiple exit codes." + (should + (string= "1 +[ Babel evaluation exited with code 2 ] +3 +[ Babel evaluation exited with code 4 ]" + (progn (org-babel-eval-wipe-error-buffer) + (org-babel-execute:sh + "echo 1 >&2; exit 2" nil) + (org-babel-execute:sh + "echo 3 >&2; exit 4" nil) + (with-current-buffer org-babel-error-buffer-name + (buffer-string)))))) + +;;; Exit codes + +(ert-deftest ob-shell/exit-code () + "Test that the exit code shows in the error buffer after exiting +with a non-zero return code." + (should + (string= "[ Babel evaluation exited with code 1 ]" + (progn (org-babel-eval-wipe-error-buffer) + (org-babel-execute:sh + "exit 1" nil) + (with-current-buffer org-babel-error-buffer-name + (buffer-string)))))) + +(ert-deftest ob-shell/exit-code-multiple () + "Test that multiple exit codes show in the error buffer after +exiting with a non-zero return code multiple times." + (should + (string= "[ Babel evaluation exited with code 1 ] +[ Babel evaluation exited with code 2 ]" + (progn (org-babel-eval-wipe-error-buffer) + (org-babel-execute:sh + "exit 1" nil) + (org-babel-execute:sh + "exit 2" nil) + (with-current-buffer org-babel-error-buffer-name + (buffer-string)))))) (provide 'test-ob-shell) -- 2.38.1 [-- Attachment #3: Type: text/plain, Size: 246 bytes --] -- "One can begin to reason only when a clear picture has been formed in the imagination." -- Walter Warwick Sawyer, Mathematician's Delight, 1943 Rudolf Adamkovič <salutis@me.com> [he/him] Studenohorská 25 84103 Bratislava Slovakia ^ permalink raw reply related [flat|nested] 37+ messages in thread
* Re: Org 9.6-pre and Bash sessions 2022-11-03 16:30 ` Rudolf Adamkovič @ 2022-11-04 2:52 ` Ihor Radchenko 2022-11-05 1:12 ` Rudolf Adamkovič 0 siblings, 1 reply; 37+ messages in thread From: Ihor Radchenko @ 2022-11-04 2:52 UTC (permalink / raw) To: Rudolf Adamkovič; +Cc: Ihor Radchenko, emacs-orgmode Rudolf Adamkovič <salutis@me.com> writes: > Ihor Radchenko <yantar92@posteo.net> writes: > >> Feel free to change it in the patch together with tests. > > Thank you for driving the discussion and for giving me the opportunity > to solve the problem (and not just temporarily, but with tests). What > do you think about the attached patch? Applied onto main changing the commit summary to "org-babel: ...". This change will affect all the babel backends that use `org-babel-eval'. https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=f7b16402e6a694d592210f766544f6114056b4b0 A small note: If stderr does not contain a trailing newline, it will be displayed as Stderr output with no trailing newline[ Babel evaluation exited with code 1 ] One may argue that it is ugly. On the other hand, adding an extra newline will make stderr with and without the newline indistinguishable. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Org 9.6-pre and Bash sessions 2022-11-04 2:52 ` Ihor Radchenko @ 2022-11-05 1:12 ` Rudolf Adamkovič 0 siblings, 0 replies; 37+ messages in thread From: Rudolf Adamkovič @ 2022-11-05 1:12 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Ihor Radchenko, emacs-orgmode Ihor Radchenko <yantar92@posteo.net> writes: > Applied onto main changing the commit summary to "org-babel: ...". > This change will affect all the babel backends that use > `org-babel-eval'. That makes sense. Thank you for the correction! > A small note: If stderr does not contain a trailing newline, it will > be displayed as > > Stderr output with no trailing newline[ Babel evaluation exited with > code 1 ] > > One may argue that it is ugly. On the other hand, adding an extra > newline will make stderr with and without the newline > indistinguishable. Ugly but correct. Besides, virtually every shell does the same, so every "civilized" program ends its output with a newline. Also, it makes me happy that we have this important behavior fixed in tests! Thank you for all your help! Rudy -- "Genius is 1% inspiration and 99% perspiration." -- Thomas Alva Edison, 1932 Rudolf Adamkovič <salutis@me.com> [he/him] Studenohorská 25 84103 Bratislava Slovakia ^ permalink raw reply [flat|nested] 37+ messages in thread
end of thread, other threads:[~2022-11-05 1:13 UTC | newest] Thread overview: 37+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-10-13 15:14 Org 9.6-pre and Bash sessions Rudolf Adamkovič 2022-10-13 17:07 ` Michael Welle 2022-10-14 3:44 ` Ihor Radchenko 2022-10-15 20:56 ` Rudolf Adamkovič 2022-10-17 8:34 ` Ihor Radchenko 2022-10-21 5:29 ` Ihor Radchenko 2022-10-21 13:38 ` Rudolf Adamkovič 2022-10-22 4:22 ` Ihor Radchenko 2022-10-22 9:44 ` Rudolf Adamkovič 2022-10-22 10:59 ` Ihor Radchenko 2022-10-23 4:27 ` Ihor Radchenko 2022-10-26 11:56 ` Rudolf Adamkovič 2022-10-26 13:21 ` Rudolf Adamkovič 2022-10-27 3:53 ` Ihor Radchenko 2022-10-28 13:12 ` Rudolf Adamkovič 2022-10-28 13:29 ` Ihor Radchenko 2022-10-28 21:52 ` Rudolf Adamkovič 2022-10-29 4:05 ` [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions) Ihor Radchenko 2022-10-29 6:14 ` tomas 2022-10-29 6:43 ` Samuel Wales 2022-10-29 9:00 ` tomas 2022-10-29 9:09 ` Ihor Radchenko 2022-10-29 9:18 ` tomas 2022-10-30 3:31 ` Ihor Radchenko 2022-10-30 6:11 ` tomas 2022-10-30 7:09 ` Ihor Radchenko 2022-10-30 8:18 ` tomas 2022-10-29 11:58 ` Max Nikulin 2022-10-30 3:37 ` Ihor Radchenko 2022-10-30 20:28 ` Tim Cross 2022-10-31 1:13 ` Org babel API (was: [FR] Display stderr contents after executing shell blocks (even when stdout :results output is requested) (was: Org 9.6-pre and Bash sessions)) Ihor Radchenko 2022-10-31 2:03 ` Tim Cross 2022-10-31 3:12 ` Ihor Radchenko 2022-10-29 4:05 ` Org 9.6-pre and Bash sessions Ihor Radchenko 2022-11-03 16:30 ` Rudolf Adamkovič 2022-11-04 2:52 ` Ihor Radchenko 2022-11-05 1:12 ` Rudolf Adamkovič
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).