Make org-mode multi line shell session work sensibly

Make org-mode multi line shell session work sensibly

[Samuel Loury]

[-- Attachment #1: Type: text/plain, Size: 3884 bytes --]

Hi,

I've been trying to work more and more with org-babel recently, and
found out that the session in the shell block does not appear to work
well.

The symptom is the following: When I try to run a code block with noop
lines, like comment, the output gets broken

--8<---------------cut here---------------start------------->8---
#+BEGIN_SRC sh :results output :exports both :session c66d4791-003d-42af-a4e0-366f5542c604
  echo a
  # noop
  echo b
#+END_SRC

#+RESULTS:
:
: a
: $ b
--8<---------------cut here---------------end--------------->8---

Of course, I would expect
--8<---------------cut here---------------start------------->8---
#+RESULTS:
: a
: b
--8<---------------cut here---------------end--------------->8---

Actually, I went through the code and I think I understand why it has
this behavior, see here for my analysis ->
https://konubinix.eu/braindump/posts/c66d4791-003d-42af-a4e0-366f5542c604/

I tried to patch the code of org-babel-comint-with-output to make it
work correctly, see here ->
https://konubinix.eu/braindump/posts/25b52cc8-71f8-420f-9161-5c60030cede9/

But it feels like it is more of a work around than an elegant solution
to the issue. I am already facing some corner cases and it does not
handle yet the continuation lines, starting with ">".

I was thinking that maybe, in org-babel-sh-evaluate, instead of sending
the lines one by one and interpreting the result, the code could write a
temporary file with the body and source it. Basically all the evaluation
would be

--8<---------------cut here---------------start------------->8---
$ source tmp-file-XXXX
$ echo org-babel-sh-eoe
--8<---------------cut here---------------end--------------->8---

It still would suffer from the no newline behavior I am describing here
-> https://konubinix.eu/braindump/posts/f6206181-09d2-4449-b437-960be2b5dfd2/
But, one step at a time, right?

Then I took though about it. I cannot believe I am the only person
trying to run multi line shell programs with comment in session, am I?

So here are a few questions: Is there anyone with the same use case as I
have? If so, how do you cope with the problem I am facing? If I want to
fix it, what do you think would be the more elegant solution: fixing the
line by line parsing like I was trying to do or sourcing the code to
avoid using line by line parsing?

org-mode is really great, org-babel is awesome and I use it more and
more in my daily life, I hope we will be able to fix this annoying
behavior together.

Thank you for your time,
My best,

PS: the real life use case I am using is the following one: I want to
have a few org-babel snippets to ease my code reviews, as presented in here:
https://konubinix.eu/braindump/posts/564621c3-4a9a-4574-abf8-bd9831b29aa4/

the first snippet has the following output

--8<---------------cut here---------------start------------->8---
: 
: $ $ $ $ $ Ready to make the review of XXX on top of YYY
--8<---------------cut here---------------end--------------->8---

instead of the expected

--8<---------------cut here---------------start------------->8---
: Ready to make the review of XXX on top of YYY
--8<---------------cut here---------------end--------------->8---

And the second one with the header ":results output file", have the
following result

--8<---------------cut here---------------start------------->8---
[[file:
the-directory/diff.diff]]
--8<---------------cut here---------------end--------------->8---

instead of the expected

--8<---------------cut here---------------start------------->8---
[[file:the-directory/diff.diff]]
--8<---------------cut here---------------end--------------->8---

With my workaround, the output is ok.
-- 
Konubinix
GPG Key    : 7439106A
Fingerprint: 5993 BE7A DA65 E2D9 06CE  5C36 75D2 3CED 7439 106A

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Make org-mode multi line shell session work sensibly

[Ihor Radchenko]

Fixed.

Samuel Loury <konubinix@gmail.com> writes:

> The symptom is the following: When I try to run a code block with noop
> lines, like comment, the output gets broken
>
> --8<---------------cut here---------------start------------->8---
> #+BEGIN_SRC sh :results output :exports both :session c66d4791-003d-42af-a4e0-366f5542c604
>   echo a
>   # noop
>   echo b
> #+END_SRC
>
> #+RESULTS:
> :
> : a
> : $ b
> --8<---------------cut here---------------end--------------->8---

This has been fixed on main in a series of other related bugfixes.
Not on bugfix though, so I just cherry-picked the relevant commits from
main. Your example is now working fine on bugfix.

> Then I took though about it. I cannot believe I am the only person
> trying to run multi line shell programs with comment in session, am I?
>
> So here are a few questions: Is there anyone with the same use case as I
> have? If so, how do you cope with the problem I am facing? If I want to
> fix it, what do you think would be the more elegant solution: fixing the
> line by line parsing like I was trying to do or sourcing the code to
> avoid using line by line parsing?

Well. This issue has been introduced in the last minor release when we
attempted to fix another bug with Org babel being too aggressive when
removing anything that looks like prompt, even when it was not prompt.
This whole comint thing is tricky.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>