[BUG] ob-shell async session does not work with multiple blocks running [9.7.16 (9.7.16-ba9734 @ /nix/store/6la93ywyb1h055phgyqba7h7qay60p0m-emacs-packages-deps/share/emacs/site-lisp/elpa/org-9.7.16/)]

[BUG] ob-shell async session does not work with multiple blocks running [9.7.16 (9.7.16-ba9734 @ /nix/store/6la93ywyb1h055phgyqba7h7qay60p0m-emacs-packages-deps/share/emacs/site-lisp/elpa/org-9.7.16/)]

[ParetoOptimal]

Problem: Running two shell source blocks with the same session doesn't
properly extract out results from the sentinel value relating to each
respective source block.

This could be a known issue which might make this an enhancement request
rather than a bug report.

Expected: Running two source blocks at once to properly deal with only
their values from the "foo-comint-indirect" buffer.

Given:

#+begin_src sh :session foo :async
sleep 5
echo "almost done"
sleep 3
echo "done"
#+end_src

#+begin_src sh :session foo :async
sleep 1
echo "almost done"
sleep 6
echo "done"
#+end_src

Then running these right after another using:

M-<			;; beginning-of-buffer
C-c C-v C-n		;; org-babel-next-src-block
C-c C-c			;; org-ctrl-c-ctrl-c
C-c C-v C-n		;; org-babel-next-src-block
C-c C-c			;; org-ctrl-c-ctrl-c

You never get the results for the last source block, but there are 3
stages:

1. After evaluating each block, but before either is done:

#+begin_src sh :session foo :async
sleep 5
echo "almost done"
sleep 3
echo "done"
#+end_src

#+RESULTS:
: 4466c6fe-e7b5-4c14-9cae-d8f066ec6ebf

#+begin_src sh :session foo :async
sleep 1
echo "almost done"
sleep 6
echo "done"
#+end_src

#+RESULTS:
: 17c8a088-82c4-4695-90f6-fe3cba9fcfd7

2. After the first is finished running (notice the clobbered results
pulled in from the other block, this is the bug)

#+begin_src sh :session foo :async
sleep 5
echo "almost done"
sleep 3
echo "done"
#+end_src

#+RESULTS:
: echo 'ob_comint_async_shell_start_17c8a088-82c4-4695-90f6-fe3cba9fcfd7'
: sleep 1
: echo "almost done"
: sleep 6
: echo "done"
: echo 'ob_comint_async_shell_end_17c8a088-82c4-4695-90f6-fe3cba9fcfd7'
: almost done
: done

#+begin_src sh :session foo :async
sleep 1
echo "almost done"
sleep 6
echo "done"
#+end_src

#+RESULTS:
: 17c8a088-82c4-4695-90f6-fe3cba9fcfd7

3. Something is done to fixup the results from the first source block,
but the second source block never gets updated with it's results

#+begin_src sh :session foo :async
sleep 5
echo "almost done"
sleep 3
echo "done"
#+end_src

#+RESULTS:
: almost done
: done

#+begin_src sh :session foo :async
sleep 1
echo "almost done"
sleep 6
echo "done"
#+end_src

#+RESULTS:
: 17c8a088-82c4-4695-90f6-fe3cba9fcfd7

Here is what the foo-comint-indirect buffer looks like:

org_babel_sh_prompt> echo 'ob_comint_async_shell_start_4466c6fe-e7b5-4c14-9cae-d8f066ec6ebf'
sleep 5
echo "almost done"
sleep 3
echo "done"
echo 'ob_comint_async_shell_end_4466c6fe-e7b5-4c14-9cae-d8f066ec6ebf'
ob_comint_async_shell_start_4466c6fe-e7b5-4c14-9cae-d8f066ec6ebf
org_babel_sh_prompt> echo 'ob_comint_async_shell_start_17c8a088-82c4-4695-90f6-fe3cba9fcfd7'
sleep 1
echo "almost done"
sleep 6
echo "done"
echo 'ob_comint_async_shell_end_17c8a088-82c4-4695-90f6-fe3cba9fcfd7'
org_babel_sh_prompt> almost done
org_babel_sh_prompt> org_babel_sh_prompt> done
org_babel_sh_prompt> ob_comint_async_shell_end_4466c6fe-e7b5-4c14-9cae-d8f066ec6ebf
org_babel_sh_prompt> ob_comint_async_shell_start_17c8a088-82c4-4695-90f6-fe3cba9fcfd7
org_babel_sh_prompt> org_babel_sh_prompt> almost done
org_babel_sh_prompt> org_babel_sh_prompt> done
org_babel_sh_prompt> ob_comint_async_shell_end_17c8a088-82c4-4695-90f6-fe3cba9fcfd7
org_babel_sh_prompt>

Thanks for the great feature. Just allowing for long running shell
blocks to not block doing things in emacs was a great addition.


Emacs  : GNU Emacs 29.4 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo version 1.18.2, Xaw3d scroll bars)
Package: Org mode version 9.7.16 (9.7.16-ba9734 @ /nix/store/6la93ywyb1h055phgyqba7h7qay60p0m-emacs-packages-deps/share/emacs/site-lisp/elpa/org-9.7.16/)

Re: [BUG] ob-shell async session does not work with multiple blocks running [9.7.16 (9.7.16-ba9734 @ /nix/store/6la93ywyb1h055phgyqba7h7qay60p0m-emacs-packages-deps/share/emacs/site-lisp/elpa/org-9.7.16/)]

[Ihor Radchenko]

ParetoOptimal <pareto.optimal@mailfence.com> writes:

> Problem: Running two shell source blocks with the same session doesn't
> properly extract out results from the sentinel value relating to each
> respective source block.
>
> This could be a known issue which might make this an enhancement request
> rather than a bug report.
>
> Expected: Running two source blocks at once to properly deal with only
> their values from the "foo-comint-indirect" buffer.
> ...

Confirmed.

Matthew, may you please take a look?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] ob-shell async session does not work with multiple blocks running [9.7.16 (9.7.16-ba9734 @ /nix/store/6la93ywyb1h055phgyqba7h7qay60p0m-emacs-packages-deps/share/emacs/site-lisp/elpa/org-9.7.16/)]

[Matt]

 ---- On Sat, 23 Nov 2024 19:43:05 +0100  Ihor Radchenko  wrote --- 
 > ParetoOptimal pareto.optimal@mailfence.com> writes:
 > 
 > > Problem: Running two shell source blocks with the same session doesn't
 > > properly extract out results from the sentinel value relating to each
 > > respective source block.
 > >
 > > This could be a known issue which might make this an enhancement request
 > > rather than a bug report.
 > >
 > > Expected: Running two source blocks at once to properly deal with only
 > > their values from the "foo-comint-indirect" buffer.
 > > ...
 > 
 > Confirmed.
 > 
 > Matthew, may you please take a look?
 
Thank you for the bug report and for the confirmation.  Unfortunately, I haven't much time at the moment to address this.

It's not clear to me what causes the observed behavior.  

Assigning '1' and '2' to the echoes in each block shows that the results for the second block are placed in the first:

#+begin_src sh :session foo :async
sleep 5
echo "almost done1"
sleep 3
echo "done1"
#+end_src

#+RESULTS:
: almost done2
: done2

#+begin_src sh :session foo :async
sleep 1
echo "almost done2"
sleep 6
echo "done2"
#+end_src

#+RESULTS:
: bebfd924-c3b8-4ad4-aec0-23f322d5af0f