> Matt matt@excalamus.com> writes:
 >
 > I see only two options to fix it: remove a space from the concat expression (which I did in my latest patch) or remove a space from `org-babel-sh-prompt'.

Unfortunately, I was mistaken and the second option (removing the space from `org-babel-sh-prompt') doesn't fix the issue.  The TLDR is that the code in `org-babel-comint-async-filter' which grabs the region between the indicators (incorrectly) fails to include the prompt's trailing space.

#+begin_longwinded_explanation
I'll first explain why removing the space from `org-babel-sh-prompt' doesn't fix the issue because it well also highlight the underlying problem.

If we remove the space from the `org-babel-sh-prompt', then `comint-prompt-regexp' becomes "^org_babel_sh_prompt> *" (with one space).   This would work if the string passed to the `ob-shell-async-chunk-callback' stayed the same.  It doesn't (this is where my reasoning and testing failed).  Changing the `org-babel-sh-prompt' to "org_babel_sh_prompt>" (without a space) causes the following string to be passed to the callback:

"org_babel_sh_prompt>1
org_babel_sh_prompt>2
org_babel_sh_prompt"

Note that the final prompt doesn't have a ">" and therefore the `comint-prompt-regexp' (which becomes "^org_babel_sh_prompt> * (with one space)) used in the callback fails to match it.  When we remove the space from the `org-babel-sh-prompt', the session buffer looks like this:

"sh-5.1$ PROMPT_COMMAND=;PS1="org_babel_sh_prompt>";PS2=
org_babel_sh_prompt>echo 'ob_comint_async_shell_start_39610981-1020-4baf-9dfb-f96d10af1cf8'
echo 1
echo 2
echo 'ob_comint_async_shell_end_39610981-1020-4baf-9dfb-f96d10af1cf8'
ob_comint_async_shell_start_39610981-1020-4baf-9dfb-f96d10af1cf8
org_babel_sh_prompt>1
org_babel_sh_prompt>2
org_babel_sh_prompt>ob_comint_async_shell_end_39610981-1020-4baf-9dfb-f96d10af1cf8
org_babel_sh_prompt>"

The `org-babel-comint-async-filter' is what calls the `ob-shell-async-chunk-callback' (ob-comint.el:284).  It monitors for the end indicator.  When that appears, it passes the region between the beginning of the end indicator **less 1** and the character after the end of the start indicator to the callback.  For a clean run of `test-ob-shell/session-async-evaluation', the beginning of the end indicator is at 361 and the character after the end of the start indicator is at 298.  This is the string I gave above which is missing the ">".  

In order to make the second option work, we'd need to change the "less 1" part of `org-babel-comint-async-filter' from (- (match-beginning 0) 1) to (match-beginning 0).   It turns out that's actually all we need to do.

When `org-babel-sh-prompt' is "org_babel_sh_prompt> " (with one space), then the session buffer looks like:

"sh-5.1$ PROMPT_COMMAND=;PS1="org_babel_sh_prompt> ";PS2=
org_babel_sh_prompt> echo 'ob_comint_async_shell_start_3270ed43-a99b-423f-a5fa-b15fb2e4ae26'
echo 1
echo 2
echo 'ob_comint_async_shell_end_3270ed43-a99b-423f-a5fa-b15fb2e4ae26'
ob_comint_async_shell_start_3270ed43-a99b-423f-a5fa-b15fb2e4ae26
org_babel_sh_prompt> 1
org_babel_sh_prompt> 2
org_babel_sh_prompt> ob_comint_async_shell_end_3270ed43-a99b-423f-a5fa-b15fb2e4ae26
org_babel_sh_prompt> "

The region passed to the callback is then defined as 366 to 300, or

"org_babel_sh_prompt> 1
org_babel_sh_prompt> 2
org_babel_sh_prompt>"  (<-- no space)

This looks okay at first glance.  However, **the last line is not a valid prompt**.  A prompt must end in a space!  When the `org-babel-sh-prompt' is set to  "org_babel_sh_prompt> " (with one space), the `comint-prompt-regexp' is "^org_babel_sh_prompt>  *" (with two spaces).  This means that the `comint-prompt-regexp' matches on a trailing space which the **region passed to the callback doesn't have**.  Therefore, the match fails.

Instead, if we modify the `org-babel-comint-async-filter' like

modified   lisp/ob-comint.el
@@ -273,7 +273,7 @@ STRING contains the output originally inserted into the comint buffer."
 		   (res-str-raw
 		    (buffer-substring
 		     ;; move point to beginning of indicator
-                     (- (match-beginning 0) 1)
+                     (match-beginning 0)
 		     ;; find the matching start indicator
 		     (cl-loop
                       do (re-search-backward indicator)

then the region passed to the callback will be from 367 to 300, or

"org_babel_sh_prompt> 1
org_babel_sh_prompt> 2
org_babel_sh_prompt> " (<-- with one space)

The `comint-prompt-regexp' will now match the last prompt in the region.

With this change, the `org-babel-sh-prompt' keeps the trailing space (like it should), the `comint-prompt-regexp' becomes "^org_babel_sh_prompt>  *" (with two spaces, requiring a prompt to have a trailing space like it should), the `ob-shell-async-chunk-callback' can use `comint-prompt-regexp' without modification, and the tests all pass.
#+end_longwinded_explanation

I've attached an updated diff.  If everyone is satisfied with this, I'll do a proper commit and then handle moving the uuid code like we talked about earlier in the thread.