emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
From: Matt <matt@excalamus.com>
To: "Ihor Radchenko" <yantar92@posteo.net>
Cc: "emacs-orgmode" <emacs-orgmode@gnu.org>
Subject: Re: [PATCH] Async evaluation in ob-shell
Date: Wed, 08 Feb 2023 23:33:13 -0500	[thread overview]
Message-ID: <1863472efe9.10fdd5ba4258906.5972264927968042941@excalamus.com> (raw)
In-Reply-To: <87v8kd8zzw.fsf@localhost>

[-- Attachment #1: Type: text/plain, Size: 1387 bytes --]

I've attached two patches which replace the previous.

I found cleaning the output was dramatically helped by calling `buffer-substring-no-properties' instead of `buffer-substring' in `org-babel-comint-async-filter'.  I'm not sure why `buffer-substring' was originally used.  `make test' shows no failures, so I assume it doesn't make a difference...?

 ---- On Tue, 07 Feb 2023 06:40:51 -0500  Ihor Radchenko  wrote --- 
 > That's likely because of the same reasons why prompt did not get cleaned
 > in synchronous blocks earlier. See `org-babel-comint-with-output'.

That, my friend, was a wild ride.

I'm curious about people's feelings on `org-babel-comint-with-output'.

Personally, I get the heebie-jeebies.  I can't shake feeling that there's a better way, especially since `org-babel-comint-async-filter' achieves similar ends.  My hunch is that other Babel languages may want async and that now would be a good time to consolidate the common functionalities of `org-babel-comint-with-output' and  `org-babel-comint-async-filter' .   Maybe even unify the API.  So far, `org-babel-comint-with-output' touches 9 languages and `org-babel-comint-async-filter' appears to touch 2 (soon to be 3).   I suspect those numbers will only grow.

I also can't shake the feeling that I might become ob-comint maintainer at some point...(not yet!).  I'm curious what people's thoughts are.





[-- Attachment #2: 01-ob-shell-remove-properties-from-callback-string.patch --]
[-- Type: application/octet-stream, Size: 997 bytes --]

diff --git a/lisp/ob-comint.el b/lisp/ob-comint.el
index 54bf5127e..679757735 100644
--- a/lisp/ob-comint.el
+++ b/lisp/ob-comint.el
@@ -233,7 +233,7 @@ STRING contains the output originally inserted into the comint buffer."
       (goto-char (point-min))
       (while (re-search-forward indicator nil t)
 	;; update dangling
-	(setq new-dangling (buffer-substring (point) (point-max)))
+	(setq new-dangling (buffer-substring-no-properties (point) (point-max)))
 	(cond ((equal (match-string 1) "end")
 	       ;; save UUID for insertion later
 	       (push (match-string 2) uuid-list))
@@ -271,7 +271,7 @@ STRING contains the output originally inserted into the comint buffer."
 	  (when (equal (match-string 1) "end")
 	    (let* ((uuid (match-string-no-properties 2))
 		   (res-str-raw
-		    (buffer-substring
+		    (buffer-substring-no-properties
 		     ;; move point to beginning of indicator
                      (- (match-beginning 0) 1)
 		     ;; find the matching start indicator

[-- Attachment #3: 02-ob-shell-async-non-file.patch --]
[-- Type: application/octet-stream, Size: 2022 bytes --]

diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el
index 9e7b45a89..23cb9683b 100644
--- a/lisp/ob-shell.el
+++ b/lisp/ob-shell.el
@@ -269,12 +269,18 @@ var of the same value."
 	    (set-marker comint-last-output-start (point))
 	    (get-buffer (current-buffer)))))))
 
+(defconst ob-shell-async-indicator "echo 'ob_comint_async_shell_%s_%s'")
+
+(defun ob-shell-async-chunk-callback (string)
+  (replace-regexp-in-string (concat org-babel-sh-prompt "*") "" string))
+
 (defun org-babel-sh-evaluate (session body &optional params stdin cmdline)
   "Pass BODY to the Shell process in BUFFER.
 If RESULT-TYPE equals `output' then return a list of the outputs
 of the statements in BODY, if RESULT-TYPE equals `value' then
 return the value of the last statement in BODY."
   (let* ((shebang (cdr (assq :shebang params)))
+         (async (org-babel-comint-use-async params))
 	 (results-params (cdr (assq :result-params params)))
 	 (value-is-exit-status
 	  (or (and
@@ -305,6 +311,23 @@ return the value of the last statement in BODY."
                           (list shell-command-switch
                                 (concat (file-local-name script-file)  " " cmdline)))))
 		(buffer-string))))
+           (async
+            (let ((uuid (org-id-uuid)))
+              (org-babel-comint-async-register
+               session
+               (current-buffer)
+               "ob_comint_async_shell_\\(.+?\\)_\\(.+\\)"
+               'ob-shell-async-chunk-callback
+               nil)
+              (org-babel-comint-async-delete-dangling-and-eval
+                  session
+                (insert (format ob-shell-async-indicator "start" uuid))
+                (comint-send-input nil t)
+                (insert (org-trim body))
+                (comint-send-input nil t)
+                (insert (format ob-shell-async-indicator "end" uuid))
+                (comint-send-input nil t))
+              uuid))
 	   (session			; session evaluation
 	    (mapconcat
 	     #'org-babel-sh-strip-weird-long-prompt

  reply	other threads:[~2023-02-09  4:34 UTC|newest]

Thread overview: 72+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-02-06 19:39 [PATCH] Async evaluation in ob-shell Matt
2023-02-07 11:40 ` Ihor Radchenko
2023-02-09  4:33   ` Matt [this message]
2023-02-09 11:24     ` Ihor Radchenko
2023-02-10 22:19       ` Matt
2023-02-11 11:44         ` Ihor Radchenko
2023-02-12 19:32           ` Matt
2023-02-15 15:08             ` Ihor Radchenko
2023-02-16  4:02               ` Matt
2023-02-17 10:44                 ` Ihor Radchenko
2023-02-19 23:14                   ` Matt
2023-02-20 11:24                     ` Ihor Radchenko
2023-02-20 17:24                       ` Matt
2023-02-22 10:30                         ` Ihor Radchenko
2023-03-02  1:36                           ` Matt
2023-03-03 14:52                             ` Ihor Radchenko
2023-03-03 17:53                               ` Matt
2023-03-05 12:15                                 ` Ihor Radchenko
2023-03-06  6:45                                   ` Matt
2023-03-07 12:45                                     ` Ihor Radchenko
2023-03-09 17:36                                       ` Matt
2023-03-10  1:52                                         ` Max Nikulin
2023-03-12 16:28                                         ` Jack Kamm
2023-03-18 10:48                                         ` Ihor Radchenko
2023-03-21 20:29                                           ` Matt
2023-03-22 12:12                                             ` Ihor Radchenko
2023-03-23 11:50                                             ` Ihor Radchenko
2023-03-23 19:35                                               ` Matt
2023-03-24  9:13                                                 ` Ihor Radchenko
2023-03-28  2:53                                                   ` Matt
2023-03-28 10:06                                                     ` Ihor Radchenko
2023-04-17 15:31                                                   ` Matt
2023-04-17 18:55                                                     ` Ihor Radchenko
2023-04-17 18:56                                                       ` Matt
2023-04-17 19:05                                                         ` Ihor Radchenko
2023-03-23  3:25       ` [SUGGESTION] ob-shell async result output should not contains shell prompt Christopher M. Miles
2023-03-23  4:21         ` Matt
2023-03-23 11:12           ` Christopher M. Miles
2023-03-23 16:23             ` Matt
2023-03-24 11:20               ` Ihor Radchenko
2023-03-23 16:26             ` Remove "shell" as a supported Babel language within ob-shell.el (was Re: [SUGGESTION] ob-shell async result output should not contains shell prompt) Matt
2023-03-24  1:53               ` Remove "shell" as a supported Babel language within ob-shell.el Christopher M. Miles
2023-03-24 11:38               ` Remove "shell" as a supported Babel language within ob-shell.el (was Re: [SUGGESTION] ob-shell async result output should not contains shell prompt) Ihor Radchenko
2023-03-25  5:47                 ` Samuel Wales
2023-03-25 18:07                   ` Ihor Radchenko
2023-03-28  2:33                 ` Matt
2023-02-11 20:56 ` [PATCH] Async evaluation in ob-shell jackkamm
2023-02-12 19:02   ` Matt
2023-02-13  3:16     ` Jack Kamm
2023-02-13 17:07       ` [BUG] shell sessions started outside of Babel broken Matt
2023-02-15  6:19         ` Jack Kamm
2023-02-16 12:53           ` Ihor Radchenko
2023-02-19 15:04             ` Jack Kamm
2023-02-20 11:22               ` Ihor Radchenko
2023-02-21  5:23                 ` Jack Kamm
2023-02-22 10:38                   ` Ihor Radchenko
2023-03-25 16:55               ` Jack Kamm
2023-03-25 16:59                 ` [PATCH] Fix externally started sessions with ob-python Jack Kamm
2023-02-13 20:11       ` [BUG] conda doesn't work in ob-shell sessions Matt
2023-02-15  6:21         ` Jack Kamm
2024-01-18 11:55           ` Ihor Radchenko
2024-01-21 22:48             ` Jack Kamm
2024-01-22  3:42               ` Jack Kamm
2024-01-22 11:59                 ` Ihor Radchenko
2024-01-23  6:09                   ` Jack Kamm
2024-01-24 15:22                     ` Ihor Radchenko
2024-01-25 19:14                       ` Matt
2024-01-25 20:36                         ` Ihor Radchenko
2024-01-26  0:42                       ` Jack Kamm
2024-01-27 10:25                         ` Matt
2024-02-09 16:37                           ` Ihor Radchenko
2024-01-23 18:51               ` Suhail Singh

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://www.orgmode.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1863472efe9.10fdd5ba4258906.5972264927968042941@excalamus.com \
    --to=matt@excalamus.com \
    --cc=emacs-orgmode@gnu.org \
    --cc=yantar92@posteo.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).