From: Matt <matt@excalamus.com>
To: "Ihor Radchenko" <yantar92@posteo.net>
Cc: "emacs-orgmode" <emacs-orgmode@gnu.org>
Subject: Re: [BUG] Prompt appears in async shell results
Date: Sun, 18 Feb 2024 13:09:34 +0100 [thread overview]
Message-ID: <18dbc1f273c.11687295c1395973.3345700621594100711@excalamus.com> (raw)
In-Reply-To: <87ttmn9fg0.fsf@localhost>
[-- Attachment #1: Type: text/plain, Size: 2253 bytes --]
---- On Mon, 05 Feb 2024 15:00:18 +0100 Ihor Radchenko wrote ---
> Yes. The right fix would be extracting the filter from
> `org-babel-comint-with-output' and re-using it.
This message documents my progress so that I may switch focus to Bruno's patches in "Asynchronous blocks for everything" attention (https://list.orgmode.org/65cfa0d8.050a0220.cb569.ce34@mx.google.com/T/#u).
Attached is a failed attempt at extracting the filter in =org-babel-comint-with-output=. It tries to extract the filter more-or-less directly:
1. take the filter code from =org-babel-comint-with-output= and put it
into a separate function, =org-babel-comint-process-output-filter=
2. call =org-babel-comint-process-output-filter= from
=org-babel-comint-with-output= and =org-babel-comint-async-filter=
The unmodified =org-babel-comint-with-output= has a comment that says, "Filter out prompts". This is misleading. The filter code does two things: removes prompts *and* removes echoed input.
The problem is the filter which removes echoes uses the body of the source block. It's unclear how to give =org-babel-comint-async-filter= the block body. =org-babel-comint-async-filter= is a =comint-output-filter-function= which receives a single input, "a string containing the text as originally inserted" in the comint buffer.
Thoughts:
- Split prompt filtering and input echoing into two filters
+ this seems to imply a =-hook= or =-functions= type implementation
+ where could input echo filter go? Where has access to the block body?
- creating a generic prompt filter duplicates =ob-shell-async-chunk-callback= or, more fundamentally, =org-babel-comint-async-chunk-callback=
- What would it take to consolidate output filtering? In addition to prompt filtering and input echo filtering, ob-shell filters the =org-babel-sh-eoe-indicator=. I'm sure there's other filtering that happens on block output. Wouldn't it be nice if that were in a single place, like right before results are inserted?
Please feel free to provide feedback and suggestions.
--
Matt Trzcinski
Emacs Org contributor (ob-shell)
Learn more about Org mode at https://orgmode.org
Support Org development at https://liberapay.com/org-mode
[-- Attachment #2: v01-refactor-filter.diff --]
[-- Type: application/octet-stream, Size: 5443 bytes --]
diff --git a/lisp/ob-comint.el b/lisp/ob-comint.el
index 1ec84e865..fd80880e9 100644
--- a/lisp/ob-comint.el
+++ b/lisp/ob-comint.el
@@ -74,6 +74,35 @@ This is useful when prompt unexpectedly changes."
(setq comint-prompt-regexp org-babel-comint-prompt-regexp-old
org-babel-comint-prompt-regexp-old tmp))))
+(defun org-babel-comint-process-output-filter (from-string &optional to-remove remove-echo)
+ "Remove string FROM-STRING from TO-REMOVE.
+
+TO-REMOVE defaults to `comint-prompt-regexp'. REMOVE-ECHO
+removes echoed commands some shells may print and defaults to t."
+ (let* ((to-remove (or to-remove comint-prompt-regexp))
+ (remove-echo (or remove-echo t)))
+ ;; remove TO-REMOVE from FROM-STRING
+ (while (string-match-p to-remove from-string)
+ (setq from-string
+ (replace-regexp-in-string
+ (format
+ "\\(?:%s\\)?\\(?:%s\\)[ \t]*"
+ "org-babel-comint-process-output-filter-separator\n"
+ to-remove)
+ "org-babel-comint-process-output-filter-separator\n"
+ from-string)))
+ ;; remove echo
+ (when (and remove-echo from-string ; <-- FIXME from-string needs to be block body
+ (string-match
+ (replace-regexp-in-string
+ "\n" "[\r\n]+" (regexp-quote (or from-string "")))
+ from-string))
+ (setq from-string (substring from-string (match-end 0))))
+ (delete "" (split-string
+ from-string
+ "org-babel-comint-process-output-filter-separator\n"))
+ ))
+
(defmacro org-babel-comint-with-output (meta &rest body)
"Evaluate BODY in BUFFER and return process output.
Will wait until EOE-INDICATOR appears in the output, then return
@@ -139,31 +168,35 @@ or user `keyboard-quit' during execution of body."
(goto-char (process-mark (get-buffer-process (current-buffer))))
(insert dangling-text)
- ;; Filter out prompts.
- (while (string-match-p comint-prompt-regexp string-buffer)
- (setq string-buffer
- (replace-regexp-in-string
- ;; Sometimes, we get multiple agglomerated
- ;; prompts together in a single output:
- ;; "prompt prompt prompt output"
- ;; Or even "<whitespace>prompt<whitespace>prompt ...>.
- ;; Remove them progressively, so that
- ;; possible "^" in the prompt regexp gets to
- ;; work as we remove the heading prompt
- ;; instance.
- (format "\\(?:%s\\)?\\(?:%s\\)[ \t]*" ,org-babel-comint-prompt-separator comint-prompt-regexp)
- ,org-babel-comint-prompt-separator
- string-buffer)))
- ;; remove echo'd FULL-BODY from input
- (when (and ,remove-echo ,full-body
- (string-match
- (replace-regexp-in-string
- "\n" "[\r\n]+" (regexp-quote (or ,full-body "")))
- string-buffer))
- (setq string-buffer (substring string-buffer (match-end 0))))
- (delete "" (split-string
- string-buffer
- ,org-babel-comint-prompt-separator))))))
+ (org-babel-comint-process-output-filter string-buffer)
+
+ ;; ;; Filter out prompts.
+ ;; (while (string-match-p comint-prompt-regexp string-buffer)
+ ;; (setq string-buffer
+ ;; (replace-regexp-in-string
+ ;; ;; Sometimes, we get multiple agglomerated
+ ;; ;; prompts together in a single output:
+ ;; ;; "prompt prompt prompt output"
+ ;; ;; Or even "<whitespace>prompt<whitespace>prompt ...>.
+ ;; ;; Remove them progressively, so that
+ ;; ;; possible "^" in the prompt regexp gets to
+ ;; ;; work as we remove the heading prompt
+ ;; ;; instance.
+ ;; (format "\\(?:%s\\)?\\(?:%s\\)[ \t]*" ,org-babel-comint-prompt-separator comint-prompt-regexp)
+ ;; ,org-babel-comint-prompt-separator
+ ;; string-buffer)))
+ ;; ;; remove echo'd FULL-BODY from input
+ ;; (when (and ,remove-echo ,full-body
+ ;; (string-match
+ ;; (replace-regexp-in-string
+ ;; "\n" "[\r\n]+" (regexp-quote (or ,full-body "")))
+ ;; string-buffer))
+ ;; (setq string-buffer (substring string-buffer (match-end 0))))
+ ;; (delete "" (split-string
+ ;; string-buffer
+ ;; ,org-babel-comint-prompt-separator))
+
+ ))))
(defun org-babel-comint-input-command (buffer cmd)
"Pass CMD to BUFFER.
@@ -324,8 +357,9 @@ STRING contains the output originally inserted into the comint buffer."
until (and (equal (match-string 1) "start")
(equal (match-string 2) uuid))
finally return (+ 1 (match-end 0)))))
- ;; Apply callback to clean up the result
- (res-str (funcall org-babel-comint-async-chunk-callback
+ ;; Apply filter to clean up the result
+ (res-str ;(funcall org-babel-comint-async-chunk-callback
+ (org-babel-comint-process-output-filter
res-str-raw)))
;; Search for uuid in associated org-buffers to insert results
(cl-loop for buf in org-buffers
next prev parent reply other threads:[~2024-02-18 12:10 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-04 17:48 [BUG] Prompt appears in async shell results Matt
2024-02-05 14:02 ` Ihor Radchenko
2024-02-18 12:09 ` Matt [this message]
2024-02-19 11:10 ` Ihor Radchenko
2024-03-17 19:36 ` Matt
2024-03-17 20:01 ` bug? org-babel-comint-with-output return value type Matt
2024-03-19 14:20 ` Ihor Radchenko
2024-03-27 10:59 ` [BUG] Prompt appears in async shell results Ihor Radchenko
2024-03-29 11:15 ` Matt
2024-03-29 11:23 ` Ihor Radchenko
-- strict thread matches above, loose matches on Subject: below --
2024-03-23 8:17 Matt
2024-03-23 14:16 ` Ihor Radchenko
2024-03-24 13:29 ` Matt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=18dbc1f273c.11687295c1395973.3345700621594100711@excalamus.com \
--to=matt@excalamus.com \
--cc=emacs-orgmode@gnu.org \
--cc=yantar92@posteo.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).