From: Nick Savage <firstname.lastname@example.org> To: Ihor Radchenko <email@example.com> Cc: "Emanuel Berg via General discussions about Org-mode." <firstname.lastname@example.org> Subject: Re: babel output seems to drop anything before % (in session) Date: Thu, 6 May 2021 19:44:32 -0400 [thread overview] Message-ID: <email@example.com> (raw) In-Reply-To: <87wnsc2gp2.fsf@localhost> [-- Attachment #1: Type: text/plain, Size: 2654 bytes --] So I have a patch written that doesn't completely fix the problem, but maybe makes enough progress that someone else can figure it out. The issue is that comint-prompt-regexp is reading the "% " as a prompt, and taking everyone off before it. I've added another parameter to org-babel-comint-with-output in the "meta" to pass our own regexp to replace comint-prompt-regexp, which works except when it doesn't. The regexp I've added is just "\n" now, so the newline characters are removed. The tests that Daniele added as a patch don't quite pass though with this. The issue is that something in the way the output is posted in the output buffer includes the prompt occasionally is included in a line and occasionally not. It seems the first time the block of code is executed, it is included (and therefore needs to be removed) and each other time it is not, so it is only the first time that it is run is not working properly. This is obviously way too fragile to actually merge, but I was hoping the work I've done so far is enough to help someone else make progress. I'll probably take another stab at this tomorrow (since it's bugging me), but thought I'd share what I have for now. Cheers, Nick On 5/6/21 8:24 AM, Ihor Radchenko wrote: > "Nicholas Savage" <firstname.lastname@example.org> writes: > >> I can confirm this too on the latest master. >> >> I took a quick peek this morning, and my suspicion is that the problem is somewhere within org-babel-comint-with-output in lisp/ob-comint.el, but I'm not positive at this point. > I confirm as well. I also saw an anomaly in the comint buffer. Note that > all the output lines, except "five 0% six" are after the shell prompt. > As I remember, the code expects the result to be exactly at the prompt > line. So, for some reason the first command ("echo five 0% six") of the > second block does not get inserted at the empty line. > > echo one 0% two > yantar92@yantar92-laptop ~/.data/1e/90360c-ef36-4d20-8706-990ae2530cbf $ one 0% two > echo tree 0% four > yantar92@yantar92-laptop ~/.data/1e/90360c-ef36-4d20-8706-990ae2530cbf $ tree 0% four > echo 'org_babel_sh_eoe' > yantar92@yantar92-laptop ~/.data/1e/90360c-ef36-4d20-8706-990ae2530cbf $ org_babel_sh_eoe > yantar92@yantar92-laptop ~/.data/1e/90360c-ef36-4d20-8706-990ae2530cbf $ echo five 0% six > five 0% six > echo seven 0% eight > yantar92@yantar92-laptop ~/.data/1e/90360c-ef36-4d20-8706-990ae2530cbf $ seven 0% eight > echo 'org_babel_sh_eoe' > yantar92@yantar92-laptop ~/.data/1e/90360c-ef36-4d20-8706-990ae2530cbf $ org_babel_sh_eoe > yantar92@yantar92-laptop ~/.data/1e/90360c-ef36-4d20-8706-990ae2530cbf $ > > Best, > Ihor [-- Attachment #2: 0001-ob-shell.el-Fix-bug-where-shell-output-was-incorrect.patch --] [-- Type: text/x-patch, Size: 3692 bytes --] From 6c7d39bfb9be38b54d23fcffbb09f1fcb96751f4 Mon Sep 17 00:00:00 2001 From: Nicholas Savage <email@example.com> Date: Thu, 6 May 2021 19:17:33 -0400 Subject: [PATCH] ob-shell.el: Fix bug where shell output was incorrectly truncated on special characters. * lisp/ob-comint.el (org-babel-comint-with-output): Add fifth meta optional argument for providing a custom prompt regexp. lisp/ob-shell.el (org-babel-sh-evaluate): Implements using new argument to prevent incorrect truncation on special characters. If shell output included special characters that also occasionally are included in shell prompts, such as "#" or "%", a regexp was tripping up on them and cutting out part of the line. As ob-shell already cuts out the shell prompts, this was not necessary and instead we can just use \n as the separator. Original functionality was retained for other ob-* files in case this was necessary. --- lisp/ob-comint.el | 20 ++++++++++++-------- lisp/ob-shell.el | 2 +- 2 files changed, 13 insertions(+), 9 deletions(-) diff --git a/lisp/ob-comint.el b/lisp/ob-comint.el index 18d4f3c93..27ad6efd7 100644 --- a/lisp/ob-comint.el +++ b/lisp/ob-comint.el @@ -57,13 +57,14 @@ executed inside the protection of `save-excursion' and (defmacro org-babel-comint-with-output (meta &rest body) "Evaluate BODY in BUFFER and return process output. -Will wait until EOE-INDICATOR appears in the output, then return -all process output. If REMOVE-ECHO and FULL-BODY are present and -non-nil, then strip echo'd body from the returned output. META -should be a list containing the following where the last two -elements are optional. +Will wait until EOE-INDICATOR appears in the output, then return all +process output. If REMOVE-ECHO and FULL-BODY are present and non-nil, +then strip echo'd body from the returned output. PROMPT-REGEXP is a +filter that, if provided, overrides the default regexp that tries to +filter out the shell prompt. META should be a list containing the +following where the last three elements are optional. - (BUFFER EOE-INDICATOR REMOVE-ECHO FULL-BODY) + (BUFFER EOE-INDICATOR REMOVE-ECHO FULL-BODY PROMPT-REGEXP) This macro ensures that the filter is removed in case of an error or user `keyboard-quit' during execution of body." @@ -71,7 +72,10 @@ or user `keyboard-quit' during execution of body." (let ((buffer (nth 0 meta)) (eoe-indicator (nth 1 meta)) (remove-echo (nth 2 meta)) - (full-body (nth 3 meta))) + (full-body (nth 3 meta)) + (prompt-regexp (if (nth 4 meta) + (nth 4 meta) + comint-prompt-regexp))) `(org-babel-comint-in-buffer ,buffer (let* ((string-buffer "") (comint-output-filter-functions @@ -111,7 +115,7 @@ or user `keyboard-quit' during execution of body." "\n" "[\r\n]+" (regexp-quote (or ,full-body ""))) string-buffer)) (setq string-buffer (substring string-buffer (match-end 0)))) - (split-string string-buffer comint-prompt-regexp))))) + (split-string string-buffer ,prompt-regexp))))) (def-edebug-spec org-babel-comint-with-output (sexp body)) (defun org-babel-comint-input-command (buffer cmd) diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el index 3eed0c164..9ec0425cb 100644 --- a/lisp/ob-shell.el +++ b/lisp/ob-shell.el @@ -263,7 +263,7 @@ return the value of the last statement in BODY." #'org-trim (butlast (org-babel-comint-with-output - (session org-babel-sh-eoe-output t body) + (session org-babel-sh-eoe-output t body "\n") (dolist (line (append (split-string (org-trim body) "\n") (list org-babel-sh-eoe-indicator))) (insert line) -- 2.20.1
next prev parent reply other threads:[~2021-05-06 23:45 UTC|newest] Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-05-05 15:15 Daniele Pizzolli 2021-05-06 2:35 ` John Corless 2021-05-06 11:39 ` Nicholas Savage 2021-05-06 12:24 ` Ihor Radchenko 2021-05-06 23:44 ` Nick Savage [this message] 2021-05-15 20:47 ` Bastien 2021-06-09 3:41 ` Kaushal Modi 2021-06-09 3:54 ` Kaushal Modi 2021-06-09 4:01 ` Kaushal Modi 2021-11-09 21:21 ` Kaushal Modi 2021-11-10 4:13 ` Ihor Radchenko
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: https://www.orgmode.org/ * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --subject='Re: babel output seems to drop anything before % (in session)' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Code repositories for project(s) associated with this inbox: https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).