From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id gOC5MAetgWY0egAAqHPOHw:P1 (envelope-from ) for ; Sun, 30 Jun 2024 19:07:52 +0000 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0.migadu.com with LMTPS id gOC5MAetgWY0egAAqHPOHw (envelope-from ) for ; Sun, 30 Jun 2024 21:07:51 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=gUO4h8di; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1719774471; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=LumzaeDfi1Sa2xyyL7l2R028i7rImeYEtF0y0b4Lhdc=; b=Vte/KRTlT/is44qs0orqKE4OvFejD0Hh9pFhQHCr+T46JX5HPE6IxkEDDUtb00LKl67ktN Rmjixda0jSzNH+aD27+hRIGxnP8bBktYRqzDjv7zqAcrcGIRLJ7XM8/AS9Y7RNY3HIc+9U Y4P6Qn2+EGUmCMW0+tDwZkiKttBUhdPIvMhNRxNlKCoM6xg0MJJ0L6e3/S2JTUPwEcgHp0 v5BkbChUiYEegLwRKaDIgrNUJTzy6N9eW2u4nyeJzrDbdY+8QelZARDVNLOrvJ6YWkhcOy 4PrjK18lGQvE8pxipycvL6Z2M0+gjqybpNUO2OrANKeISdi1fRCQmgLQhDuqHA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=gUO4h8di; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.net ARC-Seal: i=1; s=key1; d=yhetil.org; t=1719774471; a=rsa-sha256; cv=none; b=T5KpVLC9uLRtlZPFNWwiNPj2lihAGoYVqPkRrw4hQAmG+2IzFCsWHVD0GCwpD3zmda4NWC 2+JTExYEJGHXAwg4lA9L4VhMnWGv/zLexja2cbtl1N4SExliRtr50D9t9yOWZoBnYwlRis BdKTEK1HupmBLAQoNmJ36uke7pH7CH8lnoUQ9cA50r52Mn/hfBskc8O51UvJg5dzznBzp0 /XtELQ9CLAYVaiUpyzwcO9/NiX5q6NHPox6AY9VL+AaabuYSEGXjJztXak0zO4jIKBdIH8 8haazWIRyFpzGxjpYtHcwYySOjmxqj1rsDvvHjA6OXwCKz19BzA4gyCvtFDqcA== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 6EA6722233 for ; Sun, 30 Jun 2024 21:07:51 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sNztT-00081w-K6; Sun, 30 Jun 2024 15:07:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sNzt2-0007xd-54 for emacs-orgmode@gnu.org; Sun, 30 Jun 2024 15:07:00 -0400 Received: from mout02.posteo.de ([185.67.36.66]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sNzsz-0001Og-B9 for emacs-orgmode@gnu.org; Sun, 30 Jun 2024 15:06:55 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 4D77C240103 for ; Sun, 30 Jun 2024 21:06:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1719774411; bh=wbIFeWRjPUa6BxVTWkJCGufOI3kLW8E8rIGjZIsKvR4=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type: From; b=gUO4h8die1gHTcorIgn5wQObBhX9+6S7xd4jr7hVjjpZuNKfsB+cqHQN6zFb7OD5i XDQrOwv0Dmp3Vb9xQ1Y9Tkg7DiPGbc/deDrxNuAQCJzJUkAc4kIQI9zbrU3B/vIdh5 unDT0eylqPdZOyS+LZxMqNBH1whGQThL8pk1VfG+gBh+g9ag9hS2/L2xn8WiyqOwnS LJe1G6jJtX0p24J//TvifAZNCFdjdNqoWpd4k4XdYDAKyYmfHmLhkuyhp78WMy97L1 hlJx16ZZMV3Ec3eEaMhsOLnp/Aw72kcFOn0ZRmxDdUe+hc0bHK/aKHqQwPfO8uGL0x z+12GdkKUDeEQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4WBzDp3wwyz9rxK; Sun, 30 Jun 2024 21:06:50 +0200 (CEST) From: Ihor Radchenko To: "Cook, Malcolm" Cc: Matthew Trzcinski , Org-mode Subject: RE: org-babel-execute-src-block filters characters from :session *shell* output In-Reply-To: <87o782gx7o.fsf@localhost> References: <87ikybk44c.fsf@localhost> <87o782gx7o.fsf@localhost> Date: Sun, 30 Jun 2024 19:08:28 +0000 Message-ID: <871q4el043.fsf@localhost> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=185.67.36.66; envelope-from=yantar92@posteo.net; helo=mout02.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Spam-Score: -9.60 X-Migadu-Queue-Id: 6EA6722233 X-Migadu-Scanner: mx10.migadu.com X-Migadu-Spam-Score: -9.60 X-TUID: BSVGP6GI8cB5 --=-=-= Content-Type: text/plain Ihor Radchenko writes: > Again, we know this problem, and we normally force the prompt to be > something more unique when creating sessions. But we do not touch the > prompt when you point Org to an existing interactive shell buffer - that > could be a surprise if Org mode changes the prompts by force in the > shells you use interactively. ... on the other hand, Org mode simply hanging or emitting garbage is also not great. So, maybe changing the prompt by force is good? I am attaching tentative series of patches that will do exactly this - setup prompt even in the existing shell buffers. I also changed the babel prompt to be (1) shorter; (2) more "Babel"y. --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline; filename=0001-Rename-org-babel-comint-prompt-regexp-old-to-.-fallb.patch Content-Transfer-Encoding: quoted-printable >From aee2dd3f8ad0f1658ba2a222195351da9c9b4bf6 Mon Sep 17 00:00:00 2001 Message-ID: From: Ihor Radchenko Date: Sun, 30 Jun 2024 20:36:25 +0200 Subject: [PATCH 1/4] Rename `org-babel-comint-prompt-regexp-old' to `...-fallback' * lisp/ob-comint.el (org-babel-comint-prompt-regexp-fallback): Rename to more accurate name. Leave the old name as an alias. (org-babel-comint--set-fallback-prompt): Use the new name. (org-babel-comint-with-output): (org-babel-comint-wait-for-output): * lisp/ob-clojure.el (ob-clojure-eval-with-inf-clojure): * lisp/ob-haskell.el (org-babel-interpret-haskell): * lisp/ob-ruby.el (org-babel-ruby-initiate-session): * lisp/ob-shell.el (org-babel-sh-initiate-session): --- lisp/ob-clojure.el | 4 ++-- lisp/ob-comint.el | 19 ++++++++++--------- lisp/ob-haskell.el | 2 +- lisp/ob-ruby.el | 2 +- lisp/ob-shell.el | 2 +- 5 files changed, 15 insertions(+), 14 deletions(-) diff --git a/lisp/ob-clojure.el b/lisp/ob-clojure.el index c7ebbbb95..98a66d1ec 100644 --- a/lisp/ob-clojure.el +++ b/lisp/ob-clojure.el @@ -237,8 +237,8 @@ (defun ob-clojure-eval-with-inf-clojure (expanded param= s) "clojure" (format "clojure -A%s" alias) cmd0) cmd0))) - (setq - org-babel-comint-prompt-regexp-old comint-prompt-regexp + (setq-local + org-babel-comint-prompt-regexp-fallback comint-prompt-regexp comint-prompt-regexp inf-clojure-comint-prompt-regexp) (funcall-interactively #'inf-clojure cmd) (goto-char (point-max)))) diff --git a/lisp/ob-comint.el b/lisp/ob-comint.el index 764927af7..4d4c90c5a 100644 --- a/lisp/ob-comint.el +++ b/lisp/ob-comint.el @@ -58,8 +58,9 @@ (defmacro org-babel-comint-in-buffer (buffer &rest body) (let ((comint-input-filter (lambda (_input) nil))) ,@body)))))) =20 -(defvar-local org-babel-comint-prompt-regexp-old nil +(defvar org-babel-comint-prompt-regexp-fallback nil "Fallback regexp used to detect prompt.") +(defvaralias 'org-babel-comint-prompt-regexp-old 'org-babel-comint-prompt-= regexp-fallback) =20 (defcustom org-babel-comint-fallback-regexp-threshold 5.0 "Waiting time until trying to use fallback regexp to detect prompt. @@ -69,11 +70,11 @@ (defcustom org-babel-comint-fallback-regexp-threshold 5= .0 :package-version '(Org . "9.7")) =20 (defun org-babel-comint--set-fallback-prompt () - "Swap `comint-prompt-regexp' and `org-babel-comint-prompt-regexp-old'." - (when org-babel-comint-prompt-regexp-old + "Swap `comint-prompt-regexp' and `org-babel-comint-prompt-regexp-fallbac= k'." + (when org-babel-comint-prompt-regexp-fallback (let ((tmp comint-prompt-regexp)) - (setq comint-prompt-regexp org-babel-comint-prompt-regexp-old - org-babel-comint-prompt-regexp-old tmp)))) + (setq comint-prompt-regexp org-babel-comint-prompt-regexp-fallback + org-babel-comint-prompt-regexp-fallback tmp)))) =20 (defun org-babel-comint--prompt-filter (string &optional prompt-regexp) "Remove PROMPT-REGEXP from STRING. @@ -144,7 +145,7 @@ (defmacro org-babel-comint-with-output (meta &rest body) (accept-process-output (get-buffer-process (current-buffer)) org-babel-comint-fallback-regexp-threshold) - (when (and org-babel-comint-prompt-regexp-old + (when (and org-babel-comint-prompt-regexp-fallback (> (float-time (time-since start-time)) org-babel-comint-fallback-regexp-threshold) (progn @@ -154,7 +155,7 @@ (defmacro org-babel-comint-with-output (meta &rest body) (re-search-forward (regexp-quote ,eoe-indicator) nil t) (re-search-forward - org-babel-comint-prompt-regexp-old nil t))))) + org-babel-comint-prompt-regexp-fallback nil = t))))) (org-babel-comint--set-fallback-prompt)))) ;; replace cut dangling text (goto-char (process-mark (get-buffer-process (current-buffer)))) @@ -189,14 +190,14 @@ (defun org-babel-comint-wait-for-output (buffer) (accept-process-output (get-buffer-process buffer) org-babel-comint-fallback-regexp-threshold) - (when (and org-babel-comint-prompt-regexp-old + (when (and org-babel-comint-prompt-regexp-fallback (> (float-time (time-since start-time)) org-babel-comint-fallback-regexp-threshold) (progn (goto-char comint-last-input-end) (save-excursion (re-search-forward - org-babel-comint-prompt-regexp-old nil t)))) + org-babel-comint-prompt-regexp-fallback nil t)))) (org-babel-comint--set-fallback-prompt)))))) =20 (defun org-babel-comint-eval-invisibly-and-wait-for-file diff --git a/lisp/ob-haskell.el b/lisp/ob-haskell.el index 05f340fa0..a96a28b6b 100644 --- a/lisp/ob-haskell.el +++ b/lisp/ob-haskell.el @@ -153,7 +153,7 @@ (defun org-babel-interpret-haskell (body params) (add-hook 'inferior-haskell-hook (lambda () (setq-local - org-babel-comint-prompt-regexp-old comint-prompt-regexp + org-babel-comint-prompt-regexp-fallback comint-prompt-regexp comint-prompt-regexp (concat haskell-prompt-regexp "\\|^=CE=BB?> ")))) (org-babel-haskell-with-session session params diff --git a/lisp/ob-ruby.el b/lisp/ob-ruby.el index d920fb585..24b31c18c 100644 --- a/lisp/ob-ruby.el +++ b/lisp/ob-ruby.el @@ -192,7 +192,7 @@ (defun org-babel-ruby-initiate-session (&optional sessi= on params) (when new-session? (with-current-buffer session-buffer (setq-local - org-babel-comint-prompt-regexp-old comint-prompt-regexp + org-babel-comint-prompt-regexp-fallback comint-prompt-reg= exp comint-prompt-regexp (concat "^" org-babel-ruby-prompt)) (insert org-babel-ruby-define-prompt ";") (insert "_org_prompt_mode=3Dconf.prompt_mode;conf.prompt_m= ode=3D:CUSTOM;") diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el index 32b029044..9222379a3 100644 --- a/lisp/ob-shell.el +++ b/lisp/ob-shell.el @@ -294,7 +294,7 @@ (defun org-babel-sh-initiate-session (&optional session= _params) (alist-get t org-babel-shell-set-prompt-commands)) org-babel-sh-prompt)) (setq-local - org-babel-comint-prompt-regexp-old comint-prompt-regexp + org-babel-comint-prompt-regexp-fallback comint-prompt-regexp comint-prompt-regexp (concat "^" (regexp-quote org-babel-sh-prompt) " *")) --=20 2.45.2 --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0002-ob-shell-Arrange-unique-prompt-in-manually-started-s.patch >From c950bb79ad515e53bdac3d79da7b4352aa264e8a Mon Sep 17 00:00:00 2001 Message-ID: In-Reply-To: References: From: Ihor Radchenko Date: Sun, 30 Jun 2024 20:37:36 +0200 Subject: [PATCH 2/4] ob-shell: Arrange unique prompt in manually started shell buffers * lisp/ob-shell.el (org-babel-sh--prompt-initialized): New variable flag that indicates whether ob-shell changed the prompt in current comint buffer. (org-babel-sh-initiate-session): Set unique prompt in existing sessions if it is not yet done. Link: https://orgmode.org/list/87o782gx7o.fsf@localhost --- lisp/ob-shell.el | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el index 9222379a3..7b0d6ddab 100644 --- a/lisp/ob-shell.el +++ b/lisp/ob-shell.el @@ -276,16 +276,25 @@ (defvar org-babel-sh-eoe-output "org_babel_sh_eoe" (defvar org-babel-sh-prompt "org_babel_sh_prompt> " "String to set prompt in session shell.") +(defvar-local org-babel-sh--prompt-initialized nil + "When non-nil, ob-shell already initialized the prompt in current buffer.") + (defalias 'org-babel-shell-initiate-session #'org-babel-sh-initiate-session) (defun org-babel-sh-initiate-session (&optional session _params) "Initiate a session named SESSION according to PARAMS." (when (and session (not (string= session "none"))) (save-window-excursion - (or (org-babel-comint-buffer-livep session) + (or (and (org-babel-comint-buffer-livep session) + (buffer-local-value + 'org-babel-sh--prompt-initialized + (get-buffer session)) + session) (progn - (shell session) - ;; Set unique prompt for easier analysis of the output. - (org-babel-comint-wait-for-output (current-buffer)) + (if (org-babel-comint-buffer-livep session) + (set-buffer session) + (shell session) + ;; Set unique prompt for easier analysis of the output. + (org-babel-comint-wait-for-output (current-buffer))) (org-babel-comint-input-command (current-buffer) (format @@ -298,6 +307,7 @@ (defun org-babel-sh-initiate-session (&optional session _params) comint-prompt-regexp (concat "^" (regexp-quote org-babel-sh-prompt) " *")) + (setq org-babel-sh--prompt-initialized t) ;; Needed for Emacs 23 since the marker is initially ;; undefined and the filter functions try to use it without ;; checking. -- 2.45.2 --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0003-org-babel-sh-initiate-session-Fix-setting-non-standa.patch >From 21d9bcdc22c0b601e40316dd6df7394344d4cccf Mon Sep 17 00:00:00 2001 Message-ID: <21d9bcdc22c0b601e40316dd6df7394344d4cccf.1719774270.git.yantar92@posteo.net> In-Reply-To: References: From: Ihor Radchenko Date: Sun, 30 Jun 2024 20:59:22 +0200 Subject: [PATCH 3/4] org-babel-sh-initiate-session: Fix setting non-standard prompt * lisp/ob-shell.el (org-babel-sh-initiate-session): Set `comint-prompt-regexp' early, _before_ evaluating prompt change. This way, we make sure that comint is not stuck trying to search for the old prompt if the new prompt no longer matches `comint-prompt-regexp'. --- lisp/ob-shell.el | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el index 7b0d6ddab..f148fead7 100644 --- a/lisp/ob-shell.el +++ b/lisp/ob-shell.el @@ -295,6 +295,11 @@ (defun org-babel-sh-initiate-session (&optional session _params) (shell session) ;; Set unique prompt for easier analysis of the output. (org-babel-comint-wait-for-output (current-buffer))) + (setq-local + org-babel-comint-prompt-regexp-fallback comint-prompt-regexp + comint-prompt-regexp + (concat "^" (regexp-quote org-babel-sh-prompt) + " *")) (org-babel-comint-input-command (current-buffer) (format @@ -302,11 +307,6 @@ (defun org-babel-sh-initiate-session (&optional session _params) org-babel-shell-set-prompt-commands)) (alist-get t org-babel-shell-set-prompt-commands)) org-babel-sh-prompt)) - (setq-local - org-babel-comint-prompt-regexp-fallback comint-prompt-regexp - comint-prompt-regexp - (concat "^" (regexp-quote org-babel-sh-prompt) - " *")) (setq org-babel-sh--prompt-initialized t) ;; Needed for Emacs 23 since the marker is initially ;; undefined and the filter functions try to use it without -- 2.45.2 --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline; filename=0004-org-babel-sh-prompt-Use-cuneiform-break-symbol-as-un.patch Content-Transfer-Encoding: quoted-printable >From 5b98fef45f262e46323241c0482bf81573cdb763 Mon Sep 17 00:00:00 2001 Message-ID: <5b98fef45f262e46323241c0482bf81573cdb763.1719774270.git.yantar= 92@posteo.net> In-Reply-To: References: From: Ihor Radchenko Date: Sun, 30 Jun 2024 21:00:17 +0200 Subject: [PATCH 4/4] =3D?UTF-8?q?org-babel-sh-prompt:=3D20Use=3D20cuneiform= =3D20"b?=3D =3D?UTF-8?q?reak"=3D20=3DF0=3D92=3D86=3DB8=3D20symbol=3D20as=3D20unique=3D= 20prompt?=3D MIME-Version: 1.0 Content-Type: text/plain; charset=3DUTF-8 Content-Transfer-Encoding: 8bit * lisp/ob-shell.el (org-babel-sh-prompt): Change the value of unique prompt from long prompt string to a much shorter, but still unlikely to appear in output, single-char prompt. Cuneiform because Sumerian, Babylon, Tower ob Babel :) --- lisp/ob-shell.el | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el index f148fead7..05bded918 100644 --- a/lisp/ob-shell.el +++ b/lisp/ob-shell.el @@ -273,7 +273,7 @@ (defvar org-babel-sh-eoe-indicator "echo 'org_babel_sh_= eoe'" "String to indicate that evaluation has completed.") (defvar org-babel-sh-eoe-output "org_babel_sh_eoe" "String to indicate that evaluation has completed.") -(defvar org-babel-sh-prompt "org_babel_sh_prompt> " +(defvar org-babel-sh-prompt "=F0=92=86=B8 " "String to set prompt in session shell.") =20 (defvar-local org-babel-sh--prompt-initialized nil --=20 2.45.2 --=-=-= Content-Type: text/plain -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at --=-=-=--