From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id ACFoDUAvUmMnuQAAbAwnHQ (envelope-from ) for ; Fri, 21 Oct 2022 07:33:52 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id oG63DEAvUmNhZwAAG6o9tA (envelope-from ) for ; Fri, 21 Oct 2022 07:33:52 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id CF16C2970D for ; Fri, 21 Oct 2022 07:33:51 +0200 (CEST) Received: from localhost ([::1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1olkfG-0007DF-Vk for larch@yhetil.org; Fri, 21 Oct 2022 01:33:51 -0400 Received: from [::1] (helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1olkaR-0003e0-8w for larch@yhetil.org; Fri, 21 Oct 2022 01:28:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1olkaN-0003ds-PU for emacs-orgmode@gnu.org; Fri, 21 Oct 2022 01:28:48 -0400 Received: from mout01.posteo.de ([185.67.36.65]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1olkaL-0006be-DA for emacs-orgmode@gnu.org; Fri, 21 Oct 2022 01:28:47 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 59AE7240026 for ; Fri, 21 Oct 2022 07:28:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1666330123; bh=hY6NmILlCrCV0N94CzrG3OydVYrsxOeLh30cxA8Hk7w=; h=From:To:Cc:Subject:Date:From; b=eEZAyeHWi7mm30OFJ0k/S4lkQaNF5bk4eHleQjD9aVnRVrtuOozEr03XxqoyCy/cl DqElnNKaR7xTghWYYaE4uMSa6LhfSSD9ZUdqFXKrVhKR0BQDA9ygSPFD7KyZlZ1nFk y/N2Ux9H+HW2MsfS5WVIxsdi/UWsDOGqff/cUQJnOCPx/ENgMMIJVwwXLmo1kiFxwi ns4TB3Ug85MQOdp9+c4gFsbmywHmwPHR/A/FoYBYxKqRkMHSE6wOlgM/iJ20syDHK2 zobmLSjI/D2FtFKGdAYuYEZP+h1XmE7rLyp9A+g/rYpWUXXjo/wpMKTnYlWWDdungj gBtzetKXvAWZw== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4MttL059qtz9rxP; Fri, 21 Oct 2022 07:28:40 +0200 (CEST) From: Ihor Radchenko To: Rudolf =?utf-8?Q?Adamkovi=C4=8D?= Cc: Ihor Radchenko , emacs-orgmode@gnu.org Subject: Re: Org 9.6-pre and Bash sessions In-Reply-To: <87r0z6j1b1.fsf@localhost> References: <87edvbysqc.fsf@localhost> <87r0z6j1b1.fsf@localhost> Date: Fri, 21 Oct 2022 05:29:21 +0000 Message-ID: <87pmel68y6.fsf@localhost> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=185.67.36.65; envelope-from=yantar92@posteo.net; helo=mout01.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1666330432; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=ZKq686IEKAdqzMLyveCWwV6TgKAjPW9fLNRsLmysVpU=; b=QXcAd19hXe3wGpbv7c+lcpW27mpW/dPQtqINaZEvdymU8oH/Cf/xpukGEm2vwVOq2k/8pu vldguhlP2pbVsOhS0CULQKTMFiDNlHja3DkrLXB+r8deAqv8vSQJKCHOWsmvPBjby6NOZS k+wg+tGIpbesa8XzSj931GcX5TGJggRv9ewrKfZKCuYDwn5rWtKonz+1WSJH/lhe0UPtLT rKWF5e7b/ySw+39Y64aV3EaivGP8Z2l71ha+7u0JhGW3SLG2ApeVNh916kOTSEDZhQlh9e YjmXA/W3H55fi6PRO1AkyQKNuM/efb32Hui2M3sf8GtIx3odCgvDV5GSZcfIQw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1666330432; a=rsa-sha256; cv=none; b=FWPpjcP7iWTZP1MucnKXFDAIxcwdNM828xn6O9KQDvKzrPFXgF3W9eeOSK7Z6prVXNVUKr PX1D2pYcjWpjD27/bx4dGJ4VYAs+huDmVvlcZYMqIdcQqI0dqk5Nsf4pUcmnoknMkwsl3S W+DieL0k5SJdjjeTcTqP7u0SDf7gIZT4wpr80ZWc3IGZt2FEuihYSVhOOPlYDl8dhwFHas /1noLs313xIbceCnqbW4pvCnIq0xF/+t75WF8h+/AszZVRk/gwh49Ir4gYr+5wDnx9lzRV 8CO/EUstsapsAJ98x1P27peOIaUM/AK+2CYAmN53QtdyHPa3jjKVvn5BDrQBUA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=eEZAyeHW; dmarc=pass (policy=none) header.from=posteo.net; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -3.64 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=eEZAyeHW; dmarc=pass (policy=none) header.from=posteo.net; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: CF16C2970D X-Spam-Score: -3.64 X-Migadu-Scanner: scn1.migadu.com X-TUID: J/xcBZy22/Bw --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ihor Radchenko writes: > Rudolf Adamkovi=C4=8D writes: > >> : >> : > > hello >> >> on the subsequent runs. >> >> Better than the new version but still wrong. :) > > And this is what drove me crazy during debugging. I do not understand > the logic of all that ob-comint code. > > I have identified that the hang happens because Org does not change PS2 > prompt. Just PS1. But fixing this would yield to > > : org_babel_sh_prompt> org_babel_sh_prompt> hello > > Then, I tried to see how the original code works. And it does not > :facepalm: > > I asked emacs-devel > https://yhetil.org/emacs-devel/87y1tgqhmc.fsf@localhost/T/#u See the attached tentative patch. I'd appreciate some testing. Hopefully, I did not break anything. Comint is tricky. --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline; filename=0001-ob-shell-Fix-multi-line-scripts-in-sessions.patch Content-Transfer-Encoding: quoted-printable >From 4c6eead351cbdce1b9210a738c65b3a139d1cc0c Mon Sep 17 00:00:00 2001 Message-Id: <4c6eead351cbdce1b9210a738c65b3a139d1cc0c.1666330028.git.yantar= 92@posteo.net> From: Ihor Radchenko Date: Fri, 21 Oct 2022 13:21:57 +0800 Subject: [PATCH] ob-shell: Fix multi-line scripts in sessions MIME-Version: 1.0 Content-Type: text/plain; charset=3DUTF-8 Content-Transfer-Encoding: 8bit * lisp/ob-comint.el (org-babel-comint-with-output): Cleanup empty output. Such output is spitted unnecessarily for multi-line scripts. * lisp/ob-shell.el (org-babel-shell-set-prompt-commands): Disable PS2 and equivalent prompts. Make sure that PROMPT_COMMAND does not interfere with PS1 setting in POSIX shells. (org-babel-sh-evaluate): Do not send input line-by-line. Instead, let `org-babel-coming-with-output' handle waiting for the output as well as recording it. Update to the new `org-babel-coming-with-output' behavior of cleaning empty outputs. * testing/lisp/test-ob-shell.el (test-ob-shell/session): Add test. Reported-by: Rudolf Adamkovi=C4=8D Link: https://list.orgmode.org/orgmode/m2r0zboix1.fsf@me.com/ --- lisp/ob-comint.el | 2 +- lisp/ob-shell.el | 29 ++++++++++++++--------------- testing/lisp/test-ob-shell.el | 6 +++++- 3 files changed, 20 insertions(+), 17 deletions(-) diff --git a/lisp/ob-comint.el b/lisp/ob-comint.el index 36a55d36c..ec7d3f6c9 100644 --- a/lisp/ob-comint.el +++ b/lisp/ob-comint.el @@ -109,7 +109,7 @@ (defmacro org-babel-comint-with-output (meta &rest body) "\n" "[\r\n]+" (regexp-quote (or ,full-body ""))) string-buffer)) (setq string-buffer (substring string-buffer (match-end 0)))) - (split-string string-buffer comint-prompt-regexp))))) + (delete "" (split-string string-buffer comint-prompt-regexp)))))) =20 (defun org-babel-comint-input-command (buffer cmd) "Pass CMD to BUFFER. diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el index 4d579ae87..d38d2d335 100644 --- a/lisp/ob-shell.el +++ b/lisp/ob-shell.el @@ -47,10 +47,15 @@ (defvar org-babel-default-header-args:shell '()) (defvar org-babel-shell-names) =20 (defconst org-babel-shell-set-prompt-commands - '(("fish" . "function fish_prompt\n\techo \"%s\"\nend") - ("csh" . "set prompt=3D\"%s\"") + '(;; Fish has no PS2 equivalent. + ("fish" . "function fish_prompt\n\techo \"%s\"\nend") + ;; prompt2 is like PS2 in POSIX shells. + ("csh" . "set prompt=3D\"%s\"\nset prompt2=3D\"\"") + ;; PowerShell, similar to fish, does not have PS2 equivalent. ("posh" . "function prompt { \"%s\" }") - (t . "PS1=3D\"%s\"")) + ;; PROMPT_COMMAND can override PS1 settings. Disable it. + ;; Disable PS2 to avoid garbage in multi-line inputs. + (t . "PROMPT_COMMAND=3D;PS1=3D\"%s\";PS2=3D")) "Alist assigning shells with their prompt setting command. =20 Each element of the alist associates a shell type from @@ -299,20 +304,14 @@ (defun org-babel-sh-evaluate (session body &optional = params stdin cmdline) #'org-babel-sh-strip-weird-long-prompt (mapcar #'org-trim - (butlast + (butlast ; Remove eoe indicator (org-babel-comint-with-output (session org-babel-sh-eoe-output t body) - (dolist (line (append (split-string (org-trim body) "\n") - (list org-babel-sh-eoe-indicator))) - (insert line) - (comint-send-input nil t) - (while (save-excursion - (goto-char comint-last-input-end) - (not (re-search-forward - comint-prompt-regexp nil t))) - (accept-process-output - (get-buffer-process (current-buffer)))))) - 2)) + (insert (org-trim body) "\n" + org-babel-sh-eoe-indicator) + (comint-send-input nil t)) + ;; Remove `org-babel-sh-eoe-indicator' output line. + 1)) "\n")) ;; External shell script, with or without a predefined ;; shebang. diff --git a/testing/lisp/test-ob-shell.el b/testing/lisp/test-ob-shell.el index a0d5a8d22..05c369174 100644 --- a/testing/lisp/test-ob-shell.el +++ b/testing/lisp/test-ob-shell.el @@ -46,7 +46,11 @@ (ert-deftest test-ob-shell/session () ob-comint.el, which was not previously tested." (let ((res (org-babel-execute:sh "echo 1; echo 2" '((:session . "yes")))= )) (should res) - (should (listp res)))) + (should (listp res))) + ;; Test multi-line input. + (let ((res (org-babel-execute:sh "if true; then\necho \"yes\"\nfi" '((:s= ession . "yes"))))) + (should res) + (should (string=3D "yes" res)))) =20 ; A list of tests using the samples in ob-shell-test.org (ert-deftest ob-shell/generic-uses-no-arrays () --=20 2.35.1 --=-=-= Content-Type: text/plain -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at --=-=-=--