From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:403:478a::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id EDNhLqJb7GQuAwAA9RJhRA:P1 (envelope-from ) for ; Mon, 28 Aug 2023 10:32:34 +0200 Received: from aspmx1.migadu.com ([2001:41d0:403:478a::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id EDNhLqJb7GQuAwAA9RJhRA (envelope-from ) for ; Mon, 28 Aug 2023 10:32:34 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 691C2505A2 for ; Mon, 28 Aug 2023 10:32:34 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=qbfgKWHX; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.net ARC-Seal: i=1; s=key1; d=yhetil.org; t=1693211554; a=rsa-sha256; cv=none; b=cMKbwEv7SNNxohobVs/cMNcAwdqifZ5GPw86BhwjJaUjVTUsFGaHn2pL39dXMW8IJZxQoW s3I9zzzLL4/8kRY/QFOTUxQz1rHunUjC5eOwZHIFyMQfNvoxnFuq08nyg7XkuFoYg+8wd3 c9Oz+cbf++WWZIcw8Sy5pHVFUpWfB1sWmG8sV/yLbrXYvLlgBFegL2hVyHeMT/ehtpGfW1 cEnGu9mneQvoMdRH3DOX44bF6OIQba1iMz2bW1/h2Izli1uGwp85ejpfmydOqZl0H8Yflo uN9MFAd/8+VANyzvs0zawvdAPVyDF5Wk6SUTdV6t+E2cmxlqCsZU8bmkM2VA3w== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=qbfgKWHX; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1693211554; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=cnFt/jNziW90Ffpzs4M2Inx661jPe5/dez0Xz8Q1NJE=; b=moqk0gAzOnsaeLMlufmD4Yn3Jro0CfmxPA+JOy+wP32i86UGRmfyoGE8T6WerdmxBW9OTK 6aln6km/EQYwFJR8XF9+9wAbEMcWOUrarRo/rVhs3P56d8m4nDnh9QlqVKnHql02nZQF/s o87VQtmgSXoJQv//8FUAoXvN8ibJn9BawudTvS2uCfBAWWCs77l2qd/kazBbSK27tPcyrV EiyX8RzFS7VrVA3CuJ8Lzl8CJZllb1tHxUhtTiYz2MWd7a0td6Dk9em5KVYDmJCnK5a3Kk No/l93Rz4BZ3hca840abqskgsadfCRIxdLvTQlLpB6JvTBrYFYk1/boukpnv5A== Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qaXes-0001JX-Tr; Mon, 28 Aug 2023 04:31:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qaXei-0001JA-KN for emacs-orgmode@gnu.org; Mon, 28 Aug 2023 04:31:28 -0400 Received: from mout01.posteo.de ([185.67.36.65]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qaXef-0005lg-96 for emacs-orgmode@gnu.org; Mon, 28 Aug 2023 04:31:28 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 90514240027 for ; Mon, 28 Aug 2023 10:31:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1693211482; bh=yvbT5glrcYxNK9PbzCvGmm93HIJoWqAF/xfwJrY7sqw=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=qbfgKWHXTYE8/wbm95zCWzZu5yUsm9j8yxwKafLbn6pTdR/aq3YVQQpe07mSdVmsP StpGf5ZpncAHmZa7CDYKZZklY+Wg4O9luAf6eJREf6/IuoW+M6E8rOmBgcADAkt10R KiH64qYOSuaNQkvnjkOa4x3Qqdn/VkzLQjzBsRy1Ov0cQv+O+g6x/sqUEdfcpeiS5T QOrFzgpM+cZTh+BoeAcqLPzdCks0iHoWaoITTf04JXVylhT0TksP23wW2QlTULgYry ZX1G7rBmRrzEyai+uM6wEm7A9iIyX8Ot8IcNzGimol92L81Z7giACUQhT1sEB7rVsR Xp+xVtOAPGk2Q== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4RZ3gF5q8Qz6vSl; Mon, 28 Aug 2023 10:31:21 +0200 (CEST) From: Ihor Radchenko To: Jack Kamm Cc: emacs-orgmode@gnu.org Subject: Re: [BUG] FAILED test-ob-python/session-multiline In-Reply-To: <87fs449xoy.fsf@gmail.com> References: <873506j7ky.fsf@localhost> <87il91ajq7.fsf@gmail.com> <87ledxix4l.fsf@localhost> <87fs449xoy.fsf@gmail.com> Date: Mon, 28 Aug 2023 08:31:52 +0000 Message-ID: <87edjn4lfr.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=185.67.36.65; envelope-from=yantar92@posteo.net; helo=mout01.posteo.de X-Spam_score_int: -53 X-Spam_score: -5.4 X-Spam_bar: ----- X-Spam_report: (-5.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Scanner: mx0.migadu.com X-Spam-Score: -6.43 X-Migadu-Queue-Id: 691C2505A2 X-Migadu-Spam-Score: -6.43 X-TUID: 2+QiUlVS1MAD Jack Kamm writes: > This one might take some time to fix, since it's hard to reproduce and > I'm not sure the cause of it. But here are 3 different solutions I am > considering now: > > 1. In addition to printing `org-babel-python-eoe-indicator' after > execution, we could also print out a "beginning of execution" > indicator before execution, and then capture the output between the > beginning and end indicators. This is how the async session > execution works, and should avoid any possibility of capturing > prompts. This idea looks interesting. Although I would not be so sure that it will fix things - I have learned that comint has many edge cases we may not easily anticipate. For example, see the discussion in https://yhetil.org/emacs-devel/87y1tgqhmc.fsf@localhost/ > 2. Instead of relying on our own custom `org-babel-python-send-string', > we could try switching to python.el's > `python-shell-send-string-no-output', which is probably more > robust. This would also allow removing the ugly > `org-babel-python-eoe-indicator' we currently print. Beware non-standard REPL prompts. When we have something other than >>>, the logic may have hard time figuring out output boundaries without deterministic separators like eoe-indicator. > Downside is that the output would not be echoed into the session > anymore. To fix that, we could manually insert the captured output > into the comint session buffer after execution. Alternatively, we > could add an argument to `python-shell-send-string-no-output' to > avoid suppressing output, submit it upstream to python.el, and then > backport to Org to support older emacs versions. If we can (eventually) remove some custom code from Org and move it to Emacs, it will be the best for working towards RMS request https://orgmode.org/list/E1kIPh1-0001Lu-Rg@fencepost.gnu.org > 3. Revisit a series of commits I made in 2020, which was supposed to > make session evaluation more robust, and was inspired by > `python-shell-send-string-no-output': > https://git.sr.ht/~bzg/org-mode/commit/4df12ea39 > > However, I had to partially revert that work, due to compatibility > issue with emacs 26.3: > https://list.orgmode.org/871rjcan53.fsf@kyleam.com/ > > I think we no longer support emacs 26.3, so I could potentially > revisit this now -- but it's been a few years and will take some > time to refresh my memory about this. Note that I changed `org-babel-comint-wait-for-output' and some other ob-comint machinery since that time. In particular, I tried to address various edge cases we encountered with bash sessions. See the commentary in `org-babel-comint-with-output'. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at