From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:306:2d92::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id QNBnHn98LGVX9gAAauVa8A:P1 (envelope-from ) for ; Mon, 16 Oct 2023 01:57:51 +0200 Received: from aspmx1.migadu.com ([2001:41d0:306:2d92::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id QNBnHn98LGVX9gAAauVa8A (envelope-from ) for ; Mon, 16 Oct 2023 01:57:51 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 40389412A7 for ; Mon, 16 Oct 2023 01:57:51 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=nc9DsYSm; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1697414271; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=lU0A6el8rP8l0e1yOe+aoO+KUKtExusouSGPFW5owGQ=; b=umUc7v5bUNZ+hRk9NOwnPOMY2XxDuEDiJroCXXmcnLIBOxrtLAOjxBepBsQjPX6wt/nev1 0Zh9Y1t7CbGePINQTTIkQlT8GFDR9/czjqtEzq1wG1m6gQ7OGqdz1CUhKcfUe+ZNr+gkIy ywn1M5lTAoLIbS3VgTclmGnsOkbQZ3YbOeecnWPrzYPCGkuqgDUj0q/xfZn1NR8s5Vh7J4 U9/rqp2jq0o3cbwPlZ2PW3F08zfBtN2MwKPa4imUe8gGdGb7QuviLrmWcVqGaVEwvY2RfG ELS4aawb+WIplgsiWDZXxOEPGYCiHoNU2cIREzFamr2Ofwd0/b2x8xCstRTMLQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=nc9DsYSm; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=key1; d=yhetil.org; t=1697414271; a=rsa-sha256; cv=none; b=ufIGNSsc+bUu+b3tY9DZ3rK3/g8+wjwhnoaXi0Gyf6q9RHfcPEAf11c7niQ2Q47VNr3ay7 UmqloTZ6lgBTAlyqaZdBBlSP1KpMoDy2GfZNKVfr8pqaaKMJrJUOkUbb3Y2+DCXtg8sIfv joTeNBlGreaCix45iYdfkvcTEvUe8W9vDMIL4AznO9ymJ/H7A5evOfDoDYqNKf/gl3LLu6 Dmlgpa1yBg7eGp7nfQ+wdRh7Dne3VMVzvDfphL5e6Mdg+3ilWMySE++k0VOuDOVkUTRZkT Epy1K0Xotw+zpVXkRbXCZh8YhvHRd9eL5WPfKiKh6fw/L22cI5PbW1y0BeMYhw== Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qsAyf-00072N-V9; Sun, 15 Oct 2023 19:56:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qsAye-00072D-Jd for emacs-orgmode@gnu.org; Sun, 15 Oct 2023 19:56:56 -0400 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qsAyc-0001sp-JK for emacs-orgmode@gnu.org; Sun, 15 Oct 2023 19:56:56 -0400 Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-6ba172c5f3dso865500b3a.0 for ; Sun, 15 Oct 2023 16:56:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697414213; x=1698019013; darn=gnu.org; h=mime-version:message-id:date:references:in-reply-to:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=lU0A6el8rP8l0e1yOe+aoO+KUKtExusouSGPFW5owGQ=; b=nc9DsYSmwXUT+EzKE52nowoKU+jWQeKvy3cm5E884HohMtOmJLX2kknCFHXAV468Lk vPcHtNvTbtl3107H0hS29fCfkXy7Swnc614I86u17L0bZZP9qX2mNEde7NvzeJIYGSQ2 k5a2p67tRCXRoy8gtNeLpAnFS58609xTNWaykLgqp9inbWh+4QF2dfHchr1/fhmmnyN/ iQ8pb8klZuNI/69kZhp9i4tCO0uk9C4fUz1aWOkV4zz/3Yzz51u5fJnDNp8cMfoGydTd zU/7UMkOI+7jbOdkdvbXIRzaRZMMj6HNIhs/NK9Jg+2GyJoBpflrAOhf7/QUwQ2fRhQ7 6KeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697414213; x=1698019013; h=mime-version:message-id:date:references:in-reply-to:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=lU0A6el8rP8l0e1yOe+aoO+KUKtExusouSGPFW5owGQ=; b=glPoGAbHrU/Khc9AR5VZH4O91XlpZmFMwz2q1cydrhIY+42+ql/ywkUafzpDURceo0 Aow6wGpA+6xY/ToosiGqeRTo+1IOmmrAZlQytXccXI/CZ5NUkSgBmk+zstAb12JihFKp kVrWyZsZJHDgXH/vim2FkYCL4KpxUjvoOcki2bdLB6rcqP4fFmYEjELnWl4zmTKygjLq 6a6de4dqsM7Jr9a1v67eoT49iv9V9dIPtGm+18505aQzalsH51G9wZTm93N+vB8oNJ0H JkrLZtUqNZTLzytVKrYi01csKdr+gMVN876KNwUZf7MwrKSJgdtLN8iXiQoU1LdrrULK k+Mw== X-Gm-Message-State: AOJu0Yz8HdDkq+Nkg/CweuT27579+lFqun4a9OVn2C7ksN037Sn12tPK M+DRZmzmP+qFe1rZWePRJCg= X-Google-Smtp-Source: AGHT+IE3JBxiJ/u6DBfOLTtC7eAoOuBG7Fwbx0RIWA5SJZDghdjSf/SyqP1NwzFv13HlRCjqSPCTgg== X-Received: by 2002:a05:6a00:22ce:b0:6bd:66ed:c4e with SMTP id f14-20020a056a0022ce00b006bd66ed0c4emr1854881pfj.9.1697414212806; Sun, 15 Oct 2023 16:56:52 -0700 (PDT) Received: from localhost (157-131-78-143.fiber.dynamic.sonic.net. [157.131.78.143]) by smtp.gmail.com with ESMTPSA id b37-20020a631b65000000b005b3cc663c8csm2401074pgm.21.2023.10.15.16.56.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 15 Oct 2023 16:56:52 -0700 (PDT) From: Jack Kamm To: Ihor Radchenko Cc: emacs-orgmode@gnu.org Subject: Re: [BUG] FAILED test-ob-python/session-multiline In-Reply-To: <87edjn4lfr.fsf@localhost> References: <873506j7ky.fsf@localhost> <87il91ajq7.fsf@gmail.com> <87ledxix4l.fsf@localhost> <87fs449xoy.fsf@gmail.com> <87edjn4lfr.fsf@localhost> Date: Sun, 15 Oct 2023 16:56:51 -0700 Message-ID: <87edhvign0.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=2607:f8b0:4864:20::434; envelope-from=jackkamm@gmail.com; helo=mail-pf1-x434.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -7.26 X-Spam-Score: -7.26 X-Migadu-Queue-Id: 40389412A7 X-Migadu-Scanner: mx2.migadu.com X-TUID: cB/D6rB/5LJV Ihor Radchenko writes: >> 1. In addition to printing `org-babel-python-eoe-indicator' after >> execution, we could also print out a "beginning of execution" >> indicator before execution, and then capture the output between the >> beginning and end indicators. This is how the async session >> execution works, and should avoid any possibility of capturing >> prompts. > > This idea looks interesting. Although I would not be so sure that it > will fix things - I have learned that comint has many edge cases we may > not easily anticipate. > > For example, see the discussion in > https://yhetil.org/emacs-devel/87y1tgqhmc.fsf@localhost/ I think this strategy could work better in ob-python than ob-shell because ob-python sends code to a temp file and executes the whole file at once, which should prevent prompts arising between commands. I will probably try this approach next, if the fix I just sent here doesn't work out: https://list.orgmode.org/87h6mrihfg.fsf@gmail.com/ >> Alternatively, we could add an argument to >> `python-shell-send-string-no-output' to avoid suppressing output, >> submit it upstream to python.el, and then backport to Org to >> support older emacs versions. > > If we can (eventually) remove some custom code from Org and move it to > Emacs, it will be the best for working towards RMS request > https://orgmode.org/list/E1kIPh1-0001Lu-Rg@fencepost.gnu.org I started down this path here: https://lists.gnu.org/archive/html/emacs-devel/2023-10/msg00004.html But I haven't followed up because I started to have some doubts. In particular, `python-shell-send-string-no-output' will terminate once it detects a prompt, so if some output looks like it ends in a prompt then it will terminate prematurely. Whereas in our current indicator-based approach, the user accidentally emitting `org-babel-python-eoe-indicator' is unlikely. Another approach I have considered is to redirect sys.stdout from within Python. In particular, set it to a custom class inheriting from IOBase during the block's execution, that both prints and saves the output. I think this approach could ultimately be more robust, and without needing to print an ugly indicator token, but it could be complicated to do it right.