From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:403:4789::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id IPO3EWmO62RkUAAAauVa8A:P1 (envelope-from ) for ; Sun, 27 Aug 2023 19:56:57 +0200 Received: from aspmx1.migadu.com ([2001:41d0:403:4789::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id IPO3EWmO62RkUAAAauVa8A (envelope-from ) for ; Sun, 27 Aug 2023 19:56:57 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id F1FEF43BD4 for ; Sun, 27 Aug 2023 19:56:56 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=mORWdCpv; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1693159017; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=u+zvSavtLrx7xj4ecbAsxdPvzp1nAMR+RWwdy/KlHeU=; b=SEklZw/h34wa+Hv0pgOcOj4bNPeH1kPUJvOAlXzVgZQEQ//tji0o450s0EMiSiQ8IvUbgg Dlo2pAB3CslZKWgbH2tzI7IB5ALfKbN74cCRRCgKq6ovblUay3HbHL1VgAxKO/HQZDhJ1n B9mIFDWRjSzCizLrYqFHu+QFd/YOSMt8D+Edqx2histyxPeD04/L9WjwKgskqBGy/qiLus OpSqoUhh5IoxxiXkrr8NnIcz7FbYzfZy11Xaf4stEUmLqagY2Q9mltk966AvTx4We3A/Bd XX53b/ia9ldeW4qwwxkjjrg7jOSQ/xe7o2b3xoafvhPGxsXqdLnOLK02FXoSTQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1693159017; a=rsa-sha256; cv=none; b=XDnMh0Jnum9TCluaasU923tgr0XFYGhYzVk9KWKa1Q2nRSwUe+rdA9R5NT2cVVLYCV4lHr Hq5rFHat5+UXO9FgLlJktoX0R7O7B1tnQnQflbzXR6J240riQwn6auf61nO++Lo77yHkZu 7XH5qpuUMAsDaqVuYpFeaWDxKz3/kJ2a1vCbYpAwhiL+dYLs43JrP5GLe7yUN4rszTrtuI rB5JU4kpZ1BgMDMu49OBUtfYfDF91eNctm3WlLJT2wjZzhn8VRxVS9FGTt0Fe90SkGu9Ty Fvc4xbtWGsslsDG2WQtTvwcAh4BuNUG5dXIZe2bARxMYSmeZ50W3xs969k/10Q== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=mORWdCpv; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gmail.com Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qaJzY-0006Wg-BX; Sun, 27 Aug 2023 13:56:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qaJzX-0006W6-AF for emacs-orgmode@gnu.org; Sun, 27 Aug 2023 13:56:03 -0400 Received: from mail-oi1-x235.google.com ([2607:f8b0:4864:20::235]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qaJzU-0007fC-Ou for emacs-orgmode@gnu.org; Sun, 27 Aug 2023 13:56:02 -0400 Received: by mail-oi1-x235.google.com with SMTP id 5614622812f47-3a8036d805eso1983142b6e.3 for ; Sun, 27 Aug 2023 10:56:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693158959; x=1693763759; h=mime-version:message-id:date:references:in-reply-to:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=u+zvSavtLrx7xj4ecbAsxdPvzp1nAMR+RWwdy/KlHeU=; b=mORWdCpvquk0boht9FgCvrd6IVcoUiUbt9X6LdoLZ3uQkaDYGXBr24AOo7DdGh5w0B 2yEOq+Vs7fP00fhYZY/BGPVyrwNssrIEsI6EoF9xXkcttP/Uxow9rPNjj4xsaEZUHx/3 2VAzMmxsidB73DznTHmQ222czo6p3n/1tgHo3vml0HFzFkc10l6kCVtjzId9rGwkLGRJ Tz/H2g0HpGtTNj00zKKqbgs84zW/jnH2fm9YMvetOGxOzQjtr2UJ7ZJ8FqqOcBIKpLH0 WY/Zu7z88xMYZ054HBfPewChXwxgR4Ow2B+eypzyFR3NeXvPnTmUs0vBpPEPmNTggXbX Zavw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693158959; x=1693763759; h=mime-version:message-id:date:references:in-reply-to:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=u+zvSavtLrx7xj4ecbAsxdPvzp1nAMR+RWwdy/KlHeU=; b=lMYYuIx6r5pDB5ONO79TvRgFeMjDf4R2OiFoy/wYDDtDmsAonWwkHvWHQQOwtIy/Nn EeSdjdZiCDNZZ+rBMC0nKUSEdzjWp1gHHnizA9pXdGUEFaYLAn04yHPp6qeoqqnbIIQC hJg6YbhpTsDRCF3lHF3Ho4UcBuCctWk7799wwZLLmLu7y/sq4UsHFSrWgpNDFCR0vLlR vJyU8x8kL15YL7vDmSqlMMkb8B78xeq4cTE580BAl17YlQFMh6hZJp/PRvwNV1kc6gDt grf0nHulGZg2Bt+TbBqpWbx55IQpS0/EP5baLq7vEwAwwSbSNKB+Eda8UIg1hSn4VLy0 CLmQ== X-Gm-Message-State: AOJu0Yw7dje4vZFfOuNK8iYXXblQvtzInivbBP8ZNh0todWV+AJiq901 /+q6VviGlf/2weKFWLA2w5p4bmKtVI4= X-Google-Smtp-Source: AGHT+IFqCai1UdtoXaG6dFYY8ce9enwOT4hPjI8KuUyQKYStBtOQTS64nBdGwClsu79JXeT7KODvPw== X-Received: by 2002:a05:6808:2223:b0:3a7:2d4c:5d4e with SMTP id bd35-20020a056808222300b003a72d4c5d4emr12667197oib.7.1693158958790; Sun, 27 Aug 2023 10:55:58 -0700 (PDT) Received: from localhost (157-131-78-143.fiber.dynamic.sonic.net. [157.131.78.143]) by smtp.gmail.com with ESMTPSA id v24-20020a62a518000000b00684ca1b45b9sm5230400pfm.149.2023.08.27.10.55.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 27 Aug 2023 10:55:58 -0700 (PDT) From: Jack Kamm To: Ihor Radchenko Cc: emacs-orgmode@gnu.org Subject: Re: [BUG] FAILED test-ob-python/session-multiline In-Reply-To: <87ledxix4l.fsf@localhost> References: <873506j7ky.fsf@localhost> <87il91ajq7.fsf@gmail.com> <87ledxix4l.fsf@localhost> Date: Sun, 27 Aug 2023 10:55:57 -0700 Message-ID: <87fs449xoy.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=2607:f8b0:4864:20::235; envelope-from=jackkamm@gmail.com; helo=mail-oi1-x235.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -6.59 X-Spam-Score: -6.59 X-Migadu-Queue-Id: F1FEF43BD4 X-Migadu-Scanner: mx1.migadu.com X-TUID: yboeh0kv9kWr Ihor Radchenko writes: > Jack Kamm writes: > >>> FAILED 376/1256 test-ob-python/session-multiline (0.011955 sec) at ../lisp/test-ob-python.el:105 >> >> Hmmm. Do you have an idea of how long this has been happening, and how >> frequently it breaks? > > For months. > >> My first suspicion is the large ob-python commit I pushed on Tuesday: >> >> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=579e8c572345c42ad581d3ddf0f484567d55a787 > > So, should not be the recent commit. This one might take some time to fix, since it's hard to reproduce and I'm not sure the cause of it. But here are 3 different solutions I am considering now: 1. In addition to printing `org-babel-python-eoe-indicator' after execution, we could also print out a "beginning of execution" indicator before execution, and then capture the output between the beginning and end indicators. This is how the async session execution works, and should avoid any possibility of capturing prompts. 2. Instead of relying on our own custom `org-babel-python-send-string', we could try switching to python.el's `python-shell-send-string-no-output', which is probably more robust. This would also allow removing the ugly `org-babel-python-eoe-indicator' we currently print. Downside is that the output would not be echoed into the session anymore. To fix that, we could manually insert the captured output into the comint session buffer after execution. Alternatively, we could add an argument to `python-shell-send-string-no-output' to avoid suppressing output, submit it upstream to python.el, and then backport to Org to support older emacs versions. 3. Revisit a series of commits I made in 2020, which was supposed to make session evaluation more robust, and was inspired by `python-shell-send-string-no-output': https://git.sr.ht/~bzg/org-mode/commit/4df12ea39 However, I had to partially revert that work, due to compatibility issue with emacs 26.3: https://list.orgmode.org/871rjcan53.fsf@kyleam.com/ I think we no longer support emacs 26.3, so I could potentially revisit this now -- but it's been a few years and will take some time to refresh my memory about this. Of these options, Option 1 is the easiest, and the most certain to solve this bug. Options 2 and 3 are more difficult and riskier, but would have other benefits if they work: we can remove the ugly `org-babel-python-eoe-indicator' that is currently printed to the session, and/or reduce long-term maintenance burden by relying on python.el's implementation for capturing output.