From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id 0NnEGFnxwGVY3AAAqHPOHw:P1 (envelope-from ) for ; Mon, 05 Feb 2024 15:31:53 +0100 Received: from aspmx1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0.migadu.com with LMTPS id 0NnEGFnxwGVY3AAAqHPOHw (envelope-from ) for ; Mon, 05 Feb 2024 15:31:53 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=exAn7pvF; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1707143513; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=b4QoaxVuUUcAV4VJBvBDyJj72RM8tFJ5kPuZSoq9aUQ=; b=aY90s5mZHVAhdu18bZlC8jV0jkf4w609e1SiB9iYYPv2IhAsVuDOQ/oZnjGdgRVBJvLS8U U61jP7MyL03xQh3K6BwUddd6KhHIaYHopbEfoP5CVRJN5x96HAzwEs2vGXHgg/nj7GsjgY LpyIsUJdIZkesa0uqERlJEDrjNRwKVC4lFVpFyhp+Sw19OVnkwKH6U9SYdMBIjgYdPAI92 QLmVh5ncU2frQMs+6bWxlD1+tJQ/T9Lulxx9Qlgppx9Slr7LjjivlI3zHM9a8FxgARQ1EH HKxmf2iFBSC4bEUNLaixQXFAeC3psfLvrkKuyX5m+e3plPG9mLt7pGXt60FDSQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=exAn7pvF; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.net ARC-Seal: i=1; s=key1; d=yhetil.org; t=1707143513; a=rsa-sha256; cv=none; b=VvGRX40N3Ha9lcQbZpnbS2fz38CdY/znV125ehzqG8rx6bEFLuU1RtreF7Gioy/ljG8TvW zQcxomq1d4/bJzXqH4GKFGc0iODC25k2rPdSYLB1YMc2Psmxb8QIIIP+WWLL6/l18rFkRO SRsVunfr3/0b+1SO3HbWzTTaiqnWvWeTMjGtENyHwEpMM8gRmby30Lgzxx/7X3aGEkBjnd xK8UEBM6KZtDsP3YQ44GvG4srlpV1ociSCr7PafJtpjk3ipBsMHjgFk6vkjHHRqmaE0hDP xZrHhDyidf+fSpANICtZKBwtp91Ze64eOFRacV9moa6BWZFLe7H1KUUhJXeW9g== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 3606C68739 for ; Mon, 5 Feb 2024 15:31:53 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rWzv5-0003hj-L8; Mon, 05 Feb 2024 09:26:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rWzv3-0003ds-M0 for emacs-orgmode@gnu.org; Mon, 05 Feb 2024 09:25:57 -0500 Received: from mout02.posteo.de ([185.67.36.66]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rWzuz-0006AB-3A for emacs-orgmode@gnu.org; Mon, 05 Feb 2024 09:25:56 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 668A3240105 for ; Mon, 5 Feb 2024 15:25:50 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1707143150; bh=v80M/PDW30qcMDiH76DZ3lgQr1f7Ntln4CZhsZow0CA=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type: From; b=exAn7pvFJywO2VdfgUs4zRw2gYNvuviaPF5Sjbc9oJNQVlJWv/GtOGS0eaHW56HLp DKVlJsoPTitO4epSjlf3yc7/Z69Lnu6ywjNJn5kKP+LYJ3wpqVnhfZqpU9cpitH7ql +GjxAcH8JCY2oJMtrQ9TGJXrtD5WSrRk8B9TqR1wfSyFOBPizcitlhFMjG8+01Q0Xc jMQG50YqFdvZG3edCwd4Dod5cK8N0DNu3HtRJzRT2KFHN3wkrboJs+yuDUy5jw3QUu +yp03Jkp4tkFGpQOqU321A92846BB2XQJGww3cmUJ0QkkK52yBKXdwxhspYO5DQd8m i0XlUaWK63mZw== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4TT7vx3V7kz9rxF; Mon, 5 Feb 2024 15:25:49 +0100 (CET) From: Ihor Radchenko To: Jack Kamm Cc: Bruno Barbier , emacs-orgmode@gnu.org Subject: Re: [BUG] Unexpected result when evaluating python src block asynchronously [9.7-pre (release_9.6.17-1131-gc9ed03.dirty @ /home/yantar92/.emacs.d/straight/build/org/)] In-Reply-To: <874jen8zec.fsf@gmail.com> References: <87o7d0mm54.fsf@localhost> <65bbb108.050a0220.b60fd.6790@mx.google.com> <87jznm8hcu.fsf@gmail.com> <875xz42rp9.fsf@localhost> <874jen8zec.fsf@gmail.com> Date: Mon, 05 Feb 2024 14:29:19 +0000 Message-ID: <87o7cv9e80.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=185.67.36.66; envelope-from=yantar92@posteo.net; helo=mout02.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -9.89 X-Migadu-Queue-Id: 3606C68739 X-Spam-Score: -9.89 X-Migadu-Scanner: mx11.migadu.com X-TUID: t4Cj3Dt6Sjvk Jack Kamm writes: >> So, we have to insert some kind of indicator for async result. > > I meant that we could return something like "async:uuid-abcd-1234" or > "async:/path/to/tmpfile", so that `org-babel-comint-async-filter' could > still find the result. Of course, we could. But that would not solve all the possible problems. In particular, when header arguments tell Org babel to write result to a file, returning UUID will still not work. >> Of course, the existing scheme of coordination between >> `org-babel-insert-result' and `org-babel-comint-async-filter' is >> erroneous: >> >> 1. We have the problem with :results file value discussed here >> 2. We have a worse problem with :results file :file foo when the result >> may not be unique >> 3. We have :results append/prepend completely broken because >> `org-babel-comint-async-filter' simply calls >> `org-babel-insert-result' implicitly assuming that the existing >> indicator is replaced. >> >> The whole thing should be re-designed. > ... > A bit of a tangent, but if you are thinking about re-designing this, > then it may be worth considering ob-jupyter's implementation of async > sessions [1]. In particular, I believe it leaves a marker [2] where it > needs to insert the future result. I don't remember the details, > e.g. how it keeps track of which marker is for which result. But it > seems neat, and might work better for some cases such as > appending/prepending results. Markers are not as reliable as you think. If text around marker gets deleted, the marker will still exist potentially causing the async result to be inserted in the middle of unexpected place. Having an actual text indicator is more reliable - if user removes it before the async evaluation is completed, we will not write anything unexpected. Also, https://github.com/emacs-jupyter/jupyter/blob/master/ob-jupyter.el#L540 ;; KLUDGE: Remove the file result-parameter so that ;; `org-babel-insert-result' doesn't attempt to handle it while ;; async results are pending. Do the same in the synchronous ;; case, but not if link or graphics are also result-parameters, ;; only in Org >= 9.2, since those in combination with file mean ;; to interpret the result as a file link, a useful meaning that ;; doesn't interfere with Jupyter style result insertion. They also had to work around the same problem. > I agree that it would be good to redesign it, but am not sure where to > start. For example, 1. Change `org-babel-comint-async-register' to return UUID and to store PARAMS as passed by the backend (current approach with PARAMS being derived from src blocks prevents backends to transform src block PARAMS dynamically). 2. Change `org-babel-insert-result' to handle :async t specially, inserting something reliable, like #+async: in place of result without performing extra transformations. 3. Change `org-babel-insert-result' to accept an internal parameter that will make it replace #+async: keyword rather than perform normal result insertion. 4. Change `org-babel-comint-async-filter' to use the previously passed PARAMS, remove :async t from them, and arrange the call to `org-babel-insert-result' to replace the #+async: keyword. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at