From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id mIWiF3YZ02U9gAEAqHPOHw:P1 (envelope-from ) for ; Mon, 19 Feb 2024 10:03:50 +0100 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0.migadu.com with LMTPS id mIWiF3YZ02U9gAEAqHPOHw (envelope-from ) for ; Mon, 19 Feb 2024 10:03:50 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=Evsx23ur; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1708333430; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=JkCiLMtloqgwQvm73aI7TL04sMXpSz6txf5ylLtTwy4=; b=RbpwsorvYK78wQHPrkmQrWUKvp38n5/TT/og4tYaQaNbLhvKCt4DuFJRvuEe5mGPtWGhan 6POZ2/OxV48l+ONQxDvszAiQbKbDF6ixFWBR76BaLN0bhWqwHRMuQYFqD18a+m9xZckNtN qI0vOdFozFntnbXgbVZgRYSbxqjYQH1DouL+63A38D1WyjIxffPW0nxYCpqzR+FOTPfPVi GrEo/xRwMWU5E1vsiAHwnOoXIoDBrQVH8K8HdWSzbB0zlWhqXLgRC4SRNzTFsDXu2Qlhon 6ClCKUuPSXr7zR3ZAAw+mXwfJpv6jO2FAlycQ959+PbD6xCv9lSD8+BUrKo8TA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=Evsx23ur; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.net ARC-Seal: i=1; s=key1; d=yhetil.org; t=1708333430; a=rsa-sha256; cv=none; b=IQkfh4oNQWd0qTUtxi//6bG59XhjXCQpOOBODIyC2b0fvaoCdL4x6pwMn+rbZHDyeejpAD VJWeT6LRkERnkl7noJ7aRi82d4g+UCEDqt9BqDT+RVJ0nbOhxnQ9PHJvLL72cvv9J3atHi KKASOYR5KgDqAngnE8V0AMvcVptUszKl9Cmw/C3WyXoieS5DM7HjKXYSrw/Y3VMIU/ky9V D6lQVILfYqwHk3b8oQhJS/D1xnfl4Yg0/GOb3grMGiE3TcvzIKtecAOv5vLw453C+6ReIW qnZ5Kces8Yas/mdNEwr26SnNCBm8CprBYVW6oEAKOtSbGMU4KGy9MHZ5s8zfRg== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 11DB723BED for ; Mon, 19 Feb 2024 10:03:49 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rbzYC-00061z-9U; Mon, 19 Feb 2024 04:03:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rbzY7-00061c-OQ for emacs-orgmode@gnu.org; Mon, 19 Feb 2024 04:02:58 -0500 Received: from mout01.posteo.de ([185.67.36.65]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rbzY3-000847-Gg for emacs-orgmode@gnu.org; Mon, 19 Feb 2024 04:02:54 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 2526C240029 for ; Mon, 19 Feb 2024 10:02:48 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1708333368; bh=COWbVoIlRTUZJJ52MKGdqZ4rXdpNn2Qgy83x2rV7/6Y=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type: From; b=Evsx23urdeI0YQee+dDpo6Ka+2ygng+//M90EXx/GbPOBv3t2xPr8nZaWOBs+wpTl hE+lPGaXUeFDOsuFTFMK6iXIjb2Xa5UgyDq/EDMjKJd4ijw7wmb0jnEjZuGgNgzSg+ 8KcKK+jr08xQ2++lESllQXb/Q7FeTlbr+ZrRjWBQOOrpSVipSYlOYGNX6ZmDFMPMDF PS6nL8GXqWvyJxSFW4NGxOS6DpwV47THqyicQC+LKd4UigMZakPzCrX8ZhfMJTW4UY JmEjwq5JHXDkW9DvFwHoBWXM13t+Iga/84ZcyOKF2FAQesAatWVaPcm257wP7PmOrD VGOmyuh4G2Zag== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4Tdc4l0qBKz6tvl; Mon, 19 Feb 2024 10:02:46 +0100 (CET) From: Ihor Radchenko To: Bruno Barbier Cc: Matt , Jack Kamm , emacs-orgmode Subject: Re: Asynchronous blocks for everything (was Re: [BUG] Unexpected result when evaluating python src block asynchronously [9.7-pre (release_9.6.17-1131-gc9ed03.dirty @ /home/yantar92/.emacs.d/straight/build/org/)]) In-Reply-To: <65cfa0d8.050a0220.cb569.ce34@mx.google.com> References: <87o7d0mm54.fsf@localhost> <65bbb108.050a0220.b60fd.6790@mx.google.com> <87jznm8hcu.fsf@gmail.com> <875xz42rp9.fsf@localhost> <874jen8zec.fsf@gmail.com> <87o7cv9e80.fsf@localhost> <65c2875f.050a0220.caf6d.8291@mx.google.com> <18dae5cab1d.bf1c7563863897.4896289306902277373@excalamus.com> <65cfa0d8.050a0220.cb569.ce34@mx.google.com> Date: Mon, 19 Feb 2024 09:06:31 +0000 Message-ID: <87frxohlgo.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=185.67.36.65; envelope-from=yantar92@posteo.net; helo=mout01.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Scanner: mx10.migadu.com X-Migadu-Spam-Score: -4.77 X-Spam-Score: -4.77 X-Migadu-Queue-Id: 11DB723BED X-TUID: s7a2AmRNKyZv Bruno Barbier writes: > Hi Matt, Jack, Ihor, > > Sorry for the late reply. Cleaning the code took me longer than > expected. Thanks for the code! It is a lot more that I expected. I have many questions ;) > The new API itself is more about how to wait for and display one block > result. So, it's not really aware of sessions. I think that it is important to think about sessions. For non-sessions, we may execute the code in parallel, disregarding already running execution. In contrast, for sessions, we need to maintain some queue and wait until previous execution finishes before running next (if multiple session blocks are executed in quick succession). It may also be necessary to provide an UI listing the queued and running execution. Users should be able to stop "stale" processes if they are defunc (consider infinite loop). >> Also interesting, I think it's worth exploring/testing this overlay idea >> out. Does that mean that output is asynchronously printing into the Org >> buffer? It sounds cool but I wonder if it might cause problems while >> trying to edit another part of the buffer. > > Currently, I've limited the progress feedback to fit on one line to > avoid anoying vertical display jumps. When the computation is > successful, the result is inserted as usual, and, that may be annoying; > when it updates a previous result of the same height, it's ok. Else, we > could fold it to stay on one line too (using the overlay), until the > user explicitly request to see it. Let's not worry about "jumps" yet. It is a minor issue that can be easily solved. What is more important is when users, for example, remove the whole subtree containing the place where the execution result is to be inserted. Or when users perform edits at or around that place where the result is to be inserted. Or consider subtree with pending result refiled elsewhere (different file or different place in the same file); or archived. Or consider user running an src block, quickly editing it, and then running again. What should we do then? Should we stop the first running process? > So, here we go. You'll find attach a set of patchs. It works for me with > Emacsc 30.50 and 9.7-pre (from today). I have several general questions: - what if append/prepend to result asynchronously? - what if we replace the result, call async evaluation two times, and they arrive in opposite order (first called is calculated after the second)? - on failure, Org babel sometimes uses ~org-babel-eval-error-notify~. How will it interact with asynchronous evaluation? What if we have multiple simultaneously arriving error notifications? Example: try running #+begin_src bash idontexist #+end_src > I didn't check yet that the code and the commits follow all the > guidelines. This is just a preliminary version for feedbacks. > Corrections/critiques are welcome, but don't check the details until I > check them myself. > So, I decided to rewrite the whole thing, taking code from the > synchronous case (following `org-babel-python-evaluate-session'). I > also created a package that contains all the functions that should be > reusable for any language. The patch [1] adds some generic functions to > help dealing with asynchronicity (process, comint, etc.). The patch [2] > shows a new possible way to execute python code blocks, both > synchronously and asynchronously, with or without sessions. You should > just need to open [3] and follow what's written there, and execute the > existing bash and python code blocks. Note that we already have a WIP an asynchronous API in the works. Check out `org-async-call' in https://code.tecosaur.net/tec/org-mode/src/branch/dev/lisp/org-macs.el#L377 If possible, we should reuse it. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at