From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id CfnQAs2R8V9QXwAA0tVLHw (envelope-from ) for ; Sun, 03 Jan 2021 09:43:41 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id GCnVOcyR8V/8cwAA1q6Kng (envelope-from ) for ; Sun, 03 Jan 2021 09:43:40 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 4DE3C9403D5 for ; Sun, 3 Jan 2021 09:43:40 +0000 (UTC) Received: from localhost ([::1]:54214 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kvzvG-0006Wd-1m for larch@yhetil.org; Sun, 03 Jan 2021 04:43:38 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:36246) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kvzud-0006WK-Ox for emacs-orgmode@gnu.org; Sun, 03 Jan 2021 04:42:59 -0500 Received: from mail-pf1-x42f.google.com ([2607:f8b0:4864:20::42f]:41454) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kvzub-0005Lc-MB for emacs-orgmode@gnu.org; Sun, 03 Jan 2021 04:42:59 -0500 Received: by mail-pf1-x42f.google.com with SMTP id t8so14538140pfg.8 for ; Sun, 03 Jan 2021 01:42:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=references:user-agent:from:to:cc:subject:date:in-reply-to :message-id:mime-version; bh=tE/Roe97JcwHvDSt2PjYywZEGTxE5InBhnQFr1Ht5qY=; b=REiLIFyywNQREM7ymhO2QP5neXcjsqJvtVdhJrvEFjDA3NvWh4wyvNcZe/D9o+uxpL om/udxy01P/P/0SGYnl+KXiaso7WEhMFNi8uZuoAl9doIixjv+QBj+dAGjT3V2M7UrZv B7ekFtcJzxcfSUcNOY7Wite99SojExR8japj90kGp1EzXdPlZnfHti99EoEwKVZxwt5B L+G4JbWxeNxyKJAXAzGqCxNbvAmGFSdiOx8wKFBoDWU0InKl7c0ojjQRYeB3wQkn0zPZ lpEqKfmpJWDSfnx+6FZ8uPA1Z/0a0VpBUEr1ukd4gBTnx3PL+hPm9hhkulA7ygOFtAX7 RBtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:references:user-agent:from:to:cc:subject:date :in-reply-to:message-id:mime-version; bh=tE/Roe97JcwHvDSt2PjYywZEGTxE5InBhnQFr1Ht5qY=; b=nEOHm+q7Wzv0Rp2fP/JRt1UGrzMUqYpVVJ1syk8h5dYWDly6T+vvRHOuKwAb0FkwTx iaj5f8TFMwAmdrV0Q1sSS6Kt+uo7T1Hg53uxeL+L+1xmSaYBlQ+ZBj+yi2Hyr5ABSqtH vasIZWlTJu8hchcdwBwj+MKHQL0O4hyvl4QnvGp+T4FwjTS/m0faWj42reNK8zgj+koz bCIQnK/zCOXYG95jL2VYr+twiV5vNwui4iJ793zinH0ZPcBImew1s9ecrZJeUVYwJV3E wWUIwaGqtjjLwCEzjv6iOmBhLfPsgUmONjOLnl/3PMo/52F1oBpuQDMkmoylhm1rNKvc Gr3g== X-Gm-Message-State: AOAM532424n9Xf8aY5A8apKRqH2QCpZTwmG0fEKNyc1Ci0fGOMgeB7Nw 5j1iSo+1QVw50kZzOT3ENbQ= X-Google-Smtp-Source: ABdhPJwRAcuWYN8nrDgWmTNluH0DJ1Z+Qg/IVEvTZX78hjNZo5n4zEdUN0OJPSDEtEvcRm29iuAlYA== X-Received: by 2002:a63:5407:: with SMTP id i7mr42411015pgb.418.1609666975801; Sun, 03 Jan 2021 01:42:55 -0800 (PST) Received: from localhost (180-150-91-8.b4965b.per.nbn.aussiebb.net. [180.150.91.8]) by smtp.gmail.com with ESMTPSA id c23sm43057815pgc.72.2021.01.03.01.42.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 03 Jan 2021 01:42:55 -0800 (PST) References: <87h7qi2l2m.fsf@gmail.com> User-agent: mu4e 1.4.13; emacs 27.1 From: TEC To: Jack Kamm Subject: Re: [PATCH] Async session eval (2nd attempt) Date: Sun, 03 Jan 2021 16:51:15 +0800 In-reply-to: <87h7qi2l2m.fsf@gmail.com> Message-ID: <87lfdatkys.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=2607:f8b0:4864:20::42f; envelope-from=tecosaur@gmail.com; helo=mail-pf1-x42f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: emacs-orgmode@gnu.org Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -3.03 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=REiLIFyy; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Queue-Id: 4DE3C9403D5 X-Spam-Score: -3.03 X-Migadu-Scanner: scn1.migadu.com X-TUID: 1kdky0tbwjv5 Hi Jack, I love the look of this! Thanks for submitting a patch. Sorry it's taken so long for someone to take a look at it, I think a lot of the 'main' Org people have been pretty busy over the last few months. I just tried to give this a shot. First up, I had to remove the ORG-NEWS part of the patch to be able to provide it. It would be nice if you could update the patch so this applies cleanly. #+begin_example error: patch failed: etc/ORG-NEWS:88 error: etc/ORG-NEWS: patch does not apply #+end_example To test this, after applying your patch (with ORG-NEWS removed), I started emacs -Q, loaded Org, and opened a new file. I was initially unable to get this to seem to work, until I changed the :results type to "output". See a excerpt from my test file below: ----- excerpt start ----- #+begin_src python :async :session blah :results output from time import sleep a=2 sleep(2) print("Hi") #+end_src #+RESULTS: : Hi #+begin_src python :async :session blah return(a) #+end_src #+RESULTS: : /tmp/babel-62cQRX/python-EfJ4o4 #+begin_src python :async :session blah :results output print(a) #+end_src #+RESULTS: : 2 ----- excerpt end ----- I'm surprised this didn't work with the non-output block though. Other than this, I'm rather happy to see that when I tried to execute two long running blocks at once, the second one was not executed until the first completed :) Finally, I see that this requires :session to be set in order to work. Might it be possible to have this work for non-session blocks too? It seems odd that what I'd imagine is the harder case (session blocks) is supported, but one-shot (non-session) blocks aren't. Thanks again for your work, and I look forward to seeing what else you have in the future! -- Timothy p.s. After this is merged, it would be great to see support for other languages grow :) Jack Kamm writes: > This patch adds asynchronous evaluation for session blocks in > Python. It also adds functionality to implement async session eval for > other languages using ob-comint.el. > > To test the attached patch, add ":async" to a Python session block > with a long computation (or "time.sleep") in it. Upon evaluation, your > Emacs won't freeze to wait for the result -- instead, a placeholder > will be inserted, and replaced with the true result when it's ready. > > I'll note how this is different from some related projects. ob-async > implements asynchronous evaluation for Babel, but it doesn't work with > sessions. emacs-jupyter, ein, and ob-ipython all implement > asynchronous session evaluation, but only for Jupyter kernels. Jupyter > is great for some cases, but sometimes I prefer to use the built-in > org-babel languages without jupyter. > > The new functionality is mainly implemented in > `org-babel-comint-async-filter', which I've defined in ob-comint.el, > and added as a hook to `comint-output-filter-functions'. Whenever new > output is added to the comint buffer, the filter scans for an > indicator token (this is inspired by > `org-babel-comint-with-output'). Upon encountering the token, the > filter uses a regular expression to extract a UUID or temp-file > associated with the result, then searches for the appropriate location > to add the result to. > > This is my 2nd attempt at this patch [0]. I have also ported it to an > external package [1], but would like to have this functionality in Org > proper, to permit better code reuse between async and sync > implementations. The external package also includes an R > implementation that I regularly use, as well as a Ruby implementation, > but I've left these out to keep this initial patch smaller, and also I > need to confirm copyright assignment on the Ruby implementation which > was externally contributed. > > [0] https://orgmode.org/list/87muj04xim.fsf@jaheira.i-did-not-set--mail-host-address--so-tickle-me/ > [1] https://github.com/jackkamm/ob-session-async