From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id yD4tDVtWumJCmwAAbAwnHQ (envelope-from ) for ; Tue, 28 Jun 2022 03:16:11 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id eP4rDVtWumJPoQAA9RJhRA (envelope-from ) for ; Tue, 28 Jun 2022 03:16:11 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 84C0463D1 for ; Tue, 28 Jun 2022 03:16:09 +0200 (CEST) Received: from localhost ([::1]:43242 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o5zpo-0006Q9-Aj for larch@yhetil.org; Mon, 27 Jun 2022 21:16:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38604) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o5zoz-0006Pk-Cl for emacs-orgmode@gnu.org; Mon, 27 Jun 2022 21:15:17 -0400 Received: from mail-ej1-x632.google.com ([2a00:1450:4864:20::632]:44791) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1o5zow-0003PY-Nl for emacs-orgmode@gnu.org; Mon, 27 Jun 2022 21:15:17 -0400 Received: by mail-ej1-x632.google.com with SMTP id sb34so22536907ejc.11 for ; Mon, 27 Jun 2022 18:15:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=andrew.cmu.edu; s=google-2021; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=sEWH1giGqPXH9pS+3Vcq9J8+lqE1Hoaw0a6sT3pTNcU=; b=jIrBKWUJehjxU93Pr2GLyKfK1rApuOxqbfNbBayzaVtzUn9IdsrP4DBDqc6aShNeSS 0fflRip4t9eAxrVYVqH/eSIkvK4hUcqYOCSDfbf1ysgfPcjbRXczeR814gVeY/4Lo0Ds jrVKWLWM43jGtrvwVc4x0HHXxCF0YO1RnG2OodPIH0qcDH/KXrLR6ho3FYmrOlimLmaq cYuLswKaAU0YbOpqKjFD7+k5fn/w1IOb9aOfRXLDthb6BUALBfeJWR0qdAsV5rlPfkCa YA25ta9pU8DIusIG2D2UqbhM4/V3QmRZ71n1kLZr+Hw3VcdF79dfOb+EVf3O48q/bQsI jVoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=sEWH1giGqPXH9pS+3Vcq9J8+lqE1Hoaw0a6sT3pTNcU=; b=EcqJGJtmYuXEeCWs+T+4JAyELr8NnBuYJAzZuOQQ6nlfd+5z8kzfd3WXIUPwClT1no 3hOHKRo18diY5jJW1g/BaLHEflD9n8kJXE6arTwFQ+AW49CljLI4JO1/nPttBQ0+O61W 83UNUvlh6BqpktwrXWa1MRvAHACWGNI4lcWElyWzl5tpjXuQ9NmcN8QdMMJyL0IP/wQm 8AVu3T+p4HcstCGFnuHKdpUcufFFhmkto+pt7I669Vh5b4yaSA3l6Gvcivg83sA8Bvy7 TAz/ErUTLmS1OuYyMCzYxWfoqEOoyM8y/3QsH8jGs+wYkuKpTeSU+LO+zBrmQTSRMQxi HQUA== X-Gm-Message-State: AJIora9ETRAXOkIFwugCohm9zqhMy4sWxDv1JapNGThpl5FjrMEbCpvg is0FvRiEMvyClzEyS/lLNqMSbVzlfOByteZ1u3k= X-Google-Smtp-Source: AGRyM1saKPkw1ZgNq26mg7kYLORh2TUQg4eqdlzAtbMOjhvfIG/RmGkg7/t11WneyXxaHMb7ABD600lsg3mR+hWz1q4= X-Received: by 2002:a17:906:1501:b0:715:76d0:862a with SMTP id b1-20020a170906150100b0071576d0862amr14735166ejd.681.1656378911542; Mon, 27 Jun 2022 18:15:11 -0700 (PDT) MIME-Version: 1.0 References: <87fsjs3xws.fsf@localhost> <87y1xio2cn.fsf@localhost> <87k091k3md.fsf@gmail.com> In-Reply-To: <87k091k3md.fsf@gmail.com> From: John Kitchin Date: Mon, 27 Jun 2022 21:15:00 -0400 Message-ID: Subject: Re: We have asynchronous sessions, why have anything else? To: Tim Cross Cc: emacs-orgmode@gnu.org Content-Type: multipart/alternative; boundary="000000000000a2cf4505e277c4f6" Received-SPF: pass client-ip=2a00:1450:4864:20::632; envelope-from=johnrkitchin@gmail.com; helo=mail-ej1-x632.google.com X-Spam_score_int: -2 X-Spam_score: -0.3 X-Spam_bar: / X-Spam_report: (-0.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URI_DOTEDU=1.246 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1656378970; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=sEWH1giGqPXH9pS+3Vcq9J8+lqE1Hoaw0a6sT3pTNcU=; b=pM4JQOvOf6AnoWJZhleIHwVdBCmQWWrsvnLnyf6GIBzH8BMjbr16iJaneUQp0zh4axJgoR hA2B1s8zBoWJp8itRUIr41w6HfopLN4y7TVmLBborK0nv77UGGtV3d8VPsna/XiBGIoJvY K14qee76lCEqd4GE9xwKjPegP7mGDkHRUdWkhdt0KWli40dkOcioKcHEeGYJsHbNhYWctq 3dxni2SwIT4BrFchNtWlN3pUfIutfsXFUpxwspkOck433R8FPlRzrZy11pnFk+Re2AAUdG tvY12uG0VeMg0kqU2M0w6q80hQRfljKEtkjwGIzYlvA3vMnZFbJioqb/odE9vw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1656378970; a=rsa-sha256; cv=none; b=ZRV5IiIiWtEomP1Qf3EvAliViEieLPa/eb9eDXJDc4ebVpXj9yn99cFkGdAnLueFB1x8KN a+OtmyigC6kiY4BcXbn+hgQ6O23vmb40zbrdMfPqz/AxOF04RKIhQiFbR1bTVkQ0Ds0YkT tImbUNo+4+2QpBLpQYg6y2rmY6IKZUuZ27bgoq7RoLPj5INnu0Kjo08+HIuQtz6inZqdex z+PuDSDePrbJkNaXt3Fv4sWJ1hXP+aRy2Cc7DNs9XNBcEYrD2moxGCqREsxocbte7BOvcv L+0+uCmQ9DYNFFf7Z908YEIxP2fPII4SaSgO/+KhX41viqWZkag+LkYqeTdPFQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=andrew.cmu.edu header.s=google-2021 header.b=jIrBKWUJ; dmarc=pass (policy=none) header.from=andrew.cmu.edu; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.25 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=andrew.cmu.edu header.s=google-2021 header.b=jIrBKWUJ; dmarc=pass (policy=none) header.from=andrew.cmu.edu; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 84C0463D1 X-Spam-Score: -4.25 X-Migadu-Scanner: scn0.migadu.com X-TUID: 3KQ46S/X5ueN --000000000000a2cf4505e277c4f6 Content-Type: text/plain; charset="UTF-8" The Jupyter project is one approach to this. It currently has dozens of kernels for different languages, and new kernels can certainly be made. The emacs-jupyter package provides one implementation of an interface. It is complex, and relies on a compiled module for the zeromq message passing library. I am not advocating for this as the solution for org-babel, but it is an interesting case study. You can even connect to remote kernels. I use it a lot. On Mon, Jun 27, 2022 at 8:56 PM Tim Cross wrote: > > Tom Gillespie writes: > > >> I am not even sure if all the babel backends support try-except. > >> Think about ob-gnuplot or, say, ob-latex. > > > > Indeed many do not. Defining some standard "features" > > for org babel language implementations is something that > > is definitely of interest so that we can provide clear interfaces > > for things like stdio, error handling, return values, async, > > file output, remote execution, sessions, return value caching, > > module discovery/tangling, execution from file vs stdin, execution > > without a file system path, runtime environment specification, > > and much more. However, at the moment there is only a preliminary > > survey of a subset of these that was put together by Ian Martins. > > > > https://orgmode.org/worg/org-contrib/babel/languages/lang-compat.html > > > >> the two could be unified if we expand the functionality of the async > filter > > > > While this might be possible, I would definitely hold off on this because > > the changes in semantics absolutely will break many users' blocks. We > > barely knew what the impact of changing the default return value for > shell > > blocks would be. > > > > I absolutely look forward to the day when this can be done safely and > > with confidence, but I think we need a much stronger handle on babel > > interfaces in general before such a change could even be considered. > > > > At the moment each ob lang impl pretty much has to be considered > > to be completely unique, even if the text looks like bash for e.g. > > shell, comint, and screen. Users are known to rely on undocumented > > quirks of the ob lang impls that can differ wildly in their semantics. > > > > Well said Tom. > > As you point out, there are numerous deficiencies with the current > implementation, despite the fact it all sort of works. To get the sort > of improvements and consistency users want, I suspect this needs more > than just small tweaks around the edges. > > To some extent, I see the current babel implementation as similar to a > prototype. It has worked well and we have learnt a lot about what people > want to use it for and the type of functionality they are wanting and > what some of the core challenges are. Now comes the next step, which is > definitely non-trivial. We need to take all that knowledge and > consolidate it into a single model from which we can define the > interfaces and associated APIs. A big job which will take considerable > time. > > -- John ----------------------------------- Professor John Kitchin (he/him/his) Doherty Hall A207F Department of Chemical Engineering Carnegie Mellon University Pittsburgh, PA 15213 412-268-7803 @johnkitchin https://kitchingroup.cheme.cmu.edu https://pointbreezepubs.gumroad.com/ pycse bookstore --000000000000a2cf4505e277c4f6 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
The Jupyter project is one approach to this. It currently= has dozens of kernels for different languages, and new kernels can certain= ly be made. The emacs-jupyter package provides one implementation of an int= erface. It is complex, and relies on a compiled module for the zeromq messa= ge passing library. =C2=A0

I am not advocating for this as the solution for org-babel, but it is an= interesting case study. You can even connect to remote kernels.=C2=A0

I use it a lot.=C2=A0
<= div>
On= Mon, Jun 27, 2022 at 8:56 PM Tim Cross <theophilusx@gmail.com> wrote:
=
Tom Gillespie <tgb= ugs@gmail.com> writes:

>> I am not even sure if all the babel backends support try-except. >> Think about ob-gnuplot or, say, ob-latex.
>
> Indeed many do not. Defining some standard "features"
> for org babel language implementations is something that
> is definitely of interest so that we can provide clear interfaces
> for things like stdio, error handling, return values, async,
> file output, remote execution, sessions, return value caching,
> module discovery/tangling, execution from file vs stdin, execution
> without a file system path, runtime environment specification,
> and much more. However, at the moment there is only a preliminary
> survey of a subset of these that was put together by Ian Martins.
>
> https://orgmode.org/worg/o= rg-contrib/babel/languages/lang-compat.html
>
>> the two could be unified if we expand the functionality of the asy= nc filter
>
> While this might be possible, I would definitely hold off on this beca= use
> the changes in semantics absolutely will break many users' blocks.= We
> barely knew what the impact of changing the default return value for s= hell
> blocks would be.
>
> I absolutely look forward to the day when this can be done safely and<= br> > with confidence, but I think we need a much stronger handle on babel > interfaces in general before such a change could even be considered. >
> At the moment each ob lang impl pretty much has to be considered
> to be completely unique, even if the text looks like bash for e.g.
> shell, comint, and screen. Users are known to rely on undocumented
> quirks of the ob lang impls that can differ wildly in their semantics.=
>

Well said Tom.

As you point out, there are numerous deficiencies with the current
implementation, despite the fact it all sort of works. To get the sort
of improvements and consistency users want, I suspect this needs more
than just small tweaks around the edges.

To some extent, I see the current babel implementation as similar to a
prototype. It has worked well and we have learnt a lot about what people want to use it for and the type of functionality they are wanting and
what some of the core challenges are. Now comes the next step, which is
definitely non-trivial. We need to take all that knowledge and
consolidate it into a single model from which we can define the
interfaces and associated APIs. A big job which will take considerable
time.

--
J= ohn

-----------------------------------
Professor John Kitchin (h= e/him/his)
Doherty Hall A207F
Department of Chemical Engineering
C= arnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
https://kitchingroup.cheme.cmu.edu
https://pointbreezepu= bs.gumroad.com/=C2=A0pycse bookstore

--000000000000a2cf4505e277c4f6--