From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id SHV8B+p9HGTCBwEASxT56A (envelope-from ) for ; Thu, 23 Mar 2023 17:27:22 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id 2OJRB+p9HGRuDwAAauVa8A (envelope-from ) for ; Thu, 23 Mar 2023 17:27:22 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id A7C046FF9 for ; Thu, 23 Mar 2023 17:27:21 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pfNm8-0003rY-DO; Thu, 23 Mar 2023 12:26:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pfNm6-0003r5-JL for emacs-orgmode@gnu.org; Thu, 23 Mar 2023 12:26:50 -0400 Received: from sender4-op-o15.zoho.com ([136.143.188.15]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pfNm4-0002DB-GS for emacs-orgmode@gnu.org; Thu, 23 Mar 2023 12:26:50 -0400 ARC-Seal: i=1; a=rsa-sha256; t=1679588782; cv=none; d=zohomail.com; s=zohoarc; b=ZKMOJitc8fUEmI2I3j2oxYl2ueGMaGUlebM7LhK+Ti9knKR7RvBYTT1VX8utV0WTQcDlTaJLAIVdWMw3LaM235wO4S5nnYfbXU2QlcpM2owxvl+bJCVxiVpHpcPTBgvbqz8cngTfzMvaE1ThKMWtBRYJ7ZacnvJ8vUb/BMsfw0E= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1679588782; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; bh=HmK+DSyvNFnQftLH+xXId53D00vtXnuvsGaD0TehAHw=; b=QajzpcqxdGm/+hyCXlSSKugI4dAmDnNlF7MTYZYvaAnyBZdK8d2lgMNy92rpIFIzAF0+J1mpVGssXTkwEiz9o+h8QI4SLS4RaLRQUsbF8RtzknDU/xnxCifAAyk9YorIEsuhCiFTfVPmxdI7oYW/BR6TOnOlG/NQ+SQqAqxWwaU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=excalamus.com; spf=pass smtp.mailfrom=matt@excalamus.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1679588782; s=zmail; d=excalamus.com; i=matt@excalamus.com; h=Date:Date:From:From:To:To:Cc:Cc:Message-ID:In-Reply-To:References:Subject:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To; bh=HmK+DSyvNFnQftLH+xXId53D00vtXnuvsGaD0TehAHw=; b=PlWnzXvql7vJOGkK64HsUgQ2r2guD9XniOkA250Ad09ubXEiTK5t9dAqZw39eH5d blM43BFK5wxaPAmUF0cMewCa+qI9yOkFik/pOSQZsT9afltjwzpsLdr+OfuHbGWsg6/ /CRNEvtJuppJyBWmUtSx31Vo5mfavL/fdgOdP5tY= Received: from mail.zoho.com by mx.zohomail.com with SMTP id 1679588780934266.7439643436983; Thu, 23 Mar 2023 09:26:20 -0700 (PDT) Date: Thu, 23 Mar 2023 12:26:20 -0400 From: Matt To: "numbchild" Cc: "Ihor Radchenko" , "emacs-orgmode" Message-ID: <1870f4aeb76.12af86c2f745535.7020028564160040455@excalamus.com> In-Reply-To: References: <186283d230a.129f5feb61660123.3289004102603503414@excalamus.com> <87v8kd8zzw.fsf@localhost> <1863472efe9.10fdd5ba4258906.5972264927968042941@excalamus.com> <87y1p7axpe.fsf@localhost> <1870cb30c41.ba75a358545366.6275787333945468963@excalamus.com> Subject: Remove "shell" as a supported Babel language within ob-shell.el (was Re: [SUGGESTION] ob-shell async result output should not contains shell prompt) MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Importance: Medium User-Agent: Zoho Mail X-Mailer: Zoho Mail Received-SPF: pass client-ip=136.143.188.15; envelope-from=matt@excalamus.com; helo=sender4-op-o15.zoho.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Seal: i=2; s=key1; d=yhetil.org; t=1679588841; a=rsa-sha256; cv=pass; b=WwZyDs4+k2CYeqFv0nm9FtdKEqRm29M/wEL9/RrjS/SBxULdXy1xPgVyGjk3GdRglnMfag t0ZwCyerF+Ld/1N3sT3ARYg8l5BDU/AJOqJV/yZXH2sbp2gJN383lCsxcb4UAZynxCsPE/ BgwTWySC2/Nxv0qbnBULTGPcXvr44yN+WVZX+7+kGtCH5D9EJzbBc7oYsyTMynpNflnUoY BtHhZ4M1ycoHkoszPiKyju6LB7wMW4Wj6dr8frQTBoVkGFoTjO/34SmioncJe+uNyIQNXM tItgq4p2Wde5mdZswjMUb3RSiYpE9OwLXgZsxT0X5Gs7TxM//VMYCXWCj64C0A== ARC-Authentication-Results: i=2; aspmx1.migadu.com; dkim=pass header.d=excalamus.com header.s=zmail header.b=PlWnzXvq; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; arc=pass ("zohomail.com:s=zohoarc:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1679588841; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=HmK+DSyvNFnQftLH+xXId53D00vtXnuvsGaD0TehAHw=; b=icOSHFgkQ27gvdBQBM59CwQcymMToanST0aNNFfPdMkwFwcWLdCRoxXvrvN8XyweF4L4Ga MRuu04XillBRLujnRJkiA5HUQhepGEgxU9c/fuaQqydqHB1aP4TpuYRlQJ0ytQ2/6a475l kdAYh8E1sBRadOsNgCd4brzUs3UNhmgvodBCP8+dh0WEtRevAzsWUVE7TvJkg+GYSfHd+w lDFYfD1rWu7bR2iVEJprd4JOwqtMLrvmz8Sjdgz49H+R3zrQU0/w1lkYzx6eHQ505WYrgj 7yD0fbcp5Pp/JBA5us+LrBrYvaKdIlSfL+nRGUv3ZO98EBSjxhUZG7iJh0m20w== Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=excalamus.com header.s=zmail header.b=PlWnzXvq; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; arc=pass ("zohomail.com:s=zohoarc:i=1") X-Migadu-Scanner: scn1.migadu.com X-Migadu-Spam-Score: -3.24 X-Spam-Score: -3.24 X-Migadu-Queue-Id: A7C046FF9 X-TUID: FyTRQ4NPY4Qs > Matt matt@excalamus.com> writes: > > > Is there a reason you're using "shell" instead of one of the shells li= sted in `org-babel-shell-names'? I'm still curious why you're using "shell". I want to know if it's somethi= ng you're using for a specific reason. There's no wrong answer! I ask because I have an agenda: as far as I can tell, "shell" as a Babel la= nguage is a historical accident. =20 #+begin_longwinded_explanation Originally, ob-shell.el was called ob-sh.el. The main function was called = `org-babel-execute:sh' and only /usr/bin/env sh was supported. Over time i= t became clear that to support other shells, the "sh" name shouldn't be use= d for the package or the main function. That is, 'sh' refers to a specific= binary and, if other binaries such as bash, dash, csh, etc. were to be sup= ported, it would be misleading for the Babel language to refer to a specifi= c shell, 'sh'. So, the terminology was changed to something more general, = "shell". The package was renamed to "ob-shell.el", the "namespace" updated= to "shell" (for example, `org-babel-execute:shell'), and the package load = call changed from (sh . t) to (shell . t). This officially happened with O= rg 8.2 (ORG-NEWS noted the change in commit 1a9a6666dcb6d34bcbdff45445c827e= d99dbf0b8, Tue Jan 21 09:52:31 2014). I think this gave people the (unders= tandable) impression that "shell" was a valid Babel language, in addition t= o those listed in `org-babel-shell-names'. =20 And this is where the accident happened: "shell" as a Babel language only *= *happens**to work. The Babel framework looks for a function prototype like= "org-babel-execute:". When ob-sh.el was changed to ob-shell.el,= the function `org-babel-execute:sh' became `org-babel-execute:shell'. A = call like follows is perfectly legal as far as the Babel framework is conce= rned: #+begin_src shell echo "hello, world" #+end_src When such a block is run, Babel looks for a function called `org-babel-exec= ute:shell'. Running the block prior to Org 8.2 should have failed because = no `org-babel-execute:shell' function existed. The name change happened to= source Fri Dec 13 09:52:05 2013 in commit 7a6c0e35415c4a173d101336029262f3= a09abb91. After the name change, the function existed and a block using "s= hell" would execute! =20 The "shell" language specifier, as far as I can tell, was never really inte= ntionally supported. Instead, it just happened to work. It happened to wo= rk because, as far back as the first org-babel-sh.el commit, the process bu= ffer is created using the `shell' function. I don't know the history of `s= hell', but presently the documentation says, Program used comes from variable =E2=80=98explicit-shell-file-name=E2=80=99= , or (if that is nil) from the ESHELL environment variable, or (if that is nil) from =E2=80=98shell-file-name=E2=80=99. That is, the `shell' command falls back to `shell-file-name'. I assume tha= t `shell' has always had that, or a similar, fallback. The `shell-file-nam= e' is a direct path to an executable. This means that when "shell" is use= d for the language, `shell-file-name' is called and **any** startup script,= such as .bash_profile or .bashrc, is called. The prompt could be set to *= *anything** and Emacs will never know, and can never know, what the prompt = is without the user explicitly informing Emacs. Aside from the code change which allowed "shell" to work, "official" suppor= t of "shell" comes from Org manual commit 9d072ad67517875069f913315d762c9bb= 1e9c3ec, Sun Dec 17 11:06:05 2017 (for example, https://git.savannah.gnu.or= g/cgit/emacs/org-mode.git/tree/doc/org-manual.org?id=3Df7aa8c19f5170dbf0953= 8686fb569f9b60acbd6c#n18410). This appears unconnected with the code chang= e. The addition to the manual happened 4 years after the code name change = and none of the commit messages around the time of code change suggest that= "shell" was intended to work as a language. In fact, I found this email f= rom Eric Schulte (creator of Babel and maintainer at the time of the code c= hange) which suggests that "shell" is in fact not supported or intented as = a language (https://lists.gnu.org/archive/html/emacs-orgmode/2013-12/msg002= 94.html): In response to the statement, "a coworker used "#+BEGIN_SRC shell" where he should have written "#+BEGIN_= SRC sh" Eric says, "[The suggested work around] would protect against this particular error" #+end_longwinded_explanation Regardless of whether "shell" was intended to work as a Babel language, the= fact remains that it does work and that it's been advertised in the manual= (at least) for 6 years. What are the pros and cons of "shell"? What benefit does "shell" provide? - The "shell" language allows an arbitrary executable to be run. This mean= s that shells other than those given in `org-babel-shell-names' can be run.= People using a non-supported shell could still benefit from ob-shell. What downsides does "shell" bring? - "shell" falls back to `shell-file-name' which can be an arbitrary executa= ble. Whenever I hear "runs an arbitrary executable", my ears perk up and I= start to sweat. There may be security considerations. =20 - If that executable is a shell, then the prompt gets set independently fro= m Emacs. For the prompt to be filtered from the output, users would need t= o provide Emacs with the correct regexp. A recent thread discussed creatin= g a header arg to address this: https://list.orgmode.org/87ttzgeg3w.fsf@loc= alhost/ - We would get bug reports about non-supported shells which kind of work, b= ut have issues because they're not supported - Maintence associated with supporting arbitrary (shell) executables As the current maintainer of ob-shell, I'm in favor of removing "shell" as = a Babel language. The cons appear to far outweigh the pros. However, I'm = aware others may have good use for it. It's been a part of Org for nearly = a decade. I'm sure it's part of people's workflow, especially since it's b= een in the manual for 6 years. Are there any pros, cons, use-cases, or con= siderations I've overlooked?