From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0.migadu.com ([2001:41d0:303:5f26::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id IGYXOmBQoGUtCwEAkFu2QA (envelope-from ) for ; Thu, 11 Jan 2024 21:32:33 +0100 Received: from aspmx1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0.migadu.com with LMTPS id KIX/NGBQoGXHXAEAqHPOHw (envelope-from ) for ; Thu, 11 Jan 2024 21:32:32 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=excalamus.com header.s=zmail header.b=LeQlMYzh; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=none; arc=pass ("zohomail.com:s=zohoarc:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1705005152; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=rrNg31L7W8OX8wZKfgonDQULbFzOLz8ZZDFEGY9tB30=; b=mbQKYODWweN60wtxnRwtzhupKc5h7MGcyEz3VLStDIAHXBVWpT2LBzKho42FOFezj2LIB4 yiwpnzoINXYHFHfBeaHceV7c3qPnhxvkSckJ9LYqOLqreydHw1lr+P3C/r1VJxDFEDyPS0 9IRVr/XE+XYVmh1Rx8LQA+UuGkvM1PhhVTOaZm4KSg/Lby9n/LAGdiWQLn0elgful5cy7P nwBCHhaY4gsCHLXlodENVE4xCkN4SccPu6PtX2wL4WfcZU56n2hvByfq61140jtv/Un1d5 Yoz6lp8MzSZTDWnn+8lkxW2uyf2H1jlMNFsuYzcjy5ly1xG3X8cTu+USPhhrCw== ARC-Authentication-Results: i=2; aspmx1.migadu.com; dkim=pass header.d=excalamus.com header.s=zmail header.b=LeQlMYzh; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=none; arc=pass ("zohomail.com:s=zohoarc:i=1") ARC-Seal: i=2; s=key1; d=yhetil.org; t=1705005152; a=rsa-sha256; cv=pass; b=R9imk29cfzzPvZSvJ0Whm+9EwpwI2LZVzBfKqbsYm7HWDeYbsGBFrUZ6aV09h/CksKmCMb Rjrlh5z7FtipJwID8UL7zoRatsEG1mro9jHaloFYajJxLnnKhaXtT/eUm1oakQkjRpTkkN sdZxZuApIVs8WyoDJhWDGJQKfLZNbo2L86B4OP3bChU3F2Nx2eTkMU/7r5mIR7CexJnB2n 32bC5dR7VqBVJHRGcTwc40LLAQzY8bCFu7XHS1ez9D8qpQVEpM1pGaJYVL+zjsVH3GiAgO GToCQA/nv7iCBrghb0RVh0mHG7ep38itf4MhF1BGoXqMoIeqojailhZAaB7dxw== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 53BE2389EE for ; Thu, 11 Jan 2024 21:32:32 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rO1i6-000084-LT; Thu, 11 Jan 2024 15:31:30 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rO1i2-00007D-6B for emacs-orgmode@gnu.org; Thu, 11 Jan 2024 15:31:27 -0500 Received: from sender4-op-o15.zoho.com ([136.143.188.15]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rO1hm-00015C-Ib for emacs-orgmode@gnu.org; Thu, 11 Jan 2024 15:31:25 -0500 ARC-Seal: i=1; a=rsa-sha256; t=1705005061; cv=none; d=zohomail.com; s=zohoarc; b=P9GzdRsfZQUWlXvIvKSK7AdbQGlocOtj1iMaTXlTi/4EY8MMwwQq1flzalltCPLgG7lWPzDFjpJVrVyg2ocQcPxgs2AOTMtYD5LNYmf4T24NBSvp5+be5eKyMp08OCgwNSZ1W7JD5sYbrwhMoM3xvLVUaSNcEn4ocRBcyMdM53U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1705005061; h=Content-Type:Content-Transfer-Encoding:Date:Date:From:From:MIME-Version:Message-ID:Subject:Subject:To:To:Message-Id:Reply-To:Cc; bh=rrNg31L7W8OX8wZKfgonDQULbFzOLz8ZZDFEGY9tB30=; b=IfliyFLZSYelTBaESyHPVpU+KKnRiD1P7cBNpBguyjiK6GZJgFERf6JebCakNP8HVGnx8bTZk/kqRhUwCc3qXRmejjOEQkyqZHaBXmaVu9G248FNPTyOa7pXyDO6NwNaGljKmussMbJu0RpeuFJX4bdM/QcZ7FjEzVT5zlLS8jQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=excalamus.com; spf=pass smtp.mailfrom=matt@excalamus.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1705005061; s=zmail; d=excalamus.com; i=matt@excalamus.com; h=Date:Date:From:From:To:To:Message-ID:In-Reply-To:Subject:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To:Cc; bh=rrNg31L7W8OX8wZKfgonDQULbFzOLz8ZZDFEGY9tB30=; b=LeQlMYzhItf9KN7tsg1XMxPyxwkpIS8JD9YHhhAEKsRcxkINFToa8RFuiSqqCZml 4NYTBP8HeCzmWaz49lo/V1mqmNd4B2mcaPdem3CNp1eE6rfphbAO568b3Ux7AYUo7Xz S6jpt0oYrFhyM7wz2rqucLywFCNg7GDO1aJdpTU4= Received: from mail.zoho.com by mx.zohomail.com with SMTP id 1705005059481812.7784140548301; Thu, 11 Jan 2024 12:30:59 -0800 (PST) Date: Thu, 11 Jan 2024 21:30:59 +0100 From: Matt To: "emacs-orgmode" Message-ID: <18cfa388d12.f0069ffe919377.6846036599039377431@excalamus.com> In-Reply-To: Subject: ob-shell: proposal to remove "posh" MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Importance: Medium User-Agent: Zoho Mail X-Mailer: Zoho Mail Received-SPF: pass client-ip=136.143.188.15; envelope-from=matt@excalamus.com; helo=sender4-op-o15.zoho.com X-Spam_score_int: 0 X-Spam_score: -0.1 X-Spam_bar: / X-Spam_report: (-0.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URI_TRY_3LD=1.997 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -10.74 X-Spam-Score: -10.74 X-Migadu-Queue-Id: 53BE2389EE X-Migadu-Scanner: mx11.migadu.com X-TUID: 56hBN6dE4dfC Hi, I would like people's thoughts on removing the "posh" language header. ob-shell.el supports a "posh" shell. What is "posh?" * Posh is not PowerShell "posh" was added to =3Dob-shell=3D in fb09863f with no commit message on De= cember 13, 2013 (Friday the 13th!). https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=3Dfb09863fb= b35bf15bcf78262b6e31b8b8b8617e7 A mail message the same day says, > Eric Schulte writes: > >> How about the following resolution? We rename ob-sh.el to ob-shell.el= . > >> New "shell" code blocks could use the value of the > >> `org-babel-sh-command' environment variable. Then sh, bash, zsh, csh, > >> ash, dash (am I missing any other common ones) use the specific shell > >> specified. > > I've also seen ksh, mksh, posh (the latter specifically for POSIX > compatibility checks). https://list.orgmode.org/878uvychr1.fsf@pinto.chemeng.ucl.ac.uk/T/#mc20039f= 988519d409294dc604b5e0dc0f439307b There was discussion about different shells, Eric asked for others, "posh" = was mentioned as "specially for POSIX compatibility checks", and then a "po= sh" was added to ob-shell.el by Eric (fb09863f). https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=3Dfb09863fb= b35bf15bcf78262b6e31b8b8b8617e7 Around that time are a few stack exchange answers suggesting to use posh, t= he "Policy-compliant Ordinary SHell", to test for posix compliance. Debian= distributed it by saying, "using posh as your /bin/sh may reveal breakage.= " It seems that posh was used to check for POSIX compliance. It's still a= vailable on Debian. https://unix.stackexchange.com/questions/48786/how-can-i-test-for-posix-com= pliance-of-shell-scripts https://web.archive.org/web/20130930231522/https://packages.debian.org/sid/= posh https://packages.debian.org/sid/posh Wikipedia claims PowerShell is based on POSIX. However, it's not clear if = it's "POSIX compliant" (and, if it were, I would be shocked if someone seri= ously suggested using it for POSIX compatibility checks). https://en.wikipedia.org/wiki/PowerShell#Grammar Checking the Microsoft documentation, I find no mention of "posh." PowerSh= ell 7 introduced a "pwsh.exe" binary. However, the same page has a referen= ce for Windows PowerShell 5.1, corresponding to "powershell.exe". This is = also corroborated by Wikipedia which says "PowerShell 7 is the replacement = for PowerShell Core 6.x products as well as Windows PowerShell 5.1, which i= s the last supported Windows PowerShell version." "posh" was added to =3Do= b-shell=3D on December 13, 2013. This would correspond to Powershell 3 or = 4 which, AFAIU, both have the binary "powershell.exe." https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.co= re/about/about_pwsh?view=3Dpowershell-7.4 https://en.wikipedia.org/wiki/PowerShell#PowerShell_7 It's possible that Eric meant to add PowerShell and abbreviated it to "posh= ." However, it appears much more likely that it refers to the "Policy-comp= liant Ordinary SHell." (https://salsa.debian.org/clint/posh) * Now what? So, if we're agreed that "posh" is not PowerShell, what should we do about = this? I propose dropping "posh" from =3Dorg-babel-shell-names=3D and removing it = from =3Dorg-babel-shell-set-prompt-commands=3D. That is, let's not explici= tly support the "Policy-compliant Ordinary SHell" or PowerShell. We never supported PowerShell intentionally and only came to it by accident= . I don't think anyone has used the "Policy-compliant Ordinary SHell" in a= t least 2 years. Die-hard adherents to either can still use them, even if = we removed the "posh" language header. ** We didn't begin "supporting" PowerShell intentionally (AFAICT) Up to 2020 and through most of 2022, Powershell wasn't considered supported= . https://list.orgmode.org/87pn707jdz.fsf@gnu.org/T/#u August 26, 2022 changed that with commit a35d1636. When =3Dorg-babel-shell= -set-prompt-commands=3D was added to set the shell prompt, PowerShell synta= x was included for completeness because people thought "posh" was PowerShel= l. AFAIK, no one explicitly asked for PowerShell support as an end-user (t= hen or after). The inclusion of PowerShell appears based on the misunderst= anding of "posh" as PowerShell. Since then, the handful of list messages m= entioning PowerShell assume it's supported. That "support" is all based on= commit a35d1636. https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=3Da35d16368= 5908386833a3d549ed110931bf3915a https://list.orgmode.org/?q=3Dd%3A17%2FJune%2F2022..+AND+powershell https://list.orgmode.org/?q=3Dd%3A17%2FJune%2F2022..+AND+posh AFAICT, PowerShell is mainly (only?) used for tests by people on the mailin= g list to check that ob-shell can run them. Occasionally, (once every few = years) someone specifically asks about it or comments on how it's not (well= ) supported. Only after commit a35d1636 did people begin saying that Power= Shell is supported. People do use PowerShell. However, there are only 36 = hits for it on the list in (since Babel was made in 2009) and only a handfu= l of side comments since the change. ** No one uses the "Policy-compliant Ordinary SHell" The change on August 26, 2022 should have caused a breaking error for someo= ne using the "Policy-compliant Ordinary SHell." The prompt for "posh" in "= org-babel-shell-set-prompt-commands" is valid PowerShell syntax (AFAIKT) an= d invalid bash/dash syntax: function prompt { "org_babel_sh_prompt> " } It's not clear to me what this would do in the "Policy-compliant Ordinary S= Hell." In Bash, functions are defined using a form like: function fname [()] compound-command [ redirections ] It says, "Note that for historical reasons, in the most common usage the cu= rly braces that surround the body of the function must be separated from th= e body by blanks or newlines. This is because the braces are reserved words= and are only recognized as such when they are separated from the command l= ist by whitespace or another shell metacharacter. Also, when using the brac= es, the list must be terminated by a semicolon, a =E2=80=98&=E2=80=99, or a= newline." https://www.gnu.org/software/bash/manual/bash.html#Shell-Functions So, for 'bash --posix', the syntax is invalid. Dash simply errors out with= "dash: 1: function: not found". Whatever it does on a POSIX-y shell, it's= a show-stopper. I don't have "posh" available, but I've run the following using dash and ba= sh. It causes Emacs to hang until you hit C-g. Maybe the "Policy-complian= t Ordinary SHell" is different and the command works? Since it fails for '= bash --posix' and dash, I doubt it. Here's what I ran: (org-babel-do-load-languages 'org-babel-load-languages '((shell . t))) (defconst org-babel-shell-set-prompt-commands '((t . "function prompt { \"%s\" }"))) #+begin_src dash :session *dash* echo "hi" #+end_src Canceling and switching to the shell that was created, the prompt is "> " (= the PS2 prompt). This swallows all input that's not an error. =3Dorg-babe= l-execute:shell=3D calls =3Dorg-babel-sh-initiate-session=3D which calls = =3Dorg-babel-comint-wait-for-output=3D which waits for the =3Dcomint-prompt= -regexp=3D which is "org_babel_sh_prompt> ", not "> ", so Emacs hangs. There have been no complaints from "Policy-compliant Ordinary SHell" users = after August 26, 2022 about "posh" being broken. The talk of "posh" (and P= owerShell, specifically) since August 26, 2022 is often as an aside. I conclude that no one uses (or at least cares much about using) the "Polic= y-compliant Ordinary SHell." ** Both the "Policy-compliant Ordinary SHell" and PowerShell are usable wit= hout "posh" Since the "Policy-compliant Ordinary SHell" is POSIX, it should be straight= -forward to use with the "shell" language by replacing "shell-file-name." = The "shell" language header will hit the default prompt command which is ge= neral POSIX syntax (that is, it changes PS1 and PS2). A shebang should als= o work. I had access to a Windows 10 machine recently and verified the following ma= kes PowerShell "work". I use quotes because the following outputs a banner= and often fails to remove the prompt, just like the current "posh" behavio= r. (org-babel-do-load-languages 'org-babel-load-languages '((shell . t))) ;;; Setup for non-sessions (setq shell-file-name "C:/Windows/System32/WindowsPowerShell/v1.0/powershel= l.exe") #+begin_src shell :results output echo "hello" #+end_src ;;; Setup for sessions ;; required by `shell.el' in `shell' command (setq explicit-shell-file-name "C:/Windows/System32/WindowsPowerShell/v1.0/powershell.exe") (setq explicit-powershell.exe-args '("-NoExit")) ;; required by `ob-shell.el' in Org 9.6.6 (distributed with Emacs ;; 29.1 which is the latest build of Emacs for Windows) (setq org-babel-prompt-command "function prompt { \"org_babel_sh_prompt> \"= }") #+begin_src shell :results output :session *powershell* echo "world" #+end_src * My stance The unfortunate reality is we "support" PowerShell currently. We have code= explicitly to handle PowerShell. Changing that could, technically, break = someone's workflow. Here's how I reason it: First, AFAIK no PowerShell user would refer to it as "posh". At best they = may say "pwsh" and most likely "powershell". We could keep "posh" for the "Policy-compliant Ordinary SHell" and add lang= uages for "pwsh" or "powershell". This would need to be communicated Power= Shell users. I don't know what other keyword we could use for the "Policy-compliant Ordi= nary SHell". We could keep "posh" for PowerShell, drop all "support" for t= he "Policy-compliant Ordinary SHell", and commit to PowerShell only. This = would need to be communicated to the "Policy-compliant Ordinary SHell" user= s. In any case, we need to communicate our decision, in ORG-NEWS, through our = mailing list communications, and definitely by updating the comment. The cost of keeping PowerShell means maintaining it and whatever weirdo stu= ff comes with it. I suggest we make the "t" in =3Dorg-babel-shell-set-prompt-commands=3D a ca= tch-all for anything with PS1/PS2 (anything POSIX-y?) and remove the "posh"= language header. The "t" case should handle bash, dash, and (I suspect) t= he "Policy-compliant Ordinary SHell". This has the potential to break some= one using "posh" for PowerShell. However, I see no indication of anyone se= riously using it. We've said for years we don't support it. The only plac= es I'm aware of saying we support PowerShell is the code comment and a few = people on the mailing list. Moreover, a "breakage" happens anyway, for eit= her "Policy-compliant Ordinary SHell" users (as currently likely exists) or= PowerShell users (if we give a new keyword). If we're going to have to pu= t a work-around or message out there anyway, I figure let's see about remov= ing any semblance of supporting PowerShell until we talk it over. Fortunately, when I wrote the Worg page, I put "the "Policy-compliant Ordin= ary SHell" for "posh." So, the only place someone might think that PowerSh= ell is supported is from the code comment or from the mailing list. ** PowerShell or cmd.exe? I'm not claiming that people haven't wanted PowerShell support. It's been = discussed a few times on the list. It always reduces to "it would be nice,= however we lack a developer or the equipment, so sadly no." As current ob-shell maintainer, here's how I see it: I like the idea of supporting PowerShell. I like the idea of supporting cm= d.exe much, much more. Both are associated with a non-free system. Providing support (even partia= lly) for non-free systems is good because it provides an opportunity to tea= ch people about software freedom. AFAIKT, both PowerShell and cmd are MIT licensed: - https://github.com/microsoft/terminal - https://github.com/PowerShell/PowerShell The thought of compiling either for a GNU system is...ugh. But maybe someo= ne else has gotten them working? Otherwise, it looks like Microsoft distri= butes a developer VM image of Windows. All together, this means there's no *technical* barrier preventing us from = running (and hence developing for) PowerShell or cmd. It's important to note that Emacs defaults to cmdproxy.exe on Windows. Usi= ng "shell" in Babel on Windows returns from cmd.exe (through cmdproxy.exe).= I believe it's best to support the defaults first. Because it's the default, I've always used cmd when using Emacs on Windows.= I would personally much prefer support for cmd.exe over PowerShell. That= is, I'd be happy to work on PowerShell stuff after cmd stuff and both afte= r cleaning up ob-shell, ob-comint, and ob-eval. Thoughts overall? -- Matt Trzcinski Emacs Org maintainer (ob-shell) Learn more about Org mode at https://orgmode.org Support Org development at=C2=A0https://liberapay.com/org-mode