From: Matt <matt@excalamus.com>
To: "emacs-orgmode" <emacs-orgmode@gnu.org>
Subject: ob-shell: proposal to remove "posh"
Date: Thu, 11 Jan 2024 21:30:59 +0100 [thread overview]
Message-ID: <18cfa388d12.f0069ffe919377.6846036599039377431@excalamus.com> (raw)
In-Reply-To:
Hi,
I would like people's thoughts on removing the "posh" language header.
ob-shell.el supports a "posh" shell. What is "posh?"
* Posh is not PowerShell
"posh" was added to =ob-shell= in fb09863f with no commit message on December 13, 2013 (Friday the 13th!).
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=fb09863fbb35bf15bcf78262b6e31b8b8b8617e7
A mail message the same day says,
> Eric Schulte writes:
> >> How about the following resolution? We rename ob-sh.el to ob-shell.el.
> >> New "shell" code blocks could use the value of the
> >> `org-babel-sh-command' environment variable. Then sh, bash, zsh, csh,
> >> ash, dash (am I missing any other common ones) use the specific shell
> >> specified.
>
> I've also seen ksh, mksh, posh (the latter specifically for POSIX
> compatibility checks).
https://list.orgmode.org/878uvychr1.fsf@pinto.chemeng.ucl.ac.uk/T/#mc20039f988519d409294dc604b5e0dc0f439307b
There was discussion about different shells, Eric asked for others, "posh" was mentioned as "specially for POSIX compatibility checks", and then a "posh" was added to ob-shell.el by Eric (fb09863f).
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=fb09863fbb35bf15bcf78262b6e31b8b8b8617e7
Around that time are a few stack exchange answers suggesting to use posh, the "Policy-compliant Ordinary SHell", to test for posix compliance. Debian distributed it by saying, "using posh as your /bin/sh may reveal breakage." It seems that posh was used to check for POSIX compliance. It's still available on Debian.
https://unix.stackexchange.com/questions/48786/how-can-i-test-for-posix-compliance-of-shell-scripts
https://web.archive.org/web/20130930231522/https://packages.debian.org/sid/posh
https://packages.debian.org/sid/posh
Wikipedia claims PowerShell is based on POSIX. However, it's not clear if it's "POSIX compliant" (and, if it were, I would be shocked if someone seriously suggested using it for POSIX compatibility checks).
https://en.wikipedia.org/wiki/PowerShell#Grammar
Checking the Microsoft documentation, I find no mention of "posh." PowerShell 7 introduced a "pwsh.exe" binary. However, the same page has a reference for Windows PowerShell 5.1, corresponding to "powershell.exe". This is also corroborated by Wikipedia which says "PowerShell 7 is the replacement for PowerShell Core 6.x products as well as Windows PowerShell 5.1, which is the last supported Windows PowerShell version." "posh" was added to =ob-shell= on December 13, 2013. This would correspond to Powershell 3 or 4 which, AFAIU, both have the binary "powershell.exe."
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_pwsh?view=powershell-7.4
https://en.wikipedia.org/wiki/PowerShell#PowerShell_7
It's possible that Eric meant to add PowerShell and abbreviated it to "posh." However, it appears much more likely that it refers to the "Policy-compliant Ordinary SHell." (https://salsa.debian.org/clint/posh)
* Now what?
So, if we're agreed that "posh" is not PowerShell, what should we do about this?
I propose dropping "posh" from =org-babel-shell-names= and removing it from =org-babel-shell-set-prompt-commands=. That is, let's not explicitly support the "Policy-compliant Ordinary SHell" or PowerShell.
We never supported PowerShell intentionally and only came to it by accident. I don't think anyone has used the "Policy-compliant Ordinary SHell" in at least 2 years. Die-hard adherents to either can still use them, even if we removed the "posh" language header.
** We didn't begin "supporting" PowerShell intentionally (AFAICT)
Up to 2020 and through most of 2022, Powershell wasn't considered supported.
https://list.orgmode.org/87pn707jdz.fsf@gnu.org/T/#u
August 26, 2022 changed that with commit a35d1636. When =org-babel-shell-set-prompt-commands= was added to set the shell prompt, PowerShell syntax was included for completeness because people thought "posh" was PowerShell. AFAIK, no one explicitly asked for PowerShell support as an end-user (then or after). The inclusion of PowerShell appears based on the misunderstanding of "posh" as PowerShell. Since then, the handful of list messages mentioning PowerShell assume it's supported. That "support" is all based on commit a35d1636.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=a35d163685908386833a3d549ed110931bf3915a
https://list.orgmode.org/?q=d%3A17%2FJune%2F2022..+AND+powershell
https://list.orgmode.org/?q=d%3A17%2FJune%2F2022..+AND+posh
AFAICT, PowerShell is mainly (only?) used for tests by people on the mailing list to check that ob-shell can run them. Occasionally, (once every few years) someone specifically asks about it or comments on how it's not (well) supported. Only after commit a35d1636 did people begin saying that PowerShell is supported. People do use PowerShell. However, there are only 36 hits for it on the list in (since Babel was made in 2009) and only a handful of side comments since the change.
** No one uses the "Policy-compliant Ordinary SHell"
The change on August 26, 2022 should have caused a breaking error for someone using the "Policy-compliant Ordinary SHell." The prompt for "posh" in "org-babel-shell-set-prompt-commands" is valid PowerShell syntax (AFAIKT) and invalid bash/dash syntax:
function prompt { "org_babel_sh_prompt> " }
It's not clear to me what this would do in the "Policy-compliant Ordinary SHell." In Bash, functions are defined using a form like:
function fname [()] compound-command [ redirections ]
It says, "Note that for historical reasons, in the most common usage the curly braces that surround the body of the function must be separated from the body by blanks or newlines. This is because the braces are reserved words and are only recognized as such when they are separated from the command list by whitespace or another shell metacharacter. Also, when using the braces, the list must be terminated by a semicolon, a ‘&’, or a newline."
https://www.gnu.org/software/bash/manual/bash.html#Shell-Functions
So, for 'bash --posix', the syntax is invalid. Dash simply errors out with "dash: 1: function: not found". Whatever it does on a POSIX-y shell, it's a show-stopper.
I don't have "posh" available, but I've run the following using dash and bash. It causes Emacs to hang until you hit C-g. Maybe the "Policy-compliant Ordinary SHell" is different and the command works? Since it fails for 'bash --posix' and dash, I doubt it. Here's what I ran:
(org-babel-do-load-languages 'org-babel-load-languages '((shell . t)))
(defconst org-babel-shell-set-prompt-commands
'((t . "function prompt { \"%s\" }")))
#+begin_src dash :session *dash*
echo "hi"
#+end_src
Canceling and switching to the shell that was created, the prompt is "> " (the PS2 prompt). This swallows all input that's not an error. =org-babel-execute:shell= calls =org-babel-sh-initiate-session= which calls =org-babel-comint-wait-for-output= which waits for the =comint-prompt-regexp= which is "org_babel_sh_prompt> ", not "> ", so Emacs hangs.
There have been no complaints from "Policy-compliant Ordinary SHell" users after August 26, 2022 about "posh" being broken. The talk of "posh" (and PowerShell, specifically) since August 26, 2022 is often as an aside.
I conclude that no one uses (or at least cares much about using) the "Policy-compliant Ordinary SHell."
** Both the "Policy-compliant Ordinary SHell" and PowerShell are usable without "posh"
Since the "Policy-compliant Ordinary SHell" is POSIX, it should be straight-forward to use with the "shell" language by replacing "shell-file-name." The "shell" language header will hit the default prompt command which is general POSIX syntax (that is, it changes PS1 and PS2). A shebang should also work.
I had access to a Windows 10 machine recently and verified the following makes PowerShell "work". I use quotes because the following outputs a banner and often fails to remove the prompt, just like the current "posh" behavior.
(org-babel-do-load-languages 'org-babel-load-languages '((shell . t)))
;;; Setup for non-sessions
(setq shell-file-name "C:/Windows/System32/WindowsPowerShell/v1.0/powershell.exe")
#+begin_src shell :results output
echo "hello"
#+end_src
;;; Setup for sessions
;; required by `shell.el' in `shell' command
(setq explicit-shell-file-name
"C:/Windows/System32/WindowsPowerShell/v1.0/powershell.exe")
(setq explicit-powershell.exe-args '("-NoExit"))
;; required by `ob-shell.el' in Org 9.6.6 (distributed with Emacs
;; 29.1 which is the latest build of Emacs for Windows)
(setq org-babel-prompt-command "function prompt { \"org_babel_sh_prompt> \" }")
#+begin_src shell :results output :session *powershell*
echo "world"
#+end_src
* My stance
The unfortunate reality is we "support" PowerShell currently. We have code explicitly to handle PowerShell. Changing that could, technically, break someone's workflow.
Here's how I reason it:
First, AFAIK no PowerShell user would refer to it as "posh". At best they may say "pwsh" and most likely "powershell".
We could keep "posh" for the "Policy-compliant Ordinary SHell" and add languages for "pwsh" or "powershell". This would need to be communicated PowerShell users.
I don't know what other keyword we could use for the "Policy-compliant Ordinary SHell". We could keep "posh" for PowerShell, drop all "support" for the "Policy-compliant Ordinary SHell", and commit to PowerShell only. This would need to be communicated to the "Policy-compliant Ordinary SHell" users.
In any case, we need to communicate our decision, in ORG-NEWS, through our mailing list communications, and definitely by updating the comment.
The cost of keeping PowerShell means maintaining it and whatever weirdo stuff comes with it.
I suggest we make the "t" in =org-babel-shell-set-prompt-commands= a catch-all for anything with PS1/PS2 (anything POSIX-y?) and remove the "posh" language header. The "t" case should handle bash, dash, and (I suspect) the "Policy-compliant Ordinary SHell". This has the potential to break someone using "posh" for PowerShell. However, I see no indication of anyone seriously using it. We've said for years we don't support it. The only places I'm aware of saying we support PowerShell is the code comment and a few people on the mailing list. Moreover, a "breakage" happens anyway, for either "Policy-compliant Ordinary SHell" users (as currently likely exists) or PowerShell users (if we give a new keyword). If we're going to have to put a work-around or message out there anyway, I figure let's see about removing any semblance of supporting PowerShell until we talk it over.
Fortunately, when I wrote the Worg page, I put "the "Policy-compliant Ordinary SHell" for "posh." So, the only place someone might think that PowerShell is supported is from the code comment or from the mailing list.
** PowerShell or cmd.exe?
I'm not claiming that people haven't wanted PowerShell support. It's been discussed a few times on the list. It always reduces to "it would be nice, however we lack a developer or the equipment, so sadly no."
As current ob-shell maintainer, here's how I see it:
I like the idea of supporting PowerShell. I like the idea of supporting cmd.exe much, much more.
Both are associated with a non-free system. Providing support (even partially) for non-free systems is good because it provides an opportunity to teach people about software freedom.
AFAIKT, both PowerShell and cmd are MIT licensed:
- https://github.com/microsoft/terminal
- https://github.com/PowerShell/PowerShell
The thought of compiling either for a GNU system is...ugh. But maybe someone else has gotten them working? Otherwise, it looks like Microsoft distributes a developer VM image of Windows.
All together, this means there's no *technical* barrier preventing us from running (and hence developing for) PowerShell or cmd.
It's important to note that Emacs defaults to cmdproxy.exe on Windows. Using "shell" in Babel on Windows returns from cmd.exe (through cmdproxy.exe). I believe it's best to support the defaults first.
Because it's the default, I've always used cmd when using Emacs on Windows. I would personally much prefer support for cmd.exe over PowerShell. That is, I'd be happy to work on PowerShell stuff after cmd stuff and both after cleaning up ob-shell, ob-comint, and ob-eval.
Thoughts overall?
--
Matt Trzcinski
Emacs Org maintainer (ob-shell)
Learn more about Org mode at https://orgmode.org
Support Org development at https://liberapay.com/org-mode
next reply other threads:[~2024-01-11 20:32 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-01-11 20:30 Matt [this message]
2024-01-12 10:04 ` ob-shell: proposal to remove "posh" Morgan Willcock
2024-01-12 15:49 ` Matt
2024-01-13 10:43 ` Matt
2024-01-13 18:02 ` Ihor Radchenko
2024-01-15 20:11 ` Matt
2024-01-15 21:16 ` Ihor Radchenko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=18cfa388d12.f0069ffe919377.6846036599039377431@excalamus.com \
--to=matt@excalamus.com \
--cc=emacs-orgmode@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).