From: Matt <matt@excalamus.com>
To: "Osher Jacob" <osherz5@gmail.com>
Cc: "emacs-orgmode" <emacs-orgmode@gnu.org>
Subject: Re: [BUG] ob-shell doesn't evaluate last line on Windows (cmd/cmdproxy) [9.6.1 ( @ c:/Users/Osher/AppData/Roaming/.emacs.d/elpa/org-9.6.1/)]
Date: Wed, 18 Jan 2023 00:09:19 -0500 [thread overview]
Message-ID: <185c34814a4.bcdd7b94578193.327779579906739164@excalamus.com> (raw)
In-Reply-To: <CAGsxwFZGTCAG0EEUBaHLJSn+sLXC7iFjYJTdi7XjEftvU8rhRQ@mail.gmail.com>
---- On Tue, 17 Jan 2023 14:53:39 -0500 Osher Jacob wrote ---
> changing shell-command-switch to "/k" or "-k", I get a similar output:
Thanks for checking that.
> You also mentioned the source code block is being passed through the "-c" flag as a command-line argument.I might be misunderstanding something here, but it seems like it is being passed through the stdin of the shell process when the calls process-file -> call-process are being made.
That the block source is passed through "-c" is an educated guess. I keep finding myself in C land where it's less clear to me what precisely happens. You're correct that `call-process' acts on the block source. It's not clear to me whether that's done through stdin or whether being passed through stdin would even help us. Here's what I'm seeing.
Evaluating the block goes through 4 lisp commands and then into C:
1. Calls `(org-babel-eval shell-file-name (org-trim body))'
This evaluates as...
(org-babel-eval
"cmdproxy.exe" ; command, whatever `shell-file-name' is
(org-trim body) ; query, the block body
)
The query parameter (the block body) gets inserted into a temp buffer on which `org-babel--shell-command-on-region' is called.
2. Calls `(org-babel--shell-command-on-region command error-buffer)' on a temp buffer containing block body.
This evaluates as...
(org-babel--shell-command-on-region
command ; "cmdproxy.exe"
error-buffer ; #<buffer *Org-Babel Error*>
)
This in turn calls `process-file' where INPUT-FILE is a temp file containing the contents of the buffer `org-babel--shell-command-on-region' was called on (that is, the temp file contains the block body).
3. Calls `(process-file shell-file-name input-file (if error-file (list t error-file) t) nil shell-command-switch command)'
This evaluates as...
(process-file
"cmdproxy.exe" ; shell-file-name
"/path/to/temp/containing/block/source" ; input-file
(t "/tmp/babel jTCHe/ob-error-yHOivA") ; output destination (like call-process's DESTINATION)
nil ; display
"-c" ; args
"cmdproxy.exe" ; args
)
Of course, I imagine the paths would be Windows paths when run on Windows.
According to the documentation, the args are passed to the process (cmdproxy.exe) "verbatim without filename handling, as `call-process' does." The `call-process' documentation says, "remaining arguments ARGS are strings passed as command arguments to PROGRAM."
To me, that sounds like cmdproxy.exe gets "passed into" cmdproxy.exe. This would explain the nested shell calls. If all this were written out, I imagine it would look something like:
cmdproxy.exe -c cmdproxy.exe
What's not clear to me is how the INPUT-FILE is handled.
The `process-file' documentation says "normally". I assume it's how `call-process' handles INPUT-FILE, but the documentation doesn't really address it.
Finally, `process-file' finally calls `call-process'.
4. Calls `(apply 'call-process program (or lc infile) (if stderr-file (list (car buffer) stderr-file) buffer) display args)'
This evaluates as...
(apply 'call-process
program ; cmdproxy
(or lc infile) ; local-copy of the INFILE, "/path/to/temp/containing/block/source"
(if stderr-file (list (car buffer) stderr-file) buffer) ; (t "/tmp/babel jTCHe/ob-error-yHOivA" )
display ; nil
args) ; ("-c" "cmdproxy.exe")
How this actually works is non-trivial (https://git.savannah.gnu.org/cgit/emacs.git/tree/src/callproc.c#n250) and not something I understand at the moment. We can see here that indeed a call like `cmdproxy.exe -c cmdproxy.exe' is being made. Still, I'm not sure how the INPUT-FILE gets processed. For example, is it passed in the second cmdproxy call? Or, maybe it gets called first and then the second call to cmdproxy happens and hangs? I don't know.
> Any ideas on how to proceed from here?
I have two ideas.
1. Another naive work around attempt. Again, I'm going from memory, documentation, and what I have previously written.
I have in my init a command to open a terminal when working on Windows that looks like:
(start-process "cmd" nil "cmd.exe" "/C" "start" "cmd.exe" "/K" "cd" dir)
This starts a cmd process with /c which terminates the prompt after the command that follows is run. The command That follows starts another cmd process with /k which changes the directory and leaves the terminal open.
I have another command to open QGIS that does the same thing:
(start-process "cmd" nil "cmd.exe" "/C" "start" "\"qgis\"" "cmd.exe" "/K" "C:\\Program Files\\QGIS 3.22.3\\bin\\qgis.bat")
It's not clear to me why the extra call to a second cmd with /k is needed. Maybe it's copy-pasta or Windows being Windows.
Anyway, I mention these because I wonder if we might be able to do something similar.
Try changing `shell-file-name' to "cmd.exe" (ideally, the full path) and `shell-command-switch' to "/k" (or maybe "-k"). My hope would be that the final call would look something like,
cmd.exe /k cmd.exe /k <executes-input-file-as-stdin-or-whatever>
Given my analysis, I'm not sure if there's a way we could trick the call into being `cmd.exe /c cmd.exe /k input-file'.
2. We could write some ob-shell code to explicitly handle Windows cmd and powershell. For example, call `process-file' similar to the stdin/cmdline branch of `org-babel-sh-evaluate'.
I'm open to this, but, like I've said, I don't have a Windows machine to work on right now. I'd not be able to verify it. I, and I'm sure others, would be happy to guide you if that's something you'd like to help implement.
Thoughts?
next prev parent reply other threads:[~2023-01-18 5:10 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-16 16:27 [BUG] ob-shell doesn't evaluate last line on Windows (cmd/cmdproxy) [9.6.1 ( @ c:/Users/Osher/AppData/Roaming/.emacs.d/elpa/org-9.6.1/)] Osher Jacob
2023-01-16 21:40 ` Matt
2023-01-17 1:07 ` Matt
2023-01-17 19:53 ` Osher Jacob
2023-01-18 5:09 ` Matt [this message]
2023-01-18 9:05 ` Ihor Radchenko
2023-01-19 16:28 ` Osher Jacob
2023-01-20 4:29 ` Matt
2023-01-20 9:27 ` Ihor Radchenko
2023-01-23 3:12 ` Matt
2023-01-23 11:42 ` Ihor Radchenko
2023-01-26 4:04 ` Matt
2023-01-26 9:51 ` Ihor Radchenko
2023-01-30 6:00 ` Matt
2023-01-30 14:00 ` Ihor Radchenko
2023-01-30 17:08 ` Matt
2023-02-01 12:05 ` Ihor Radchenko
2023-02-01 20:21 ` Matt
2023-01-20 9:24 ` Ihor Radchenko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=185c34814a4.bcdd7b94578193.327779579906739164@excalamus.com \
--to=matt@excalamus.com \
--cc=emacs-orgmode@gnu.org \
--cc=osherz5@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).