From: "Eric Schulte" <schulte.eric@gmail.com>
To: "Sébastien Vauban" <wxhgmqzgwmuf@spammotel.com>
Cc: emacs-orgmode@gnu.org
Subject: Re: [Babel] Executing sh code within sessions
Date: Fri, 05 Nov 2010 15:38:29 -0600 [thread overview]
Message-ID: <877hgrqwek.fsf@gmail.com> (raw)
In-Reply-To: 80oca4rs9e.fsf@mundaneum.com
Hi Seb,
Yes, I've noticed the shell session evaluation problems as well. I
think one possible solution is for us to move from the current schema of
echoing a babel-specific marker to indicate that evaluation has
completed (which isn't working very well and is fairly ugly to boot) and
move towards something more like what Dan has done with R, where the
output is written to a file, and we check for that file to be created to
signal the end of session based evaluation.
This is certainly something that needs to be addressed, however I for
one won't have any time over the next week (I'm attending a conference).
Rest assured this issue is in our inbox. :)
Best Regards -- Eric
Sébastien Vauban <wxhgmqzgwmuf@spammotel.com> writes:
> Hi Eric,
>
> As requested, I'm here emphasizing the important part of this post:
>
> When executing sh code block within a session, it never terminates, echoing...
>
> : executing Sh code block...
>
> in the minibuffer.
>
> The =sva= shell buffer contains:
>
> #+begin_src sh
> echo In $(pwd):
>
> /Sebastien@MEDIACENTER:...ples/Org-scraps 0$ In /cygdrive/c/home/sva/Examples/Org-scraps:
> /Sebastien@MEDIACENTER:...ples/Org-scraps 0$ echo In $(pwd):
> In /cygdrive/c/home/sva/Examples/Org-scraps:
> /Sebastien@MEDIACENTER:...ples/Org-scraps 0$ ls *.org
> Agenda-Sorting-Strategy.org
> Clock-Report.org
> org-beamer-fpu-rules.org
> org-hist.org
> /Sebastien@MEDIACENTER:...ples/Org-scraps 0$ echo 'org_babel_sh_eoe'
> org_babel_sh_eoe
> /Sebastien@MEDIACENTER:...ples/Org-scraps 0$
> #+end_src
>
> Does that ring a bell to you?
>
> Details below, if needed.
>
> Best regards,
> Seb
>
> Sébastien Vauban wrote:
>> #+TITLE: Executing sh-code within a session
>> #+AUTHOR: Seb Vauban
>> #+LANGUAGE: en_US
>>
>> * Executing sh-code within a session
>>
>> ** Context
>>
>> - git pull done this morning (2010-11-04 Thu).
>>
>> - EmacsW32 + Cygwin bash as inferior shell
>>
>> #+begin_src emacs-lisp
>> (setq shell-file-name "bash")
>> (setenv "SHELL" "bash")
>> (setq shell-command-switch "-c")
>> (setq w32-quote-process-args t)
>> (setq explicit-shell-file-name "bash")
>> #+end_src
>>
>> ** First time
>>
>> The first time I execute the following:
>>
>> #+begin_src sh :session sva
>> echo In $(pwd):
>> ls *.org
>> #+end_src
>>
>> I get the following error:
>>
>> #+begin_src emacs-lisp
>> Debugger entered--Lisp error: (error "Marker does not point anywhere")
>> ansi-color-apply-on-region(#<marker in no buffer> #<marker at 17 in sva>)
>> ansi-color-process-output("")
>> run-hook-with-args(ansi-color-process-output "")
>> comint-send-input(nil t)
>> (lambda (line) (insert line) (comint-send-input nil t) (sleep-for 0.25))("echo In $(pwd):")
>> mapc((lambda (line) (insert line) (comint-send-input nil t) (sleep-for 0.25)) ("echo In $(pwd):" "ls *.org" "echo 'org_babel_sh_eoe'"))
>> (progn (goto-char (process-mark ...)) (let (... ...) (setq dangling-text ...) (delete-region start end)) (mapc (lambda ... ... ... ...) (append ... ...)) (while (progn ... ...) (accept-process-output ...)) (goto-char (process-mark ...)) (insert dangling-text))
>> (unwind-protect (progn (goto-char ...) (let ... ... ...) (mapc ... ...) (while ... ...) (goto-char ...) (insert dangling-text)) (remove-hook (quote comint-output-filter-functions) (quote my-filt)))
>> (progn (fset (quote my-filt) (function* ...)) (add-hook (quote comint-output-filter-functions) (quote my-filt)) (unwind-protect (progn ... ... ... ... ... ...) (remove-hook ... ...)))
>> (unwind-protect (progn (fset ... ...) (add-hook ... ...) (unwind-protect ... ...)) (if --cl-letf-bound-- (fset ... --cl-letf-save--) (fmakunbound ...)))
>> (let* ((--cl-letf-bound-- ...) (--cl-letf-save-- ...)) (unwind-protect (progn ... ... ...) (if --cl-letf-bound-- ... ...)))
>> (letf ((... ...)) (add-hook (quote comint-output-filter-functions) (quote my-filt)) (unwind-protect (progn ... ... ... ... ... ...) (remove-hook ... ...)))
>> (letf* ((... ...)) (add-hook (quote comint-output-filter-functions) (quote my-filt)) (unwind-protect (progn ... ... ... ... ... ...) (remove-hook ... ...)))
>> (flet ((my-filt ... ...)) (add-hook (quote comint-output-filter-functions) (quote my-filt)) (unwind-protect (progn ... ... ... ... ... ...) (remove-hook ... ...)))
>> (let ((string-buffer "") dangling-text raw) (flet (...) (add-hook ... ...) (unwind-protect ... ...)) (if (and t body ...) (setq raw ...)) (split-string string-buffer comint-prompt-regexp))
>> (progn (unless (org-babel-comint-buffer-livep session) (error "buffer %s doesn't exist or has no process" session)) (set-buffer session) (let (... dangling-text raw) (flet ... ... ...) (if ... ...) (split-string string-buffer comint-prompt-regexp)))
>> (unwind-protect (progn (unless ... ...) (set-buffer session) (let ... ... ... ...)) (set-match-data save-match-data-internal (quote evaporate)))
>> (let ((save-match-data-internal ...)) (unwind-protect (progn ... ... ...) (set-match-data save-match-data-internal ...)))
>> (save-match-data (unless (org-babel-comint-buffer-livep session) (error "buffer %s doesn't exist or has no process" session)) (set-buffer session) (let (... dangling-text raw) (flet ... ... ...) (if ... ...) (split-string string-buffer comint-prompt-regexp)))
>> (save-excursion (save-match-data (unless ... ...) (set-buffer session) (let ... ... ... ...)))
>> (org-babel-comint-in-buffer session (let (... dangling-text raw) (flet ... ... ...) (if ... ...) (split-string string-buffer comint-prompt-regexp)))
>> (org-babel-comint-with-output (session org-babel-sh-eoe-output t body) (mapc (lambda ... ... ... ...) (append ... ...)))
>> (butlast (org-babel-comint-with-output (session org-babel-sh-eoe-output t body) (mapc ... ...)) 2)
>> (mapcar (function org-babel-trim) (butlast (org-babel-comint-with-output ... ...) 2))
>> (mapconcat (function org-babel-sh-strip-weird-long-prompt) (mapcar (function org-babel-trim) (butlast ... 2)) "\n")
>> (if (not session) (org-babel-eval org-babel-sh-command (org-babel-trim body)) (mapconcat (function org-babel-sh-strip-weird-long-prompt) (mapcar ... ...) "\n"))
>> ((lambda (results) (when results ...)) (if (not session) (org-babel-eval org-babel-sh-command ...) (mapconcat ... ... "\n")))
>> org-babel-sh-evaluate(#<buffer sva> "echo In $(pwd):\nls *.org\n" ("replace"))
>> (org-babel-reassemble-table (org-babel-sh-evaluate session full-body result-params) (org-babel-pick-name (cdr ...) (cdr ...)) (org-babel-pick-name (cdr ...) (cdr ...)))
>> (let* ((session ...) (result-params ...) (full-body ...)) (org-babel-reassemble-table (org-babel-sh-evaluate session full-body result-params) (org-babel-pick-name ... ...) (org-babel-pick-name ... ...)))
>> org-babel-execute:sh("echo In $(pwd):\nls *.org\n" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:comments . "") (:shebang . "") (:cache . "no") (:noweb . "no") (:tangle . "no") (:exports . "code") (:results . "replace") (:session . "sva") (:hlines . "no")))
>> funcall(org-babel-execute:sh "echo In $(pwd):\nls *.org\n" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:comments . "") (:shebang . "") (:cache . "no") (:noweb . "no") (:tangle . "no") (:exports . "code") (:results . "replace") (:session . "sva") (:hlines . "no")))
>> ((lambda (result) (cond ... ... ...)) (funcall cmd body params))
>> (setq result ((lambda ... ...) (funcall cmd body params)))
>> (if (and (not arg) new-hash (equal new-hash old-hash)) (save-excursion (goto-char ...) (end-of-line 1) (forward-char 1) (setq result ...) (message ...) result) (message "executing %s code block%s..." (capitalize lang) (if ... ... "")) (setq result (... ...)) (org-babel-insert-result result result-params info new-hash indent lang) (run-hooks (quote org-babel-after-execute-hook)) result)
>> (progn (fset (quote call-process-region) (function* ...)) (unless (fboundp cmd) (error "No org-babel-execute function for %s!" lang)) (if (and ... new-hash ...) (save-excursion ... ... ... ... ... result) (message "executing %s code block%s..." ... ...) (setq result ...) (org-babel-insert-result result result-params info new-hash indent lang) (run-hooks ...) result))
>> (unwind-protect (progn (fset ... ...) (unless ... ...) (if ... ... ... ... ... ... result)) (if --cl-letf-bound-- (fset ... --cl-letf-save--) (fmakunbound ...)))
>> (let* ((--cl-letf-bound-- ...) (--cl-letf-save-- ...)) (unwind-protect (progn ... ... ...) (if --cl-letf-bound-- ... ...)))
>> (letf ((... ...)) (unless (fboundp cmd) (error "No org-babel-execute function for %s!" lang)) (if (and ... new-hash ...) (save-excursion ... ... ... ... ... result) (message "executing %s code block%s..." ... ...) (setq result ...) (org-babel-insert-result result result-params info new-hash indent lang) (run-hooks ...) result))
>> (letf* ((... ...)) (unless (fboundp cmd) (error "No org-babel-execute function for %s!" lang)) (if (and ... new-hash ...) (save-excursion ... ... ... ... ... result) (message "executing %s code block%s..." ... ...) (setq result ...) (org-babel-insert-result result result-params info new-hash indent lang) (run-hooks ...) result))
>> (flet ((call-process-region ... ...)) (unless (fboundp cmd) (error "No org-babel-execute function for %s!" lang)) (if (and ... new-hash ...) (save-excursion ... ... ... ... ... result) (message "executing %s code block%s..." ... ...) (setq result ...) (org-babel-insert-result result result-params info new-hash indent lang) (run-hooks ...) result))
>> (unwind-protect (flet (...) (unless ... ...) (if ... ... ... ... ... ... result)) (setq call-process-region (quote org-babel-call-process-region-original)))
>> (let* ((lang ...) (params ...) (cache\? ...) (result-params ...) (new-hash ...) (old-hash ...) (body ...) (cmd ...) (dir ...) (default-directory ...) (org-babel-call-process-region-original ...) (indent ...) result) (unwind-protect (flet ... ... ...) (setq call-process-region ...)))
>> (progn (let* (... ... ... ... ... ... ... ... ... ... ... ... result) (unwind-protect ... ...)))
>> (if (org-babel-confirm-evaluate info) (progn (let* ... ...)))
>> (when (org-babel-confirm-evaluate info) (let* (... ... ... ... ... ... ... ... ... ... ... ... result) (unwind-protect ... ...)))
>> (let ((info ...)) (when (org-babel-confirm-evaluate info) (let* ... ...)))
>> org-babel-execute-src-block(nil ("sh" "echo In $(pwd):\nls *.org\n" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:comments . "") (:shebang . "") (:cache . "no") (:noweb . "no") (:tangle . "no") (:exports . "code") (:results . "replace") (:session . "sva") (:hlines . "no")) "" nil 0))
>> (progn (org-babel-execute-src-block current-prefix-arg info) t)
>> (if info (progn (org-babel-execute-src-block current-prefix-arg info) t) nil)
>> (let ((info ...)) (if info (progn ... t) nil))
>> org-babel-execute-src-block-maybe()
>> (or (org-babel-execute-src-block-maybe) (org-babel-lob-execute-maybe))
>> org-babel-execute-maybe()
>> call-interactively(org-babel-execute-maybe)
>> (cond ((commandp org-speed-command) (setq this-command org-speed-command) (call-interactively org-speed-command)) ((functionp org-speed-command) (funcall org-speed-command)) ((and org-speed-command ...) (eval org-speed-command)) (t (let ... ...)))
>> (cond ((and org-use-speed-commands ...) (cond ... ... ... ...)) ((and ... ... ... ...) (let ... ... ... ... ...)) (t (setq org-table-may-need-update t) (self-insert-command N) (org-fix-tags-on-the-fly) (if org-self-insert-cluster-for-undo ...)))
>> org-self-insert-command(1)
>> call-interactively(org-self-insert-command nil nil)
>> #+end_src
>>
>> ** Next times
>>
>> The second time I execute it:
>>
>> #+begin_src sh :session sva
>> echo In $(pwd):
>> ls *.org
>> #+end_src
>>
>> it never terminates, echoing:
>>
>> : executing Sh code block...
>>
>> in the minibuffer.
>>
>> The =sva= shell buffer contains:
>>
>> #+begin_src sh
>> echo In $(pwd):
>>
>> /Sebastien@MEDIACENTER:...ples/Org-scraps 0$ In /cygdrive/c/home/sva/Examples/Org-scraps:
>> /Sebastien@MEDIACENTER:...ples/Org-scraps 0$ echo In $(pwd):
>> In /cygdrive/c/home/sva/Examples/Org-scraps:
>> /Sebastien@MEDIACENTER:...ples/Org-scraps 0$ ls *.org
>> Agenda-Sorting-Strategy.org
>> Clock-Report.org
>> org-beamer-fpu-rules.org
>> org-hist.org
>> /Sebastien@MEDIACENTER:...ples/Org-scraps 0$ echo 'org_babel_sh_eoe'
>> org_babel_sh_eoe
>> /Sebastien@MEDIACENTER:...ples/Org-scraps 0$
>> #+end_src
>>
>> ** Value of my Bash prompt
>>
>> It now is defined as such:
>>
>> #+begin_src sh
>> # my format of the prompt: mono-line and (often) colored
>> function my_prompt_command ()
>> {
>> RET=$?
>>
>> # replace the $HOME prefix by ~ in the current directory
>> if [[ $HOME == ${PWD:0:${#HOME}} ]]; then
>> NEWPWD="~${PWD:${#HOME}}"
>> else
>> NEWPWD=$PWD
>> fi
>>
>> # how many characters of the $PWD should be kept
>> local pwd_max_length=15
>>
>> if [[ ${#NEWPWD} -gt $pwd_max_length ]]; then
>> local pwd_offset=$(( ${#NEWPWD} - $pwd_max_length ))
>> NEWPWD="...${NEWPWD:$pwd_offset:$pwd_max_length}"
>> fi
>>
>> # prompt character
>> if [[ $(whoami) = "root" ]]; then
>> local PROMPTCHAR="#"
>> else
>> local PROMPTCHAR="$"
>> fi
>>
>> setenv PS1 "/\u@\h:${NEWPWD} ${RET}${PROMPTCHAR} "
>> }
>>
>> # shell prompt
>> setenv PROMPT_COMMAND my_prompt_command
>> #+end_src
>>
>> So, it now definitively is:
>>
>> - mono-line
>> - uncolored
>>
>> That should be the basic requirements I needed to meet, right?
>>
>> ** Variable comint-prompt-regexp
>>
>> Value of the =comint-prompt-regexp= variable in my shell, used by Org-babel to
>> digest output from the shell:
>>
>> #+begin_src emacs-lisp
>> (switch-to-buffer "sva")
>> (describe-variable 'comint-prompt-regexp)
>> #+end_src
>>
>> #+results:
>> #+begin_example
>> comint-prompt-regexp is a variable defined in `comint.el'.
>> Its value is
>> "^[^#$%>\n]*[#$%>] *"
>> Local in buffer sva; global value is "^"
>>
>> Documentation:
>> Regexp to recognize prompts in the inferior process.
>> Defaults to "^", the null string at BOL.
>>
>> This variable is only used if the variable
>> `comint-use-prompt-regexp' is non-nil.
>>
>> Good choices:
>> Canonical Lisp: "^[^> \n]*>+:? *" (Lucid, franz, kcl, T, cscheme, oaklisp)
>> Lucid Common Lisp: "^\\(>\\|\\(->\\)+\\) *"
>> franz: "^\\(->\\|<[0-9]*>:\\) *"
>> kcl: "^>+ *"
>> shell: "^[^#$%>\n]*[#$%>] *"
>> T: "^>+ *"
>>
>> This is a good thing to set in mode hooks.
>> #+end_example
>>
>> This regexp should match my own prompt as it appears in your *shell*
>> buffers locally (product of my personal =.bashrc=) configuration.
>>
>> It does, right?
>>
>> ** What about shell-prompt-pattern?
>>
>> Browsing along, it seems there is another interesting value to look at:
>> =shell-prompt-pattern=.
>>
>> #+begin_src emacs-lisp
>> (switch-to-buffer "sva")
>> (describe-variable 'shell-prompt-pattern)
>> #+end_src
>>
>> #+results:
>> #+begin_example
>> shell-prompt-pattern is a variable defined in `shell.el'.
>> Its value is
>> "^[^#$%>\n]*[#$%>] *"
>>
>> Documentation:
>> Regexp to match prompts in the inferior shell.
>> Defaults to "^[^#$%>\n]*[#$%>] *", which works pretty well.
>> This variable is used to initialize `comint-prompt-regexp' in the
>> shell buffer.
>>
>> If `comint-use-prompt-regexp' is nil, then this variable is only used
>> to determine paragraph boundaries. See Info node `Shell Prompts' for
>> how Shell mode treats paragraphs.
>>
>> The pattern should probably not match more than one line. If it does,
>> Shell mode may become confused trying to distinguish prompt from input
>> on lines which don't start with a prompt.
>>
>> This is a fine thing to set in your `.emacs' file.
>>
>> You can customize this variable.
>> #+end_example
>>
>> I don't know if this was important, but -- as you can see -- I've set it to
>> the same value as =comint-prompt-regexp=.
>>
>> ** Back to comint's prompt regexp!
>>
>> #+begin_src emacs-lisp
>> (switch-to-buffer "sva")
>> (describe-variable 'comint-use-prompt-regexp)
>> #+end_src
>>
>> #+results:
>> #+begin_example
>> comint-use-prompt-regexp is a variable defined in `comint.el'.
>> Its value is nil
>>
>> Documentation:
>> If non-nil, use `comint-prompt-regexp' to recognize prompts.
>> If nil, then program output and user-input are given different `field'
>> properties, which Emacs commands can use to distinguish them (in
>> particular, common movement commands such as `beginning-of-line'
>> respect field boundaries in a natural way).
>>
>> You can customize this variable.
>> #+end_example
>>
>> NIL!? Important?
>>
>> It should not. The problems seem to be somewhere else... but where? Can you
>> give me a hint?[1]
>>
>> Best regards,
>> Seb
>>
>> Footnotes:
>> [1] Sorry for this looooong mail. Really. But I hope its Org form will be on
>> the "good" side...
next prev parent reply other threads:[~2010-11-05 21:43 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-11-04 11:24 Executing sh code within sessions Sébastien Vauban
2010-11-05 10:15 ` [Babel] " Sébastien Vauban
2010-11-05 21:38 ` Eric Schulte [this message]
2010-11-06 8:14 ` Sébastien Vauban
2010-11-17 12:57 ` Sébastien Vauban
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=877hgrqwek.fsf@gmail.com \
--to=schulte.eric@gmail.com \
--cc=emacs-orgmode@gnu.org \
--cc=wxhgmqzgwmuf@spammotel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).