Executing sh code within sessions

["Sébastien Vauban" <wxhgmqzgwmuf-geNee64TY+gS+FvcfC7Uqw@public.gmane.org>]

#+TITLE:     Executing sh-code within a session
#+AUTHOR:    Seb Vauban
#+LANGUAGE:  en_US

* Executing sh-code within a session

** Context

- git pull done this morning (2010-11-04 Thu).

- EmacsW32 + Cygwin bash as inferior shell

  #+begin_src emacs-lisp
  (setq shell-file-name "bash")
  (setenv "SHELL" "bash")
  (setq shell-command-switch "-c")
  (setq w32-quote-process-args t)
  (setq explicit-shell-file-name "bash")
  #+end_src

** First time

The first time I execute the following:

#+begin_src sh :session sva
echo In $(pwd):
ls *.org
#+end_src

I get the following error:

#+begin_src emacs-lisp
Debugger entered--Lisp error: (error "Marker does not point anywhere")
  ansi-color-apply-on-region(#<marker in no buffer> #<marker at 17 in sva>)
  ansi-color-process-output("")
  run-hook-with-args(ansi-color-process-output "")
  comint-send-input(nil t)
  (lambda (line) (insert line) (comint-send-input nil t) (sleep-for 0.25))("echo In $(pwd):")
  mapc((lambda (line) (insert line) (comint-send-input nil t) (sleep-for 0.25)) ("echo In $(pwd):" "ls *.org" "echo 'org_babel_sh_eoe'"))
  (progn (goto-char (process-mark ...)) (let (... ...) (setq dangling-text ...) (delete-region start end)) (mapc (lambda ... ... ... ...) (append ... ...)) (while (progn ... ...) (accept-process-output ...)) (goto-char (process-mark ...)) (insert dangling-text))
  (unwind-protect (progn (goto-char ...) (let ... ... ...) (mapc ... ...) (while ... ...) (goto-char ...) (insert dangling-text)) (remove-hook (quote comint-output-filter-functions) (quote my-filt)))
  (progn (fset (quote my-filt) (function* ...)) (add-hook (quote comint-output-filter-functions) (quote my-filt)) (unwind-protect (progn ... ... ... ... ... ...) (remove-hook ... ...)))
  (unwind-protect (progn (fset ... ...) (add-hook ... ...) (unwind-protect ... ...)) (if --cl-letf-bound-- (fset ... --cl-letf-save--) (fmakunbound ...)))
  (let* ((--cl-letf-bound-- ...) (--cl-letf-save-- ...)) (unwind-protect (progn ... ... ...) (if --cl-letf-bound-- ... ...)))
  (letf ((... ...)) (add-hook (quote comint-output-filter-functions) (quote my-filt)) (unwind-protect (progn ... ... ... ... ... ...) (remove-hook ... ...)))
  (letf* ((... ...)) (add-hook (quote comint-output-filter-functions) (quote my-filt)) (unwind-protect (progn ... ... ... ... ... ...) (remove-hook ... ...)))
  (flet ((my-filt ... ...)) (add-hook (quote comint-output-filter-functions) (quote my-filt)) (unwind-protect (progn ... ... ... ... ... ...) (remove-hook ... ...)))
  (let ((string-buffer "") dangling-text raw) (flet (...) (add-hook ... ...) (unwind-protect ... ...)) (if (and t body ...) (setq raw ...)) (split-string string-buffer comint-prompt-regexp))
  (progn (unless (org-babel-comint-buffer-livep session) (error "buffer %s doesn't exist or has no process" session)) (set-buffer session) (let (... dangling-text raw) (flet ... ... ...) (if ... ...) (split-string string-buffer comint-prompt-regexp)))
  (unwind-protect (progn (unless ... ...) (set-buffer session) (let ... ... ... ...)) (set-match-data save-match-data-internal (quote evaporate)))
  (let ((save-match-data-internal ...)) (unwind-protect (progn ... ... ...) (set-match-data save-match-data-internal ...)))
  (save-match-data (unless (org-babel-comint-buffer-livep session) (error "buffer %s doesn't exist or has no process" session)) (set-buffer session) (let (... dangling-text raw) (flet ... ... ...) (if ... ...) (split-string string-buffer comint-prompt-regexp)))
  (save-excursion (save-match-data (unless ... ...) (set-buffer session) (let ... ... ... ...)))
  (org-babel-comint-in-buffer session (let (... dangling-text raw) (flet ... ... ...) (if ... ...) (split-string string-buffer comint-prompt-regexp)))
  (org-babel-comint-with-output (session org-babel-sh-eoe-output t body) (mapc (lambda ... ... ... ...) (append ... ...)))
  (butlast (org-babel-comint-with-output (session org-babel-sh-eoe-output t body) (mapc ... ...)) 2)
  (mapcar (function org-babel-trim) (butlast (org-babel-comint-with-output ... ...) 2))
  (mapconcat (function org-babel-sh-strip-weird-long-prompt) (mapcar (function org-babel-trim) (butlast ... 2)) "\n")
  (if (not session) (org-babel-eval org-babel-sh-command (org-babel-trim body)) (mapconcat (function org-babel-sh-strip-weird-long-prompt) (mapcar ... ...) "\n"))
  ((lambda (results) (when results ...)) (if (not session) (org-babel-eval org-babel-sh-command ...) (mapconcat ... ... "\n")))
  org-babel-sh-evaluate(#<buffer sva> "echo In $(pwd):\nls *.org\n" ("replace"))
  (org-babel-reassemble-table (org-babel-sh-evaluate session full-body result-params) (org-babel-pick-name (cdr ...) (cdr ...)) (org-babel-pick-name (cdr ...) (cdr ...)))
  (let* ((session ...) (result-params ...) (full-body ...)) (org-babel-reassemble-table (org-babel-sh-evaluate session full-body result-params) (org-babel-pick-name ... ...) (org-babel-pick-name ... ...)))
  org-babel-execute:sh("echo In $(pwd):\nls *.org\n" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:comments . "") (:shebang . "") (:cache . "no") (:noweb . "no") (:tangle . "no") (:exports . "code") (:results . "replace") (:session . "sva") (:hlines . "no")))
  funcall(org-babel-execute:sh "echo In $(pwd):\nls *.org\n" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:comments . "") (:shebang . "") (:cache . "no") (:noweb . "no") (:tangle . "no") (:exports . "code") (:results . "replace") (:session . "sva") (:hlines . "no")))
  ((lambda (result) (cond ... ... ...)) (funcall cmd body params))
  (setq result ((lambda ... ...) (funcall cmd body params)))
  (if (and (not arg) new-hash (equal new-hash old-hash)) (save-excursion (goto-char ...) (end-of-line 1) (forward-char 1) (setq result ...) (message ...) result) (message "executing %s code block%s..." (capitalize lang) (if ... ... "")) (setq result (... ...)) (org-babel-insert-result result result-params info new-hash indent lang) (run-hooks (quote org-babel-after-execute-hook)) result)
  (progn (fset (quote call-process-region) (function* ...)) (unless (fboundp cmd) (error "No org-babel-execute function for %s!" lang)) (if (and ... new-hash ...) (save-excursion ... ... ... ... ... result) (message "executing %s code block%s..." ... ...) (setq result ...) (org-babel-insert-result result result-params info new-hash indent lang) (run-hooks ...) result))
  (unwind-protect (progn (fset ... ...) (unless ... ...) (if ... ... ... ... ... ... result)) (if --cl-letf-bound-- (fset ... --cl-letf-save--) (fmakunbound ...)))
  (let* ((--cl-letf-bound-- ...) (--cl-letf-save-- ...)) (unwind-protect (progn ... ... ...) (if --cl-letf-bound-- ... ...)))
  (letf ((... ...)) (unless (fboundp cmd) (error "No org-babel-execute function for %s!" lang)) (if (and ... new-hash ...) (save-excursion ... ... ... ... ... result) (message "executing %s code block%s..." ... ...) (setq result ...) (org-babel-insert-result result result-params info new-hash indent lang) (run-hooks ...) result))
  (letf* ((... ...)) (unless (fboundp cmd) (error "No org-babel-execute function for %s!" lang)) (if (and ... new-hash ...) (save-excursion ... ... ... ... ... result) (message "executing %s code block%s..." ... ...) (setq result ...) (org-babel-insert-result result result-params info new-hash indent lang) (run-hooks ...) result))
  (flet ((call-process-region ... ...)) (unless (fboundp cmd) (error "No org-babel-execute function for %s!" lang)) (if (and ... new-hash ...) (save-excursion ... ... ... ... ... result) (message "executing %s code block%s..." ... ...) (setq result ...) (org-babel-insert-result result result-params info new-hash indent lang) (run-hooks ...) result))
  (unwind-protect (flet (...) (unless ... ...) (if ... ... ... ... ... ... result)) (setq call-process-region (quote org-babel-call-process-region-original)))
  (let* ((lang ...) (params ...) (cache\? ...) (result-params ...) (new-hash ...) (old-hash ...) (body ...) (cmd ...) (dir ...) (default-directory ...) (org-babel-call-process-region-original ...) (indent ...) result) (unwind-protect (flet ... ... ...) (setq call-process-region ...)))
  (progn (let* (... ... ... ... ... ... ... ... ... ... ... ... result) (unwind-protect ... ...)))
  (if (org-babel-confirm-evaluate info) (progn (let* ... ...)))
  (when (org-babel-confirm-evaluate info) (let* (... ... ... ... ... ... ... ... ... ... ... ... result) (unwind-protect ... ...)))
  (let ((info ...)) (when (org-babel-confirm-evaluate info) (let* ... ...)))
  org-babel-execute-src-block(nil ("sh" "echo In $(pwd):\nls *.org\n" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:comments . "") (:shebang . "") (:cache . "no") (:noweb . "no") (:tangle . "no") (:exports . "code") (:results . "replace") (:session . "sva") (:hlines . "no")) "" nil 0))
  (progn (org-babel-execute-src-block current-prefix-arg info) t)
  (if info (progn (org-babel-execute-src-block current-prefix-arg info) t) nil)
  (let ((info ...)) (if info (progn ... t) nil))
  org-babel-execute-src-block-maybe()
  (or (org-babel-execute-src-block-maybe) (org-babel-lob-execute-maybe))
  org-babel-execute-maybe()
  call-interactively(org-babel-execute-maybe)
  (cond ((commandp org-speed-command) (setq this-command org-speed-command) (call-interactively org-speed-command)) ((functionp org-speed-command) (funcall org-speed-command)) ((and org-speed-command ...) (eval org-speed-command)) (t (let ... ...)))
  (cond ((and org-use-speed-commands ...) (cond ... ... ... ...)) ((and ... ... ... ...) (let ... ... ... ... ...)) (t (setq org-table-may-need-update t) (self-insert-command N) (org-fix-tags-on-the-fly) (if org-self-insert-cluster-for-undo ...)))
  org-self-insert-command(1)
  call-interactively(org-self-insert-command nil nil)
#+end_src

** Next times

The second time I execute it:

#+begin_src sh :session sva
echo In $(pwd):
ls *.org
#+end_src

it never terminates, echoing:

: executing Sh code block...

in the minibuffer.

The =sva= shell buffer contains:

#+begin_src sh
echo In $(pwd):

/Sebastien@MEDIACENTER:...ples/Org-scraps 0$ In /cygdrive/c/home/sva/Examples/Org-scraps:
/Sebastien@MEDIACENTER:...ples/Org-scraps 0$ echo In $(pwd):
In /cygdrive/c/home/sva/Examples/Org-scraps:
/Sebastien@MEDIACENTER:...ples/Org-scraps 0$ ls *.org
Agenda-Sorting-Strategy.org
Clock-Report.org
org-beamer-fpu-rules.org
org-hist.org
/Sebastien@MEDIACENTER:...ples/Org-scraps 0$ echo 'org_babel_sh_eoe'
org_babel_sh_eoe
/Sebastien@MEDIACENTER:...ples/Org-scraps 0$
#+end_src

** Value of my Bash prompt

It now is defined as such:

#+begin_src sh
# my format of the prompt: mono-line and (often) colored
function my_prompt_command ()
{
    RET=$?

    # replace the $HOME prefix by ~ in the current directory
    if [[ $HOME == ${PWD:0:${#HOME}} ]]; then
        NEWPWD="~${PWD:${#HOME}}"
    else
        NEWPWD=$PWD
    fi

    # how many characters of the $PWD should be kept
    local pwd_max_length=15

    if [[ ${#NEWPWD} -gt $pwd_max_length ]]; then
        local pwd_offset=$(( ${#NEWPWD} - $pwd_max_length ))
        NEWPWD="...${NEWPWD:$pwd_offset:$pwd_max_length}"
    fi

    # prompt character
    if [[ $(whoami) = "root" ]]; then
        local PROMPTCHAR="#"
    else
        local PROMPTCHAR="$"
    fi

    setenv PS1 "/\u@\h:${NEWPWD} ${RET}${PROMPTCHAR} "
}

# shell prompt
setenv PROMPT_COMMAND my_prompt_command
#+end_src

So, it now definitively is:

- mono-line
- uncolored

That should be the basic requirements I needed to meet, right?

** Variable comint-prompt-regexp

Value of the =comint-prompt-regexp= variable in my shell, used by Org-babel to
digest output from the shell:

#+begin_src emacs-lisp
(switch-to-buffer "sva")
(describe-variable 'comint-prompt-regexp)
#+end_src

#+results:
#+begin_example
comint-prompt-regexp is a variable defined in `comint.el'.
Its value is 
"^[^#$%>\n]*[#$%>] *"
Local in buffer sva; global value is "^"

Documentation:
Regexp to recognize prompts in the inferior process.
Defaults to "^", the null string at BOL.

This variable is only used if the variable
`comint-use-prompt-regexp' is non-nil.

Good choices:
  Canonical Lisp: "^[^> \n]*>+:? *" (Lucid, franz, kcl, T, cscheme, oaklisp)
  Lucid Common Lisp: "^\\(>\\|\\(->\\)+\\) *"
  franz: "^\\(->\\|<[0-9]*>:\\) *"
  kcl: "^>+ *"
  shell: "^[^#$%>\n]*[#$%>] *"
  T: "^>+ *"

This is a good thing to set in mode hooks.
#+end_example

This regexp should match my own prompt as it appears in your *shell*
buffers locally (product of my personal =.bashrc=) configuration.

It does, right?

** What about shell-prompt-pattern?

Browsing along, it seems there is another interesting value to look at:
=shell-prompt-pattern=.

#+begin_src emacs-lisp
(switch-to-buffer "sva")
(describe-variable 'shell-prompt-pattern)
#+end_src

#+results:
#+begin_example
shell-prompt-pattern is a variable defined in `shell.el'.
Its value is 
"^[^#$%>\n]*[#$%>] *"

Documentation:
Regexp to match prompts in the inferior shell.
Defaults to "^[^#$%>\n]*[#$%>] *", which works pretty well.
This variable is used to initialize `comint-prompt-regexp' in the
shell buffer.

If `comint-use-prompt-regexp' is nil, then this variable is only used
to determine paragraph boundaries.  See Info node `Shell Prompts' for
how Shell mode treats paragraphs.

The pattern should probably not match more than one line.  If it does,
Shell mode may become confused trying to distinguish prompt from input
on lines which don't start with a prompt.

This is a fine thing to set in your `.emacs' file.

You can customize this variable.
#+end_example

I don't know if this was important, but -- as you can see -- I've set it to
the same value as =comint-prompt-regexp=.

** Back to comint's prompt regexp!

#+begin_src emacs-lisp
(switch-to-buffer "sva")
(describe-variable 'comint-use-prompt-regexp)
#+end_src

#+results:
#+begin_example
comint-use-prompt-regexp is a variable defined in `comint.el'.
Its value is nil

Documentation:
If non-nil, use `comint-prompt-regexp' to recognize prompts.
If nil, then program output and user-input are given different `field'
properties, which Emacs commands can use to distinguish them (in
particular, common movement commands such as `beginning-of-line'
respect field boundaries in a natural way).

You can customize this variable.
#+end_example

NIL!?  Important?

It should not. The problems seem to be somewhere else... but where?  Can you
give me a hint?[1]

Best regards,
  Seb

Footnotes:
[1] Sorry for this looooong mail. Really. But I hope its Org form will be on
the "good" side...
-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode