From: Kyle Meyer <kyle@kyleam.com>
To: Richard Stanton <stanton@haas.berkeley.edu>
Cc: emacs-orgmode@gnu.org, John Kitchin <jkitchin@andrew.cmu.edu>
Subject: Re: Extraneous output from Python code blocks using :session option
Date: Fri, 13 Mar 2015 02:58:35 -0400 [thread overview]
Message-ID: <87d24dtmck.fsf@kmlap.domain.org> (raw)
In-Reply-To: <87bnjxq59p.fsf@kyleam.com> (Kyle Meyer's message of "Thu, 12 Mar 2015 17:23:30 -0400")
[-- Attachment #1: Type: text/plain, Size: 1049 bytes --]
Kyle Meyer <kyle@kyleam.com> wrote:
> Richard Stanton <stanton@haas.berkeley.edu> wrote:
>> This looks great. While we're patching this code, why does having
>> blank lines inside function definitions cause such problems in
>> :session mode?
>
> I think this is because the lines are being sent one by one, so sending
> the blank line causes an issue, like typing enter with the cursor at _:
>
> >>> def abc():
> >>> _
>
> ...
> File "<stdin>", line 2
>
> ^
> IndentationError: expected an indented block
> >>>
>
> I'll take a look at fixing this. python.el in Emacs 24 seems to send
> functions with blank lines over fine, so perhaps I can use it as an
> example.
I've attached two patches. The first one is a proposed way to deal with
the indentation issues in sessions. It is very similar to what
python.el does for multiline input (use a temporary file and then
execute that from the shell). The second is an update of my previous
patch to remove shell prompt characters that are leaking into the
output.
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ob-python.el-Allow-indented-code-in-sessions.patch --]
[-- Type: text/x-diff, Size: 2950 bytes --]
From 28f271314f7bff7c54696defe8e451da69cd2d6c Mon Sep 17 00:00:00 2001
From: Kyle Meyer <kyle@kyleam.com>
Date: Fri, 13 Mar 2015 02:45:04 -0400
Subject: [PATCH 1/2] ob-python.el: Allow indented code in sessions
* lisp/ob-python.el (org-babel-python-evaluate-session): Recognize
indented code in session and treat it differently to avoid syntax
errors.
For session blocks with results set to 'output', go through an
intermediate file when there is indented code to prevent
indentation-related errors when sending code to the Python shell. For
session blocks with results set to 'value', issue an user-error.
These (usually) would have resulted in a syntax error in the shell
anyway, and the '_' variable can't be used to get the last value here
(as it is for non-indented code) because it isn't set when executing
the code through a file.
---
lisp/ob-python.el | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/lisp/ob-python.el b/lisp/ob-python.el
index dd3cc66..8c51679 100644
--- a/lisp/ob-python.el
+++ b/lisp/ob-python.el
@@ -238,6 +238,14 @@ (defconst org-babel-python-pp-wrapper-method
open('%s', 'w').write( pprint.pformat(main()) )")
+(defconst org-babel-python-indented-lines-session-method
+ (concat "fname= '%s'; fh = open(fname); "
+ "exec(compile(fh.read(), fname, 'exec')); "
+ "fh.close()")
+ "Single line to execute indented Python code in session.
+%s will be formatted with the file name of the file containing
+ the code.")
+
(defun org-babel-python-evaluate
(session body &optional result-type result-params preamble)
"Evaluate BODY as Python code."
@@ -303,6 +311,13 @@ (defun org-babel-python-evaluate-session
(mapc (lambda (line) (insert line) (funcall send-wait))
(split-string body "[\r\n]"))
(funcall send-wait)))
+ (indented-p (org-babel-python--indented-p body))
+ (body (if indented-p
+ (let ((tmp-file (org-babel-temp-file "python-")))
+ (with-temp-file tmp-file (insert body))
+ (format org-babel-python-indented-lines-session-method
+ tmp-file))
+ body))
(results
(case result-type
(output
@@ -317,6 +332,8 @@ (defun org-babel-python-evaluate-session
(funcall send-wait))
2) "\n"))
(value
+ (when indented-p
+ (user-error "Value output limited to unindented lines with session"))
(let ((tmp-file (org-babel-temp-file "python-")))
(org-babel-comint-with-output
(session org-babel-python-eoe-indicator nil body)
@@ -339,6 +356,13 @@ (defun org-babel-python-read-string (string)
(match-string 1 string)
string))
+(defun org-babel-python--indented-p (input)
+ "Return true if any line in INPUT is indented."
+ (with-temp-buffer
+ (insert input)
+ (goto-char (point-min))
+ (re-search-forward "^\\s-" nil t)))
+
(provide 'ob-python)
--
2.3.1
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-ob-python.el-Strip-leading-session-characters.patch --]
[-- Type: text/x-diff, Size: 2314 bytes --]
From 3d433fac1162f1544fbcfc7e4a8675a258764f34 Mon Sep 17 00:00:00 2001
From: Kyle Meyer <kyle@kyleam.com>
Date: Fri, 13 Mar 2015 02:46:38 -0400
Subject: [PATCH 2/2] ob-python.el: Strip leading session characters
* lisp/ob-python.el (org-babel-python-evaluate-session): Strip extra leading
"..." and ">>>" from session output.
---
lisp/ob-python.el | 34 ++++++++++++++++++++++++----------
1 file changed, 24 insertions(+), 10 deletions(-)
diff --git a/lisp/ob-python.el b/lisp/ob-python.el
index 8c51679..170b6cd 100644
--- a/lisp/ob-python.el
+++ b/lisp/ob-python.el
@@ -321,16 +321,17 @@ (defun org-babel-python-evaluate-session
(results
(case result-type
(output
- (mapconcat
- #'org-babel-trim
- (butlast
- (org-babel-comint-with-output
- (session org-babel-python-eoe-indicator t body)
- (funcall input-body body)
- (funcall send-wait) (funcall send-wait)
- (insert org-babel-python-eoe-indicator)
- (funcall send-wait))
- 2) "\n"))
+ (org-babel-python--strip-session-chars
+ (mapconcat
+ #'org-babel-chomp
+ (butlast
+ (org-babel-comint-with-output
+ (session org-babel-python-eoe-indicator t body)
+ (funcall input-body body)
+ (funcall send-wait) (funcall send-wait)
+ (insert org-babel-python-eoe-indicator)
+ (funcall send-wait))
+ 2) "\n")))
(value
(when indented-p
(user-error "Value output limited to unindented lines with session"))
@@ -363,6 +364,19 @@ (defun org-babel-python--indented-p (input)
(goto-char (point-min))
(re-search-forward "^\\s-" nil t)))
+(defun org-babel-python--strip-session-chars (string)
+ "Remove leading '>>>' and '...' from Python session output.
+`org-babel-comint-with-output' splits by
+`comint-prompt-regexp' (which includes '>>>' and '...'), but, in
+some situations, these still make it through at the start of the
+output string."
+ (with-temp-buffer
+ (insert string)
+ (goto-char (point-min))
+ (when (looking-at "\\s-*\n\\(\\(>>> \\)\\|\\(\\.\\.\\. \\)\\)*")
+ (delete-region (match-beginning 0) (match-end 0)))
+ (buffer-string)))
+
(provide 'ob-python)
--
2.3.1
[-- Attachment #4: Type: text/plain, Size: 9 bytes --]
--
Kyle
next prev parent reply other threads:[~2015-03-13 6:54 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-03-11 3:38 Extraneous output from Python code blocks using :session option Richard Stanton
2015-03-11 18:29 ` Kyle Meyer
2015-03-11 18:54 ` Richard Stanton
2015-03-12 15:17 ` John Kitchin
2015-03-12 15:34 ` Richard Stanton
2015-03-12 21:23 ` Kyle Meyer
2015-03-13 6:58 ` Kyle Meyer [this message]
2015-03-13 8:21 ` Nicolas Goaziou
2015-03-13 14:40 ` Kyle Meyer
2015-03-14 8:26 ` Nicolas Goaziou
2015-03-16 0:40 ` Kyle Meyer
2015-03-12 21:21 ` Kyle Meyer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87d24dtmck.fsf@kmlap.domain.org \
--to=kyle@kyleam.com \
--cc=emacs-orgmode@gnu.org \
--cc=jkitchin@andrew.cmu.edu \
--cc=stanton@haas.berkeley.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).