* [BUG][babel] ":result output table" doesn't work for python code blocks
@ 2011-04-15 0:46 Ethan Ligon
2011-04-15 2:26 ` Eric Schulte
0 siblings, 1 reply; 9+ messages in thread
From: Ethan Ligon @ 2011-04-15 0:46 UTC (permalink / raw)
To: emacs-orgmode
The results from python code blocks aren't correctly returned as tables
when they should be. To wit:
#+begin_src python :results output table
print "A,B,C"
#+end_src
#+results:
: A,B,C
where I'd expect to see
#+results:
| A | B | C |
Eric Fraga seems to have noticed the same problem, but with octave code
blocks in
http://thread.gmane.org/gmane.emacs.orgmode/36091
but so far there's been no useful response to his post.
Thanks for any help!
-Ethan Ligon
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [BUG][babel] ":result output table" doesn't work for python code blocks
2011-04-15 0:46 [BUG][babel] ":result output table" doesn't work for python code blocks Ethan Ligon
@ 2011-04-15 2:26 ` Eric Schulte
2011-04-15 10:35 ` Eric S Fraga
0 siblings, 1 reply; 9+ messages in thread
From: Eric Schulte @ 2011-04-15 2:26 UTC (permalink / raw)
To: Ethan Ligon; +Cc: emacs-orgmode
Ethan Ligon <ligon@are.berkeley.edu> writes:
> The results from python code blocks aren't correctly returned as tables
> when they should be. To wit:
>
> #+begin_src python :results output table
> print "A,B,C"
> #+end_src
>
> #+results:
> : A,B,C
>
> where I'd expect to see
>
> #+results:
> | A | B | C |
>
In many languages ":results output" will always return a scalar, the
thinking being, that what was printed to STDOUT was a series of strings
and thus should be inserted into the buffer as a string, rather than for
example
#+begin_src python
return [1, 2, 3]
#+end_src
#+results:
| 1 | 2 | 3 |
in which case the value returned is a list, and is thus inserted into
the buffer as a list.
There are currently options for printing to standard out and having the
results inserted as a list. For example,
#+begin_src python :results output raw
print "| 1 | 2 | 3 |"
#+end_src
#+results:
| 1 | 2 | 3 |
That said, I agree that in examples like yours above the returned value
should be a table given that the ":results table" is explicitly stated.
I've just pushed up a patch after which the following is possible.
#+begin_src python :results output table
print "[A, B, C]"
#+end_src
#+results:
| A | B | C |
Note that your exact example above would still not return a list as the
printed value does not look like a python list.
Cheers -- Eric
--
Eric Schulte
http://cs.unm.edu/~eschulte/
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [BUG][babel] ":result output table" doesn't work for python code blocks
2011-04-15 2:26 ` Eric Schulte
@ 2011-04-15 10:35 ` Eric S Fraga
2011-04-15 16:13 ` Eric Schulte
0 siblings, 1 reply; 9+ messages in thread
From: Eric S Fraga @ 2011-04-15 10:35 UTC (permalink / raw)
To: Eric Schulte; +Cc: emacs-orgmode
"Eric Schulte" <schulte.eric@gmail.com> writes:
[...]
> That said, I agree that in examples like yours above the returned value
> should be a table given that the ":results table" is explicitly stated.
> I've just pushed up a patch after which the following is possible.
Eric,
It would appear that this change you've made is only for python)? Is
there any chance of having the same for octave, please? But only if it
is easy to do as =:results output raw= with carefully formatted output
does the job for me for the moment!
Thanks!
--
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1
: using Org-mode version 7.5 (release_7.5.181.g60864)
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [BUG][babel] ":result output table" doesn't work for python code blocks
2011-04-15 10:35 ` Eric S Fraga
@ 2011-04-15 16:13 ` Eric Schulte
2011-04-20 20:28 ` Eric S Fraga
0 siblings, 1 reply; 9+ messages in thread
From: Eric Schulte @ 2011-04-15 16:13 UTC (permalink / raw)
To: emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 1070 bytes --]
Eric S Fraga <e.fraga@ucl.ac.uk> writes:
> "Eric Schulte" <schulte.eric@gmail.com> writes:
>
> [...]
>
>> That said, I agree that in examples like yours above the returned value
>> should be a table given that the ":results table" is explicitly stated.
>> I've just pushed up a patch after which the following is possible.
>
> Eric,
>
> It would appear that this change you've made is only for python)? Is
> there any chance of having the same for octave, please? But only if it
> is easy to do as =:results output raw= with carefully formatted output
> does the job for me for the moment!
>
Hi Eric,
I do not have a local copy of octave, so I'm less confident making
changes to that file, but the attached patch attempts to make the same
changes in ob-octave which were made in ob-python.
Could you please test this patch for both external and session based
evaluation and let me know if it works (I'm more hopeful that the
external evaluation will work as expected than the session evaluation).
Once this is working I'll commit it to the core.
Thanks -- Eric
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ob-octave-allow-collecting-tabular-results-when-resu.patch --]
[-- Type: text/x-diff, Size: 3905 bytes --]
From 7477c3ac10d28342ccf993ea36b41ebfcab015ac Mon Sep 17 00:00:00 2001
From: Eric Schulte <schulte.eric@gmail.com>
Date: Fri, 15 Apr 2011 10:11:18 -0600
Subject: [PATCH] ob-octave: allow collecting tabular results when ":results output"
* lisp/ob-octave.el (org-babel-octave-evaluate-external-process): Dump
output to temp file, and read results from that file as with ":results value".
(org-babel-octave-evaluate-session): Read results from temporary
file with both ":results output" and ":results value".
---
lisp/ob-octave.el | 55 ++++++++++++++++++++++++++++------------------------
1 files changed, 30 insertions(+), 25 deletions(-)
diff --git a/lisp/ob-octave.el b/lisp/ob-octave.el
index 3430dea..064677e 100644
--- a/lisp/ob-octave.el
+++ b/lisp/ob-octave.el
@@ -164,18 +164,18 @@ value of the last statement in BODY, as elisp."
(defun org-babel-octave-evaluate-external-process (body result-type matlabp)
"Evaluate BODY in an external octave process."
- (let ((cmd (if matlabp
+ (let ((tmp-file (org-babel-temp-file "octave-"))
+ (cmd (if matlabp
org-babel-matlab-shell-command
org-babel-octave-shell-command)))
(case result-type
- (output (org-babel-eval cmd body))
- (value (let ((tmp-file (org-babel-temp-file "octave-")))
- (org-babel-eval
- cmd
- (format org-babel-octave-wrapper-method body
- (org-babel-process-file-name tmp-file 'noquote)
- (org-babel-process-file-name tmp-file 'noquote)))
- (org-babel-octave-import-elisp-from-file tmp-file))))))
+ (output (with-temp-file tmp-file (insert (org-babel-eval cmd body))))
+ (value (org-babel-eval
+ cmd
+ (format org-babel-octave-wrapper-method body
+ (org-babel-process-file-name tmp-file 'noquote)
+ (org-babel-process-file-name tmp-file 'noquote)))))
+ (org-babel-octave-import-elisp-from-file tmp-file)))
(defun org-babel-octave-evaluate-session
(session body result-type &optional matlabp)
@@ -194,7 +194,8 @@ value of the last statement in BODY, as elisp."
(format org-babel-matlab-emacs-link-wrapper-method
body
(org-babel-process-file-name tmp-file 'noquote)
- (org-babel-process-file-name tmp-file 'noquote) wait-file) "\n")
+ (org-babel-process-file-name tmp-file 'noquote)
+ wait-file) "\n")
(mapconcat
#'org-babel-chomp
(list (format org-babel-octave-wrapper-method
@@ -221,21 +222,25 @@ value of the last statement in BODY, as elisp."
org-babel-octave-eoe-output)
t full-body)
(insert full-body) (comint-send-input nil t)))) results)
- (case result-type
- (value
- (org-babel-octave-import-elisp-from-file tmp-file))
- (output
- (progn
- (setq results
- (if matlabp
- (cdr (reverse (delq "" (mapcar
- #'org-babel-octave-read-string
- (mapcar #'org-babel-trim raw)))))
- (cdr (member org-babel-octave-eoe-output
- (reverse (mapcar
- #'org-babel-octave-read-string
- (mapcar #'org-babel-trim raw)))))))
- (mapconcat #'identity (reverse results) "\n"))))))
+ (if (or (member "code" result-params)
+ (member "pp" result-params)
+ (member "scalar" result-params)
+ (and (member "output" result-params)
+ (not (member "table" result-params))))
+ (progn
+ (setq results
+ (if matlabp
+ (cdr (reverse (delq "" (mapcar
+ #'org-babel-octave-read-string
+ (mapcar #'org-babel-trim raw)))))
+ (cdr (member org-babel-octave-eoe-output
+ (reverse (mapcar
+ #'org-babel-octave-read-string
+ (mapcar #'org-babel-trim raw)))))))
+ (mapconcat #'identity (reverse results) "\n"))
+ (when (equal results-type 'output)
+ (with-temp-file tmp-file (insert raw)))
+ (org-babel-octave-import-elisp-from-file tmp-file))))
(defun org-babel-octave-import-elisp-from-file (file-name)
"Import data from FILE-NAME.
--
1.7.1
[-- Attachment #3: Type: text/plain, Size: 47 bytes --]
--
Eric Schulte
http://cs.unm.edu/~eschulte/
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [BUG][babel] ":result output table" doesn't work for python code blocks
2011-04-15 16:13 ` Eric Schulte
@ 2011-04-20 20:28 ` Eric S Fraga
2011-04-26 19:31 ` Eric Schulte
0 siblings, 1 reply; 9+ messages in thread
From: Eric S Fraga @ 2011-04-20 20:28 UTC (permalink / raw)
To: Eric Schulte; +Cc: emacs-orgmode
"Eric Schulte" <schulte.eric@gmail.com> writes:
> Eric S Fraga <e.fraga@ucl.ac.uk> writes:
>
>> "Eric Schulte" <schulte.eric@gmail.com> writes:
>>
>> [...]
>>
>>> That said, I agree that in examples like yours above the returned value
>>> should be a table given that the ":results table" is explicitly stated.
>>> I've just pushed up a patch after which the following is possible.
>>
>> Eric,
>>
>> It would appear that this change you've made is only for python)? Is
>> there any chance of having the same for octave, please? But only if it
>> is easy to do as =:results output raw= with carefully formatted output
>> does the job for me for the moment!
>>
>
> Hi Eric,
>
> I do not have a local copy of octave, so I'm less confident making
> changes to that file, but the attached patch attempts to make the same
> changes in ob-octave which were made in ob-python.
>
> Could you please test this patch for both external and session based
> evaluation and let me know if it works (I'm more hopeful that the
> external evaluation will work as expected than the session
> evaluation).
Eric,
thanks for this.
I haven't tried the session based evaluation but the external one,
unfortunately, doesn't work as I would expect. In the following, I
would expect both source code blocks to give me the same output with the
first, value based, being the correct output.
--8<---------------cut here---------------start------------->8---
#+srcname: valueresult
#+begin_src octave :results value table :cache yes
values = [];
for i=1:15
values(end+1,:) = [i, i^3];
endfor
ans = values
#+end_src
#+results[7a87a711b7814ecf23ec5d3741a5fec3ffff3e44]: valueresult
| 1 | 1 |
| 2 | 8 |
| 3 | 27 |
| 4 | 64 |
| 5 | 125 |
| 6 | 216 |
| 7 | 343 |
| 8 | 512 |
| 9 | 729 |
| 10 | 1000 |
| 11 | 1331 |
| 12 | 1728 |
| 13 | 2197 |
| 14 | 2744 |
| 15 | 3375 |
#+srcname: outputresult
#+begin_src octave :results output table :cache yes
values = [];
for i=1:15
values(end+1,:) = [i, i^3];
endfor
disp(values)
#+end_src
#+results[3104573dcf8ec30de1a26aae7051a9e8fce6a92a]: outputresult
| 1 1 |
| 2 8 |
| 3 27 |
| 4 64 |
| 5 125 |
| 6 216 |
| 7 343 |
| 8 512 |
| 9 729 |
| 10 1000 |
| 11 1331 |
| 12 1728 |
| 13 2197 |
| 14 2744 |
| 15 3375 |
--8<---------------cut here---------------end--------------->8---
Thanks,
eric
--
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1
: using Org-mode version 7.5 (release_7.5.183.g1997.dirty)
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [BUG][babel] ":result output table" doesn't work for python code blocks
2011-04-20 20:28 ` Eric S Fraga
@ 2011-04-26 19:31 ` Eric Schulte
2011-04-27 10:09 ` Eric S Fraga
0 siblings, 1 reply; 9+ messages in thread
From: Eric Schulte @ 2011-04-26 19:31 UTC (permalink / raw)
To: emacs-orgmode
Eric S Fraga <e.fraga@ucl.ac.uk> writes:
> "Eric Schulte" <schulte.eric@gmail.com> writes:
>
>> Eric S Fraga <e.fraga@ucl.ac.uk> writes:
>>
>>> "Eric Schulte" <schulte.eric@gmail.com> writes:
>>>
>>> [...]
>>>
>>>> That said, I agree that in examples like yours above the returned value
>>>> should be a table given that the ":results table" is explicitly stated.
>>>> I've just pushed up a patch after which the following is possible.
>>>
>>> Eric,
>>>
>>> It would appear that this change you've made is only for python)? Is
>>> there any chance of having the same for octave, please? But only if it
>>> is easy to do as =:results output raw= with carefully formatted output
>>> does the job for me for the moment!
>>>
>>
>> Hi Eric,
>>
>> I do not have a local copy of octave, so I'm less confident making
>> changes to that file, but the attached patch attempts to make the same
>> changes in ob-octave which were made in ob-python.
>>
>> Could you please test this patch for both external and session based
>> evaluation and let me know if it works (I'm more hopeful that the
>> external evaluation will work as expected than the session
>> evaluation).
>
> Eric,
>
> thanks for this.
>
> I haven't tried the session based evaluation but the external one,
> unfortunately, doesn't work as I would expect. In the following, I
> would expect both source code blocks to give me the same output with the
> first, value based, being the correct output.
>
Hi Eric,
Does the matlab/octave `disp' function display tabular data in the same
manner as it is written literally in source code?
This is part of the issue with the "output table" :results combination,
namely what constitutes a table for printed output. In source code the
answer is obvious, namely whatever the language's interpreter would read
as a literal table, however with printed output there are many possible
ways to represent tabular data, but none *is* tabular in the way that
source code can *be* tabular.
If the printed output is exactly the same as how a table would be
written in matlab/octave source code, then does my patched version work?
If not, then rather than writing another table parser, perhaps the data
could be printed as an Org-mode table, and then the "output raw"
:results combination could be used, or the output could be sent through
another code block to convert the string to a table.
Hope this helps, Best -- Eric
>
>
> #+srcname: valueresult
> #+begin_src octave :results value table :cache yes
> values = [];
> for i=1:15
> values(end+1,:) = [i, i^3];
> endfor
> ans = values
> #+end_src
>
> #+results[7a87a711b7814ecf23ec5d3741a5fec3ffff3e44]: valueresult
> | 1 | 1 |
> | 2 | 8 |
> | 3 | 27 |
> | 4 | 64 |
> | 5 | 125 |
> | 6 | 216 |
> | 7 | 343 |
> | 8 | 512 |
> | 9 | 729 |
> | 10 | 1000 |
> | 11 | 1331 |
> | 12 | 1728 |
> | 13 | 2197 |
> | 14 | 2744 |
> | 15 | 3375 |
>
>
> #+srcname: outputresult
> #+begin_src octave :results output table :cache yes
> values = [];
> for i=1:15
> values(end+1,:) = [i, i^3];
> endfor
> disp(values)
> #+end_src
>
> #+results[3104573dcf8ec30de1a26aae7051a9e8fce6a92a]: outputresult
> | 1 1 |
> | 2 8 |
> | 3 27 |
> | 4 64 |
> | 5 125 |
> | 6 216 |
> | 7 343 |
> | 8 512 |
> | 9 729 |
> | 10 1000 |
> | 11 1331 |
> | 12 1728 |
> | 13 2197 |
> | 14 2744 |
> | 15 3375 |
>
> Thanks,
> eric
--
Eric Schulte
http://cs.unm.edu/~eschulte/
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [BUG][babel] ":result output table" doesn't work for python code blocks
2011-04-26 19:31 ` Eric Schulte
@ 2011-04-27 10:09 ` Eric S Fraga
2011-04-27 13:16 ` Eric Schulte
0 siblings, 1 reply; 9+ messages in thread
From: Eric S Fraga @ 2011-04-27 10:09 UTC (permalink / raw)
To: Eric Schulte; +Cc: emacs-orgmode
"Eric Schulte" <schulte.eric@gmail.com> writes:
[...]
>
> Hi Eric,
>
> Does the matlab/octave `disp' function display tabular data in the same
> manner as it is written literally in source code?
Hi Eric!
No, unfortunately not. Arrays (tabular data) are written out using only
whitespace for formatting:
--8<---------------cut here---------------start------------->8---
*** simple example
#+srcname: octave-simple
#+begin_src octave :results output
A = [1, 2; 3, 4];
disp(A)
#+end_src
#+results: octave-simple
: 1 2
: 3 4
--8<---------------cut here---------------end--------------->8---
> This is part of the issue with the "output table" :results combination,
> namely what constitutes a table for printed output. In source code the
> answer is obvious, namely whatever the language's interpreter would read
> as a literal table, however with printed output there are many possible
> ways to represent tabular data, but none *is* tabular in the way that
> source code can *be* tabular.
Yes, that makes sense.
> If the printed output is exactly the same as how a table would be
> written in matlab/octave source code, then does my patched version work?
> If not, then rather than writing another table parser, perhaps the data
> could be printed as an Org-mode table, and then the "output raw"
> :results combination could be used, or the output could be sent through
> another code block to convert the string to a table.
That's what I do at the moment, actually.
Mind you, org is able to take the output and convert it to a table
easily enough with =C-c |= (org-table-create-or-convert-from-region)
operating on the output if selected as a region. Would it be possible
to post-process the output from babel automatically using this method
(maybe with a hook?) with the output selected if ":results output
table", say, were specified? That alone would be sufficient for 90% of
the cases...
Apologies for the sporadic nature of my responses lately: I have been on
holiday (much needed!) and purposely without email most of the time!
Thanks,
eric
--
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1
: using Org-mode version 7.5 (release_7.5.209.g1a687)
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [BUG][babel] ":result output table" doesn't work for python code blocks
2011-04-27 10:09 ` Eric S Fraga
@ 2011-04-27 13:16 ` Eric Schulte
2011-04-27 15:28 ` Eric S Fraga
0 siblings, 1 reply; 9+ messages in thread
From: Eric Schulte @ 2011-04-27 13:16 UTC (permalink / raw)
To: emacs-orgmode
>> perhaps the data could be printed as an Org-mode table, and then the
>> "output raw" :results combination could be used, or the output could
>> be sent through another code block to convert the string to a table.
>
> That's what I do at the moment, actually.
>
> Mind you, org is able to take the output and convert it to a table
> easily enough with =C-c |= (org-table-create-or-convert-from-region)
> operating on the output if selected as a region. Would it be possible
> to post-process the output from babel automatically using this method
> (maybe with a hook?) with the output selected if ":results output
> table", say, were specified? That alone would be sufficient for 90% of
> the cases...
>
Ah, that sounds like a much more satisfying and general solution than
the previous approach of parsing output in language-specific manners.
Although I don't have time at the moment, I will put automatic
conversion of org type results onto my todo list, so that at some point
in the future, code like the following will be possible...
#+source: org-results
#+begin_src sh :results output org
echo "| 1 |"
echo "| 2 |"
#+end_src
#+begin_src emacs-lisp :var in=org-results
(listp in)
#+end_src
#+results:
: t
>
> Apologies for the sporadic nature of my responses lately: I have been on
> holiday (much needed!) and purposely without email most of the time!
>
No need to apologize! I try to do the same when on vacation.
Best -- Eric
>
> Thanks,
> eric
--
Eric Schulte
http://cs.unm.edu/~eschulte/
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [BUG][babel] ":result output table" doesn't work for python code blocks
2011-04-27 13:16 ` Eric Schulte
@ 2011-04-27 15:28 ` Eric S Fraga
0 siblings, 0 replies; 9+ messages in thread
From: Eric S Fraga @ 2011-04-27 15:28 UTC (permalink / raw)
To: Eric Schulte; +Cc: emacs-orgmode
"Eric Schulte" <schulte.eric@gmail.com> writes:
>>> perhaps the data could be printed as an Org-mode table, and then the
>>> "output raw" :results combination could be used, or the output could
>
>>> be sent through another code block to convert the string to a table.
>>
>> That's what I do at the moment, actually.
>>
>> Mind you, org is able to take the output and convert it to a table
>> easily enough with =C-c |= (org-table-create-or-convert-from-region)
>> operating on the output if selected as a region. Would it be possible
>> to post-process the output from babel automatically using this method
>> (maybe with a hook?) with the output selected if ":results output
>> table", say, were specified? That alone would be sufficient for 90% of
>> the cases...
>>
>
> Ah, that sounds like a much more satisfying and general solution than
> the previous approach of parsing output in language-specific manners.
> Although I don't have time at the moment, I will put automatic
> conversion of org type results onto my todo list, so that at some point
> in the future, code like the following will be possible...
>
> #+source: org-results
> #+begin_src sh :results output org
> echo "| 1 |"
> echo "| 2 |"
> #+end_src
>
> #+begin_src emacs-lisp :var in=org-results
> (listp in)
> #+end_src
> #+results:
> : t
Eric,
I am not sure I explained myself clearly above. What I meant is that
the output of, say, this octave code:
--8<---------------cut here---------------start------------->8---
#+srcname: octave-formatted
#+begin_src octave :results output raw
A = [1, 2; 3, 4];
disp(A)
#+end_src
#+results: octave-formatted
1 2
3 4
--8<---------------cut here---------------end--------------->8---
can subsequently be selected and converted to a table by applying the
org-table-create-or-convert-from-region command, which gives:
--8<---------------cut here---------------start------------->8---
| 1 | 2 |
| 3 | 4 |
--8<---------------cut here---------------end--------------->8---
It's not about identifying /org-like/ output but simply applying that
function automatically to the output, if "output table" has been
requested.
I hope this makes more sense?
--
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1
: using Org-mode version 7.5 (release_7.5.209.g1a687)
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2011-04-27 15:28 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-04-15 0:46 [BUG][babel] ":result output table" doesn't work for python code blocks Ethan Ligon
2011-04-15 2:26 ` Eric Schulte
2011-04-15 10:35 ` Eric S Fraga
2011-04-15 16:13 ` Eric Schulte
2011-04-20 20:28 ` Eric S Fraga
2011-04-26 19:31 ` Eric Schulte
2011-04-27 10:09 ` Eric S Fraga
2011-04-27 13:16 ` Eric Schulte
2011-04-27 15:28 ` Eric S Fraga
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).