* [BUG][babel] ":result output table" doesn't work for python code blocks @ 2011-04-15 0:46 Ethan Ligon 2011-04-15 2:26 ` Eric Schulte 0 siblings, 1 reply; 9+ messages in thread From: Ethan Ligon @ 2011-04-15 0:46 UTC (permalink / raw) To: emacs-orgmode The results from python code blocks aren't correctly returned as tables when they should be. To wit: #+begin_src python :results output table print "A,B,C" #+end_src #+results: : A,B,C where I'd expect to see #+results: | A | B | C | Eric Fraga seems to have noticed the same problem, but with octave code blocks in http://thread.gmane.org/gmane.emacs.orgmode/36091 but so far there's been no useful response to his post. Thanks for any help! -Ethan Ligon ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [BUG][babel] ":result output table" doesn't work for python code blocks 2011-04-15 0:46 [BUG][babel] ":result output table" doesn't work for python code blocks Ethan Ligon @ 2011-04-15 2:26 ` Eric Schulte 2011-04-15 10:35 ` Eric S Fraga 0 siblings, 1 reply; 9+ messages in thread From: Eric Schulte @ 2011-04-15 2:26 UTC (permalink / raw) To: Ethan Ligon; +Cc: emacs-orgmode Ethan Ligon <ligon@are.berkeley.edu> writes: > The results from python code blocks aren't correctly returned as tables > when they should be. To wit: > > #+begin_src python :results output table > print "A,B,C" > #+end_src > > #+results: > : A,B,C > > where I'd expect to see > > #+results: > | A | B | C | > In many languages ":results output" will always return a scalar, the thinking being, that what was printed to STDOUT was a series of strings and thus should be inserted into the buffer as a string, rather than for example #+begin_src python return [1, 2, 3] #+end_src #+results: | 1 | 2 | 3 | in which case the value returned is a list, and is thus inserted into the buffer as a list. There are currently options for printing to standard out and having the results inserted as a list. For example, #+begin_src python :results output raw print "| 1 | 2 | 3 |" #+end_src #+results: | 1 | 2 | 3 | That said, I agree that in examples like yours above the returned value should be a table given that the ":results table" is explicitly stated. I've just pushed up a patch after which the following is possible. #+begin_src python :results output table print "[A, B, C]" #+end_src #+results: | A | B | C | Note that your exact example above would still not return a list as the printed value does not look like a python list. Cheers -- Eric -- Eric Schulte http://cs.unm.edu/~eschulte/ ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [BUG][babel] ":result output table" doesn't work for python code blocks 2011-04-15 2:26 ` Eric Schulte @ 2011-04-15 10:35 ` Eric S Fraga 2011-04-15 16:13 ` Eric Schulte 0 siblings, 1 reply; 9+ messages in thread From: Eric S Fraga @ 2011-04-15 10:35 UTC (permalink / raw) To: Eric Schulte; +Cc: emacs-orgmode "Eric Schulte" <schulte.eric@gmail.com> writes: [...] > That said, I agree that in examples like yours above the returned value > should be a table given that the ":results table" is explicitly stated. > I've just pushed up a patch after which the following is possible. Eric, It would appear that this change you've made is only for python)? Is there any chance of having the same for octave, please? But only if it is easy to do as =:results output raw= with carefully formatted output does the job for me for the moment! Thanks! -- : Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1 : using Org-mode version 7.5 (release_7.5.181.g60864) ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [BUG][babel] ":result output table" doesn't work for python code blocks 2011-04-15 10:35 ` Eric S Fraga @ 2011-04-15 16:13 ` Eric Schulte 2011-04-20 20:28 ` Eric S Fraga 0 siblings, 1 reply; 9+ messages in thread From: Eric Schulte @ 2011-04-15 16:13 UTC (permalink / raw) To: emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 1070 bytes --] Eric S Fraga <e.fraga@ucl.ac.uk> writes: > "Eric Schulte" <schulte.eric@gmail.com> writes: > > [...] > >> That said, I agree that in examples like yours above the returned value >> should be a table given that the ":results table" is explicitly stated. >> I've just pushed up a patch after which the following is possible. > > Eric, > > It would appear that this change you've made is only for python)? Is > there any chance of having the same for octave, please? But only if it > is easy to do as =:results output raw= with carefully formatted output > does the job for me for the moment! > Hi Eric, I do not have a local copy of octave, so I'm less confident making changes to that file, but the attached patch attempts to make the same changes in ob-octave which were made in ob-python. Could you please test this patch for both external and session based evaluation and let me know if it works (I'm more hopeful that the external evaluation will work as expected than the session evaluation). Once this is working I'll commit it to the core. Thanks -- Eric [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-ob-octave-allow-collecting-tabular-results-when-resu.patch --] [-- Type: text/x-diff, Size: 3905 bytes --] From 7477c3ac10d28342ccf993ea36b41ebfcab015ac Mon Sep 17 00:00:00 2001 From: Eric Schulte <schulte.eric@gmail.com> Date: Fri, 15 Apr 2011 10:11:18 -0600 Subject: [PATCH] ob-octave: allow collecting tabular results when ":results output" * lisp/ob-octave.el (org-babel-octave-evaluate-external-process): Dump output to temp file, and read results from that file as with ":results value". (org-babel-octave-evaluate-session): Read results from temporary file with both ":results output" and ":results value". --- lisp/ob-octave.el | 55 ++++++++++++++++++++++++++++------------------------ 1 files changed, 30 insertions(+), 25 deletions(-) diff --git a/lisp/ob-octave.el b/lisp/ob-octave.el index 3430dea..064677e 100644 --- a/lisp/ob-octave.el +++ b/lisp/ob-octave.el @@ -164,18 +164,18 @@ value of the last statement in BODY, as elisp." (defun org-babel-octave-evaluate-external-process (body result-type matlabp) "Evaluate BODY in an external octave process." - (let ((cmd (if matlabp + (let ((tmp-file (org-babel-temp-file "octave-")) + (cmd (if matlabp org-babel-matlab-shell-command org-babel-octave-shell-command))) (case result-type - (output (org-babel-eval cmd body)) - (value (let ((tmp-file (org-babel-temp-file "octave-"))) - (org-babel-eval - cmd - (format org-babel-octave-wrapper-method body - (org-babel-process-file-name tmp-file 'noquote) - (org-babel-process-file-name tmp-file 'noquote))) - (org-babel-octave-import-elisp-from-file tmp-file)))))) + (output (with-temp-file tmp-file (insert (org-babel-eval cmd body)))) + (value (org-babel-eval + cmd + (format org-babel-octave-wrapper-method body + (org-babel-process-file-name tmp-file 'noquote) + (org-babel-process-file-name tmp-file 'noquote))))) + (org-babel-octave-import-elisp-from-file tmp-file))) (defun org-babel-octave-evaluate-session (session body result-type &optional matlabp) @@ -194,7 +194,8 @@ value of the last statement in BODY, as elisp." (format org-babel-matlab-emacs-link-wrapper-method body (org-babel-process-file-name tmp-file 'noquote) - (org-babel-process-file-name tmp-file 'noquote) wait-file) "\n") + (org-babel-process-file-name tmp-file 'noquote) + wait-file) "\n") (mapconcat #'org-babel-chomp (list (format org-babel-octave-wrapper-method @@ -221,21 +222,25 @@ value of the last statement in BODY, as elisp." org-babel-octave-eoe-output) t full-body) (insert full-body) (comint-send-input nil t)))) results) - (case result-type - (value - (org-babel-octave-import-elisp-from-file tmp-file)) - (output - (progn - (setq results - (if matlabp - (cdr (reverse (delq "" (mapcar - #'org-babel-octave-read-string - (mapcar #'org-babel-trim raw))))) - (cdr (member org-babel-octave-eoe-output - (reverse (mapcar - #'org-babel-octave-read-string - (mapcar #'org-babel-trim raw))))))) - (mapconcat #'identity (reverse results) "\n")))))) + (if (or (member "code" result-params) + (member "pp" result-params) + (member "scalar" result-params) + (and (member "output" result-params) + (not (member "table" result-params)))) + (progn + (setq results + (if matlabp + (cdr (reverse (delq "" (mapcar + #'org-babel-octave-read-string + (mapcar #'org-babel-trim raw))))) + (cdr (member org-babel-octave-eoe-output + (reverse (mapcar + #'org-babel-octave-read-string + (mapcar #'org-babel-trim raw))))))) + (mapconcat #'identity (reverse results) "\n")) + (when (equal results-type 'output) + (with-temp-file tmp-file (insert raw))) + (org-babel-octave-import-elisp-from-file tmp-file)))) (defun org-babel-octave-import-elisp-from-file (file-name) "Import data from FILE-NAME. -- 1.7.1 [-- Attachment #3: Type: text/plain, Size: 47 bytes --] -- Eric Schulte http://cs.unm.edu/~eschulte/ ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [BUG][babel] ":result output table" doesn't work for python code blocks 2011-04-15 16:13 ` Eric Schulte @ 2011-04-20 20:28 ` Eric S Fraga 2011-04-26 19:31 ` Eric Schulte 0 siblings, 1 reply; 9+ messages in thread From: Eric S Fraga @ 2011-04-20 20:28 UTC (permalink / raw) To: Eric Schulte; +Cc: emacs-orgmode "Eric Schulte" <schulte.eric@gmail.com> writes: > Eric S Fraga <e.fraga@ucl.ac.uk> writes: > >> "Eric Schulte" <schulte.eric@gmail.com> writes: >> >> [...] >> >>> That said, I agree that in examples like yours above the returned value >>> should be a table given that the ":results table" is explicitly stated. >>> I've just pushed up a patch after which the following is possible. >> >> Eric, >> >> It would appear that this change you've made is only for python)? Is >> there any chance of having the same for octave, please? But only if it >> is easy to do as =:results output raw= with carefully formatted output >> does the job for me for the moment! >> > > Hi Eric, > > I do not have a local copy of octave, so I'm less confident making > changes to that file, but the attached patch attempts to make the same > changes in ob-octave which were made in ob-python. > > Could you please test this patch for both external and session based > evaluation and let me know if it works (I'm more hopeful that the > external evaluation will work as expected than the session > evaluation). Eric, thanks for this. I haven't tried the session based evaluation but the external one, unfortunately, doesn't work as I would expect. In the following, I would expect both source code blocks to give me the same output with the first, value based, being the correct output. --8<---------------cut here---------------start------------->8--- #+srcname: valueresult #+begin_src octave :results value table :cache yes values = []; for i=1:15 values(end+1,:) = [i, i^3]; endfor ans = values #+end_src #+results[7a87a711b7814ecf23ec5d3741a5fec3ffff3e44]: valueresult | 1 | 1 | | 2 | 8 | | 3 | 27 | | 4 | 64 | | 5 | 125 | | 6 | 216 | | 7 | 343 | | 8 | 512 | | 9 | 729 | | 10 | 1000 | | 11 | 1331 | | 12 | 1728 | | 13 | 2197 | | 14 | 2744 | | 15 | 3375 | #+srcname: outputresult #+begin_src octave :results output table :cache yes values = []; for i=1:15 values(end+1,:) = [i, i^3]; endfor disp(values) #+end_src #+results[3104573dcf8ec30de1a26aae7051a9e8fce6a92a]: outputresult | 1 1 | | 2 8 | | 3 27 | | 4 64 | | 5 125 | | 6 216 | | 7 343 | | 8 512 | | 9 729 | | 10 1000 | | 11 1331 | | 12 1728 | | 13 2197 | | 14 2744 | | 15 3375 | --8<---------------cut here---------------end--------------->8--- Thanks, eric -- : Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1 : using Org-mode version 7.5 (release_7.5.183.g1997.dirty) ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [BUG][babel] ":result output table" doesn't work for python code blocks 2011-04-20 20:28 ` Eric S Fraga @ 2011-04-26 19:31 ` Eric Schulte 2011-04-27 10:09 ` Eric S Fraga 0 siblings, 1 reply; 9+ messages in thread From: Eric Schulte @ 2011-04-26 19:31 UTC (permalink / raw) To: emacs-orgmode Eric S Fraga <e.fraga@ucl.ac.uk> writes: > "Eric Schulte" <schulte.eric@gmail.com> writes: > >> Eric S Fraga <e.fraga@ucl.ac.uk> writes: >> >>> "Eric Schulte" <schulte.eric@gmail.com> writes: >>> >>> [...] >>> >>>> That said, I agree that in examples like yours above the returned value >>>> should be a table given that the ":results table" is explicitly stated. >>>> I've just pushed up a patch after which the following is possible. >>> >>> Eric, >>> >>> It would appear that this change you've made is only for python)? Is >>> there any chance of having the same for octave, please? But only if it >>> is easy to do as =:results output raw= with carefully formatted output >>> does the job for me for the moment! >>> >> >> Hi Eric, >> >> I do not have a local copy of octave, so I'm less confident making >> changes to that file, but the attached patch attempts to make the same >> changes in ob-octave which were made in ob-python. >> >> Could you please test this patch for both external and session based >> evaluation and let me know if it works (I'm more hopeful that the >> external evaluation will work as expected than the session >> evaluation). > > Eric, > > thanks for this. > > I haven't tried the session based evaluation but the external one, > unfortunately, doesn't work as I would expect. In the following, I > would expect both source code blocks to give me the same output with the > first, value based, being the correct output. > Hi Eric, Does the matlab/octave `disp' function display tabular data in the same manner as it is written literally in source code? This is part of the issue with the "output table" :results combination, namely what constitutes a table for printed output. In source code the answer is obvious, namely whatever the language's interpreter would read as a literal table, however with printed output there are many possible ways to represent tabular data, but none *is* tabular in the way that source code can *be* tabular. If the printed output is exactly the same as how a table would be written in matlab/octave source code, then does my patched version work? If not, then rather than writing another table parser, perhaps the data could be printed as an Org-mode table, and then the "output raw" :results combination could be used, or the output could be sent through another code block to convert the string to a table. Hope this helps, Best -- Eric > > > #+srcname: valueresult > #+begin_src octave :results value table :cache yes > values = []; > for i=1:15 > values(end+1,:) = [i, i^3]; > endfor > ans = values > #+end_src > > #+results[7a87a711b7814ecf23ec5d3741a5fec3ffff3e44]: valueresult > | 1 | 1 | > | 2 | 8 | > | 3 | 27 | > | 4 | 64 | > | 5 | 125 | > | 6 | 216 | > | 7 | 343 | > | 8 | 512 | > | 9 | 729 | > | 10 | 1000 | > | 11 | 1331 | > | 12 | 1728 | > | 13 | 2197 | > | 14 | 2744 | > | 15 | 3375 | > > > #+srcname: outputresult > #+begin_src octave :results output table :cache yes > values = []; > for i=1:15 > values(end+1,:) = [i, i^3]; > endfor > disp(values) > #+end_src > > #+results[3104573dcf8ec30de1a26aae7051a9e8fce6a92a]: outputresult > | 1 1 | > | 2 8 | > | 3 27 | > | 4 64 | > | 5 125 | > | 6 216 | > | 7 343 | > | 8 512 | > | 9 729 | > | 10 1000 | > | 11 1331 | > | 12 1728 | > | 13 2197 | > | 14 2744 | > | 15 3375 | > > Thanks, > eric -- Eric Schulte http://cs.unm.edu/~eschulte/ ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [BUG][babel] ":result output table" doesn't work for python code blocks 2011-04-26 19:31 ` Eric Schulte @ 2011-04-27 10:09 ` Eric S Fraga 2011-04-27 13:16 ` Eric Schulte 0 siblings, 1 reply; 9+ messages in thread From: Eric S Fraga @ 2011-04-27 10:09 UTC (permalink / raw) To: Eric Schulte; +Cc: emacs-orgmode "Eric Schulte" <schulte.eric@gmail.com> writes: [...] > > Hi Eric, > > Does the matlab/octave `disp' function display tabular data in the same > manner as it is written literally in source code? Hi Eric! No, unfortunately not. Arrays (tabular data) are written out using only whitespace for formatting: --8<---------------cut here---------------start------------->8--- *** simple example #+srcname: octave-simple #+begin_src octave :results output A = [1, 2; 3, 4]; disp(A) #+end_src #+results: octave-simple : 1 2 : 3 4 --8<---------------cut here---------------end--------------->8--- > This is part of the issue with the "output table" :results combination, > namely what constitutes a table for printed output. In source code the > answer is obvious, namely whatever the language's interpreter would read > as a literal table, however with printed output there are many possible > ways to represent tabular data, but none *is* tabular in the way that > source code can *be* tabular. Yes, that makes sense. > If the printed output is exactly the same as how a table would be > written in matlab/octave source code, then does my patched version work? > If not, then rather than writing another table parser, perhaps the data > could be printed as an Org-mode table, and then the "output raw" > :results combination could be used, or the output could be sent through > another code block to convert the string to a table. That's what I do at the moment, actually. Mind you, org is able to take the output and convert it to a table easily enough with =C-c |= (org-table-create-or-convert-from-region) operating on the output if selected as a region. Would it be possible to post-process the output from babel automatically using this method (maybe with a hook?) with the output selected if ":results output table", say, were specified? That alone would be sufficient for 90% of the cases... Apologies for the sporadic nature of my responses lately: I have been on holiday (much needed!) and purposely without email most of the time! Thanks, eric -- : Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1 : using Org-mode version 7.5 (release_7.5.209.g1a687) ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [BUG][babel] ":result output table" doesn't work for python code blocks 2011-04-27 10:09 ` Eric S Fraga @ 2011-04-27 13:16 ` Eric Schulte 2011-04-27 15:28 ` Eric S Fraga 0 siblings, 1 reply; 9+ messages in thread From: Eric Schulte @ 2011-04-27 13:16 UTC (permalink / raw) To: emacs-orgmode >> perhaps the data could be printed as an Org-mode table, and then the >> "output raw" :results combination could be used, or the output could >> be sent through another code block to convert the string to a table. > > That's what I do at the moment, actually. > > Mind you, org is able to take the output and convert it to a table > easily enough with =C-c |= (org-table-create-or-convert-from-region) > operating on the output if selected as a region. Would it be possible > to post-process the output from babel automatically using this method > (maybe with a hook?) with the output selected if ":results output > table", say, were specified? That alone would be sufficient for 90% of > the cases... > Ah, that sounds like a much more satisfying and general solution than the previous approach of parsing output in language-specific manners. Although I don't have time at the moment, I will put automatic conversion of org type results onto my todo list, so that at some point in the future, code like the following will be possible... #+source: org-results #+begin_src sh :results output org echo "| 1 |" echo "| 2 |" #+end_src #+begin_src emacs-lisp :var in=org-results (listp in) #+end_src #+results: : t > > Apologies for the sporadic nature of my responses lately: I have been on > holiday (much needed!) and purposely without email most of the time! > No need to apologize! I try to do the same when on vacation. Best -- Eric > > Thanks, > eric -- Eric Schulte http://cs.unm.edu/~eschulte/ ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [BUG][babel] ":result output table" doesn't work for python code blocks 2011-04-27 13:16 ` Eric Schulte @ 2011-04-27 15:28 ` Eric S Fraga 0 siblings, 0 replies; 9+ messages in thread From: Eric S Fraga @ 2011-04-27 15:28 UTC (permalink / raw) To: Eric Schulte; +Cc: emacs-orgmode "Eric Schulte" <schulte.eric@gmail.com> writes: >>> perhaps the data could be printed as an Org-mode table, and then the >>> "output raw" :results combination could be used, or the output could > >>> be sent through another code block to convert the string to a table. >> >> That's what I do at the moment, actually. >> >> Mind you, org is able to take the output and convert it to a table >> easily enough with =C-c |= (org-table-create-or-convert-from-region) >> operating on the output if selected as a region. Would it be possible >> to post-process the output from babel automatically using this method >> (maybe with a hook?) with the output selected if ":results output >> table", say, were specified? That alone would be sufficient for 90% of >> the cases... >> > > Ah, that sounds like a much more satisfying and general solution than > the previous approach of parsing output in language-specific manners. > Although I don't have time at the moment, I will put automatic > conversion of org type results onto my todo list, so that at some point > in the future, code like the following will be possible... > > #+source: org-results > #+begin_src sh :results output org > echo "| 1 |" > echo "| 2 |" > #+end_src > > #+begin_src emacs-lisp :var in=org-results > (listp in) > #+end_src > #+results: > : t Eric, I am not sure I explained myself clearly above. What I meant is that the output of, say, this octave code: --8<---------------cut here---------------start------------->8--- #+srcname: octave-formatted #+begin_src octave :results output raw A = [1, 2; 3, 4]; disp(A) #+end_src #+results: octave-formatted 1 2 3 4 --8<---------------cut here---------------end--------------->8--- can subsequently be selected and converted to a table by applying the org-table-create-or-convert-from-region command, which gives: --8<---------------cut here---------------start------------->8--- | 1 | 2 | | 3 | 4 | --8<---------------cut here---------------end--------------->8--- It's not about identifying /org-like/ output but simply applying that function automatically to the output, if "output table" has been requested. I hope this makes more sense? -- : Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1 : using Org-mode version 7.5 (release_7.5.209.g1a687) ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2011-04-27 15:28 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2011-04-15 0:46 [BUG][babel] ":result output table" doesn't work for python code blocks Ethan Ligon 2011-04-15 2:26 ` Eric Schulte 2011-04-15 10:35 ` Eric S Fraga 2011-04-15 16:13 ` Eric Schulte 2011-04-20 20:28 ` Eric S Fraga 2011-04-26 19:31 ` Eric Schulte 2011-04-27 10:09 ` Eric S Fraga 2011-04-27 13:16 ` Eric Schulte 2011-04-27 15:28 ` Eric S Fraga
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).