From mboxrd@z Thu Jan 1 00:00:00 1970 From: Eric Schulte Subject: Re: org-mode and python pandas Date: Wed, 03 Jul 2013 08:09:43 -0600 Message-ID: <8761wrpwew.fsf@gmail.com> References: <87bo6nkv0e.fsf@gmail.com> <87y59qdysz.fsf@Rainer.invalid> <87zju6jjo3.fsf@pank.eu> Mime-Version: 1.0 Content-Type: text/plain Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:54510) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UuNmK-0004HU-BJ for emacs-orgmode@gnu.org; Wed, 03 Jul 2013 10:11:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UuNmF-0004Mr-4f for emacs-orgmode@gnu.org; Wed, 03 Jul 2013 10:11:28 -0400 Received: from mail-pd0-x234.google.com ([2607:f8b0:400e:c02::234]:51987) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UuNmE-0004Mg-Qy for emacs-orgmode@gnu.org; Wed, 03 Jul 2013 10:11:23 -0400 Received: by mail-pd0-f180.google.com with SMTP id 10so134498pdi.11 for ; Wed, 03 Jul 2013 07:11:22 -0700 (PDT) In-Reply-To: (Dov Grobgeld's message of "Wed, 3 Jul 2013 12:15:18 +0300") List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Dov Grobgeld Cc: emacs-orgmode , Rasmus Dov Grobgeld writes: > Thanks for the answers, but there is still something missing in order > to get it to work. Part of it seems to be connected to the python > parsing. E.g. the following translation of Eric's sh example doesn't > output correctly with python: > > > #+BEGIN_SRC python :results output > print """,A,B,C > 0,0.628365,0.424279,0.619791 > 1,0.799666,0.527572,0.132928 > 2,0.837255,0.138906,0.408233 > 3,0.388080,0.146212,0.575346 > """ > #+END_SRC > > #+RESULTS: > : ,A,B,C > : 0,0.628365,0.424279,0.619791 > : 1,0.799666,0.527572,0.132928 > : 2,0.837255,0.138906,0.408233 > : 3,0.388080,0.146212,0.575346 > > > #+BEGIN_SRC python :results table > return """,A,B,C > 0,0.628365,0.424279,0.619791 > 1,0.799666,0.527572,0.132928 > 2,0.837255,0.138906,0.408233 > 3,0.388080,0.146212,0.575346 > """ > #+END_SRC > > #+RESULTS: > | ,A,B,C\n\n0,0.628365,0.424279,0.619791\n\n1,0.799666,0.527572,0.132928\n\n2,0.837255,0.138906,0.408233\n\n3,0.388080,0.146212,0.575346 > | > > It seems that the only way to get a table from python is by outputting > a two dimensional python structure: > > #+BEGIN_SRC python > return [[0,0.628365,0.424279,0.619791], > [1,0.799666,0.527572,0.132928]] > #+END_SRC > > #+RESULTS: > | 0 | 0.628365 | 0.424279 | 0.619791 | > | 1 | 0.799666 | 0.527572 | 0.132928 | > > This seems quite limiting.... > In most cases this is what one wants when returning data from python code. The following elisp defined a "panda" code block, which is just like python, only it assumes that the results will be these sort of human readable strings instead of python code. ;; -*- emacs-lisp -*- (defun org-babel-execute:panda (body params) (let ((results (org-babel-execute:python body (org-babel-merge-params '((:results . "scalar")) params)))) (org-babel-result-cond (cdr (assoc :result-params params)) results (let ((tmp-file (org-babel-temp-file "sh-"))) (with-temp-file tmp-file (insert results)) (org-babel-import-elisp-from-file tmp-file))))) With the above evaluated the following works #+BEGIN_SRC panda return """,A,B,C 0,0.628365,0.424279,0.619791 1,0.799666,0.527572,0.132928 2,0.837255,0.138906,0.408233 3,0.388080,0.146212,0.575346 """ #+END_SRC #+RESULTS: | | A | B | C | | 0 | 0.628365 | 0.424279 | 0.619791 | | 1 | 0.799666 | 0.527572 | 0.132928 | | 2 | 0.837255 | 0.138906 | 0.408233 | | 3 | 0.38808 | 0.146212 | 0.575346 | > > Another related question is if there is any support for header tables? > I.e. instead of this: > > | | A | B | C | > | 0 | 0.827817 | 0.664009 | 0.089161 | > | 1 | 0.170031 | 0.729214 | 0.110918 | > | 2 | 0.575918 | 0.863924 | 0.757536 | > | 3 | 0.682722 | 0.774445 | 0.992041 | > > I want this: > > | | A | B | C | > |---+----------+----------+----------| > | 0 | 0.827817 | 0.664009 | 0.089161 | > | 1 | 0.170031 | 0.729214 | 0.110918 | > | 2 | 0.575918 | 0.863924 | 0.757536 | > | 3 | 0.682722 | 0.774445 | 0.992041 | > > I guess that if I start playing around with the python ob module, it > should be possible to get this working? > See the :colnames header argument in the manual. Best, > > Regards, > Dov > > On Mon, Jul 1, 2013 at 8:04 PM, Rasmus wrote: >> Achim Gratz writes: >> >>>>> 2. Add to pandas the option of globally influencing the text >>>>> formatting so that it outputs something more parsable by org-mode. >>>> >>>> This sounds promising, if pandas support csv output that will be >>>> correctly parsed by Org-mode. >>> >>> The package already has CSV export, so one could use that. I don't know >>> if you could echo the result directly to the output, all examples >>> revolve around putting the CSV into a file. For Org, TSV output would >>> be more natural. >> >> Something like: >> >> from pandas import DataFrame >> from numpy.random import rand >> from sys import stdout >> df = DataFrame(rand(10,3), columns = list('abc')) >> df >> df.to_csv(stdout, sep="\t", header = True, cols=(1,2)) >> >> I was completely unable to get ob-python working this morning, so I >> haven't tested it. I'm using python3, build in python mode and elpy. >> >> In any case, the csv route might be better, as Pandas doesn't print >> the table if it's too big (try changing 10 to 1000 above). >> >> -- >> Powered by magic pixies! >> >> > -- Eric Schulte http://cs.unm.edu/~eschulte