Re: Org-babel `:hlines yes` no longer working for python

From: Christopher Allan Webber <cwebber@dustycloud.org>
To: Eric Schulte <schulte.eric@gmail.com>
Cc: emacs-orgmode@gnu.org
Subject: Re: Org-babel `:hlines yes` no longer working for python
Date: Mon, 28 Jun 2010 13:58:00 -0500	[thread overview]
Message-ID: <87zkyfvvbb.fsf@dustycloud.org> (raw)
In-Reply-To: <87fx07xbr8.fsf@gmail.com> (Eric Schulte's message of "Mon, 28 Jun 2010 11:17:31 -0700")

Works perfect now... Thanks! :D

"Eric Schulte" <schulte.eric@gmail.com> writes:

> Hi Chris,
>
> Thanks for catching this.  I've just pushed up a patch which should fix
> the issue.
>
> Best -- Eric
>
> Christopher Allan Webber <cwebber@dustycloud.org> writes:
>
>> Hm.  I've found a bug with this patch:
>>
>> #+begin_src python
>> return [['foo', 'bar', 'baz'], ["a", "b", "None of the above"], ['1', 2, 3]]
>> #+end_src
>>
>> #+results:
>> | foo | bar | baz                |
>> | a   | b   | hline of the above |
>> | 1   | 2   | 3                  |
>>
>> This also replaces the word "None" anywhere with hline, even in
>> strings.
>>
>>
>> "Eric Schulte" <schulte.eric@gmail.com> writes:
>>
>>> Hi,
>>>
>>> OK, I've applied this patch.
>>>
>>> Christopher Allan Webber <cwebber@dustycloud.org> writes:
>>>
>>>> Eric,
>>>>
>>>> Looks good to me!  It's abusing the None type's meaning a little, but
>>>> I think it's acceptable enough.  (If you think of hlines as rows that
>>>> are not rows, you can trick yourself into thinking it is perfectly
>>>> pythonic :))
>>>>
>>>
>>> Yea, this semantic mismatch bothered me, however it looks like Python
>>> doesn't have anything like symbols that could be used here, and I guess
>>> there isn't an issue of wanting to preserve "None" for "nil" mapping
>>> because "nil" can be represented with an empty list "[]".
>>>
>>> Thanks for bringing this up! -- Eric
>>>
>>>>
>>>>  - cwebb
>>>>
>>>> "Eric Schulte" <schulte.eric@gmail.com> writes:
>>>>
>>>>> Hi Christopher,
>>>>>
>>>>> I'm certainly no Python expert, but I implemented your idea of
>>>>> converting "hlines" to and from "None"'s (patch below [1]), and it seems
>>>>> to work (under some definition of work).  See the following example with
>>>>> the new behavior.
>>>>>
>>>>>
>>>>> --8<---------------cut here---------------start------------->8---
>>>>> #+tblname: many-cols
>>>>> | a | b | c |
>>>>> |---+---+---|
>>>>> | d | e | f |
>>>>> |---+---+---|
>>>>> | g | h | i |
>>>>>
>>>>> #+source: echo-table
>>>>> #+begin_src python :var tab=many-cols :hlines yes
>>>>>   return tab
>>>>> #+end_src
>>>>>
>>>>> #+results: echo-table
>>>>> | a | b | c |
>>>>> |---+---+---|
>>>>> | d | e | f |
>>>>> |---+---+---|
>>>>> | g | h | i |
>>>>> --8<---------------cut here---------------end--------------->8---
>>>>>
>>>>> Please, Python people, try this out and if you like the behavior then
>>>>> I'll happily apply the patch.
>>>>>
>>>>> Best -- Eric
>>>>>
>>>>> Christopher Allan Webber <cwebber@dustycloud.org> writes:
>>>>>
>>>>>> Hey Eric,
>>>>>>
>>>>>> Thanks for the super helpful reply!
>>>>>>
>>>>>> Out of curiosity, is it likely that we will ever get hline support in
>>>>>> Python and etc?  I've been pondering how it might be done, and maybe it
>>>>>> could be like this, using a '|-' string instead of a list for the row:
>>>>>>
>>>>>> [['a', 'b', 'c'], '|-', ['d', 'e', 'f'], ['g', 'h', 'i']]
>>>>>>
>>>>>> Which would produce:
>>>>>>
>>>>>> | a | b | c |
>>>>>> |---+---+---|
>>>>>> | d | e | f |
>>>>>> | g | h | i |
>>>>>>
>>>>>> Alternately maybe the same thing could be done by abusing None:
>>>>>>
>>>>>> [['a', 'b', 'c'], None, ['d', 'e', 'f'], ['g', 'h', 'i']]
>>>>>>
>>>>>> Thoughts?
>>>>>>  - cwebb
>>>>>>
>>>>>> "Eric Schulte" <schulte.eric@gmail.com> writes:
>>>>>>
>>>>>>> Hi Christopher,
>>>>>>>
>>>>>>> Thanks for pointing this out, this is an error in the documentation,
>>>>>>> which I will update.  The code you posted should generate the error you
>>>>>>> have received.
>>>>>>>
>>>>>>> Currently the only language which can handle hlines is emacs-lisp, all
>>>>>>> other languages will result in errors like the one you pasted below.
>>>>>>> That's not to say that it wouldn't be possible to add hline handling to
>>>>>>> other languages, or to maybe do something tricky like session-based
>>>>>>> evaluation in which an `hlines' variable was pre-initialized to some
>>>>>>> value, but I digress.
>>>>>>>
>>>>>>> Note that it *is* possible to have hlines in the output, using colnames,
>>>>>>> e.g.
>>>>>>>
>>>>>>>
>>>>>>> --8<---------------cut here---------------start------------->8---
>>>>>>> #+tblname: A
>>>>>>> | a | b | c |
>>>>>>> |---+---+---|
>>>>>>> | d | e | f |
>>>>>>> | g | h | i |
>>>>>>>
>>>>>>> #+begin_src python :var tab=A :colnames yes
>>>>>>> return [[val + '*' for val in row] for row in tab]
>>>>>>> #+end_src
>>>>>>>
>>>>>>> #+results:
>>>>>>> | a  | b  | c  |
>>>>>>> |----+----+----|
>>>>>>> | d* | e* | f* |
>>>>>>> | g* | h* | i* |
>>>>>>> --8<---------------cut here---------------end--------------->8---
>>>>>>>
>>>>>>> which works because the hline, and the column names, are never made
>>>>>>> available to python, rather Babel holds onto them and then re-applies
>>>>>>> them to the source block's output.
>>>>>>>
>>>>>>> or even to have an elisp block add hlines to your results
>>>>>>>
>>>>>>>
>>>>>>> --8<---------------cut here---------------start------------->8---
>>>>>>> #+tblname: many-cols
>>>>>>> | a | b | c |
>>>>>>> |---+---+---|
>>>>>>> | d | e | f |
>>>>>>> |---+---+---|
>>>>>>> | g | h | i |
>>>>>>>
>>>>>>> #+source: echo-table
>>>>>>> #+begin_src python :var tab=many-cols
>>>>>>>   return tab
>>>>>>> #+end_src
>>>>>>>
>>>>>>> #+begin_src emacs-lisp :var table=echo-table
>>>>>>>   (butlast (apply #'append (mapcar (lambda (el) (list el 'hline)) table)))
>>>>>>> #+end_src
>>>>>>>
>>>>>>> #+results:
>>>>>>> | a | b | c |
>>>>>>> |---+---+---|
>>>>>>> | d | e | f |
>>>>>>> |---+---+---|
>>>>>>> | g | h | i |
>>>>>>> --8<---------------cut here---------------end--------------->8---
>>>>>>>
>>>>>>> Thanks for pointing this out!
>>>>>>>
>>>>>>> Best -- Eric
>>>>>>>
>>>>>>> Christopher Allan Webber <cwebber@dustycloud.org> writes:
>>>>>>>
>>>>>>>> Hello all,
>>>>>>>>
>>>>>>>> I was going through the tutorial and testing the :hlines yes feature as
>>>>>>>> described in the info manual.  Unfortunately, the example given no
>>>>>>>> longer seems to work for python:
>>>>>>>>
>>>>>>>> #+tblname: many-cols
>>>>>>>> | a | b | c |
>>>>>>>> |---+---+---|
>>>>>>>> | d | e | f |
>>>>>>>> |---+---+---|
>>>>>>>> | g | h | i |
>>>>>>>>
>>>>>>>> #+source: echo-table
>>>>>>>> #+begin_src python :var tab=many-cols :hlines yes
>>>>>>>>   return tab
>>>>>>>> #+end_src
>>>>>>>>
>>>>>>>> #+results: echo-table
>>>>>>>> | a | b | c |
>>>>>>>> | d | e | f |
>>>>>>>> | g | h | i |
>>>>>>>>
>>>>>>>> In the buffer *Org-Babel Error Output* I see:
>>>>>>>>
>>>>>>>> Traceback (most recent call last):
>>>>>>>>   File "<stdin>", line 6, in <module>
>>>>>>>>   File "<stdin>", line 3, in main
>>>>>>>> NameError: global name 'hline' is not defined
>>>>>>>>
>>>>>>>> In emacs-lisp this still seems to work though.  But I also see that in
>>>>>>>> emacs lisp hlines are represented by the hline symbol.  I'm guessing
>>>>>>>> that the python equivalent was trying to do the same thing, but no hline
>>>>>>>> variable exists in python?
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>  - cwebb
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Emacs-orgmode mailing list
>>>>>>>> Please use `Reply All' to send replies to the list.
>>>>>>>> Emacs-orgmode@gnu.org
>>>>>>>> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>>>>>
>>>>> Footnotes: 
>>>>> [1]  
>>>>>
>>>>> diff --git a/lisp/babel/langs/ob-python.el b/lisp/babel/langs/ob-python.el
>>>>> index 2ce9e1d..29bb166 100644
>>>>> --- a/lisp/babel/langs/ob-python.el
>>>>> +++ b/lisp/babel/langs/ob-python.el
>>>>> @@ -96,7 +96,7 @@ called by `org-babel-execute-src-block'."
>>>>>  specifying a var of the same value."
>>>>>    (if (listp var)
>>>>>        (concat "[" (mapconcat #'org-babel-python-var-to-python var ", ") "]")
>>>>> -    (format "%S" var)))
>>>>> +    (if (equal var 'hline) "None" (format "%S" var))))
>>>>>  
>>>>>  (defun org-babel-python-table-or-string (results)
>>>>>    "If the results look like a list or tuple, then convert them into an
>>>>> @@ -110,7 +110,9 @@ Emacs-lisp table, otherwise return the results as a string."
>>>>>                   "\\[" "(" (replace-regexp-in-string
>>>>>                              "\\]" ")" (replace-regexp-in-string
>>>>>                                         ", " " " (replace-regexp-in-string
>>>>> -                                                 "'" "\"" results))))))
>>>>> +                                                 "'" "\""
>>>>> +						 (replace-regexp-in-string
>>>>> +						  "None" "hline" results t)))))))
>>>>>       results)))
>>>>>  
>>>>>  (defvar org-babel-python-buffers '(:default . nil))