Re: Org-babel `:hlines yes` no longer working for python

From: Christopher Allan Webber <cwebber@dustycloud.org>
To: Eric Schulte <schulte.eric@gmail.com>
Cc: emacs-orgmode@gnu.org
Subject: Re: Org-babel `:hlines yes` no longer working for python
Date: Mon, 28 Jun 2010 12:53:06 -0500	[thread overview]
Message-ID: <87iq53xcvx.fsf@dustycloud.org> (raw)
In-Reply-To: <87wrtk6nyp.fsf@gmail.com> (Eric Schulte's message of "Sun, 27 Jun 2010 16:43:42 -0700")

Hm.  I've found a bug with this patch:

#+begin_src python
return [['foo', 'bar', 'baz'], ["a", "b", "None of the above"], ['1', 2, 3]]
#+end_src

#+results:
| foo | bar | baz                |
| a   | b   | hline of the above |
| 1   | 2   | 3                  |

This also replaces the word "None" anywhere with hline, even in
strings.

"Eric Schulte" <schulte.eric@gmail.com> writes:

> Hi,
>
> OK, I've applied this patch.
>
> Christopher Allan Webber <cwebber@dustycloud.org> writes:
>
>> Eric,
>>
>> Looks good to me!  It's abusing the None type's meaning a little, but
>> I think it's acceptable enough.  (If you think of hlines as rows that
>> are not rows, you can trick yourself into thinking it is perfectly
>> pythonic :))
>>
>
> Yea, this semantic mismatch bothered me, however it looks like Python
> doesn't have anything like symbols that could be used here, and I guess
> there isn't an issue of wanting to preserve "None" for "nil" mapping
> because "nil" can be represented with an empty list "[]".
>
> Thanks for bringing this up! -- Eric
>
>>
>>  - cwebb
>>
>> "Eric Schulte" <schulte.eric@gmail.com> writes:
>>
>>> Hi Christopher,
>>>
>>> I'm certainly no Python expert, but I implemented your idea of
>>> converting "hlines" to and from "None"'s (patch below [1]), and it seems
>>> to work (under some definition of work).  See the following example with
>>> the new behavior.
>>>
>>>
>>> --8<---------------cut here---------------start------------->8---
>>> #+tblname: many-cols
>>> | a | b | c |
>>> |---+---+---|
>>> | d | e | f |
>>> |---+---+---|
>>> | g | h | i |
>>>
>>> #+source: echo-table
>>> #+begin_src python :var tab=many-cols :hlines yes
>>>   return tab
>>> #+end_src
>>>
>>> #+results: echo-table
>>> | a | b | c |
>>> |---+---+---|
>>> | d | e | f |
>>> |---+---+---|
>>> | g | h | i |
>>> --8<---------------cut here---------------end--------------->8---
>>>
>>> Please, Python people, try this out and if you like the behavior then
>>> I'll happily apply the patch.
>>>
>>> Best -- Eric
>>>
>>> Christopher Allan Webber <cwebber@dustycloud.org> writes:
>>>
>>>> Hey Eric,
>>>>
>>>> Thanks for the super helpful reply!
>>>>
>>>> Out of curiosity, is it likely that we will ever get hline support in
>>>> Python and etc?  I've been pondering how it might be done, and maybe it
>>>> could be like this, using a '|-' string instead of a list for the row:
>>>>
>>>> [['a', 'b', 'c'], '|-', ['d', 'e', 'f'], ['g', 'h', 'i']]
>>>>
>>>> Which would produce:
>>>>
>>>> | a | b | c |
>>>> |---+---+---|
>>>> | d | e | f |
>>>> | g | h | i |
>>>>
>>>> Alternately maybe the same thing could be done by abusing None:
>>>>
>>>> [['a', 'b', 'c'], None, ['d', 'e', 'f'], ['g', 'h', 'i']]
>>>>
>>>> Thoughts?
>>>>  - cwebb
>>>>
>>>> "Eric Schulte" <schulte.eric@gmail.com> writes:
>>>>
>>>>> Hi Christopher,
>>>>>
>>>>> Thanks for pointing this out, this is an error in the documentation,
>>>>> which I will update.  The code you posted should generate the error you
>>>>> have received.
>>>>>
>>>>> Currently the only language which can handle hlines is emacs-lisp, all
>>>>> other languages will result in errors like the one you pasted below.
>>>>> That's not to say that it wouldn't be possible to add hline handling to
>>>>> other languages, or to maybe do something tricky like session-based
>>>>> evaluation in which an `hlines' variable was pre-initialized to some
>>>>> value, but I digress.
>>>>>
>>>>> Note that it *is* possible to have hlines in the output, using colnames,
>>>>> e.g.
>>>>>
>>>>>
>>>>> --8<---------------cut here---------------start------------->8---
>>>>> #+tblname: A
>>>>> | a | b | c |
>>>>> |---+---+---|
>>>>> | d | e | f |
>>>>> | g | h | i |
>>>>>
>>>>> #+begin_src python :var tab=A :colnames yes
>>>>> return [[val + '*' for val in row] for row in tab]
>>>>> #+end_src
>>>>>
>>>>> #+results:
>>>>> | a  | b  | c  |
>>>>> |----+----+----|
>>>>> | d* | e* | f* |
>>>>> | g* | h* | i* |
>>>>> --8<---------------cut here---------------end--------------->8---
>>>>>
>>>>> which works because the hline, and the column names, are never made
>>>>> available to python, rather Babel holds onto them and then re-applies
>>>>> them to the source block's output.
>>>>>
>>>>> or even to have an elisp block add hlines to your results
>>>>>
>>>>>
>>>>> --8<---------------cut here---------------start------------->8---
>>>>> #+tblname: many-cols
>>>>> | a | b | c |
>>>>> |---+---+---|
>>>>> | d | e | f |
>>>>> |---+---+---|
>>>>> | g | h | i |
>>>>>
>>>>> #+source: echo-table
>>>>> #+begin_src python :var tab=many-cols
>>>>>   return tab
>>>>> #+end_src
>>>>>
>>>>> #+begin_src emacs-lisp :var table=echo-table
>>>>>   (butlast (apply #'append (mapcar (lambda (el) (list el 'hline)) table)))
>>>>> #+end_src
>>>>>
>>>>> #+results:
>>>>> | a | b | c |
>>>>> |---+---+---|
>>>>> | d | e | f |
>>>>> |---+---+---|
>>>>> | g | h | i |
>>>>> --8<---------------cut here---------------end--------------->8---
>>>>>
>>>>> Thanks for pointing this out!
>>>>>
>>>>> Best -- Eric
>>>>>
>>>>> Christopher Allan Webber <cwebber@dustycloud.org> writes:
>>>>>
>>>>>> Hello all,
>>>>>>
>>>>>> I was going through the tutorial and testing the :hlines yes feature as
>>>>>> described in the info manual.  Unfortunately, the example given no
>>>>>> longer seems to work for python:
>>>>>>
>>>>>> #+tblname: many-cols
>>>>>> | a | b | c |
>>>>>> |---+---+---|
>>>>>> | d | e | f |
>>>>>> |---+---+---|
>>>>>> | g | h | i |
>>>>>>
>>>>>> #+source: echo-table
>>>>>> #+begin_src python :var tab=many-cols :hlines yes
>>>>>>   return tab
>>>>>> #+end_src
>>>>>>
>>>>>> #+results: echo-table
>>>>>> | a | b | c |
>>>>>> | d | e | f |
>>>>>> | g | h | i |
>>>>>>
>>>>>> In the buffer *Org-Babel Error Output* I see:
>>>>>>
>>>>>> Traceback (most recent call last):
>>>>>>   File "<stdin>", line 6, in <module>
>>>>>>   File "<stdin>", line 3, in main
>>>>>> NameError: global name 'hline' is not defined
>>>>>>
>>>>>> In emacs-lisp this still seems to work though.  But I also see that in
>>>>>> emacs lisp hlines are represented by the hline symbol.  I'm guessing
>>>>>> that the python equivalent was trying to do the same thing, but no hline
>>>>>> variable exists in python?
>>>>>>
>>>>>> Thanks!
>>>>>>  - cwebb
>>>>>>
>>>>>> _______________________________________________
>>>>>> Emacs-orgmode mailing list
>>>>>> Please use `Reply All' to send replies to the list.
>>>>>> Emacs-orgmode@gnu.org
>>>>>> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>>>
>>> Footnotes: 
>>> [1]  
>>>
>>> diff --git a/lisp/babel/langs/ob-python.el b/lisp/babel/langs/ob-python.el
>>> index 2ce9e1d..29bb166 100644
>>> --- a/lisp/babel/langs/ob-python.el
>>> +++ b/lisp/babel/langs/ob-python.el
>>> @@ -96,7 +96,7 @@ called by `org-babel-execute-src-block'."
>>>  specifying a var of the same value."
>>>    (if (listp var)
>>>        (concat "[" (mapconcat #'org-babel-python-var-to-python var ", ") "]")
>>> -    (format "%S" var)))
>>> +    (if (equal var 'hline) "None" (format "%S" var))))
>>>  
>>>  (defun org-babel-python-table-or-string (results)
>>>    "If the results look like a list or tuple, then convert them into an
>>> @@ -110,7 +110,9 @@ Emacs-lisp table, otherwise return the results as a string."
>>>                   "\\[" "(" (replace-regexp-in-string
>>>                              "\\]" ")" (replace-regexp-in-string
>>>                                         ", " " " (replace-regexp-in-string
>>> -                                                 "'" "\"" results))))))
>>> +                                                 "'" "\""
>>> +						 (replace-regexp-in-string
>>> +						  "None" "hline" results t)))))))
>>>       results)))
>>>  
>>>  (defvar org-babel-python-buffers '(:default . nil))