[babel] python sessions

[babel] python sessions

[Andrea Crotti]

I wanted to use sessions in python to do some nice literate programming
and splitting functions, but it doesn't work as expected.

Here below a very simple example in python and ruby, where in ruby
everything seems to work well while in python it doesn't...
And by the way, what is that org_babel_python_eoe? I can't find it
documented anywhere in the manual...

I've read some time ago that python support for babel was a bit tricky,
are there workarounds to make it work anyway?

Or could you explain briefly what the problem is so that I can try to
contribute as soon as I finish this thing?

Thanks a lot,
Andrea

--8<---------------cut here---------------start------------->8---
#+begin_src python :session
  def var(x):
      return float(x ** 2)
  
#+end_src

#+results:
: org_babel_python_eoe

#+begin_src python :session
  def var2(x):
      return x ** 2 * var(x)
  
  var2(10)
#+end_src

#+results:
: org_babel_python_eoe

#+begin_src ruby :session
  def fun(x)
    x + 2
  end
#+end_src

#+results:
: nil

#+begin_src ruby :session
  def fun2(x)
    x + fun(x)
  end
  
  fun2 10
#+end_src

#+results:
: 22

--8<---------------cut here---------------end--------------->8---

Re: [babel] python sessions

[Eric Schulte]

Hi Andrea,

Andrea Crotti <andrea.crotti.0@gmail.com> writes:

> I wanted to use sessions in python to do some nice literate programming
> and splitting functions, but it doesn't work as expected.
>
> Here below a very simple example in python and ruby, where in ruby
> everything seems to work well while in python it doesn't...
> And by the way, what is that org_babel_python_eoe? I can't find it
> documented anywhere in the manual...
>

This string is used by Babel to indicate when to stop waiting for output
form the interactive python process and return control to the user.

>
> I've read some time ago that python support for babel was a bit tricky,
> are there workarounds to make it work anyway?
>

This is true, in addition to being a language which is dependent upon
whitespace characters, python has been tricky due to the many
independent inferior python modes (python.el, python-mode.el, etc...)
and to the fact that I personally and not very familiar with the
language.

I've just pushed up a patch which should improve upon the python session
behavior.  After this patch your example returns the following
results...

--8<---------------cut here---------------start------------->8---
#+begin_src python :session :results silent
  def var(x):
      return float(x ** 2)
#+end_src

#+begin_src python :session :result value
  def var2(x):
      return x ** 2 * var(x)
  
  var2(10)
#+end_src

#+results:
: 10000.0
--8<---------------cut here---------------end--------------->8---

Please let me know if you notice any other problematic behavior.

Thanks -- Eric

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: [babel] python sessions

[Andrea Crotti]

Eric Schulte <schulte.eric@gmail.com> writes:
>
> This is true, in addition to being a language which is dependent upon
> whitespace characters, python has been tricky due to the many
> independent inferior python modes (python.el, python-mode.el, etc...)
> and to the fact that I personally and not very familiar with the
> language.

That is indeed a problem (also in cedet for example) and I really would
like to have just one and working well python mode, not a thousand.. 

>
> I've just pushed up a patch which should improve upon the python session
> behavior.  After this patch your example returns the following
> results...
>
> #+begin_src python :session :results silent
>   def var(x):
>       return float(x ** 2)
> #+end_src
>
> #+begin_src python :session :result value
>   def var2(x):
>       return x ** 2 * var(x)
>   
>   var2(10)
> #+end_src
>
> #+results:
> : 10000.0
>

That example now works like a charm
But here still I get that string, but if I tangle the file I get the
correct result, any idea?

--8<---------------cut here---------------start------------->8---
#+begin_src python :session :tangle myset.py :results silent
  class MySetList(object):
  
      def __init__(self):
          self._set = []
  
      def add(self, el):
          if el not in self._set:
              self._set.append(el)
  
      # implementation of other typical set functions
#+end_src

#+begin_src python :session :tangle myset.py :results silent
  class MySetDict(object):
  
      def __init__(self):
          self._dic = {}
  
      def add(self, el):
          if el not in self._dic:
              # we only care about the keys
              self._dic[el] = None
#+end_src

#+begin_src python :session :tangle myset.py :results silent
  class MySetSet(object):
      def __init__(self):
          self._set = set()
      
      def add(self, el):
          self._set.add(el)
#+end_src


#+begin_src python :session :exports both :tangle myset.py
  import timeit
  import random
  
  NUM_ELS = 100
  
      
  def add_many_to_set(set_type):
      m = set_type()
      for i in range(NUM_ELS):
          m.add(i)
  
  
  def test_impl(set_type):
      to_import = """
  from __main__ import add_many_to_set
  from __main__ import %s"""
  
      name = set_type.__name__
      print("testing %s" % name)
      return timeit.timeit("add_many_to_set(%s)" % name,
                           setup=(to_import % name))
  
  
  test_impl(MySetList), test_impl(MySetDict), test_impl(MySetSet)
  
#+end_src
--8<---------------cut here---------------end--------------->8---

Re: [babel] python sessions

[Eric Schulte]

Andrea Crotti <andrea.crotti.0@gmail.com> writes:

> Eric Schulte <schulte.eric@gmail.com> writes:
>>
>> This is true, in addition to being a language which is dependent upon
>> whitespace characters, python has been tricky due to the many
>> independent inferior python modes (python.el, python-mode.el, etc...)
>> and to the fact that I personally and not very familiar with the
>> language.
>
> That is indeed a problem (also in cedet for example) and I really would
> like to have just one and working well python mode, not a thousand.. 
>

From what I hear the situation should improve in Emacs24, as there is a
ground up re-write which should contain much of the functionality of
python-mode.el with the Emacs-amenable license of python.el.

>
>>
>> I've just pushed up a patch which should improve upon the python session
>> behavior.  After this patch your example returns the following
>> results...
>>
>> #+begin_src python :session :results silent
>>   def var(x):
>>       return float(x ** 2)
>> #+end_src
>>
>> #+begin_src python :session :result value
>>   def var2(x):
>>       return x ** 2 * var(x)
>>   
>>   var2(10)
>> #+end_src
>>
>> #+results:
>> : 10000.0
>>
>
> That example now works like a charm
> But here still I get that string, but if I tangle the file I get the
> correct result, any idea?
>

The eoe string will only even appear in session output when there is no
other result returned by the code block.  I've just pushed up a small
change which should fix this situation.

The eoe will never appear in tangled code as it is part of the
interactive session evaluation.

Best -- Eric

>
> #+begin_src python :session :tangle myset.py :results silent
>   class MySetList(object):
>   
>       def __init__(self):
>           self._set = []
>   
>       def add(self, el):
>           if el not in self._set:
>               self._set.append(el)
>   
>       # implementation of other typical set functions
> #+end_src
>
> #+begin_src python :session :tangle myset.py :results silent
>   class MySetDict(object):
>   
>       def __init__(self):
>           self._dic = {}
>   
>       def add(self, el):
>           if el not in self._dic:
>               # we only care about the keys
>               self._dic[el] = None
> #+end_src
>
> #+begin_src python :session :tangle myset.py :results silent
>   class MySetSet(object):
>       def __init__(self):
>           self._set = set()
>       
>       def add(self, el):
>           self._set.add(el)
> #+end_src
>
>
> #+begin_src python :session :exports both :tangle myset.py
>   import timeit
>   import random
>   
>   NUM_ELS = 100
>   
>       
>   def add_many_to_set(set_type):
>       m = set_type()
>       for i in range(NUM_ELS):
>           m.add(i)
>   
>   
>   def test_impl(set_type):
>       to_import = """
>   from __main__ import add_many_to_set
>   from __main__ import %s"""
>   
>       name = set_type.__name__
>       print("testing %s" % name)
>       return timeit.timeit("add_many_to_set(%s)" % name,
>                            setup=(to_import % name))
>   
>   
>   test_impl(MySetList), test_impl(MySetDict), test_impl(MySetSet)
>   
> #+end_src

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: [babel] python sessions

[Andrea Crotti]

Eric Schulte <schulte.eric@gmail.com> writes:

>
> From what I hear the situation should improve in Emacs24, as there is a
> ground up re-write which should contain much of the functionality of
> python-mode.el with the Emacs-amenable license of python.el.
>

Nice to know, I use python-mode.el and emacs24, but I would vote for a
merge :)

>
> The eoe string will only even appear in session output when there is no
> other result returned by the code block.  I've just pushed up a small
> change which should fix this situation.
>
> The eoe will never appear in tangled code as it is part of the
> interactive session evaluation.
>
> Best -- Eric

I think the last patch created some problems, now it often (but not
always and I can't get why) when I execute the source block, and still I
can't make it work the previous example that I posted...

Thanks,
Andrea

Re: [babel] python sessions

[Eric Schulte]

[-- Attachment #1: Type: text/plain, Size: 1327 bytes --]

Andrea Crotti <andrea.crotti.0@gmail.com> writes:

> Eric Schulte <schulte.eric@gmail.com> writes:
>
>>
>> From what I hear the situation should improve in Emacs24, as there is a
>> ground up re-write which should contain much of the functionality of
>> python-mode.el with the Emacs-amenable license of python.el.
>>
>
> Nice to know, I use python-mode.el and emacs24, but I would vote for a
> merge :)
>
>>
>> The eoe string will only even appear in session output when there is no
>> other result returned by the code block.  I've just pushed up a small
>> change which should fix this situation.
>>
>> The eoe will never appear in tangled code as it is part of the
>> interactive session evaluation.
>>
>> Best -- Eric
>
> I think the last patch created some problems, now it often (but not
> always and I can't get why) when I execute the source block, and still I
> can't make it work the previous example that I posted...
>

I don't understand.  What do python code blocks often do after
application of the new patch?

Using this attached minimal emacs configuration [1] I was able to
evaluate the code blocks in this simple org-mode file [2] starting Emacs
with the following.


: emacs -Q -l minimal.el short.org

Perhaps your problems are related to use of python-mode.el?

Best -- Eric


Footnotes: 
[1]  minimal.el

[-- Attachment #2: minimal.el --]
[-- Type: application/emacs-lisp, Size: 410 bytes --]

[-- Attachment #3: Type: text/plain, Size: 16 bytes --]


[2]  short.org

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: short.org --]
[-- Type: text/x-org, Size: 240 bytes --]

#+begin_src python :session :results output
  def var(x):
      return float(x ** 2)
#+end_src

#+results:

#+begin_src python :session :result value
  def var2(x):
      return x ** 2 * var(x)
  
  var2(10)
#+end_src

#+results:
: 10000.0

[-- Attachment #5: Type: text/plain, Size: 48 bytes --]



-- 
Eric Schulte
http://cs.unm.edu/~eschulte/