Sage Math - Maybe some inspiration for org-babel

Sage Math - Maybe some inspiration for org-babel

[Darlan Cavalcante Moreira]

Hello org-users

I have been reading with attention the E-Mails about org-babel here on the list
and decided to play with it a little this weekend. It reminded me about another
great piece of software called "Sage Math" (Some screencasts here ->
http://www.sagemath.org/help-video.html) that I have tested sometime ago.

"Sage is a free open-source mathematics software system licensed under the
GPL. It combines the power of many existing open-source packages into a common
Python-based interface."

Although Sage and org-babel are very different, I think that Sage may be a good
source of inspiration for org-babel. Probably the most practical way of using
sage is though the notebook: a front-end that runs inside the browser and
communicates with the sage server (may be remote or local).

In the sage notebook one creates worksheets. Each worksheet starts with a text
area where you type the code. The main language is python, but others may be
used if specified with a comment. When the code is evaluated any output is shown
in addition to the last value. Therefore, the code

,----
| print "hello"
| 10
| print "world"
| 9
| 15
`----
will show
,----
| hello
| world
| 15
`----

This is similar to a "combination" of :results output and :results value in
org-babel. A really nice feature is that if an image is created by the last
command in the block it is automatically shown. Therefore, it doesn't matter
which library was used to generate the plot as long as it creates an image (for
instance, plotting to a file with matplotlib will shown the plot when the code
is evaluated). Maybe the same approach could be used in org-mode.

In the text area, RET only creates a new line, Shift+RET evaluates the code in
the text area and creates a new one below the output keeping all the variables
similar to the multiple source blocks within a single session in org-babel.

Two things I find interesting is that you can easily evaluate all blocks in the
worksheet (org-mode already has org-babel-execute-buffer) and if any block
starts with the comment "%hide" the code is not shown, only its output. I think
that a header argument ":hide" for a source block could be useful.


That's my two cents. It is really amazing what you can do with org-mode and
org-babel and I'm not saying in that org-babel should became more similar to
sage. I only think that getting some inspiration from sage could be nice.

Regards,

Darlan

Re: Sage Math - Maybe some inspiration for org-babel

[Eric Schulte]

Hi Darlan,

Thanks for pointing out this interesting tool.

my comment are inline below

Darlan Cavalcante Moreira <darcamo@gmail.com> writes:

> Hello org-users
>
> I have been reading with attention the E-Mails about org-babel here on the list
> and decided to play with it a little this weekend. It reminded me about another
> great piece of software called "Sage Math" (Some screencasts here ->
> http://www.sagemath.org/help-video.html) that I have tested sometime ago.
>
> "Sage is a free open-source mathematics software system licensed under the
> GPL. It combines the power of many existing open-source packages into a common
> Python-based interface."
>

This is the first I'd heard about sage, after watching the first intro
video it looks like a very relevant tool.

It seems like the biggest difference between Sage and Org-babel is that
sage wraps all of it's component-languages into a single top level
environment (which is basically python) and somehow puts the
functionality of each language behind an API of function/library calls
(I may be misunderstanding here).  Where as babel doesn't provide any
unified top level, but rather evaluates each language in it's own
separate environment and allows it's supported languages to share
results via a lowest-common-denominator of elisp and org-mode constructs
(like tables).

>
> Although Sage and org-babel are very different, I think that Sage may be a good
> source of inspiration for org-babel. Probably the most practical way of using
> sage is though the notebook: a front-end that runs inside the browser and
> communicates with the sage server (may be remote or local).
>

As soon as someone implements an emacs-run browser which can render
org-mode files on demand and supports an elisp REPL we should be very
close to similar functionality in Org-babel. :)

>
> In the sage notebook one creates worksheets. Each worksheet starts with a text
> area where you type the code. The main language is python, but others may be
> used if specified with a comment. When the code is evaluated any output is shown
> in addition to the last value. Therefore, the code
>
> ,----
> | print "hello"
> | 10
> | print "world"
> | 9
> | 15
> `----
> will show
> ,----
> | hello
> | world
> | 15
> `----
>
> This is similar to a "combination" of :results output and :results value in
> org-babel. A really nice feature is that if an image is created by the last
> command in the block it is automatically shown. Therefore, it doesn't matter
> which library was used to generate the plot as long as it creates an image (for
> instance, plotting to a file with matplotlib will shown the plot when the code
> is evaluated). Maybe the same approach could be used in org-mode.
>

I'd be interested to see how sage captures images created by it's
sub-languages.  It looks to me like sage has tighter coupling with it's
components (perhaps making more requirements of it's sub-languages) than
org-babel.  I know that for languages (like R) which support changing
the default graphical output to a file it would be possible (in fact I
believe Dan may already have this implemented to some degree) to
automatically capture any image output and place it in the org-mode
buffer.

>
> In the text area, RET only creates a new line, Shift+RET evaluates the code in
> the text area and creates a new one below the output keeping all the variables
> similar to the multiple source blocks within a single session in org-babel.
>
> Two things I find interesting is that you can easily evaluate all blocks in the
> worksheet (org-mode already has org-babel-execute-buffer) and if any block
> starts with the comment "%hide" the code is not shown, only its output. I think
> that a header argument ":hide" for a source block could be useful.
>

There have been a couple of people expressing the desire to hide the
body of source-code blocks recently.  It seems that there is some need
here which is not met by any of the existing source-block hiding methods
- pressing tab while at the head of the source-code block
- adding #+STARTUP hideblocks to the top of an org-mode file
- saving source-code blocks as library-of-babel functions

what behavior would you suggest results from a :hide header argument?

>
> That's my two cents. It is really amazing what you can do with org-mode and
> org-babel and I'm not saying in that org-babel should became more similar to
> sage. I only think that getting some inspiration from sage could be nice.
>

I agree completely, I'm sure that there are lessons to be learned from a
project with such similarities.  Now it's just a matter of getting to
know a little bit more about Sage.

Thanks -- Eric

>
> Regards,
>
> Darlan
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Remember: use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Sage Math - Maybe some inspiration for org-babel

[Darlan Cavalcante Moreira]

Hi Eric,

My comments are also inline

At Wed, 21 Oct 2009 09:11:50 -0600,
Eric Schulte wrote:
> 
> Hi Darlan,
> 
> Thanks for pointing out this interesting tool.
> 
> my comment are inline below
> 
> Darlan Cavalcante Moreira <darcamo@gmail.com> writes:
> 
> > Hello org-users
> >
> > I have been reading with attention the E-Mails about org-babel here on the list
> > and decided to play with it a little this weekend. It reminded me about another
> > great piece of software called "Sage Math" (Some screencasts here ->
> > http://www.sagemath.org/help-video.html) that I have tested sometime ago.
> >
> > "Sage is a free open-source mathematics software system licensed under the
> > GPL. It combines the power of many existing open-source packages into a common
> > Python-based interface."
> >
> 
> This is the first I'd heard about sage, after watching the first intro
> video it looks like a very relevant tool.
> 
> It seems like the biggest difference between Sage and Org-babel is that
> sage wraps all of it's component-languages into a single top level
> environment (which is basically python) and somehow puts the
> functionality of each language behind an API of function/library calls
> (I may be misunderstanding here).  Where as babel doesn't provide any
> unified top level, but rather evaluates each language in it's own
> separate environment and allows it's supported languages to share
> results via a lowest-common-denominator of elisp and org-mode constructs
> (like tables).
>

You understood right. Sage tries to integrate the different components in a
natural way with this wrapping, but it is also possible to use the components
directly as in org-mode by specifying the language with the comment %LANGUAGE at
the beginning of the text-area. However, I not sure how sage can exchange
information from different languages in this way (there is probably a way).

> >
> > Although Sage and org-babel are very different, I think that Sage may be a good
> > source of inspiration for org-babel. Probably the most practical way of using
> > sage is though the notebook: a front-end that runs inside the browser and
> > communicates with the sage server (may be remote or local).
> >
> 
> As soon as someone implements an emacs-run browser which can render
> org-mode files on demand and supports an elisp REPL we should be very
> close to similar functionality in Org-babel. :)
>

With emacs you never know. Maybe there is an elisp_REPL.el somewhere in the
internet. :)

> >
> > In the sage notebook one creates worksheets. Each worksheet starts with a text
> > area where you type the code. The main language is python, but others may be
> > used if specified with a comment. When the code is evaluated any output is shown
> > in addition to the last value. Therefore, the code
> >
> > ,----
> > | print "hello"
> > | 10
> > | print "world"
> > | 9
> > | 15
> > `----
> > will show
> > ,----
> > | hello
> > | world
> > | 15
> > `----
> >
> > This is similar to a "combination" of :results output and :results value in
> > org-babel. A really nice feature is that if an image is created by the last
> > command in the block it is automatically shown. Therefore, it doesn't matter
> > which library was used to generate the plot as long as it creates an image (for
> > instance, plotting to a file with matplotlib will shown the plot when the code
> > is evaluated). Maybe the same approach could be used in org-mode.
> >
> 
> I'd be interested to see how sage captures images created by it's
> sub-languages.  It looks to me like sage has tighter coupling with it's
> components (perhaps making more requirements of it's sub-languages) than
> org-babel.  I know that for languages (like R) which support changing
> the default graphical output to a file it would be possible (in fact I
> believe Dan may already have this implemented to some degree) to
> automatically capture any image output and place it in the org-mode
> buffer.
> 
> >
> > In the text area, RET only creates a new line, Shift+RET evaluates the code in
> > the text area and creates a new one below the output keeping all the variables
> > similar to the multiple source blocks within a single session in org-babel.
> >
> > Two things I find interesting is that you can easily evaluate all blocks in the
> > worksheet (org-mode already has org-babel-execute-buffer) and if any block
> > starts with the comment "%hide" the code is not shown, only its output. I think
> > that a header argument ":hide" for a source block could be useful.
> >
> 
> There have been a couple of people expressing the desire to hide the
> body of source-code blocks recently.  It seems that there is some need
> here which is not met by any of the existing source-block hiding methods
> - pressing tab while at the head of the source-code block
> - adding #+STARTUP hideblocks to the top of an org-mode file
> - saving source-code blocks as library-of-babel functions
> 
> what behavior would you suggest results from a :hide header argument?

My suggestion is that if a source block has the :hide header argument it should
be closed by default as if the user had pressed tab. The user could then press
tab at the "#BEGIN_SRC ..." line to show the content of the block and, maybe,
the block could be closed again if the cursor leaves the block. This can be
useful for other blocks as well.

Let me use the sage as inspiration again. In sage when you put the cursor above
the text-area a blue line is showed. If you click on this line a new text-area
is created above the current one but the really nice trick is that shift+click
creates an HTML text-area (with a tool-bar with bold, italic, centralized
buttons, etc). Then you can create a worksheet in sage such as
                           +-----------------------+
                           |    Some HTML text     |
                           | describing a problem  |
                           |    Use jsMath for     |
                           |    nice equations     |
                           |   with latex syntax   |
                           +-----------------------+
                           |%hide                  |
                           | some hidden code that |
                           | generates some output |
                           +-----------------------+
                           |    solution of the    |
                           |   problem that was    |
                           |calculated by the code |
                           +-----------------------+

That is, when you are only interested in the output and not in the code itself
you use the %hide comment. I'm not sure what is the best way to handle the :hide
argument when exporting. Maybe it is better to use it simple for folding in the
org buffer and use another option to control export of the block. For example, a
":noexport" option in addition to ":hide" and a buffer option "don't export
hidden bocks" in case the user wants to avoid exporting all blocks with
the :hide option.


> 
> >
> > That's my two cents. It is really amazing what you can do with org-mode and
> > org-babel and I'm not saying in that org-babel should became more similar to
> > sage. I only think that getting some inspiration from sage could be nice.
> >
> 
> I agree completely, I'm sure that there are lessons to be learned from a
> project with such similarities.  Now it's just a matter of getting to
> know a little bit more about Sage.
> 
> Thanks -- Eric
> 
> >
> > Regards,
> >
> > Darlan
> >
> >
> > _______________________________________________
> > Emacs-orgmode mailing list
> > Remember: use `Reply All' to send replies to the list.
> > Emacs-orgmode@gnu.org
> > http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Regards,

Darlan

Re: Sage Math - Maybe some inspiration for org-babel

[Eric Schulte]

Darlan Cavalcante Moreira <darcamo@gmail.com> writes:

> Hi Eric,
>
> My comments are also inline
>
> At Wed, 21 Oct 2009 09:11:50 -0600,
> Eric Schulte wrote:

[...]

>> what behavior would you suggest results from a :hide header argument?
>
> My suggestion is that if a source block has the :hide header argument it should
> be closed by default as if the user had pressed tab. The user could then press
> tab at the "#BEGIN_SRC ..." line to show the content of the block and, maybe,
> the block could be closed again if the cursor leaves the block. This can be
> useful for other blocks as well.
>

That seems very reasonable.  I'll pop this onto the TODO stack for
org-babel (meaning hopefully something will happen at some time).

>
> Let me use the sage as inspiration again. In sage when you put the cursor above
> the text-area a blue line is showed. If you click on this line a new text-area
> is created above the current one but the really nice trick is that shift+click
> creates an HTML text-area (with a tool-bar with bold, italic, centralized
> buttons, etc). Then you can create a worksheet in sage such as
>                            +-----------------------+
>                            |    Some HTML text     |
>                            | describing a problem  |
>                            |    Use jsMath for     |
>                            |    nice equations     |
>                            |   with latex syntax   |
>                            +-----------------------+
>                            |%hide                  |
>                            | some hidden code that |
>                            | generates some output |
>                            +-----------------------+
>                            |    solution of the    |
>                            |   problem that was    |
>                            |calculated by the code |
>                            +-----------------------+
>
> That is, when you are only interested in the output and not in the code itself
> you use the %hide comment. I'm not sure what is the best way to handle the :hide
> argument when exporting. Maybe it is better to use it simple for folding in the
> org buffer and use another option to control export of the block.

yes, I would agree that we should restrict the :hide argument to folding
in the org-mode buffer.  Currently there is an :exports header argument
which takes the self-explanatory values of
- none
- code
- results
- both

> For example, a ":noexport" option in addition to ":hide" and a buffer
> option "don't export hidden bocks" in case the user wants to avoid
> exporting all blocks with the :hide option.
>
>
>> 
>> >
>> > That's my two cents. It is really amazing what you can do with org-mode and
>> > org-babel and I'm not saying in that org-babel should became more similar to
>> > sage. I only think that getting some inspiration from sage could be nice.
>> >
>> 
>> I agree completely, I'm sure that there are lessons to be learned from a
>> project with such similarities.  Now it's just a matter of getting to
>> know a little bit more about Sage.
>> 
>> Thanks -- Eric
>> 
>> >
>> > Regards,
>> >
>> > Darlan
>> >
>> >
>> > _______________________________________________
>> > Emacs-orgmode mailing list
>> > Remember: use `Reply All' to send replies to the list.
>> > Emacs-orgmode@gnu.org
>> > http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>
> Regards,
>
> Darlan