Re: Possible Calc support for Org-Babel?

Re: Possible Calc support for Org-Babel?

[orgmode]

Hi,

addition of babel support for calc is nice.  However calc also has  
"embedded mode".  that means active calc documents can easily be  
embedded in org documents (and also latex, ...).  It is different from  
babel, but in some instances it may work better.

Re: Re: Possible Calc support for Org-Babel?

[Eric S Fraga]

"orgmode@h-rd.org" <orgmode@h-rd.org> writes:

> Hi,
>
> addition of babel support for calc is nice.  However calc also has
> "embedded mode".  that means active calc documents can easily be
> embedded in org documents (and also latex, ...).  It is different from
> babel, but in some instances it may work better.

Indeed.  Embedded mode is excellent and there is some overlap with what
babel provides.  I like them both.
-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1
: using Org-mode version 7.4 (release_7.4.576.g99675.dirty)

Re: Possible Calc support for Org-Babel?

[Eric S Fraga]

Forgot to CC the list et al.

"Eric Schulte" <schulte.eric@gmail.com> writes:

> Eric S Fraga <ucecesf@ucl.ac.uk> writes:

[...]

>> I now wonder if it would be worthwhile discussing the choices you have
>> made regarding stack versus algebraic evaluation.  I would rather have
>> the quote mean an algebraic expression, just from the simple reason that
>> these will be longer than stack operations and so the overhead of a
>> quote is smaller as a percentage of keystrokes...
>>
>
> Hmm, one point against prefixing the algebraic expressions is that they
> would be more likely to be used by themselves in an inline code block.

Yes, I guess so.  My initial thoughts were that we have plenty of
algebraic languages available already through babel (octave, R, python,
...) so why not support a stack based one more directly.   However, the
real benefit of calc is that it is *emacs* and not external!  Algebraic
is more natural to most people so making it easy for them to express
themselves makes sense.

-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 23.2.1
: using Org-mode version 7.02trans (release_7.3.10.g7f79.dirty)

Re: Possible Calc support for Org-Babel?

[Eric Schulte]

Alright,

I've just pushed up support for variables.  The following should all
work as expected (or at least as I would expect them to).  Note that
most all of these blocks would have an effect on the calc stack.  I
think the next step would be to allow different sessions to specify
different stacks.

** playing with calc support
#+begin_src emacs-lisp :results silent
  (require 'ob-calc)
#+end_src

#+begin_src calc :var some=8
  some
  some
  '*
  1
  '+
#+end_src

#+results:
: 65

#+begin_src calc
  2*(8+8)
#+end_src

#+results:
: 32

#+begin_src calc
  2*e
#+end_src

#+results:
: 5.43656365692

#+begin_src calc :var something=9
  2*something
#+end_src

#+results:
: 18

There are very likely some bugs, and as always I'm eager for a calc
power user to show me the light of how this support could be make more
"calc idiomatic".

Cheers -- Eric

Eric S Fraga <ucecesf@ucl.ac.uk> writes:

> Forgot to CC the list et al.
>
> "Eric Schulte" <schulte.eric@gmail.com> writes:
>
>> Eric S Fraga <ucecesf@ucl.ac.uk> writes:
>
> [...]
>
>>> I now wonder if it would be worthwhile discussing the choices you have
>>> made regarding stack versus algebraic evaluation.  I would rather have
>>> the quote mean an algebraic expression, just from the simple reason that
>>> these will be longer than stack operations and so the overhead of a
>>> quote is smaller as a percentage of keystrokes...
>>>
>>
>> Hmm, one point against prefixing the algebraic expressions is that they
>> would be more likely to be used by themselves in an inline code block.
>
> Yes, I guess so.  My initial thoughts were that we have plenty of
> algebraic languages available already through babel (octave, R, python,
> ...) so why not support a stack based one more directly.   However, the
> real benefit of calc is that it is *emacs* and not external!  Algebraic
> is more natural to most people so making it easy for them to express
> themselves makes sense.

Re: Possible Calc support for Org-Babel?

[Sébastien Vauban]

Eric,

> I've just pushed up support for variables. The following should all work as
> expected (or at least as I would expect them to).
>
> ** playing with calc support
> #+begin_src emacs-lisp :results silent
>   (require 'ob-calc)
> #+end_src

There must be something stupid in my config, but even a simple one like the
following does not work for me:

#+begin_src calc
1
2
'+
#+end_src

Error is:

#+begin_src emacs-lisp
executing Calc code block...
calc-push-list: Symbol's value as variable is void: calc-command-flags
#+end_src

Though, I confirm you that I have loaded both =calc= and =ob-calc=... Any idea?


> There are very likely some bugs, and as always I'm eager for a calc power
> user to show me the light of how this support could be make more "calc
> idiomatic".

Be hopeless: I won't be that one ;-(

Best regards,
  Seb

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: Possible Calc support for Org-Babel?

[Eric Schulte]

Sébastien Vauban <wxhgmqzgwmuf@spammotel.com> writes:

> There must be something stupid in my config, but even a simple one like the
> following does not work for me:
>
> #+begin_src calc
> 1
> 2
> '+
> #+end_src
>
> Error is:
>
> #+begin_src emacs-lisp
> executing Calc code block...
> calc-push-list: Symbol's value as variable is void: calc-command-flags
> #+end_src
>
> Though, I confirm you that I have loaded both =calc= and =ob-calc=... Any idea?
>

That variable is defined in my version of calc.el (distributed with the
latest version of Emacs from git).  Is that variable not defined in your
calc.el file?  Maybe the solution is to upgrade your calc.

Re: Possible Calc support for Org-Babel?

[Sébastien Vauban]

Eric,

"Eric Schulte" wrote:
> Sébastien Vauban <wxhgmqzgwmuf-geNee64TY+gS+FvcfC7Uqw@public.gmane.org> writes:
>
>> There must be something stupid in my config, but even a simple one like the
>> following does not work for me:
>>
>> #+begin_src calc
>> 1
>> 2
>> '+
>> #+end_src
>>
>> Error is:
>>
>> #+begin_src emacs-lisp
>> executing Calc code block...
>> calc-push-list: Symbol's value as variable is void: calc-command-flags
>> #+end_src
>>
>> Though, I confirm you that I have loaded both =calc= and =ob-calc=... Any idea?
>
> That variable is defined in my version of calc.el (distributed with the
> latest version of Emacs from git).  Is that variable not defined in your
> calc.el file?  Maybe the solution is to upgrade your calc.

I have "GNU Emacs 23.1.50.1 (i386-mingw-nt5.1.2600) of 2009-10-14 on
LENNART-69DE564 (patched)" (known as EmacsW32, latest version that Lennart put
online).

Library is file "c:/Program Files/Emacs/emacs/lisp/calc/calc.elc", and I see
(on line 1544 of the =.el= file):

#+begin_src emacs-lisp
(defvar calc-command-flags)
#+end_src

ELC seems well compiled from that EL, as times report:

--8<---------------cut here---------------start------------->8---
  -rw-rw-rw-  1 Fabrice Aucun 138805 2009-10-14 02:45 calc.el
  -rw-rw-rw-  1 Fabrice Aucun 103841 2009-10-14 02:46 calc.elc
--8<---------------cut here---------------end--------------->8---

Setting it myself to nil -- I have no idea why I must do this! -- :

#+begin_src emacs-lisp
(setq calc-command-flags nil)
#+end_src

has the effect that many examples now work.

Not all, though. For example,

#+begin_src calc
2*e
#+end_src

returns =nil=:

#+results:
: nil

Best regards,
  Seb

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: Possible Calc support for Org-Babel?

[Eric Schulte]

Sébastien Vauban <wxhgmqzgwmuf@spammotel.com> writes:

> Eric,
>
> "Eric Schulte" wrote:
>> Sébastien Vauban <wxhgmqzgwmuf@spammotel.com> writes:
>>
>>> There must be something stupid in my config, but even a simple one like the
>>> following does not work for me:
>>>
>>> #+begin_src calc
>>> 1
>>> 2
>>> '+
>>> #+end_src
>>>
>>> Error is:
>>>
>>> #+begin_src emacs-lisp
>>> executing Calc code block...
>>> calc-push-list: Symbol's value as variable is void: calc-command-flags
>>> #+end_src
>>>
>>> Though, I confirm you that I have loaded both =calc= and =ob-calc=... Any idea?
>>
>> That variable is defined in my version of calc.el (distributed with the
>> latest version of Emacs from git).  Is that variable not defined in your
>> calc.el file?  Maybe the solution is to upgrade your calc.
>
> I have "GNU Emacs 23.1.50.1 (i386-mingw-nt5.1.2600) of 2009-10-14 on
> LENNART-69DE564 (patched)" (known as EmacsW32, latest version that Lennart put
> online).
>
> Library is file "c:/Program Files/Emacs/emacs/lisp/calc/calc.elc", and I see
> (on line 1544 of the =.el= file):
>
> #+begin_src emacs-lisp
> (defvar calc-command-flags)
> #+end_src
>
> ELC seems well compiled from that EL, as times report:
>
>   -rw-rw-rw-  1 Fabrice Aucun 138805 2009-10-14 02:45 calc.el
>   -rw-rw-rw-  1 Fabrice Aucun 103841 2009-10-14 02:46 calc.elc
>
> Setting it myself to nil -- I have no idea why I must do this! -- :
>
> #+begin_src emacs-lisp
> (setq calc-command-flags nil)
> #+end_src
>

Hmm, that is weird.  I guess you could search for the text variable name
in your calc.el file, but I don't know how that would help you.  Since
I'm requiring calc from ob-calc.el, and I am only calling existing calc
functions I'm going to treat this as a calc bug (i.e. beyond my scope).

>
> has the effect that many examples now work.
>
> Not all, though. For example,
>
> #+begin_src calc
> 2*e
> #+end_src
>
> returns =nil=:
>
> #+results:
> : nil
>

That is the only example I gave that uses a calc default variable.
Maybe this is also a difference in our calc implementations?

Best -- Eric

>
> Best regards,
>   Seb

Possible Calc support for Org-Babel?

[Matthew Oesting]

A few of us at my institution have started using CALC in our documents (not embedded, which is far too clumsy for most of us, but C-x * u and simple embedded phrases, often to the tune of several pages) to use Emacs text files rather like Maple and Mathematica files.  We're starting to use GIT and OrgMode together to format the work and save versions and forks; while there are many CAS packages that can be used to write out memos and reports, the power of Elisp/Emacs was simply too good to pass up.

It appears that Calc syntax can be used to assign formulas to spreadsheets, but the result is difficult to read, and unsuitable for a sequence of several dozen lines.  Similarly, using embedded Ruby-symbolic code in the text produces excellent results, but we lose the ability to take advantage of Calc, which is significantly stronger for our purposes.  Other symbolic packages require more time to learn and are difficult to implement across all of our various platforms, whereas Calc is a universal and extremely flexible tool for this work.

My questions are these:
  * Has there been implemented, or would it be at all difficult to implement, a 'calc' language functionality for Org-Babel allowing the following manner of text:

> #+begin_src calc :results output 
> Fish = 2
> Dog = 2 * Fish
> 
> sqrt( Dog ) =>
> sqrt( 4 ) =>
> #+end_src
> 
> #+results:
> : sqrt( Dog ) => 2
> : sqrt( 4 ) = 2

The goal of this exercise would simply be to cut out the text from the code block, feed it to a buffer, run the Emacs Calc mode on it, and then output the appropriate results where => occurs.

 * Has there been implemented, or would it be at all difficult to implement, a 'running tag' approach to the Calc mode, essentially a #+CALC directive allowing the following manner of text, preferably with auto-update:

> This is a discussion.
>
> #+CALC: Fish = 2
>
> Fish deserves significant commentary; however, we expect that there is some command
>  that would allow us to update the entire buffer's #+CALC sequence.  Thus:
> 
> #+CALC: Dog = 2 * Fish
> #+CALC: sqrt( Dog ) =>
> #+results:
> : sqrt( Dog ) => 2
>
> If you need a different result, tweak the 'Fish =' line, above, and request a re-sequencing,
>  either via C-c C-c or via some M-x command, resulting in behavior functionally similar to
> C-x * u.

The goal of this exercise would be, upon updating, to strip all lines beginning with #+CALC and feed them to a buffer, inserting the results of => statements where the respective line occurs.  The benefit of this method over the previous is that it allows for assignments (:=) enduring across entire files (though the same could be accomplished if one SRC_BEGIN block could somehow call another.)

In conclusion, it seems to me that I cannot possibly be the first person to have done research in OrgMode, and that these problems were likely solved long ago.  When searching the manual, however, I find nothing that produces a distinctly clear and encapsulated Calc embed in OrgMode.  Do these exist, and if not, would it require mere hours of time to implement them, or weeks?  Is there a tutorial on writing OrgMode extensions?

- M

Re: Possible Calc support for Org-Babel?

[Eric Schulte]

Hi Matthew,

Matthew Oesting <oestingm@me.com> writes:

> A few of us at my institution have started using CALC in our documents
> (not embedded, which is far too clumsy for most of us, but C-x * u and
> simple embedded phrases, often to the tune of several pages) to use
> Emacs text files rather like Maple and Mathematica files.  We're
> starting to use GIT and OrgMode together to format the work and save
> versions and forks; while there are many CAS packages that can be used
> to write out memos and reports, the power of Elisp/Emacs was simply
> too good to pass up.
>
> It appears that Calc syntax can be used to assign formulas to
> spreadsheets, but the result is difficult to read, and unsuitable for
> a sequence of several dozen lines.  Similarly, using embedded
> Ruby-symbolic code in the text produces excellent results, but we lose
> the ability to take advantage of Calc, which is significantly stronger
> for our purposes.  Other symbolic packages require more time to learn
> and are difficult to implement across all of our various platforms,
> whereas Calc is a universal and extremely flexible tool for this work.
>
> My questions are these:
>   * Has there been implemented, or would it be at all difficult to
>   implement, a 'calc' language functionality for Org-Babel allowing
>   the following manner of text:
>

A Calc language has not yet been implemented, however (although I'd have
to look at Calc's support for running series of commands) I do not think
it would be difficult to implement such functionality.  In fact I think
this is a very good idea.

If you're feeling brave, it may make sense to look at a couple of the
shorter examples of languages supported by Babel (e.g. ob-dot.el) and
try to adapt their functions for working with Calc as you describe below
(see below for more information on implementing a calc interface).

>
>> #+begin_src calc :results output 
>> Fish = 2
>> Dog = 2 * Fish
>> 
>> sqrt( Dog ) =>
>> sqrt( 4 ) =>
>> #+end_src
>> 
>> #+results:
>> : sqrt( Dog ) => 2
>> : sqrt( 4 ) = 2
>
> The goal of this exercise would simply be to cut out the text from the
> code block, feed it to a buffer, run the Emacs Calc mode on it, and
> then output the appropriate results where => occurs.
>

Although I'm not familiar with using Calc as anything more than a 1-off
calculator in the bottom of the frame (i.e. M-x calc) this sounds like a
good approach to using calc to execute code blocks.

>
>  * Has there been implemented, or would it be at all difficult to
>  implement, a 'running tag' approach to the Calc mode, essentially a
>  #+CALC directive allowing the following manner of text, preferably
>  with auto-update:
>
>> This is a discussion.
>>
>> #+CALC: Fish = 2
>>
>> Fish deserves significant commentary; however, we expect that there
>> is some command that would allow us to update the entire buffer's
>> #+CALC sequence.  Thus:
>> 
>> #+CALC: Dog = 2 * Fish
>> #+CALC: sqrt( Dog ) =>
>> #+results:
>> : sqrt( Dog ) => 2
>>
>> If you need a different result, tweak the 'Fish =' line, above, and
>> request a re-sequencing, either via C-c C-c or via some M-x command,
>> resulting in behavior functionally similar to C-x * u.
>

I believe this may be much more easily implemented using the "session"
features of Org-mode code blocks, possibly in combination with the
inline code block syntax.  Please see [1] for much more information on
working with code blocks in Org-mode.

>
> The goal of this exercise would be, upon updating, to strip all lines
> beginning with #+CALC and feed them to a buffer, inserting the results
> of => statements where the respective line occurs.  The benefit of
> this method over the previous is that it allows for assignments (:=)
> enduring across entire files (though the same could be accomplished if
> one SRC_BEGIN block could somehow call another.)
>

It is currently possible for one source block to call another, see the
chapter of the manual linked above for details.

>
> In conclusion, it seems to me that I cannot possibly be the first
> person to have done research in OrgMode, and that these problems were
> likely solved long ago.  When searching the manual, however, I find
> nothing that produces a distinctly clear and encapsulated Calc embed
> in OrgMode.

You are certainly not the first to use Org-mode as a research platform,
however you may be the first calc power-user to attempt to combine calc
with Org-mode.

> Do these exist, and if not, would it require mere hours of time to
> implement them, or weeks?  Is there a tutorial on writing OrgMode
> extensions?
>

Using the Babel code block support I would anticipate that this would
take more on the order of hours rather than weeks (assuming some elisp
fluency).  See the existing template file for adding support for a new
language to Org-mode.

http://repo.or.cz/w/Worg.git/blob/HEAD:/org-contrib/babel/ob-template.el

This process should be significantly simplified by the fact that Calc is
an Emacs command (rather than an external utility).

I would be happy to help you in implementing an ob-calc.el file, and if
I find time I may take a stab at it myself.

Best -- Eric

>
> - M
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Footnotes: 
[1]  http://orgmode.org/manual/Working-With-Source-Code.html#Working-With-Source-Code

Re: Possible Calc support for Org-Babel?

[Eric Schulte]

"Eric Schulte" <schulte.eric@gmail.com> writes:

>
> Although I'm not familiar with using Calc as anything more than a 1-off
> calculator in the bottom of the frame (i.e. M-x calc) this sounds like a
> good approach to using calc to execute code blocks.
>

Did I mention I'm not familiar with Calc.

I've thrown together a very naive first pass at a function for
evaluating calc code blocks.  This inverts the normal calc (as I
understand it) use of ' prefixes and assumes that every line is an
algebraic expression unless that line is prefixed with a ' in which case
it is taken as a stack operation.

This *does* change the value of the stack, allowing multiple code blocks
to collaborate, in effect treating the stack as a session.  I'd be
interested to hear what real calc users think of this approach.

Best -- Eric

to use this evaluate the following function, and then try the subsequent
code blocks

evaluate this code block to add support for calc code blocks
#+begin_src emacs-lisp
  (defun org-babel-execute:calc (body params)
    "Execute a block of calc code with Babel."
    (mapcar
     (lambda (line)
       (when (> (length line) 0)
         (if (string= "'" (substring line 0 1))
             (funcall (lookup-key calc-mode-map (substring line 1)) nil)
           (calc-push-list (list (math-read-number (calc-eval line)))))))
     (split-string body "[\n\r]"))
    (calc-eval (calc-top 1)))
#+end_src

This block pushes 1 and 2 on the stack, then adds them
#+begin_src calc
  1
  2
  '+
#+end_src

This block evaluates 3^3 with calc pushing the result on the stack and
returning it into the Org-mode buffer
#+begin_src calc
  3^3
#+end_src

This block evaluates (2+2)*4 pushing the result on the stack, it then
calls calc-plus adding the top two elements on the stack (one of which
is left over from the previous code block).
#+begin_src calc
  (2+2)*4
  '+
#+end_src

Re: Possible Calc support for Org-Babel?

[Sébastien Vauban]

Hi Eric,

"Eric Schulte" wrote:
> I've thrown together a very naive first pass at a function for evaluating
> calc code blocks. This inverts the normal calc (as I understand it) use of '
> prefixes and assumes that every line is an algebraic expression unless that
> line is prefixed with a ' in which case it is taken as a stack operation.
>
> This *does* change the value of the stack, allowing multiple code blocks to
> collaborate, in effect treating the stack as a session. I'd be interested to
> hear what real calc users think of this approach.
>
> Best -- Eric
>
> to use this evaluate the following function, and then try the subsequent
> code blocks
>
> evaluate this code block to add support for calc code blocks
> #+begin_src emacs-lisp
>   (defun org-babel-execute:calc (body params)
>     "Execute a block of calc code with Babel."
>     (mapcar
>      (lambda (line)
>        (when (> (length line) 0)
>          (if (string= "'" (substring line 0 1))
>              (funcall (lookup-key calc-mode-map (substring line 1)) nil)
>            (calc-push-list (list (math-read-number (calc-eval line)))))))
>      (split-string body "[\n\r]"))
>     (calc-eval (calc-top 1)))
> #+end_src

Tried to run the following:

> This block pushes 1 and 2 on the stack, then adds them
> #+begin_src calc
>   1
>   2
>   '+
> #+end_src

--8<---------------cut here---------------start------------->8---
executing Calc code block...
if: Symbol's function definition is void: calc-push-list
--8<---------------cut here---------------end--------------->8---

Required =calc= by calling =C-x * *=:

--8<---------------cut here---------------start------------->8---
+-> Requiring `calc-macs'
+-> Requiring `calc-macs'...done
Loading c:/Program Files/Emacs-23/emacs/lisp/calc/calc-loaddefs.el...
C-x *  (Type ? for a list of Calc options)
+-> Requiring `calc-ext'
  +-> Requiring `calc' (already loaded)
  +-> Requiring `calc-macs' (already loaded)
+-> Requiring `calc-ext'...done
+-> Requiring `calc-ext' (already loaded)
+-> Requiring `calc-macs' (already loaded)
Loading nil...
+-> Requiring `calc-menu'
+-> Requiring `calc-menu'...done
+-> Requiring `calc' (already loaded)
+-> Requiring `calc-macs' (already loaded)
--8<---------------cut here---------------end--------------->8---

Tried again:

--8<---------------cut here---------------start------------->8---
calc-do-embedded: Syntax error
--8<---------------cut here---------------end--------------->8---


> This block evaluates 3^3 with calc pushing the result on the stack and
> returning it into the Org-mode buffer
> #+begin_src calc
>   3^3
> #+end_src
>
> This block evaluates (2+2)*4 pushing the result on the stack, it then
> calls calc-plus adding the top two elements on the stack (one of which
> is left over from the previous code block).
> #+begin_src calc
>   (2+2)*4
>   '+
> #+end_src

On the 2 lasts, I had another error:

--8<---------------cut here---------------start------------->8---
executing Calc code block...
+-> Requiring `calc' (already loaded)
+-> Requiring `calc-macs' (already loaded)
calc-push-list: Symbol's value as variable is void: calc-command-flags
--8<---------------cut here---------------end--------------->8---

Best regards,
  Seb

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: Possible Calc support for Org-Babel?

[Eric Schulte]

Sébastien Vauban <wxhgmqzgwmuf@spammotel.com> writes:

[...]
>
> executing Calc code block...
> if: Symbol's function definition is void: calc-push-list
>
> Required =calc= by calling =C-x * *=:
>

hmm, I just pushed the actual ob-calc.el file up to the repository.  Do
you still get these errors if you loaf up calc support through calling
(require 'ob-calc)?