evaluation of perl in babel

evaluation of perl in babel

[D M German]

Hi Everybody,

I looked a bit more onto the way that perl is evaluated. I know the
support of perl is minor. I understand that, so please, don't see this
message as a complaint, so this is more for discussion and potential
improvements of the Perl support in Babel.

One of the things I have noticed is that the way that Babel handles the
results coming from the code is not the best.

Let me elaborate:

At the bottom you will find a set of test that stress the different
:results types.

There are some bugs. For example, the interpretation of :results table,
vector and list.

But I think the main problem comes from the way that Babel expects the
result. In Babel, and except for :results output, the last expression in
perl is considered the input to the results. This is implementing by
saving the last expression into a variable, and printing each value
separated by a "\n" (including the last). So basically, org takes the
last expression, and outputs them to the babel input file one per line.

This places some constraints. First, it is not currently capable of
dealing with two dimensional arrays. Second, it makes it hard to create
complex output (such as HTML or LaTeX), and third, it is hard to debug
without first printing the value of the array (this output would be lost
during the evaluation, so it would have be debugged outside org).

I feel that a better approach is to use std output as the default input
to any of these :results types, and then try to parse them into the
corresponding :results types. This will allow the creation of HTML and
LaTeX from perl (which the current implementation does not allow).

So recapitulating, my suggestion is that perl should use STDOUT as the
output of the snippet in any :results type, rather than the result of
the last expression.

I know this will break backwards compatibility. One solution is to keep
the current src perl and add a new perl_stdout mode (or something like
that) that implements this.

--dmg



----------------------------------------------------------------------
#+begin_src perl :results output 
print "Test\n";
(1, 2)
#+end_src

#+RESULTS:
: Test

#+name: t_output_raw
#+begin_src perl :results raw 
print "Test\n";
(1, 2)
#+end_src

#+RESULTS: t_output_raw
1
2

#+name: t_output_table
#+begin_src perl :results table
print "Test\n";
(1, 2)
#+end_src

#+RESULTS: t_output_table
| 1\n2\n |

#+name: t_output_vector
#+begin_src perl :results vector
print "Test\n";
(1, 2)
#+end_src

#+RESULTS: t_output_vector
| 1\n2\n |


#+name: t_output_list
#+begin_src perl :results list
print "Test\n";
(1, 2)
#+end_src

#+RESULTS: t_output_list
#+begin_example
- 1
2
#+end_example

#+name: t_output_org
#+begin_src perl :results org
print "Test\n";
(1, 2)
#+end_src

#+RESULTS: t_output_org
#+BEGIN_SRC org
1
2
#+END_SRC

#+name: t_output_html
#+begin_src perl :results html
print "Test\n";
(1, 2)
#+end_src

#+RESULTS: t_output_html
#+BEGIN_HTML
1
2
#+END_HTML

#+name: t_output_latex
#+begin_src perl :results latex
print "Test\n";
(1, 2)
#+end_src

#+RESULTS: t_output_latex
#+BEGIN_LaTeX
1
2
#+END_LaTeX
----------------------------------------------------------------------


--
Daniel M. German                  "I see no good reason why the
                                   views given in this volume
                                   should shock the religious
   Charles Darwin ->               feelings of anyone."
http://turingmachine.org/
http://silvernegative.com/
dmg (at) uvic (dot) ca
replace (at) with @ and (dot) with .

 

Re: evaluation of perl in babel

[dmg]

Mm, I didn't include :results value

I think that :results value should do what it does now: return the
value of the last expression.

--dmg

On Sun, Feb 24, 2013 at 1:08 PM, D M German <dmg@uvic.ca> wrote:
>
> Hi Everybody,
>
> I looked a bit more onto the way that perl is evaluated. I know the
> support of perl is minor. I understand that, so please, don't see this
> message as a complaint, so this is more for discussion and potential
> improvements of the Perl support in Babel.
>
> One of the things I have noticed is that the way that Babel handles the
> results coming from the code is not the best.
>
> Let me elaborate:
>
> At the bottom you will find a set of test that stress the different
> :results types.
>
> There are some bugs. For example, the interpretation of :results table,
> vector and list.
>
> But I think the main problem comes from the way that Babel expects the
> result. In Babel, and except for :results output, the last expression in
> perl is considered the input to the results. This is implementing by
> saving the last expression into a variable, and printing each value
> separated by a "\n" (including the last). So basically, org takes the
> last expression, and outputs them to the babel input file one per line.
>
> This places some constraints. First, it is not currently capable of
> dealing with two dimensional arrays. Second, it makes it hard to create
> complex output (such as HTML or LaTeX), and third, it is hard to debug
> without first printing the value of the array (this output would be lost
> during the evaluation, so it would have be debugged outside org).
>
> I feel that a better approach is to use std output as the default input
> to any of these :results types, and then try to parse them into the
> corresponding :results types. This will allow the creation of HTML and
> LaTeX from perl (which the current implementation does not allow).
>
> So recapitulating, my suggestion is that perl should use STDOUT as the
> output of the snippet in any :results type, rather than the result of
> the last expression.
>
> I know this will break backwards compatibility. One solution is to keep
> the current src perl and add a new perl_stdout mode (or something like
> that) that implements this.
>
> --dmg
>
>
>
> ----------------------------------------------------------------------
> #+begin_src perl :results output
> print "Test\n";
> (1, 2)
> #+end_src
>
> #+RESULTS:
> : Test
>
> #+name: t_output_raw
> #+begin_src perl :results raw
> print "Test\n";
> (1, 2)
> #+end_src
>
> #+RESULTS: t_output_raw
> 1
> 2
>
> #+name: t_output_table
> #+begin_src perl :results table
> print "Test\n";
> (1, 2)
> #+end_src
>
> #+RESULTS: t_output_table
> | 1\n2\n |
>
> #+name: t_output_vector
> #+begin_src perl :results vector
> print "Test\n";
> (1, 2)
> #+end_src
>
> #+RESULTS: t_output_vector
> | 1\n2\n |
>
>
> #+name: t_output_list
> #+begin_src perl :results list
> print "Test\n";
> (1, 2)
> #+end_src
>
> #+RESULTS: t_output_list
> #+begin_example
> - 1
> 2
> #+end_example
>
> #+name: t_output_org
> #+begin_src perl :results org
> print "Test\n";
> (1, 2)
> #+end_src
>
> #+RESULTS: t_output_org
> #+BEGIN_SRC org
> 1
> 2
> #+END_SRC
>
> #+name: t_output_html
> #+begin_src perl :results html
> print "Test\n";
> (1, 2)
> #+end_src
>
> #+RESULTS: t_output_html
> #+BEGIN_HTML
> 1
> 2
> #+END_HTML
>
> #+name: t_output_latex
> #+begin_src perl :results latex
> print "Test\n";
> (1, 2)
> #+end_src
>
> #+RESULTS: t_output_latex
> #+BEGIN_LaTeX
> 1
> 2
> #+END_LaTeX
> ----------------------------------------------------------------------
>
>
> --
> Daniel M. German                  "I see no good reason why the
>                                    views given in this volume
>                                    should shock the religious
>    Charles Darwin ->               feelings of anyone."
> http://turingmachine.org/
> http://silvernegative.com/
> dmg (at) uvic (dot) ca
> replace (at) with @ and (dot) with .
>
>



-- 
--dmg

---
Daniel M. German
http://turingmachine.org

Re: evaluation of perl in babel

[Achim Gratz]

D M German writes:
> There are some bugs. For example, the interpretation of :results table,
> vector and list.

You may misunderstand some things, or I don't understand what you are
asking.  It is (at least currently) the responsibility of the Perl
program (or any other Babel language) to deliver the result in such a
way that it can be interpreted correctly by the result type chosen (in
other word, the program output must be valid Org syntax in the given
context).  You can't have the same program produce tables, vectors and
LaTeX output just by switching the results type.

> But I think the main problem comes from the way that Babel expects the
> result. In Babel, and except for :results output, the last expression in
> perl is considered the input to the results.

That is with the default wrapper function, which expects the program to
return something that either is a string or interpolates to a string
that Babel can interpret.  You can easily define one yourself that does
different things, like simply open the output file then select the
filehandle for output.  That's what I'd do in any case and I think it
would work just as you want.

--8<---------------cut here---------------start------------->8---
(defvar org-babel-perl-wrapper-method
  "{
  my $prog = q(%s);
  open my $BO, qq(>%s) or die qq(Perl: Could not open output file.$\\);
  select $BO;
  eval( $prog );
}")
--8<---------------cut here---------------end--------------->8---

> This is implementing by
> saving the last expression into a variable, and printing each value
> separated by a "\n" (including the last). So basically, org takes the
> last expression, and outputs them to the babel input file one per line.

Yes, not the most elegant way, not perlish and all that; but there's
nothing to stop you from making that last expression a string variable
that has collected your output.  Try the above and let me know if that
addresses your concerns, if only partly.

As for debugging, the output files that Perl produces and Babel then
interprets are kept in the temporary folder, org-babel-eval-read-file
doesn't delete them after use (it probably should, though).


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Waldorf MIDI Implementation & additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs

Re: evaluation of perl in babel

[D M German]

 Achim Gratz twisted the bytes to say:


 Achim> D M German writes:
 >> There are some bugs. For example, the interpretation of :results table,
 >> vector and list.

 Achim> You may misunderstand some things, or I don't understand what you are
 Achim> asking.  It is (at least currently) the responsibility of the Perl
 Achim> program (or any other Babel language) to deliver the result in such a
 Achim> way that it can be interpreted correctly by the result type chosen (in
 Achim> other word, the program output must be valid Org syntax in the given
 Achim> context).  You can't have the same program produce tables, vectors and
 Achim> LaTeX output just by switching the results type.

I understand. But what I want is the output to be wrapped accordingly,
and my script to deliver exactly the output as expected. So say I want
to generate HTML in my script, I can use :results output, but then I
have to change to replace the #+being_example with #+begin_HTML.

I guess that I can generate a two dimensional table with perl too
using output (printing the necessary | and \n), but then it will be
wrapped with #+begin_example.


 >> But I think the main problem comes from the way that Babel expects the
 >> result. In Babel, and except for :results output, the last expression in
 >> perl is considered the input to the results.

 Achim> That is with the default wrapper function, which expects the program to
 Achim> return something that either is a string or interpolates to a string
 Achim> that Babel can interpret.  You can easily define one yourself that does
 Achim> different things, like simply open the output file then select the
 Achim> filehandle for output.  That's what I'd do in any case and I think it
 Achim> would work just as you want.

Perhaps a string can be the solution. Ok, I am testing:

#+begin_src perl :results table
("a|b|c|\n|c|d|e")
#+end_src

#+RESULTS:
| a | b | c | \n | c | d | e |

Ok, this seems to be more useful that returning a list of values. 
Is there a way to separate two rows in the result?

Thanks again Achim,

--daniel

--
Daniel M. German
http://turingmachine.org/
http://silvernegative.com/
dmg (at) uvic (dot) ca
replace (at) with @ and (dot) with .

 

Re: evaluation of perl in babel

[dmg]

On Mon, Feb 25, 2013 at 1:54 AM, D M German <dmg@uvic.ca> wrote:
>
>  Achim> You may misunderstand some things, or I don't understand what you are
>  Achim> asking.  It is (at least currently) the responsibility of the Perl
>  Achim> program (or any other Babel language) to deliver the result in such a
>  Achim> way that it can be interpreted correctly by the result type chosen (in
>  Achim> other word, the program output must be valid Org syntax in the given
>  Achim> context).  You can't have the same program produce tables, vectors and
>  Achim> LaTeX output just by switching the results type.
>
> I understand. But what I want is the output to be wrapped accordingly,
> and my script to deliver exactly the output as expected. So say I want
> to generate HTML in my script, I can use :results output, but then I
> have to change to replace the #+being_example with #+begin_HTML.
>
> I guess that I can generate a two dimensional table with perl too
> using output (printing the necessary | and \n), but then it will be
> wrapped with #+begin_example.

Ok, I got it. What I need is to return a string with whatever I need.
A bit cumbersome, but I can live with it


#+begin_src perl :results html
"<table>
<b>a </b>
</table>
"
#+end_src

#+RESULTS:
#+BEGIN_HTML
<table>
<b>a </b>
</table>
#+END_HTML

thanks again for the explanation,

-- 
--dmg

---
Daniel M. German
http://turingmachine.org

Re: evaluation of perl in babel

[Eric Schulte]

dmg <dmg@uvic.ca> writes:

> On Mon, Feb 25, 2013 at 1:54 AM, D M German <dmg@uvic.ca> wrote:
>>
>>  Achim> You may misunderstand some things, or I don't understand what you are
>>  Achim> asking.  It is (at least currently) the responsibility of the Perl
>>  Achim> program (or any other Babel language) to deliver the result in such a
>>  Achim> way that it can be interpreted correctly by the result type chosen (in
>>  Achim> other word, the program output must be valid Org syntax in the given
>>  Achim> context).  You can't have the same program produce tables, vectors and
>>  Achim> LaTeX output just by switching the results type.
>>
>> I understand. But what I want is the output to be wrapped accordingly,
>> and my script to deliver exactly the output as expected. So say I want
>> to generate HTML in my script, I can use :results output, but then I
>> have to change to replace the #+being_example with #+begin_HTML.
>>
>> I guess that I can generate a two dimensional table with perl too
>> using output (printing the necessary | and \n), but then it will be
>> wrapped with #+begin_example.
>
> Ok, I got it. What I need is to return a string with whatever I need.
> A bit cumbersome, but I can live with it
>

It might be worth spending some time with the Org-mode manual.
http://orgmode.org/manual/Working-With-Source-Code.html

You could also use ":results org" or ":results raw", and
then return raw Org-mode from your Perl script instead of html.

Best,

>
>
> #+begin_src perl :results html
> "<table>
> <b>a </b>
> </table>
> "
> #+end_src
>
> #+RESULTS:
> #+BEGIN_HTML
> <table>
> <b>a </b>
> </table>
> #+END_HTML
>
> thanks again for the explanation,

-- 
Eric Schulte
http://cs.unm.edu/~eschulte