org-babel-read should have option NOT to interpret as elisp

org-babel-read should have option NOT to interpret as elisp

[Vladimir Alexiev]

I keep perl regexps in a table, and some of them start with "(".
I'd like these to be left alone (returned unmodified as a string).

(info "(org)var") doesn't mention special processing of table cells,
but the doc of org-babel-read says:

Convert the string value of CELL to a number if appropriate.
Otherwise if cell looks like lisp (meaning it starts with a
"(" or a "'") then read it as lisp, otherwise return it
unmodified as a string.

So it seems to me that this special behavior of org-babel-read 
should be documented in info, and controllable by a user option.

Re: org-babel-read should have option NOT to interpret as elisp

[Eric Schulte]

[-- Attachment #1: Type: text/plain, Size: 1601 bytes --]

Vladimir Alexiev <vladimir@sirma.bg> writes:

> I keep perl regexps in a table, and some of them start with "(".
> I'd like these to be left alone (returned unmodified as a string).
>
> (info "(org)var") doesn't mention special processing of table cells,
> but the doc of org-babel-read says:
>
> Convert the string value of CELL to a number if appropriate.
> Otherwise if cell looks like lisp (meaning it starts with a
> "(" or a "'") then read it as lisp, otherwise return it
> unmodified as a string.
>
> So it seems to me that this special behavior of org-babel-read 
> should be documented in info, and controllable by a user option.
>

When passing values directly through header arguments the solution would
be to wrap the string in double-quotes so that it is interpreted as a
literal string, e.g.,

#+begin_src perl :var it="(+ 1 1)" :results output
  printf "passed in %s", $it
#+end_src

#+results:
: passed in (+ 1 1)

However I do agree that this would be onerous to have to wrap every cell
of a table in double quotes...

I'm attaching a patch which inhibits the lisp evaluation of values read
from tables and lists.  This should solve your issue above.  I'm not
directly applying this patch, because it would be a breaking change for
anyone who is currently relying on the ability to fill a table or list
with to-be-evaluated emacs-lisp statements.  If anyone is in that
situation please respond to this email in the next couple of days,
otherwise I am leaning towards applying this patch to the main
repository.

Best -- Eric

p.s. this patch can be applied with the "git am" command.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ob-inhibit-lisp-evaluation-of-values-read-from-table.patch --]
[-- Type: text/x-diff, Size: 2700 bytes --]

From f57ff0b724f0420751dd90600553d988aec507f1 Mon Sep 17 00:00:00 2001
From: Eric Schulte <schulte.eric@gmail.com>
Date: Sun, 27 Feb 2011 08:47:36 -0700
Subject: [PATCH] ob: inhibit lisp evaluation of values read from tables and lists

* lisp/ob.el (org-babel-read-table): Inhibit lisp evaluation of values
  when reading from tables.
  (org-babel-read-list): Inhibit lisp evaluation of values when
  reading from lists.
  (org-babel-read): Add optional argument which can be used to inhibit
  lisp evaluation of value.
---
 lisp/ob.el |   23 ++++++++++++-----------
 1 files changed, 12 insertions(+), 11 deletions(-)

diff --git a/lisp/ob.el b/lisp/ob.el
index 6e98263..b4ce3cd 100644
--- a/lisp/ob.el
+++ b/lisp/ob.el
@@ -1386,12 +1386,13 @@ following the source block."
   "Read the table at `point' into emacs-lisp."
   (mapcar (lambda (row)
             (if (and (symbolp row) (equal row 'hline)) row
-              (mapcar #'org-babel-read row)))
+              (mapcar (lambda (el) (org-babel-read el 'inhibit-lisp-eval)) row)))
           (org-table-to-lisp)))
 
 (defun org-babel-read-list ()
   "Read the list at `point' into emacs-lisp."
-  (mapcar #'org-babel-read (mapcar #'cadr (cdr (org-list-parse-list)))))
+  (mapcar (lambda (el) (org-babel-read el 'inhibit-lisp-eval))
+	  (mapcar #'cadr (cdr (org-list-parse-list)))))
 
 (defvar org-link-types-re)
 (defun org-babel-read-link ()
@@ -1901,18 +1902,18 @@ block but are passed literally to the \"example-block\"."
 	     (apply #'string (reverse out)))))
        str))))
 
-(defun org-babel-read (cell)
+(defun org-babel-read (cell &optional inhibit-lisp-eval)
   "Convert the string value of CELL to a number if appropriate.
-Otherwise if cell looks like lisp (meaning it starts with a
-\"(\" or a \"'\") then read it as lisp, otherwise return it
-unmodified as a string.
-
-This is taken almost directly from `org-read-prop'."
+Otherwise if cell looks like lisp (meaning it starts with a \"(\"
+or a \"'\") then read it as lisp, otherwise return it unmodified
+as a string.  Optional argument NO-LISP-EVAL inhibits lisp
+evaluation for situations in which is it not appropriate."
   (if (and (stringp cell) (not (equal cell "")))
       (or (org-babel-number-p cell)
-          (if (or (equal "(" (substring cell 0 1))
-                  (equal "'" (substring cell 0 1))
-                  (equal "`" (substring cell 0 1)))
+          (if (and (not inhibit-lisp-eval)
+		   (or (equal "(" (substring cell 0 1))
+		       (equal "'" (substring cell 0 1))
+		       (equal "`" (substring cell 0 1))))
               (eval (read cell))
             (progn (set-text-properties 0 (length cell) nil cell) cell)))
     cell))
-- 
1.7.1


[-- Attachment #3: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: org-babel-read should have option NOT to interpret as elisp

[Vladimir Alexiev]

> would be onerous to have to wrap every cell
> of a table in double quotes...

I tried doing this as a workaround, but it is not 
since the quotes are passed to perl, and that's not what I want

> it would be a breaking change for
> anyone who is currently relying on the ability

Right, that's why I think it should be an option.
The default could stay as is, but should be documented.

Re: Re: org-babel-read should have option NOT to interpret as elisp

[Eric Schulte]

Vladimir Alexiev <vladimir@sirma.bg> writes:

>> would be onerous to have to wrap every cell
>> of a table in double quotes...
>
> I tried doing this as a workaround, but it is not 
> since the quotes are passed to perl, and that's not what I want
>
>> it would be a breaking change for
>> anyone who is currently relying on the ability
>
> Right, that's why I think it should be an option.
> The default could stay as is, but should be documented.
>

What syntax would you suggest to indicate that a variable is to be
passed without the possibility of elisp evaluation.  I agree making this
behavior optional would be ideal, but I couldn't think of an intuitive
syntax.

Cheers -- Eric

Re: org-babel-read should have option NOT to interpret as elisp

[Vladimir Alexiev]

> What syntax would you suggest to indicate that a variable is to be
> passed without the possibility of elisp evaluation

I think this should be done with a header arg, 
since they have very flexible setup scheme:
  see (info "(org)Using header arguments")
  "values of header arguments can be set in six different ways"

- Ideally, the arg should be attached to
  #+tblname: since it's a characteristic of the table itself
- If attached to
  #+begin_src: then it will be dissociated from the table,
  and all :var's of that line will be forced to use the same arg.
- But that's good enough since it can be set in various ways. For me most useful:
-- org-babel-default-header-args: global
-- #+BABEL: per file

The header arg should be called :read-as (or :var-read). Considerations:
- should be quite distinct so it can be used as a property name
- should use dash so it's analogous to no-expand

Its values should be:
- literal: 
  If a value looks like a number, it's read as a number.
  Else it's read as a literal string.
- elisp or nil (default):
  If a value starts with one of ('` it's read as emacs lisp.
  Else it's read as 'literal'.
- quoted: 
  If a value starts with " then it's read as a quoted string:
  start and end quotes are stripped, and \" escaped quotes are unescaped
  (this is useful for embedding leading/trailing whitespace in strings).
  Else it's read as `literal'.
- quoted_elisp: combination of `quoted' and `elisp'
(I assume that using multiple values per arg is not supported)

This above is about data coming from tables, 
since I haven't used the other two options (literal value and code block).
The chosen solution should work for those too.

Please note that org-babel-read says
  "This is taken almost directly from `org-read-prop'."
so the chosen solution should be compatible with that.
But I can't find such function.

Re: org-babel-read should have option NOT to interpret as elisp

[Sébastien Vauban]

Hi Eric,

"Eric Schulte" wrote:
> However I do agree that this would be onerous to have to wrap every cell of
> a table in double quotes...

Aren't we forced to do so, right now?  As in the following home-made example:

#+TITLE:     Show differences between files

* Code

Assuming you have two directories (namely =xhtml-dir= and =xhtml-dir-bak=,
accessible from this local directory), the following blocks of code will
compare named files (see column 2 of table) in them, in both /text/ and
/binary/ versions.

#+srcname: diff-text
#+begin_src sh :var file=""
diff -sq --strip-trailing-cr xhtml-dir/$file xhtml-dir-bak/$file > /dev/null
case "$?" in
  0) echo "identical";;
  1) echo "differ";;
  2) echo "no such file";;
  *) echo "?";;
esac
#+end_src

#+srcname: diff-binary
#+begin_src sh :var file=""
diff -sq xhtml-dir/$file xhtml-dir-bak/$file > /dev/null
case "$?" in
  0) echo "identical";;
  1) echo "differ";;
  2) echo "no such file";;
  *) echo "?";;
esac
#+end_src

* Results

| FPH | File                          | Text      | Binary    |
|-----+-------------------------------+-----------+-----------|
| M   | "Icopreations.xhtml"          | identical | identical |
| M   | "Ocuevaonspfiessai.xhtml"     | identical | differ    |
| M   | "Ocuevain-tut.xhtml"          | differ    | differ    |
| M   | "Ocuevatagessai.xhtml"        | identical | differ    |
| M   | "Ocuf2-stg.xhtml"             | differ    | differ    |
| M   | "Ocurolro.xhtml"              | identical | differ    |
| M   | "Ocuroliers.xhtml"            | differ    | differ    |
| M   | "Ocurolredit.xhtml"           | identical | differ    |
| M   | "Ocusigletiquebc1233de.xhtml" | differ    | differ    |
| M   | "Rocsaieprestations.xhtml"    | identical | identical |
#+TBLFM: $3='(sbe diff-text (file $2))::$4='(sbe diff-binary (file $2))

I had to enclose the filenames between double quotes, for the =sbe= code to
work.

Trying what you wrote on
http://eschulte.github.com/babel-dev/DONE-literal-values-from-tables.html,
that is using $$2 instead of $2 has the following behavior: when evaluating
the table, Org prompts me for a "Lisp expression"!?

Best regards,
  Seb

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: org-babel-read should have option NOT to interpret as elisp

[Eric Schulte]

Hi Seb,

earlier in this thread, I attached a patch which allows string results
in tables which look like elisp to be interpreted literally, e.g.,

#+results: elisp-looking-table
| 1 | (+ 1 1) |
| 2 | (+ 2 2) |
| 3 | (+ 3 3) |

#+begin_src perl :var data=elisp-looking-table[1,1]
  $data
#+end_src

results in

#+results:
: (+ 2 2)

rather than

#+results:
: 4

> 
> Trying what you wrote on
> http://eschulte.github.com/babel-dev/DONE-literal-values-from-tables.html,
> that is using $$2 instead of $2 has the following behavior: when evaluating
> the table, Org prompts me for a "Lisp expression"!?
>

It like doubel dollar sign syntax in table evaluation has bit-rot.  It
appears that some other part of Org-mode now interprets the double
dollar sign as a prompt to the user -- although I can't seem to find the
relevant code.  The `sbe' in-table evaluation could use some attention.

Best -- Eric

Re: Re: org-babel-read should have option NOT to interpret as elisp

[Eric Schulte]

Vladimir Alexiev <vladimir@sirma.bg> writes:

>> What syntax would you suggest to indicate that a variable is to be
>> passed without the possibility of elisp evaluation
>
> I think this should be done with a header arg, 
> since they have very flexible setup scheme:
>   see (info "(org)Using header arguments")
>   "values of header arguments can be set in six different ways"
>
> - Ideally, the arg should be attached to
>   #+tblname: since it's a characteristic of the table itself

This would be a good way to go, however there is currently no support
for attaching header arguments to tables, so this is not viable---unless
there are enough other motivating use cases for attaching header
arguments to tables and results, in which case this could be revisited.

> 
> - If attached to #+begin_src: then it will be dissociated from the
> table, and all :var's of that line will be forced to use the same arg.
> - But that's good enough since it can be set in various ways. For me
> most useful: -- org-babel-default-header-args: global -- #+BABEL: per
> file
>
> The header arg should be called :read-as (or
>:var-read). Considerations:
> - should be quite distinct so it can be used as a property name
> - should use dash so it's analogous to no-expand
>
> Its values should be:
> - literal: 
>   If a value looks like a number, it's read as a number.
>   Else it's read as a literal string.
> - elisp or nil (default):
>   If a value starts with one of ('` it's read as emacs lisp.
>   Else it's read as 'literal'.
> - quoted: 
>   If a value starts with " then it's read as a quoted string:
>   start and end quotes are stripped, and \" escaped quotes are unescaped
>   (this is useful for embedding leading/trailing whitespace in strings).
>   Else it's read as `literal'.
> - quoted_elisp: combination of `quoted' and `elisp'
> (I assume that using multiple values per arg is not supported)
>

There are currently a number of header arguments which allow multiple
arguments, e.g., :results.

>
> This above is about data coming from tables,
> since I haven't used the other two options (literal value and code block).
> The chosen solution should work for those too.
>

The problem here is the same as in our other table indexing thread,
namely it requires that some header arguments be parsed and their
results available *before* the :var header arguments are parsed.

I think that the only viable solution may involve adding something to
the actual :var header argument, although I can't think of any obvious
syntax there.

I'm thinking that any of the above would introduce undue complexity to
the header argument parsing.  For the interim, I've applied the patch
attached to my previous email and unless there is a real push-back from
users who are embedding executable elisp code into tables and lists I
think this solution is the best compromise between usability and
simplicity.

Thanks -- Eric

>
> Please note that org-babel-read says
>   "This is taken almost directly from `org-read-prop'."
> so the chosen solution should be compatible with that.
> But I can't find such function.
>

Yes, the patch attached to my previous email fixes this documentation
string.

RE: Re: org-babel-read should have option NOT to interpret as elisp

[Vladimir Alexiev]

> I've applied the patch
> attached to my previous email and unless there is a real push-back ...

Could you add handling of quoted strings?
And most importantly, document all of this in "(org)var".
Here is a merged dscription:

If a value starts with one of ('` it is read as an emacs lisp sexp.
If it starts with " then it's read as a quoted string: start/end quotes are stripped, \" and \\ are unescaped
   (this is useful for embedding leading/trailing whitespace in strings).
If it looks like a number, it's read as a number.
Else it's read as a literal string, without any quotation or escaping.
(Please note that | in tables is always interpreted as column separator and currently there is no way to quote it)

Re: RE: Re: org-babel-read should have option NOT to interpret as elisp

[Eric Schulte]

"Vladimir Alexiev" <vladimir@sirma.bg> writes:

>> I've applied the patch
>> attached to my previous email and unless there is a real push-back ...
>
> Could you add handling of quoted strings?

I believe quoted strings are already handled, e.g.,

#+results: elisp-looking-table
| 1 | (a b c)   |
| 2 | "(a b c)" |

#+headers: :var without=elisp-looking-table[0,1]
#+headers:    :var with=elisp-looking-table[1,1]
#+begin_src perl
  $with, $without
#+end_src

#+results:
: "(a b c)"
: (a b c)

> 
> And most importantly, document all of this in "(org)var".  Here is a
> merged dscription:
>

Thanks for the reminder, I have  updated the documentation.

Best -- Eric

>
> If a value starts with one of ('` it is read as an emacs lisp sexp.
> If it starts with " then it's read as a quoted string: start/end
> quotes are stripped, \" and \\ are unescaped (this is useful for
> embedding leading/trailing whitespace in strings).  If it looks like a
> number, it's read as a number.  Else it's read as a literal string,
> without any quotation or escaping.  (Please note that | in tables is
> always interpreted as column separator and currently there is no way
> to quote it)
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

RE: RE: Re: org-babel-read should have option NOT to interpret as elisp

[Vladimir Alexiev]

> I believe quoted strings are already handled, e.g.,
> #+results:
> : "(a b c)"

They are not. This leaves the quotes as part of the string.

Please add a second usage to the description:

(this is useful for having leading/trailing whitespace in a string,
or having a leading ('` yet preventing the emacs lisp treatment)

Re: RE: Re: org-babel-read should have option NOT to interpret as elisp

[Eric Schulte]

"Vladimir Alexiev" <vladimir@sirma.bg> writes:

>> I believe quoted strings are already handled, e.g.,
>> #+results:
>> : "(a b c)"
>
> They are not. This leaves the quotes as part of the string.
>

Alright, I've changed this behavior so that double-quoted strings are
read into variable values (removing the quotes).  For example

#+results: spaces-wrapped-string
- " pass through with space "


#+begin_src emacs-lisp :var res=spaces-wrapped-string[0]
  res
#+end_src

#+results:
:  pass through with space 

Best -- Eric