(Maybe) enhance `org-element-src-block-interpreter'?

(Maybe) enhance `org-element-src-block-interpreter'?

[Thorsten Jolitz]

Hi, 

a slight enhancement:

#+begin_src emacs-lisp
  (defun tj/src-block-interpreter (src-block contents)
    "Interpret SRC-BLOCK element as Org syntax.  CONTENTS is nil."
    (let ((lang (org-element-property :language src-block))
          (switches (org-element-property :switches src-block))
          (params (org-element-property :parameters src-block))
          (headers (org-element-property :header src-block))
          (value
           (let ((val (org-element-property :value src-block)))
             (cond
              ((or org-src-preserve-indentation
                   (org-element-property :preserve-indent src-block))
               val)
              ((zerop org-edit-src-content-indentation) val)
              (t
               (let ((ind (make-string
                           org-edit-src-content-indentation ?\s)))
                 (replace-regexp-in-string
                  "\\(^\\)[ \t]*\\S-" ind val nil nil 1))))))
          packed-headers)
      (concat (format "%s#+BEGIN_SRC%s\n"
                      (progn
                        (while headers
                          (setq packed-headers
                                (concat
                                 (format "#+HEADER: %s\n"
                                         (pop headers))
                                 packed-headers)))
                        (or packed-headers ""))
                      (concat (and lang (concat " " lang))
                              (and switches (concat " " switches))
                              (and params (concat " " params))))
              (org-escape-code-in-string value)
              ;; (save-match-data
              ;;   (if (looking-at "^$") "" "\n"))
              "#+END_SRC")))
#+end_src

#+results:
: tj/src-block-interpreter

Parse this src-block (with point at beg of block):

,----
| M-: (setq ptree (org-element-at-point))
`----

#+header: :results raw
#+begin_src emacs-lisp
(message "hello world")
#+end_src

#+results:
hello world

Then evaluate

#+begin_src emacs-lisp
(org-element-src-block-interpreter ptree nil)
#+end_src

#+results:
: #+BEGIN_SRC emacs-lisp
:   (message "hello world")
: #+END_SRC

#+begin_src emacs-lisp
(tj/src-block-interpreter ptree nil)
#+end_src

#+results:
: #+HEADER: :results raw
: #+BEGIN_SRC emacs-lisp
:   (message "hello world")
: #+END_SRC


-- 
cheers,
Thorsten

Re: (Maybe) enhance `org-element-src-block-interpreter'?

[Aaron Ecay]

Hi Thorsten,

2014ko abuztuak 5an, Thorsten Jolitz-ek idatzi zuen:

[...]

> 
> Parse this src-block (with point at beg of block):
> 
> ,----
> | M-: (setq ptree (org-element-at-point))
> `----
> 
> #+header: :results raw
> #+begin_src emacs-lisp
> (message "hello world")
> #+end_src
> 
> #+results:
> hello world
> 
> Then evaluate
> 
> #+begin_src emacs-lisp
> (org-element-src-block-interpreter ptree nil)
> #+end_src
> 
> #+results:
> : #+BEGIN_SRC emacs-lisp
> :   (message "hello world")
> : #+END_SRC
> 
> #+begin_src emacs-lisp
> (tj/src-block-interpreter ptree nil)
> #+end_src
> 
> #+results:
> : #+HEADER: :results raw
> : #+BEGIN_SRC emacs-lisp
> :   (message "hello world")
> : #+END_SRC

Indeed this seems like an improvement on the status quo.  But other
elements of org syntax (not just src blocks) can have a valid #+header
(and indeed other affiliated keywords, like #+attr_latex), so the fix
probably should be more general.

-- 
Aaron Ecay

Re: (Maybe) enhance `org-element-src-block-interpreter'?

[Thorsten Jolitz]

Aaron Ecay <aaronecay@gmail.com> writes:

> Hi Thorsten,
>
> 2014ko abuztuak 5an, Thorsten Jolitz-ek idatzi zuen:
>
> [...]
>
>> 
>> Parse this src-block (with point at beg of block):
>> 
>> ,----
>> | M-: (setq ptree (org-element-at-point))
>> `----
>> 
>> #+header: :results raw
>> #+begin_src emacs-lisp
>> (message "hello world")
>> #+end_src
>> 
>> #+results:
>> hello world
>> 
>> Then evaluate
>> 
>> #+begin_src emacs-lisp
>> (org-element-src-block-interpreter ptree nil)
>> #+end_src
>> 
>> #+results:
>> : #+BEGIN_SRC emacs-lisp
>> :   (message "hello world")
>> : #+END_SRC
>> 
>> #+begin_src emacs-lisp
>> (tj/src-block-interpreter ptree nil)
>> #+end_src
>> 
>> #+results:
>> : #+HEADER: :results raw
>> : #+BEGIN_SRC emacs-lisp
>> :   (message "hello world")
>> : #+END_SRC
>
> Indeed this seems like an improvement on the status quo.  But other
> elements of org syntax (not just src blocks) can have a valid #+header
> (and indeed other affiliated keywords, like #+attr_latex), so the fix
> probably should be more general.

Yes, there are other elements and more affiliated keywords. The
parser (or rather the interpreter(s)) and parts of Org Babel do not
always deal with them yet. 

This was more a 'constructive bug report', not so much an attempt of a
general fix. I just (partly) fixed this function for myself because I
needed it ...

-- 
cheers,
Thorsten

Re: (Maybe) enhance `org-element-src-block-interpreter'?

[Nicolas Goaziou]

Hello,

Thorsten Jolitz <tjolitz@gmail.com> writes:

> Aaron Ecay <aaronecay@gmail.com> writes:
>
>> Hi Thorsten,
>>
>> 2014ko abuztuak 5an, Thorsten Jolitz-ek idatzi zuen:
>>
>> [...]
>>
>>> 
>>> Parse this src-block (with point at beg of block):
>>> 
>>> ,----
>>> | M-: (setq ptree (org-element-at-point))
>>> `----
>>> 
>>> #+header: :results raw
>>> #+begin_src emacs-lisp
>>> (message "hello world")
>>> #+end_src
>>> 
>>> #+results:
>>> hello world
>>> 
>>> Then evaluate
>>> 
>>> #+begin_src emacs-lisp
>>> (org-element-src-block-interpreter ptree nil)
>>> #+end_src
>>> 
>>> #+results:
>>> : #+BEGIN_SRC emacs-lisp
>>> :   (message "hello world")
>>> : #+END_SRC
>>> 
>>> #+begin_src emacs-lisp
>>> (tj/src-block-interpreter ptree nil)
>>> #+end_src
>>> 
>>> #+results:
>>> : #+HEADER: :results raw
>>> : #+BEGIN_SRC emacs-lisp
>>> :   (message "hello world")
>>> : #+END_SRC
>>
>> Indeed this seems like an improvement on the status quo.  But other
>> elements of org syntax (not just src blocks) can have a valid #+header
>> (and indeed other affiliated keywords, like #+attr_latex), so the fix
>> probably should be more general.
>
> Yes, there are other elements and more affiliated keywords. The
> parser (or rather the interpreter(s)) and parts of Org Babel do not
> always deal with them yet. 
>
> This was more a 'constructive bug report', not so much an attempt of a
> general fix. I just (partly) fixed this function for myself because I
> needed it ...

There is no bug here.

`org-element-src-block-interpreter' is meant to create a src block and
only a src block. `org-element--interpret-affiliated-keyword' is used to
create affiliated keywords (like "#+header"). You shouldn't call any of
these anyway, since `org-element-interpret-data' is the one and only
entry point to interpret parsed data. Try


  (org-element-interpret-data ptree)


Regards,

-- 
Nicolas Goaziou

Re: (Maybe) enhance `org-element-src-block-interpreter'?

[Thorsten Jolitz]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

Hello,
>> Yes, there are other elements and more affiliated keywords. The
>> parser (or rather the interpreter(s)) and parts of Org Babel do not
>> always deal with them yet. 
>>
>> This was more a 'constructive bug report', not so much an attempt of a
>> general fix. I just (partly) fixed this function for myself because I
>> needed it ...
>
> There is no bug here.
>
> `org-element-src-block-interpreter' is meant to create a src block and
> only a src block. `org-element--interpret-affiliated-keyword' is used to
> create affiliated keywords (like "#+header"). You shouldn't call any of
> these anyway, since `org-element-interpret-data' is the one and only
> entry point to interpret parsed data. Try
>
>
>   (org-element-interpret-data ptree)

I see, thanks. I'm still getting used to the 'org-element API', but it
seems to be much easier than I thought: just use `org-element-at-point' and
`org-element-interpret-data' in most cases ...

-- 
cheers,
Thorsten

Re: (Maybe) enhance `org-element-src-block-interpreter'?

[Nicolas Goaziou]

Thorsten Jolitz <tjolitz@gmail.com> writes:

> I see, thanks. I'm still getting used to the 'org-element API', but it
> seems to be much easier than I thought: just use `org-element-at-point' and
> `org-element-interpret-data' in most cases ...

Correct.

For completeness, if you're working at the local level (i.e. with
`org-element-at-point'), available accessors are

  - `org-element-type'
  - `org-element-property'

When you're working at the global level (i.e. with
`org-element-parse-buffer'), you get another accessor,
`org-element-contents', and some tools to modify the parse tree

  - `org-element-put-property'
  - `org-element-adopt-element'
  - `org-element-insert-before'
  - `org-element-extract-element'
  - `org-element-set-element'

In both cases, `org-element-interpret-data' is the only way to turn data
back into Org syntax.


Regards,

-- 
Nicolas Goaziou

Re: (Maybe) enhance `org-element-src-block-interpreter'?

[Nicolas Goaziou]

Here goes the completeness...

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> For completeness, if you're working at the local level (i.e. with
> `org-element-at-point'), available accessors are

with `org-element-at-point' or `org-element-context'

>   - `org-element-type'
>   - `org-element-property'
>
> When you're working at the global level (i.e. with
> `org-element-parse-buffer'), you get another accessor,
> `org-element-contents',

In fact, new accessors are

  - `org-element-contents'
  - `org-element-map'

> and some tools to modify the parse tree
>
>   - `org-element-put-property'
>   - `org-element-adopt-element'
>   - `org-element-insert-before'
>   - `org-element-extract-element'
>   - `org-element-set-element'

Re: (Maybe) enhance `org-element-src-block-interpreter'?

[Thorsten Jolitz]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Here goes the completeness...
>
> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>
>> For completeness, if you're working at the local level (i.e. with
>> `org-element-at-point'), available accessors are
>
> with `org-element-at-point' or `org-element-context'
>
>>   - `org-element-type'
>>   - `org-element-property'

Accessor `org-element-contents' is badly missed here ...

E.g. I can get locally the content of a src-block (its :value), but for
most other elements (e.g. paragraph) that is not true. OTOH I cannot
reuse a src-block value as the content of a (locally created) paragraph
because this element has no :value property I could set (and its
interpreter simply inserts 'content', which is unaccessible on local
level). 

I know this is *much* easier asked as provided:
can getters and setters for element-content be introduced at the local
level too? maybe via another property shared by all elements (:content
?). The content is probably not even parsed at local level, but anyway,
maybe there is some kind of trick to make it accessible without parsing
the whole buffer?

The only thing that comes to my mind is narrow the buffer to
element-at-point and then parse only this visible buffer part and return
its content. Valid idea?

>> When you're working at the global level (i.e. with
>> `org-element-parse-buffer'), you get another accessor,
>> `org-element-contents',
>
> In fact, new accessors are
>
>   - `org-element-contents'
>   - `org-element-map'
>
>> and some tools to modify the parse tree
>>
>>   - `org-element-put-property'
>>   - `org-element-adopt-element'
>>   - `org-element-insert-before'
>>   - `org-element-extract-element'
>>   - `org-element-set-element'


-- 
cheers,
Thorsten

Re: (Maybe) enhance `org-element-src-block-interpreter'?

[Nicolas Goaziou]

Thorsten Jolitz <tjolitz@gmail.com> writes:

> Accessor `org-element-contents' is badly missed here ...
>
> E.g. I can get locally the content of a src-block (its :value), but for
> most other elements (e.g. paragraph) that is not true. OTOH I cannot
> reuse a src-block value as the content of a (locally created) paragraph
> because this element has no :value property I could set (and its
> interpreter simply inserts 'content', which is unaccessible on local
> level). 
>
> I know this is *much* easier asked as provided:
> can getters and setters for element-content be introduced at the local
> level too? 

No, `org-element-at-point' focuses on the element at point, not elements
within. It would slow it down and make caching more complicated, for
little benefit.

> maybe via another property shared by all elements (:content
> ?). The content is probably not even parsed at local level, but anyway,
> maybe there is some kind of trick to make it accessible without parsing
> the whole buffer?
>
> The only thing that comes to my mind is narrow the buffer to
> element-at-point and then parse only this visible buffer part and return
> its content. Valid idea?

You're correct. Some cleanup is needed though (org-data + section
elements).


Regards,

-- 
Nicolas Goaziou

Re: (Maybe) enhance `org-element-src-block-interpreter'?

[Thorsten Jolitz]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Thorsten Jolitz <tjolitz@gmail.com> writes:
>
>> I see, thanks. I'm still getting used to the 'org-element API', but it
>> seems to be much easier than I thought: just use
>> org-element-at-point' and
>> `org-element-interpret-data' in most cases ...
>
> Correct.
>
> For completeness, if you're working at the local level (i.e. with
> `org-element-at-point'), available accessors are
>
>   - `org-element-type'
>   - `org-element-property'
>
> When you're working at the global level (i.e. with
> `org-element-parse-buffer'), you get another accessor,
> `org-element-contents', and some tools to modify the parse tree
>
>   - `org-element-put-property'
>   - `org-element-adopt-element'
>   - `org-element-insert-before'
>   - `org-element-extract-element'
>   - `org-element-set-element'
>
> In both cases, `org-element-interpret-data' is the only way to turn data
> back into Org syntax.

That makes things much clearer, thanks.

I definitely would have used `org-element-put-property' to modify a
'local' parse-tree too, but I can just as well directly use `plist-put'
on the raw plist in its cdr - would that be the correct way?

-- 
cheers,
Thorsten

Re: (Maybe) enhance `org-element-src-block-interpreter'?

[Thorsten Jolitz]

Thorsten Jolitz <tjolitz@gmail.com> writes:

> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>
>> Thorsten Jolitz <tjolitz@gmail.com> writes:
>>
>>> I see, thanks. I'm still getting used to the 'org-element API', but it
>>> seems to be much easier than I thought: just use
>>> org-element-at-point' and
>>> `org-element-interpret-data' in most cases ...
>>
>> Correct.
>>
>> For completeness, if you're working at the local level (i.e. with
>> `org-element-at-point'), available accessors are
>>
>>   - `org-element-type'
>>   - `org-element-property'
>>
>> When you're working at the global level (i.e. with
>> `org-element-parse-buffer'), you get another accessor,
>> `org-element-contents', and some tools to modify the parse tree
>>
>>   - `org-element-put-property'
>>   - `org-element-adopt-element'
>>   - `org-element-insert-before'
>>   - `org-element-extract-element'
>>   - `org-element-set-element'
>>
>> In both cases, `org-element-interpret-data' is the only way to turn data
>> back into Org syntax.
>
> That makes things much clearer, thanks.
>
> I definitely would have used `org-element-put-property' to modify a
> 'local' parse-tree too, but I can just as well directly use `plist-put'
> on the raw plist in its cdr - would that be the correct way?

I tried it out, works very nicely in this buffer:

* ORG SCRATCH

#+name: myPicoBlock
#+header: :var X=5
#+begin_src picolisp :results value raw :tangle no
  (+ 3 X)
#+end_src

#+results:
8

Evaluating the following src-block with C-c C-c gives an error:
 
,----
| split-string: Wrong type argument: stringp, nil
`----

but it can be copied to the *scratch* buffer and evaluated there.

#+begin_src emacs-lisp :results
(defun tj/try-parser-api ()
  (let (ptree)
    (org-babel-map-src-blocks nil
      (when (string= lang "picolisp")
	(setq ptree
	      (delq nil
		    (list (org-element-at-point) ptree)))))
    (org-element-put-property (car ptree) :value " (* 5 X)\n")
    (message "%S" ptree)		    
    (mapconcat 
     'org-element-interpret-data
     ptree "\n")))
#+end_src

Then interpreting works as expected:

#+begin_src emacs-lisp :results raw
(tj/try-parser-api)
#+end_src

#+results:
#+NAME: myPicoBlock
#+HEADER: :var X=5
#+BEGIN_SRC picolisp :results value raw :tangle no
   (* 5 X)
#+END_SRC

#+results: myPicoBlock
25


and I can actually use `org-element-put-property' on the car of the
local parse-tree (but need to add an "\n"). 

-- 
cheers,
Thorsten

Re: (Maybe) enhance `org-element-src-block-interpreter'?

[Nicolas Goaziou]

Thorsten Jolitz <tjolitz@gmail.com> writes:

> I definitely would have used `org-element-put-property' to modify a
> 'local' parse-tree too, but I can just as well directly use `plist-put'
> on the raw plist in its cdr - would that be the correct way?

You shouldn't do this. Local value is cached and
`org-element-put-property' is destructive. You might break cache.

Applying `org-element-put-property' on a copy of the returned value is
OK, though.


Regards,

-- 
Nicolas Goaziou