org-export: how to copy the parsed tree?

org-export: how to copy the parsed tree?

[Vitalie Spinu]

Hi, 

Is there an easy way to copy org sub-tree in :filter-parse-tree?

The structure of the parsed tree is somewhat complicated with recursive
references to parents in multiple places. So, copy-tree infloops.

Thanks, 

   Vitalie
   

Re: org-export: how to copy the parsed tree?

[Thorsten Jolitz]

Vitalie Spinu <spinuvit@gmail.com> writes:

> Is there an easy way to copy org sub-tree in :filter-parse-tree?
>
> The structure of the parsed tree is somewhat complicated with recursive
> references to parents in multiple places. So, copy-tree infloops.

You will get a better answer most likely, but with (org-no-properties
contents) you can get the original Org-syntax of a parse-tree element,
e.g. applied to all sections (untested):

,----------------------------------------------------------------
| (defun org-myexp-section (section contents info)
|   "Transcode SECTION element into myexp syntax.
| CONTENTS is its contents, as a string or nil. INFO is ignored."
|   (if (and contents (stringp contents) (> (length contents) 0))
|       (format "%S"
|               (org-no-properties contents))
|     ""))
`----------------------------------------------------------------

-- 
cheers,
Thorsten

Re: org-export: how to copy the parsed tree?

[Thorsten Jolitz]

Thorsten Jolitz <tjolitz@gmail.com> writes:

> Vitalie Spinu <spinuvit@gmail.com> writes:
>
>> Is there an easy way to copy org sub-tree in :filter-parse-tree?
>>
>> The structure of the parsed tree is somewhat complicated with recursive
>> references to parents in multiple places. So, copy-tree infloops.
>
> You will get a better answer most likely, but with (org-no-properties
> contents) you can get the original Org-syntax of a parse-tree element,
> e.g. applied to all sections (untested):
>
> ,----------------------------------------------------------------
> | (defun org-myexp-section (section contents info)
> |   "Transcode SECTION element into myexp syntax.
> | CONTENTS is its contents, as a string or nil. INFO is ignored."
> |   (if (and contents (stringp contents) (> (length contents) 0))
> |       (format "%S"
> |               (org-no-properties contents))
> |     ""))
> `----------------------------------------------------------------

I probably misunderstood your question. I don't know if there is an easy
way to copy a subtree, but I do know about the problems with the
recursive references, so I found a way to replace them with simple
numeric :org-elem-id and :parent-id attributes:

,--------------------------------------------
| :filters-alist '((:filter-parse-tree
|     . org-myexp-filter-parse-tree-function)
`--------------------------------------------

#+begin_src emacs-lisp
(defun org-myexp-filter-parse-tree-function (tree backend info)
  "Filter complete parsed TREE ignoring BACKEND and INFO."
   ;; optional
   (org-myexp-add-children
    (org-myexp-add-parent-ids
     (org-myexp-add-ids tree backend info)
     backend info)
    backend info))


(defun org-myexp-add-ids (tree backend info)
  "Add ':org-elem-id' property to each element of parse TREE."
  (let ((counter 1)
        (structure 1))
    (org-element-map tree myexp-default-map-types
      (lambda (--elem)
          (org-element-put-property --elem :org-elem-id counter)
          (setq counter (1+ counter))
          (and (eq (org-element-type --elem) 'plain-list)
               (org-element-put-property --elem :structure-id structure)
               (setq structure (1+ structure))))))
  tree)

(defun org-myexp--collect-children (tree)
  "Return alist with '(org-elem-id . parent-id)' pairs.
The data is collected from parse TREE."
  (let (child-lst)
    (org-element-map tree 'headline
      (lambda (--headline)
        (push (cons (org-element-property :org-elem-id --headline)
                    (org-element-property :parent-id --headline))
              child-lst)))
    child-lst))

;; optional
(defun org-myexp-add-children (tree backend info)
  "Add `:children' property to each headline in parse TREE.
Assumes that all headlines are tagged with an `:org-elem-id' property
and that the circular-list read-syntax of the `:parent' attribute
has been replaced with simple integer values (the :org-elem-id of the
elements parent)."
  (let ((pairs (org-myexp--collect-children tree)))
    (org-element-map tree 'headline
      (lambda (--elem)
        (org-element-put-property
         --elem :children
         (reverse
          (delq nil
                (mapcar
                 (lambda (--pair)
                   (and (eq (cdr --pair)
                            (org-element-property :org-elem-id --elem))
                        (car --pair)))
                 pairs)))))))
  tree)

(defun org-myexp-add-parent-ids (tree backend info)
  "Add `:parent-id' and `:parent-structure-id' to parse-tree TREE."
  (org-element-map tree myexp-all-map-types
    (lambda (--elem)
      (let ((par (org-element-property :parent --elem)))
        (and (eq (org-element-type --elem) 'item)
             (eq (org-element-type par) 'plain-list)
             (org-element-put-property
              --elem :parent-structure-id
              (org-element-property :structure-id par)))
        (org-element-put-property
         --elem :parent-id
         (if (eq (org-element-type par) 'org-data)
             0
           (org-element-property :org-elem-id par)))))
    nil nil nil 'WITH-AFFILIATED)
  tree)
#+end_src

Then in the transcode functions I build the parse-tree again without
the circular :parent attribut, but with the information contained in it
now contained in :org-elem-id and :parent-id:

#+begin_src emacs-lisp
(defun org-myexp-headline (headline contents info)
  "Transcode HEADLINE element into myexp syntax.
CONTENTS is its contents, as a string or nil.  INFO is ignored."
  (let ((props org-myexp-node-properties))
    (setq org-myexp-node-properties nil)
    (format "(headline %S %s) "
            (list
             'org-elem-id
             (org-element-property :org-elem-id headline)
             'parent-id
             (org-element-property :parent-id headline)
             [...])
            (org-no-properties contents))))
#+end_src

Not really an easy way, and I needed it for a very special purpose so it
might not fit your needs. However, this can be used to convert the
circular parse-tree to a non-circular nested list that all the usual
list processing functions can digest.

But I must say that I simply transformed all the SECTIONS to Org-syntax
(with no circular refs at all), so I ended up with only a handful
transcode functions.

Doing the above on the whole parse-tree with all its circular references
might be a tedious exercise ...

-- 
cheers,
Thorsten

Re: org-export: how to copy the parsed tree?

[Nicolas Goaziou]

Hello,

Vitalie Spinu <spinuvit@gmail.com> writes:

> Is there an easy way to copy org sub-tree in :filter-parse-tree?
>
> The structure of the parsed tree is somewhat complicated with recursive
> references to parents in multiple places. So, copy-tree infloops.

This is not really an answer to your question, but do you really need to
copy the full tree? There are tools to modify it. You can also build
a new parse tree and return it.


Regards,

-- 
Nicolas Goaziou

Re: org-export: how to copy the parsed tree?

[Vitalie Spinu]

 >>> Nicolas Goaziou on Sat, 01 Mar 2014 12:36:30 +0100 wrote:

 > Hello,
 > Vitalie Spinu <spinuvit@gmail.com> writes:

 >> Is there an easy way to copy org sub-tree in :filter-parse-tree?
 >> 
 >> The structure of the parsed tree is somewhat complicated with recursive
 >> references to parents in multiple places. So, copy-tree infloops.

 > This is not really an answer to your question, but do you really need to
 > copy the full tree? There are tools to modify it. You can also build
 > a new parse tree and return it.

Thanks Nocolas. I need to copy only a subtree associated with a
headline. In a nutshell I need to "flatten" the tree of headlines for
the csv type exporter. That is, I need to replicate the parent headlines
across the children. For example
   
   * H1
     bla bla
   ** HA
      some text
   ** HB
      some other text
   
would translate into 
   
   * H1
     bla bla
   ** HA
      some text
   * H1
     bla bla
   ** HB
      some other text

I played with org-element-extract-element, org-element-set-contents and
org-element-insert-before but failed a big deal.

Main problem seems to be the lack of uniform interface to handle parents
in connection to secondary elements (which I don't understand at this
point). Is headline a secondary element?

The :parent seems to be stored in 2 places for the headline, in the
headline itself and in the :title. Most of the org code uses
org-element-put-property and org-element-property to access the parent
but some code like org-element-extract-element uses :parent from
:title. This makes it difficult to program as there are no explicit
getter/setter for the parent.

BTW, is there a way to pretty print the org tree? I think that's the
main barrier for me in understanding how it all works.

Thanks, 

  Vitalie

Re: org-export: how to copy the parsed tree?

[Nicolas Goaziou]

Vitalie Spinu <spinuvit@gmail.com> writes:

> Thanks Nocolas. I need to copy only a subtree associated with a
> headline. In a nutshell I need to "flatten" the tree of headlines for
> the csv type exporter. That is, I need to replicate the parent headlines
> across the children. For example
>    
>    * H1
>      bla bla
>    ** HA
>       some text
>    ** HB
>       some other text
>    
> would translate into 
>    
>    * H1
>      bla bla
>    ** HA
>       some text
>    * H1
>      bla bla
>    ** HB
>       some other text
>
> I played with org-element-extract-element, org-element-set-contents and
> org-element-insert-before but failed a big deal.

In this case, I think I would build the tree, since data is duplicated
(e.g. "* H1"), not moved.

> Main problem seems to be the lack of uniform interface to handle parents
> in connection to secondary elements (which I don't understand at this
> point).

There is no such thing as "secondary elements". I think you mean
"secondary strings". This is some parsed data that doesn't belong to the
main hierarchy in the tree.

I don't think any interface problem for :parent property. You simply use
`org-element-property' and `org-element-put-property'.

> Is headline a secondary element?

No. A headline is a full-fledged element. But, besides its contents, it
contains additional data (its title) which is Org syntax and therefore
qualifies as a secondary string.

> The :parent seems to be stored in 2 places for the headline, in the
> headline itself and in the :title.

You are wrong. Headline's parent is stored only in the :parent property
of the headline. You get it with:

  (org-element-property :parent headline)

Within `:title', `:parent' property refers to the parent of the objects
contained in :title, which is the headline itself:

  (eq (org-element-property :parent
                            ;; The first object in the :title property.
                            (car (org-element-property :title headline)))
      headline)  => t

There is only one parent per object/element/string.

> Most of the org code uses org-element-put-property and
> org-element-property to access the parent but some code like
> org-element-extract-element uses :parent from :title.

`org-element-extract-element' checks if the element/object you're
removing belong to a secondary string or to the contents of the current
element.

Moreover, this function removes the :parent property, so you shouldn't
bother about where it came from.

> This makes it difficult to program as there are no explicit
> getter/setter for the parent.

See above.

> BTW, is there a way to pretty print the org tree? I think that's the
> main barrier for me in understanding how it all works.

Use `pp' and set both `print-level' and `print-circle' accordingly.


Regards,

-- 
Nicolas Goaziou

Re: org-export: how to copy the parsed tree?

[Vitalie Spinu]

 >>> Nicolas Goaziou on Sat, 01 Mar 2014 22:02:55 +0100 wrote:

[...]

 > In this case, I think I would build the tree, since data is duplicated
 > (e.g. "* H1"), not moved.

Is there an idiomatic way to do this? Like constructors of org elements?
Or, one just starts with an empty '(headline) list and populate it with
org-element-set-contents and org-element-put-property?

[...]

 >> The :parent seems to be stored in 2 places for the headline, in the
 >> headline itself and in the :title.

 > You are wrong. Headline's parent is stored only in the :parent property
 > of the headline. You get it with:

 >   (org-element-property :parent headline)

 > Within `:title', `:parent' property refers to the parent of the objects
 > contained in :title, which is the headline itself:

 >   (eq (org-element-property :parent
 >                             ;; The first object in the :title property.
 >                             (car (org-element-property :title headline)))
 >       headline)  => t

It makes sense now. Thanks.

[...]

 >> BTW, is there a way to pretty print the org tree? I think that's the
 >> main barrier for me in understanding how it all works.

 > Use `pp' and set both `print-level' and `print-circle' accordingly.

Thanks. Didn't know about this.

  Vitalie

Re: org-export: how to copy the parsed tree?

[Nicolas Goaziou]

Hello,

Vitalie Spinu <spinuvit@gmail.com> writes:

> Is there an idiomatic way to do this? Like constructors of org elements?
> Or, one just starts with an empty '(headline) list

You can even start with '(org-data nil).

> and populate it with org-element-set-contents and
> org-element-put-property?

I would use `org-element-adopt-elements' instead
`org-element-set-contents', which is too low level. The former will take
care of setting the appropriate :parent property.

You can probably extract headlines with `org-element-extract-element',
duplicate them as many times as you want, and insert them back with
`org-element-adopt-elements'. This is just a rough idea; I don't have
any algorithm to offer.


Regards,

-- 
Nicolas Goaziou