emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed

* question on org-element-interpret-data and when it works
@ 2018-02-26  0:12 John Kitchin
  2018-02-26 10:10 ` Nicolas Goaziou
  2018-03-03 11:43 ` Thorsten Jolitz
  0 siblings, 2 replies; 9+ messages in thread
From: John Kitchin @ 2018-02-26  0:12 UTC (permalink / raw)
  To: org-mode-email

[-- Attachment #1: Type: text/plain, Size: 3032 bytes --]

I am trying to find some ways to programatically modify org-elements that
use fewer regexps and motion commands. It seems like org-dp (
https://github.com/tj64/org-dp) was intended to do that but it is not clear
enough how you might use it, and it also doesn't seem to support
plain-lists yet.

What I imagined happening is that I would get the element to modify as a
data structure, modify the data structure, and replace the old element with
an interpreted version of the modified data structure.

I want to do something like a radio list where only one box can be checked
at a time. Suppose I have this:

#+attr_org: :radio
- [ ] one
- [ ] two
- [ ] three

It gets represented from org-element-context as:

(plain-list
 (:type unordered :begin 579 :end 630 :contents-begin 598 :contents-end 630
:structure
((598 0 "- " nil "[ ]" nil 608)
(608 0 "- " nil "[ ]" nil 618)
(618 0 "- " nil "[ ]" nil 630))
:post-blank 0 :post-affiliated 598 :attr_org
(":radio")
:parent nil))

What I thought I could is something like (here I simulate having point in
the first item):

#+BEGIN_SRC emacs-lisp :results code
(let* ((p 600) ;where current point is
       n
       (data '(plain-list
       (:type unordered :begin 579 :end 630 :contents-begin 598
:contents-end 630 :structure
      ((598 0 "- " nil "[ ]" nil 608)
       (608 0 "- " nil "[ ]" nil 618)
       (618 0 "- " nil "[ ]" nil 630))
      :post-blank 0 :post-affiliated 598 :attr_org
      (":radio")
      :parent nil)))
       (structure (plist-get (cadr data) :structure)))
  (loop for i from 0 for item in structure
do
(if (and (>= p (first item))
(< p (seventh item)))
    ;; in the item, toggle it
    (setf (fifth item) (if (string= "[X]" (fifth item))
   "[ ]"
"[X]"))
  ;; not on the item
  (setf (fifth item) "[ ]")))

  data)
#+END_SRC

Which outputs:

#+RESULTS:
#+BEGIN_SRC emacs-lisp
(plain-list
 (:type unordered :begin 579 :end 630 :contents-begin 598 :contents-end 630
:structure
((598 0 "- " nil "[X]" nil 608)
(608 0 "- " nil "[ ]" nil 618)
(618 0 "- " nil "[ ]" nil 630))
:post-blank 0 :post-affiliated 598 :attr_org
(":radio")
:parent nil))
#+END_SRC

As a step towards getting to a data structure I could programmatically
modify, and then reinterpret, I made this little function:

(defun ointerp ()
  (interactive)
  (let ((el (org-element-context)))
    (kill-new (org-element-interpret-data el))))

It works on some things, e.g. headlines, src blocks. I put the point on one
of those things, run this command, and then I can paste it somewhere to see
that it did indeed work.

But, it does not work on plain-lists, or paragraphs. I either get an empty
string, or Wrong type argument: char-or-string-p, nil

Is it possible to do what I am describing? Am I just missing how to get the
element data in the right form?

Thanks,



John

-----------------------------------
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu

[-- Attachment #2: Type: text/html, Size: 7038 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

• * Re: question on org-element-interpret-data and when it works
    2018-02-26  0:12 question on org-element-interpret-data and when it works John Kitchin
  @ 2018-02-26 10:10 ` Nicolas Goaziou
    2018-03-03 11:43 ` Thorsten Jolitz
    1 sibling, 0 replies; 9+ messages in thread
  From: Nicolas Goaziou @ 2018-02-26 10:10 UTC (permalink / raw)
    To: John Kitchin; +Cc: org-mode-email
  
  Hello,
  
  John Kitchin <jkitchin@andrew.cmu.edu> writes:
  
  > I am trying to find some ways to programatically modify org-elements that
  > use fewer regexps and motion commands. It seems like org-dp (
  > https://github.com/tj64/org-dp) was intended to do that but it is not clear
  > enough how you might use it, and it also doesn't seem to support
  > plain-lists yet.
  
  [...]
  
  > It works on some things, e.g. headlines, src blocks. I put the point on one
  > of those things, run this command, and then I can paste it somewhere to see
  > that it did indeed work.
  >
  > But, it does not work on plain-lists, or paragraphs. I either get an empty
  > string, or Wrong type argument: char-or-string-p, nil
  >
  > Is it possible to do what I am describing? Am I just missing how to get the
  > element data in the right form?
  
  You cannot change a non-terminal element without changing its contents.
  In particular plain-lists, tables and paragraphs are fully defined by
  their contents, i.e., they do not decorate contents like headlines. In
  this case, data returned by `org-element-context' is incomplete in this
  case. You probably need to parse the buffer between :begin and :end and
  modify structure recursively.
  
  Regards,
  
  -- 
  Nicolas Goaziou
  
  ^ permalink raw reply	[flat|nested] 9+ messages in thread

• * Re: question on org-element-interpret-data and when it works
    2018-02-26  0:12 question on org-element-interpret-data and when it works John Kitchin
    2018-02-26 10:10 ` Nicolas Goaziou
  @ 2018-03-03 11:43 ` Thorsten Jolitz
    2018-03-03 12:45   ` Nicolas Goaziou
    1 sibling, 1 reply; 9+ messages in thread
  From: Thorsten Jolitz @ 2018-03-03 11:43 UTC (permalink / raw)
    To: emacs-orgmode
  
  John Kitchin <jkitchin@andrew.cmu.edu> writes:
  
  Hello John,
  
  > I am trying to find some ways to programatically modify org-elements
  > that use fewer regexps and motion commands. It seems like org-dp
  > (https://github.com/tj64/org-dp) was intended to do that
  
  thats right, that's it's exact use case
  
  > but it is not clear enough how you might use it, and it also doesn't
  > seem to support plain-lists yet.
  
  it's actually split into core and lib functionality, org-dp.el being the
  core and org-dp-lib.el being the lib.
  
  AFAIK it org-dp does work, and in its core it's based on just two
  commands, one for CREATE and one for REWIRE (=modify) org elements
  (locally, not replacing Nicolas org element framework, but rather making
  it easy to use locally without the need for a whole parse tree).
  
  The basic idea:
   - internally, all org elements look alike (plists)
   - org elements have many properties, but we are interested only in
     those used by the org element interpreter (that creates org syntax
     out of Emacs Lisp plists). These are surprisingly few.
   - luckily, plists ignore all elements that are not accessed, making
     transformation between different plists for different org elements
     much easier
  
  You should be able to do anything you want with org elements with just
  these two functions, org-dp-create and org-dp-rewire.
  
  There is a generic prompt function two. Never write interactive specs
  again, just use this one prompt functions for all org element related
  prompts (or utility commands based on the prompt workhorse).
  
  And then there is this fast and simple mapping function for org-dp:
  
  ,----[ C-h f org-dp-map RET ]
  | org-dp-map is a Lisp function in ‘org-dp.el’.
  | 
  | (org-dp-map FUN-WITH-ARGS RGXP &optional MATCH-POS BACKWARD-SEARCH-P
  | BEG END SILENT-P)
  | 
  | Apply quoted FUN-WITH-ARGS at every RGXP match.
  | 
  | [...] 
  | In contrast to other mapping functions in Org-mode, this mapping
  | function does not collect any information about mapped elements,
  | it simply moves point quickly to all positions in a buffer(range)
  | that are matched by a (forward) regexp-search and applies one of
  | ‘org-dp’’s or ‘org-dp-lib’’s functions locally at that
  | point (i.e. without any context information other than that about
  | the parsed element-at-point).
  | 
  | When calling FUN ‘org-dp-create’, or ‘org-dp-rewire’ with
  | argument ELEMENT given, no parsing at all takes places, but newly
  | created of modified elements can be inserted at point.
  | 
  | This mapping function wraps its body in ‘save-excursion’ and
  | ‘save-match-data’ calls, so point position and global match-data
  | are preserved. It does not widen the buffer before executing its
  | body, so buffer restrictions are respected. 
  `----
  
  > What I imagined happening is that I would get the element to modify as a
  > data structure, modify the data structure, and replace the old element
  > with an interpreted version of the modified data structure.
  
  Thats exactly what this central org-dp workhorse function does:
  
  ,----[ C-h f org-dp-rewire RET ]
  | org-dp-rewire is a Lisp function in ‘org-dp.el’.
  | 
  | (org-dp-rewire ELEM-TYPE &optional CONTENTS REPLACE AFFILIATED ELEMENT
  | &rest ARGS)
  | 
  | Rewire element-at-point or ELEMENT (if given).
  | 
  | [...] 
  | ELEM-TYPE is one of the types in ‘org-element-all-elements’. If
  | it is nil, the element type of the original element is used. ARGS
  | is a plist consisting of key-val pairs of all other keyword
  | arguments given, defining the (rewired) element’s properties.
  | 
  | The former value of an element property can be reused in the
  | creation of a new value by giving a ‘lambda’ expession or
  | function taking two arguments (instead of just a value) to a
  | key. The first argument will then be replaced by the property’s
  | former value when applying the function. The second argument
  | should be the parsed element itself, enabling access to its type
  | and all its properties inside of the lambda expression.
  `----
  
  But, as Nicolas said in his answer, plain lists, tables, properties
  etc are nested org elements, what is most obvious with tables:
  a table is just a container for table rows that hold the actual data
  (with some meta data in the container).
  
  Thats why org-dp-lib.el has several related functions:
  
  ,----
  | org-dp-lib.el
  | 40:(defun org-dp-wrap-in-block (&optional lines user-info &rest prompt-spec)
  | 204:(defun org-dp-toggle-headers (&optional action)
  | 290:(defun org-dp-org-props ()
  | 307:(defun org-dp-filter-node-props (filter &optional negate-p verbose-p)
  | 366:(defun org-dp-create-table (row-lst &optional tblfm table-el-p insert-p)
  | 416:(defun org-dp-create-plain-list (item-lst &optional insert-p)
  | 452:(defun org-dp-create-property-drawer (node-prop-lst &optional insert-p)
  `----
  
  They should hopefully be pretty well documented, since I use to do that.
  
  To get started with org-dp, you really need these 3 (or 4)
  functions. They spare you typing, and more important, they tell which
  properties of an org element matter for its interpretation.
  
  CREATE:
  
  ,----[ C-h f tempo-template-org-dp-create RET ]
  | tempo-template-org-dp-create is an interactive Lisp function.
  | 
  | (tempo-template-org-dp-create &optional ARG)
  | 
  | Insert org-dp-create template.
  | 
  | [back]
  `----
  
  or this one with explanations to get started (not so much for productive use):
  
  ,----[ C-h f tempo-template-org-dp-create-with-comments RET ]
  | tempo-template-org-dp-create-with-comments is an interactive Lisp
  | function.
  | 
  | (tempo-template-org-dp-create-with-comments &optional ARG)
  | 
  | Insert org-dp-create template.
  | 
  | [back]
  `----
  
  Example: you will be prompted for the org element (in this case I
  decided to use "example-block"), you don't have to type anything, just
  do this single selection when prompted.
  
  ,----
  | ;; Affiliated keywords: '(:kw1 val1 :kw2 val2 ...)
  | ;; :name "val"
  | ;; :plot "val"
  | ;; :results ("val") or ("val" "key")
  | ;; :header ("val") or ("val1" "val2")
  | ;; :caption (("val")) or (("val" "key")) or
  | ;;          (("val2" "key2") ("val2" "key2"))
  | ;; :attr_xyz ("val") or ("val1" "val2")
  | (org-dp-create 'example-block nil t ;cont ins
  | nil ;aff 
  | :switches ""
  | :preserve-indent ""
  | :value ""
  | )
  `----
  
  What you see: you can just write "Hello World" in the 
  
  ,----
  | :value ""
  `----
  
  parameter and will get an example block with "Hello World" as content
  (funny enough, some org elements have 'value' as their content, others
  'content').  Check the docstring of org-dp-create for more info on its
  args.  You can add affiliated keywords too, as demonstrated (as one
  function arg).
  
  
  REWIRE:
  
  ,----[ C-h f tempo-template-org-dp-rewire RET ]
  | tempo-template-org-dp-rewire is an interactive Lisp function.
  | 
  | (tempo-template-org-dp-rewire &optional ARG)
  | 
  | Insert org-dp-rewire template.
  | 
  | [back]
  `----
  
  or this one, if you don't just want to set properties, but need a lambda
  function to do work in calculation the new property (with access to the
  old property and the whole plist of the old element you want to
  rewire/modify):
  
  ,----[ C-h f tempo-template-org-dp-rewire-lambda RET ]
  | tempo-template-org-dp-rewire-lambda is an interactive Lisp function.
  | 
  | (tempo-template-org-dp-rewire-lambda &optional ARG)
  | 
  | Insert org-dp-rewire template with lambdas.
  | 
  | [back]
  `----
  
  Examples:
  
  ,----
  | (org-dp-rewire 'example-block nil t ;cont ins
  | nil ;aff 
  | nil ;elem 
  | :switches ""
  | :preserve-indent ""
  | :value ""
  | )
  `----
  
  ,----
  | (org-dp-rewire 'example-block nil t ;cont ins
  | nil ;aff 
  | nil ;elem 
  | :switches '(lambda (old elem)  )
  | :preserve-indent '(lambda (old elem)  )
  | :value '(lambda (old elem)  )
  `----
  
  Again, check the docstring of org-dp-rewire for more info on its args.
  
  I hope this helps already, feel free to ask any further questions (maybe
  put me in Cc since I'm not a very frecuent visitor of the list).
  
  If I find time and motivation I might look at your specific example/use
  case, but I cannot promise that.
  
  -- 
  cheers,
  Thorsten
  
  ^ permalink raw reply	[flat|nested] 9+ messages in thread

• • * Re: question on org-element-interpret-data and when it works
      2018-03-03 11:43 ` Thorsten Jolitz
    @ 2018-03-03 12:45   ` Nicolas Goaziou
      2018-03-03 13:02     ` Thorsten Jolitz
      0 siblings, 1 reply; 9+ messages in thread
    From: Nicolas Goaziou @ 2018-03-03 12:45 UTC (permalink / raw)
      To: Thorsten Jolitz; +Cc: emacs-orgmode
    
    Hello,
    
    Thorsten Jolitz <tjolitz@gmail.com> writes:
    
    > (funny enough, some org elements have 'value' as their content, others
    > 'content').
    
    Could you point out where there is such discrepancy in "org-element.el"?
    
    Thank you.
    
    Regards,
    
    -- 
    Nicolas Goaziou
    
    ^ permalink raw reply	[flat|nested] 9+ messages in thread

  • • * Re: question on org-element-interpret-data and when it works
        2018-03-03 12:45   ` Nicolas Goaziou
      @ 2018-03-03 13:02     ` Thorsten Jolitz
        2018-03-03 13:20       ` Nicolas Goaziou
        0 siblings, 1 reply; 9+ messages in thread
      From: Thorsten Jolitz @ 2018-03-03 13:02 UTC (permalink / raw)
        To: emacs-orgmode
      
      Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
      
      Hello,
      
      > Thorsten Jolitz <tjolitz@gmail.com> writes:
      >
      >> (funny enough, some org elements have 'value' as their content, others
      >> 'content').
      >
      > Could you point out where there is such discrepancy in "org-element.el"?
      
      I have defined these two constants in org-dp.el to work around this
      discrepancy (and to know which elements do not have interpreted content
      at all):
      
      ,----
      | (defconst org-dp-no-content-elems
      |   (list 'babel-call 'clock 'comment 'comment-block 'diary-sexp
      | 	'example-block 'fixed-width 'horizontal-rule 'keyword
      | 	'latex-environment 'node-property 'planning 'src-block)
      |   "List of Org elements without interpreted .")
      | 
      | (defconst org-dp-value-blocks
      |   (list 'comment-block 'example-block 'src-block)
      |   "List of Org block that have a :value instead of contents.")
      `----
      
      PS 
      should probably read "... without interpreted content" in the first
      defconst
      
      -- 
      cheers,
      Thorsten
      
      ^ permalink raw reply	[flat|nested] 9+ messages in thread

    • • * Re: question on org-element-interpret-data and when it works
          2018-03-03 13:02     ` Thorsten Jolitz
        @ 2018-03-03 13:20       ` Nicolas Goaziou
          2018-03-03 13:39         ` Thorsten Jolitz
          0 siblings, 1 reply; 9+ messages in thread
        From: Nicolas Goaziou @ 2018-03-03 13:20 UTC (permalink / raw)
          To: Thorsten Jolitz; +Cc: emacs-orgmode
        
        Thorsten Jolitz <tjolitz@gmail.com> writes:
        
        > I have defined these two constants in org-dp.el to work around this
        > discrepancy (and to know which elements do not have interpreted content
        > at all):
        >
        > ,----
        > | (defconst org-dp-no-content-elems
        > |   (list 'babel-call 'clock 'comment 'comment-block 'diary-sexp
        > | 	'example-block 'fixed-width 'horizontal-rule 'keyword
        > | 	'latex-environment 'node-property 'planning 'src-block)
        > |   "List of Org elements without interpreted .")
        > | 
        > | (defconst org-dp-value-blocks
        > |   (list 'comment-block 'example-block 'src-block)
        > |   "List of Org block that have a :value instead of contents.")
        > `----
        
        I don't understand where you think there is a discrepancy here.
        
        ^ permalink raw reply	[flat|nested] 9+ messages in thread

      • • * Re: question on org-element-interpret-data and when it works
            2018-03-03 13:20       ` Nicolas Goaziou
          @ 2018-03-03 13:39         ` Thorsten Jolitz
            2018-03-03 14:06           ` Nicolas Goaziou
            0 siblings, 1 reply; 9+ messages in thread
          From: Thorsten Jolitz @ 2018-03-03 13:39 UTC (permalink / raw)
            To: emacs-orgmode
          
          Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
          
          > Thorsten Jolitz <tjolitz@gmail.com> writes:
          >
          >> I have defined these two constants in org-dp.el to work around this
          >> discrepancy (and to know which elements do not have interpreted content
          >> at all):
          >>
          >> ,----
          >> | (defconst org-dp-no-content-elems
          >> |   (list 'babel-call 'clock 'comment 'comment-block 'diary-sexp
          >> | 	'example-block 'fixed-width 'horizontal-rule 'keyword
          >> | 	'latex-environment 'node-property 'planning 'src-block)
          >> |   "List of Org elements without interpreted .")
          >> | 
          >> | (defconst org-dp-value-blocks
          >> |   (list 'comment-block 'example-block 'src-block)
          >> |   "List of Org block that have a :value instead of contents.")
          >> `----
          >
          > I don't understand where you think there is a discrepancy here.
          
          You used the word 'discrepancy', I simply needed to know for each org
          element what is interpreted and what not. And some have a content,
          others a :value. So if I pass 'Hello World' as content to an example
          block, nothing happens, if I pass it via :value, it appears as the
          blocks ... well, content.
          
          This is no critique, and no problem, and please don't change it (since
          it would be a breaking change in this context). 
          
          With the org-dp tempo-templates, its no problem for org-dp users either,
          since these are smart: they offer you a content arg (cont) to fill, if
          it makes sense, they hide it, if not. And for some block types they
          offer the :value parameter, since it is interpreted.
          
          So I don't see any problem, just something an org-dp user probably
          should be aware of.
          
          -- 
          cheers,
          Thorsten
          
          ^ permalink raw reply	[flat|nested] 9+ messages in thread

        • • * Re: question on org-element-interpret-data and when it works
              2018-03-03 13:39         ` Thorsten Jolitz
            @ 2018-03-03 14:06           ` Nicolas Goaziou
              2018-03-03 14:47             ` Thorsten Jolitz
              0 siblings, 1 reply; 9+ messages in thread
            From: Nicolas Goaziou @ 2018-03-03 14:06 UTC (permalink / raw)
              To: Thorsten Jolitz; +Cc: emacs-orgmode
            
            Thorsten Jolitz <tjolitz@gmail.com> writes:
            
            > You used the word 'discrepancy',
            
            True. I inferred it from
            
              (funny enough, some org elements have 'value' as their content, others
              'content').
            
            which, IMO, sounds like it is a surprising fact.
            
            > I simply needed to know for each org
            > element what is interpreted and what not. And some have a content,
            > others a :value.
            
            As in every AST, some nodes are terminal (no contents), and some are not
            (contents).
            
            This distinction is made in `org-element-greater-elements', i.e.,
            non-terminal elements. See also `org-element-recursive-objects' for
            non-terminal objects.
            
            > So if I pass 'Hello World' as content to an example
            > block, nothing happens, if I pass it via :value, it appears as the
            > blocks ... well, content.
            
            Contents imply Org syntax. This would defeat the purpose of an example
            block.
            
            ^ permalink raw reply	[flat|nested] 9+ messages in thread

          • • * Re: question on org-element-interpret-data and when it works
                2018-03-03 14:06           ` Nicolas Goaziou
              @ 2018-03-03 14:47             ` Thorsten Jolitz
                0 siblings, 0 replies; 9+ messages in thread
              From: Thorsten Jolitz @ 2018-03-03 14:47 UTC (permalink / raw)
                To: emacs-orgmode
              
              Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
              
              > Thorsten Jolitz <tjolitz@gmail.com> writes:
              >
              >> You used the word 'discrepancy',
              >
              > True. I inferred it from
              >
              >   (funny enough, some org elements have 'value' as their content, others
              >   'content').
              >
              > which, IMO, sounds like it is a surprising fact.
              >
              >> I simply needed to know for each org
              >> element what is interpreted and what not. And some have a content,
              >> others a :value.
              >
              > As in every AST, some nodes are terminal (no contents), and some are not
              > (contents).
              >
              > This distinction is made in `org-element-greater-elements', i.e.,
              > non-terminal elements. See also `org-element-recursive-objects' for
              > non-terminal objects.
              >
              >> So if I pass 'Hello World' as content to an example
              >> block, nothing happens, if I pass it via :value, it appears as the
              >> blocks ... well, content.
              >
              > Contents imply Org syntax. This would defeat the purpose of an example
              > block.
              
              Ok, so its just a matter of wording.  
              
              On the computer science side of things, content seems to be org elements
              or objects contained in other org elements (like table rows in a table),
              and on the laymans side of things the text inside of an example block
              looks very much like the blocks content too (while its technically named
              'value' in this case).
              
              -- 
              cheers,
              Thorsten
              
              ^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2018-03-03 14:47 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-02-26  0:12 question on org-element-interpret-data and when it works John Kitchin
2018-02-26 10:10 ` Nicolas Goaziou
2018-03-03 11:43 ` Thorsten Jolitz
2018-03-03 12:45   ` Nicolas Goaziou
2018-03-03 13:02     ` Thorsten Jolitz
2018-03-03 13:20       ` Nicolas Goaziou
2018-03-03 13:39         ` Thorsten Jolitz
2018-03-03 14:06           ` Nicolas Goaziou
2018-03-03 14:47             ` Thorsten Jolitz

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).