* Re: [ANN] Org Elements in contrib
2011-11-21 18:50 [ANN] Org Elements in contrib Nicolas Goaziou
@ 2011-11-21 22:02 ` Martyn Jago
2011-11-22 5:17 ` Eric Abrahamsen
` (3 subsequent siblings)
4 siblings, 0 replies; 9+ messages in thread
From: Martyn Jago @ 2011-11-21 22:02 UTC (permalink / raw)
To: emacs-orgmode
Nicolas Goaziou <n.goaziou@gmail.com> writes:
Hi Nicolas
> Hello,
>
>
> I've added org-element.el in contrib directory. It is a complete parser
> and interpreter for Org syntax.
>
> While it was written to be extensible, it is also an attempt to
> normalize current syntax and provide guidance for its evolution.
>
> Org syntax can be divided into three categories: "Greater elements",
> "Elements" and "Objects".
>
> An object can be defined anywhere on a line. It may span over more than
> a line but never contains a blank one. Objects belong to the following
> types: `emphasis', `entity', `export-snippet', `footnote-reference',
> `inline-babel-call', `inline-src-block', `latex-fragment', `line-break',
> `link', `macro', `radio-target', `statistics-cookie', `subscript',
> `superscript', `target', `time-stamp' and `verbatim'.
>
> An element always starts and ends at the beginning of a line. The only
> element's type containing objects is called a `paragraph'. Other types
> are: `comment', `comment-block', `example-block', `export-block',
> `fixed-width', `horizontal-rule', `keyword', `latex-environment',
> `babel-call', `property-drawer', `quote-section', `src-block', `table'
> and `verse-block'.
>
> Elements containing paragraphs are called greater elements. Concerned
> types are: `center-block', `drawer', `dynamic-block',
> `footnote-definition', `headline', `inlinetask', `item', `plain-list',
> `quote-block' and `special-block'.
>
> Greater elements (excepted `headline' and `item' types) and elements
> (excepted `keyword', `babel-call', and `property-drawer' types) can have
> a fixed set of keywords as attributes. Those are called "affiliated
> keywords", to distinguish them from others keywords, which are
> full-fledged elements. In particular, the "name" affiliated keyword
> allows to label almost any element in an Org buffer.
>
> Notwithstanding affiliated keywords, each greater element, element and
> object has a fixed set of properties attached to it. Among them, three
> are shared by all types: `:begin' and `:end', which refer to the
> beginning and ending buffer positions of the considered element or
> object, and `:post-blank', which holds the number of blank lines, or
> white spaces, at its end.
>
> Some elements also have special properties whose value can hold objects
> themselves (i.e. an item tag, an headline name, a table cell). Such
> values are called "secondary strings".
>
> Lisp-wise, an element or an object can be represented as a list. It
> follows the pattern (TYPE PROPERTIES CONTENTS), where: TYPE is a symbol
> describing the Org element or object. PROPERTIES is the property list
> attached to it. See docstring of appropriate parsing function to get an
> exhaustive list. CONTENTS is a list of elements, objects or raw strings
> contained in the current element or object, when applicable.
>
> An Org buffer is a nested list of such elements and objects, whose type
> is `org-data' and properties is nil.
>
> The first part of this file implements a parser and an interpreter for
> each type of Org syntax.
>
> The next two parts introduce two accessors and a function retrieving the
> smallest element containing point (respectively
> `org-element-get-property', `org-element-get-contents' and
> `org-element-at-point').
>
> The following part creates a fully recursive buffer parser. It also
> provides a tool to map a function to elements or objects matching some
> criteria in the parse tree. Functions of interest are
> `org-element-parse-buffer', `org-element-map' and, to a lesser extent,
> `org-element-parse-secondary-string'.
>
> The penultimate part is the cradle of an interpreter for the obtained
> parse tree: `org-element-interpret-data' (and its relative,
> `org-element-interpret-secondary').
>
> The library ends by furnishing a set of interactive tools for element's
> navigation and manipulation.
>
> More specifically, that last part includes some tools like
> `org-element-forward', `org-element-backward',
> `org-element-drag-forward', `org-element-drag-backward',
> `org-element-mark-element', `org-element-up',
> `org-element-unindent-buffer'...
>
> For the impatient (well, not quite as you're still reading this), you
> can evaluate the following examples in an Org buffer :
>
> (org-element-parse-buffer)
> (org-element-parse-buffer 'headline)
> (org-element-parse-buffer 'headline 'visible-only)
>
> Also, the following code will parse the buffer, interpret the parsed
> tree, and create a canonical copy of it (no indentation, lowercased
> blocks, standard keywords):
>
> #+begin_src org
> (let ((out (org-element-interpret-data (org-element-parse-buffer))))
> (switch-to-buffer (get-buffer-create "*Bijectivep*"))
> (erase-buffer)
> (insert out)
> (goto-char (point-min))
> (org-mode))
> #+end_src
>
> Beside allowing to add keywords like "#+name:", "#+caption:" or
> "#+attr_latex:" to almost any Org element, it also introduces two less
> noticable changes:
>
> 1. "#+label:" keywords are deprecated in favor of "#+name:". Though,
> for now, "label" is still considered as a synonym of "name".
>
> 2. Protected HTML snippets (like @<b>) are no longer supported, as
> they were too specific.
>
> Instead, a general mechanism to inline back-end specific commands
> is created. Thus, the HTML back-end will see "<b>some text<\b>"
> while the LaTeX one will only see "some text" if the buffer
> contains:
>
> @html{<b>}some text@html{<\b>}
>
> Syntax is heavier, but a configurable variable allows to define
> shortcuts, allowing to reduce it to, for example, @h{<b>}. No
> shortcut is provided by default.
>
> Also, the syntax is experimental, and may change if proven to be
> inadequate.
>
>
> I will commit a generic exporter built on top of Elements, along with
> a LaTeX back-end, in a couple of days.
>
> Feedback is welcome.
>
>
> Regards,
Sounds Awesome - and ultimately very testable. I look forward
to the in-depth conversation but it certainly sounds like a move in the
right direction. Good job.
Best, Martyn
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [ANN] Org Elements in contrib
2011-11-21 18:50 [ANN] Org Elements in contrib Nicolas Goaziou
2011-11-21 22:02 ` Martyn Jago
@ 2011-11-22 5:17 ` Eric Abrahamsen
2011-11-22 14:02 ` Brian Wightman
` (2 subsequent siblings)
4 siblings, 0 replies; 9+ messages in thread
From: Eric Abrahamsen @ 2011-11-22 5:17 UTC (permalink / raw)
To: emacs-orgmode
On Tue, Nov 22 2011, Nicolas Goaziou wrote:
> Hello,
>
>
> I've added org-element.el in contrib directory. It is a complete parser
> and interpreter for Org syntax.
This looks great! Is the eventual idea to use this library for other
basic org functionality? It seems ideal for replacing the guts of things
like `org-scan-tags' and family.
--
GNU Emacs 24.0.91.1 (i686-pc-linux-gnu, GTK+ Version 2.24.6)
of 2011-11-07 on pellet
Org-mode version 7.7 (release_7.7.578.ge3ec9)
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [ANN] Org Elements in contrib
2011-11-21 18:50 [ANN] Org Elements in contrib Nicolas Goaziou
2011-11-21 22:02 ` Martyn Jago
2011-11-22 5:17 ` Eric Abrahamsen
@ 2011-11-22 14:02 ` Brian Wightman
2011-11-22 16:00 ` Thomas S. Dye
2012-06-10 16:40 ` Michael Brand
4 siblings, 0 replies; 9+ messages in thread
From: Brian Wightman @ 2011-11-22 14:02 UTC (permalink / raw)
To: Nicolas Goaziou; +Cc: Org Mode List
On Mon, Nov 21, 2011 at 12:50 PM, Nicolas Goaziou <n.goaziou@gmail.com> wrote:
> I've added org-element.el in contrib directory. It is a complete parser
> and interpreter for Org syntax.
>
> While it was written to be extensible, it is also an attempt to
> normalize current syntax and provide guidance for its evolution.
This is also a good thing (IMO) from the viewpoint of being able to
provide external (non-org-mode) parsers for org-mode files. If
org-element is the canonical definition of org syntax (not semantics,
just syntax), all other parsers must adhere to this or they are in
error.
A slight evolution of this would be to have this definition defined in
the tests, and have org-element, or any other parser, adhere to those
tests. Defining a canon for org-mode syntax is, in any form, a big
step forward.
Brian
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [ANN] Org Elements in contrib
2011-11-21 18:50 [ANN] Org Elements in contrib Nicolas Goaziou
` (2 preceding siblings ...)
2011-11-22 14:02 ` Brian Wightman
@ 2011-11-22 16:00 ` Thomas S. Dye
2011-11-23 20:12 ` Wes Hardaker
2012-06-10 16:40 ` Michael Brand
4 siblings, 1 reply; 9+ messages in thread
From: Thomas S. Dye @ 2011-11-22 16:00 UTC (permalink / raw)
To: Nicolas Goaziou; +Cc: Org Mode List
Aloha Nicolas,
This looks brilliant. The interactive functions seem to know their way
around my various org-mode files. Looking forward to the generic
exporter and the LaTeX back-end.
All the best,
Tom
Nicolas Goaziou <n.goaziou@gmail.com> writes:
> Hello,
>
>
> I've added org-element.el in contrib directory. It is a complete parser
> and interpreter for Org syntax.
>
> While it was written to be extensible, it is also an attempt to
> normalize current syntax and provide guidance for its evolution.
>
> Org syntax can be divided into three categories: "Greater elements",
> "Elements" and "Objects".
>
> An object can be defined anywhere on a line. It may span over more than
> a line but never contains a blank one. Objects belong to the following
> types: `emphasis', `entity', `export-snippet', `footnote-reference',
> `inline-babel-call', `inline-src-block', `latex-fragment', `line-break',
> `link', `macro', `radio-target', `statistics-cookie', `subscript',
> `superscript', `target', `time-stamp' and `verbatim'.
>
> An element always starts and ends at the beginning of a line. The only
> element's type containing objects is called a `paragraph'. Other types
> are: `comment', `comment-block', `example-block', `export-block',
> `fixed-width', `horizontal-rule', `keyword', `latex-environment',
> `babel-call', `property-drawer', `quote-section', `src-block', `table'
> and `verse-block'.
>
> Elements containing paragraphs are called greater elements. Concerned
> types are: `center-block', `drawer', `dynamic-block',
> `footnote-definition', `headline', `inlinetask', `item', `plain-list',
> `quote-block' and `special-block'.
>
> Greater elements (excepted `headline' and `item' types) and elements
> (excepted `keyword', `babel-call', and `property-drawer' types) can have
> a fixed set of keywords as attributes. Those are called "affiliated
> keywords", to distinguish them from others keywords, which are
> full-fledged elements. In particular, the "name" affiliated keyword
> allows to label almost any element in an Org buffer.
>
> Notwithstanding affiliated keywords, each greater element, element and
> object has a fixed set of properties attached to it. Among them, three
> are shared by all types: `:begin' and `:end', which refer to the
> beginning and ending buffer positions of the considered element or
> object, and `:post-blank', which holds the number of blank lines, or
> white spaces, at its end.
>
> Some elements also have special properties whose value can hold objects
> themselves (i.e. an item tag, an headline name, a table cell). Such
> values are called "secondary strings".
>
> Lisp-wise, an element or an object can be represented as a list. It
> follows the pattern (TYPE PROPERTIES CONTENTS), where: TYPE is a symbol
> describing the Org element or object. PROPERTIES is the property list
> attached to it. See docstring of appropriate parsing function to get an
> exhaustive list. CONTENTS is a list of elements, objects or raw strings
> contained in the current element or object, when applicable.
>
> An Org buffer is a nested list of such elements and objects, whose type
> is `org-data' and properties is nil.
>
> The first part of this file implements a parser and an interpreter for
> each type of Org syntax.
>
> The next two parts introduce two accessors and a function retrieving the
> smallest element containing point (respectively
> `org-element-get-property', `org-element-get-contents' and
> `org-element-at-point').
>
> The following part creates a fully recursive buffer parser. It also
> provides a tool to map a function to elements or objects matching some
> criteria in the parse tree. Functions of interest are
> `org-element-parse-buffer', `org-element-map' and, to a lesser extent,
> `org-element-parse-secondary-string'.
>
> The penultimate part is the cradle of an interpreter for the obtained
> parse tree: `org-element-interpret-data' (and its relative,
> `org-element-interpret-secondary').
>
> The library ends by furnishing a set of interactive tools for element's
> navigation and manipulation.
>
> More specifically, that last part includes some tools like
> `org-element-forward', `org-element-backward',
> `org-element-drag-forward', `org-element-drag-backward',
> `org-element-mark-element', `org-element-up',
> `org-element-unindent-buffer'...
>
> For the impatient (well, not quite as you're still reading this), you
> can evaluate the following examples in an Org buffer :
>
> (org-element-parse-buffer)
> (org-element-parse-buffer 'headline)
> (org-element-parse-buffer 'headline 'visible-only)
>
> Also, the following code will parse the buffer, interpret the parsed
> tree, and create a canonical copy of it (no indentation, lowercased
> blocks, standard keywords):
>
> #+begin_src org
> (let ((out (org-element-interpret-data (org-element-parse-buffer))))
> (switch-to-buffer (get-buffer-create "*Bijectivep*"))
> (erase-buffer)
> (insert out)
> (goto-char (point-min))
> (org-mode))
> #+end_src
>
> Beside allowing to add keywords like "#+name:", "#+caption:" or
> "#+attr_latex:" to almost any Org element, it also introduces two less
> noticable changes:
>
> 1. "#+label:" keywords are deprecated in favor of "#+name:". Though,
> for now, "label" is still considered as a synonym of "name".
>
> 2. Protected HTML snippets (like @<b>) are no longer supported, as
> they were too specific.
>
> Instead, a general mechanism to inline back-end specific commands
> is created. Thus, the HTML back-end will see "<b>some text<\b>"
> while the LaTeX one will only see "some text" if the buffer
> contains:
>
> @html{<b>}some text@html{<\b>}
>
> Syntax is heavier, but a configurable variable allows to define
> shortcuts, allowing to reduce it to, for example, @h{<b>}. No
> shortcut is provided by default.
>
> Also, the syntax is experimental, and may change if proven to be
> inadequate.
>
>
> I will commit a generic exporter built on top of Elements, along with
> a LaTeX back-end, in a couple of days.
>
> Feedback is welcome.
>
>
> Regards,
--
Thomas S. Dye
http://www.tsdye.com
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [ANN] Org Elements in contrib
2011-11-22 16:00 ` Thomas S. Dye
@ 2011-11-23 20:12 ` Wes Hardaker
0 siblings, 0 replies; 9+ messages in thread
From: Wes Hardaker @ 2011-11-23 20:12 UTC (permalink / raw)
To: Thomas S. Dye; +Cc: Org Mode List, Nicolas Goaziou
>>>>> On Tue, 22 Nov 2011 06:00:49 -1000, tsd@tsdye.com (Thomas S. Dye) said:
TSD> This looks brilliant. The interactive functions seem to know their way
TSD> around my various org-mode files. Looking forward to the generic
TSD> exporter and the LaTeX back-end.
Most specifically, the hardest part of writing the org-export-generic
functionality was the parser. When I started on it, it very quickly
showed that a generic parser with an API was needed (but I didn't have
the time to do that). This sounds like a perfect replacement for the
guts of the org-export-generic parser.
--
Wes Hardaker
My Pictures: http://capturedonearth.com/
My Thoughts: http://pontifications.hardakers.net/
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [ANN] Org Elements in contrib
2011-11-21 18:50 [ANN] Org Elements in contrib Nicolas Goaziou
` (3 preceding siblings ...)
2011-11-22 16:00 ` Thomas S. Dye
@ 2012-06-10 16:40 ` Michael Brand
2012-06-12 12:32 ` Nicolas Goaziou
4 siblings, 1 reply; 9+ messages in thread
From: Michael Brand @ 2012-06-10 16:40 UTC (permalink / raw)
To: Nicolas Goaziou; +Cc: Org Mode List
Hi Nicolas
On Mon, Nov 21, 2011 at 7:50 PM, Nicolas Goaziou <n.goaziou@gmail.com> wrote:
> I've added org-element.el in contrib directory. It is a complete parser
> and interpreter for Org syntax.
>
> [...]
>
> the following code will parse the buffer, interpret the parsed
> tree, and create a canonical copy of it (no indentation, lowercased
> blocks, standard keywords):
>
> #+begin_src org
> (let ((out (org-element-interpret-data (org-element-parse-buffer))))
> (switch-to-buffer (get-buffer-create "*Bijectivep*"))
> (erase-buffer)
> (insert out)
> (goto-char (point-min))
> (org-mode))
> #+end_src
>
> [...]
>
> Feedback is welcome.
Good to mention is also this code to show the pretty print formatted
lisp object representation of the current Org buffer:
#+BEGIN_SRC emacs-lisp
(let ((out (org-element-parse-buffer)))
(setq eval-expression-print-length nil)
(setq eval-expression-print-level nil)
(switch-to-buffer (get-buffer-create "*prettyprint*"))
(erase-buffer)
(insert (pp-to-string out))
(goto-char (point-min))
(emacs-lisp-mode)
(setq truncate-lines t))
#+END_SRC
Not sure if the following should be supported by
org-element-interpret-data:
#+BEGIN_SRC org
,variable declaration in some programming languages:
,- Perl :: the variable declaration can be implicit
,- C ::
, #+BEGIN_SRC C
, time_t variable_name;
, #+END_SRC
#+END_SRC
The above command for Bijectivep of (org-element-interpret-data
(org-element-parse-buffer)) shows BEGIN_SRC on the same line as
"- C ::" but I expect it on a separate line like in the original. The above
command for prettyprint of (org-element-parse-buffer) alone looks ok
to me.
Michael
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [ANN] Org Elements in contrib
2012-06-10 16:40 ` Michael Brand
@ 2012-06-12 12:32 ` Nicolas Goaziou
2012-06-12 14:14 ` Michael Brand
0 siblings, 1 reply; 9+ messages in thread
From: Nicolas Goaziou @ 2012-06-12 12:32 UTC (permalink / raw)
To: Michael Brand; +Cc: Org Mode List
Hello,
Michael Brand <michael.ch.brand@gmail.com> writes:
> Not sure if the following should be supported by
> org-element-interpret-data:
>
> #+BEGIN_SRC org
> ,variable declaration in some programming languages:
> ,- Perl :: the variable declaration can be implicit
> ,- C ::
> , #+BEGIN_SRC C
> , time_t variable_name;
> , #+END_SRC
> #+END_SRC
This bug should be fixed now. Thank you for reporting it.
Regards,
--
Nicolas Goaziou
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [ANN] Org Elements in contrib
2012-06-12 12:32 ` Nicolas Goaziou
@ 2012-06-12 14:14 ` Michael Brand
0 siblings, 0 replies; 9+ messages in thread
From: Michael Brand @ 2012-06-12 14:14 UTC (permalink / raw)
To: Nicolas Goaziou; +Cc: Org Mode List
Hi Nicolas
On Tue, Jun 12, 2012 at 2:32 PM, Nicolas Goaziou <n.goaziou@gmail.com> wrote:
> This bug should be fixed now.
I confirm that it is fixed. And as I see from your commit: Thanks to
your modular and therefore stable design it is fixed also for other
cases that I didn't test before with table or list as the only body
element. And the regression tests in general I appreciate a lot too.
Michael
^ permalink raw reply [flat|nested] 9+ messages in thread