Re: Thoughts on the standardization of Org

[Tim Cross <theophilusx@gmail.com>]

I think there are a couple of important points to consider in
discussions of this type. I should state up-front, I am somewhat
sceptical regarding an org-mode which is separate or independent of
Emacs. Much of what makes org-mode so powerful and useful is due to
features of Emacs. While most, if not all, of these features could be
implemented in other solutions, the amount of work and level of
maintenance should not be under-estimated.

While standards and formal standardisation of something can be
important, it is often a dual edge sword. Over the last 30+ years
involved in technology, I have seen many good ideas come undone as the
result of a standardisation effort. consider for example, the results of
the lisp standardisation that resulted in common Lisp, what happened
with CORBA, xml-rpc and the move to REST based APIs. XHTML and the
breakdown of standardisation processes within the W3C and the
development of HTML5. Sometimes, we can emerge from a standardisation
process with a clearer, consistent standard that is easy to implement
and use. Other times, we can emerge with a complex, difficult to
implement and confusing standard which can kill or stifle further
development. The trick with standardisation seem to be getting the
balance right between clarity and complexity and focusing on the key
requirements, avoiding the trap of trying to cover everything.

One of the things I like a lot about org-mode is that it is not terribly
prescriptive. It provides a collection of features and functions which
you are able to assemble according to your own needs and preferences.
While I think it is important to have a clear idea of basic syntax for
each of these elements and how they relate to each other, I'm less
convinced we would want to prescribe an overly formal specification for
how an org document should be structured i.e. the org DOM idea. The
existing draft syntax document is probably sufficiently prescriptive
here already.

The four areas which I think would provide the greatest benefit would
be to

- Finalise the draft org syntax document on Worg, possibly adding it to
  the manual once complete. A considerable amount of work has already
  been put into this document and I think it is a good start.

- Define a specification for a property API which compliant org-mode
  implementation should support. This could be based on the existing
  ELisp mapping API.

- Define a specification for an element mapping API which compliant
  org-mode implementation should support. Again, this could be based on
  the existing ELisp element mapping API.

- Define a set of org reference documents. These would be documents that
  all compliant parsers should be able to process successfully. It might
  also be worthwhile including some documents with common errors which
  parses should be able to handle and recover from in a graceful manner.
  Those developing external tools can then use these documents as a
  guide and for testing their implementations.

Tim

Asa Zeren <asaizeren@gmail.com> writes:

> Hi,
>
> Even though I am new to the org-mode community, I would like to share
> some thoughts on the specification of org-mode, especially since I
> have seen some recent discussion of it in relation to registering org
> as a MIME type.
>
> First, I would like to repeat the importance of developing standards
> for org-mode. If we want to expand the influence of org, tooling must
> expand beyond Emacs. While Emacs is an amazing tool, (a) we cannot
> convince the entire world to use Emacs and (b) org-mode should be
> integrated into tooling unrelated to text editing, and is outside of
> the Emacs-Lisp environment. Without additional org implementations,
> this is impossible. If org catches on before it is standardized, we
> end up in the situation of Markdown, with many competing standards and
> non-standards. Hence, standardization is essential.
>
> Standardizing org is much harder than standardizing something like
> Markdown, but I think by breaking it down as follows will maximize the
> portability of org while not compromising on development of org.
>
> I see three areas of standardization, which I think should be
> standardized separately:
>  - Org DOM
>  - Org Syntax
>  - Org Standard Environments
>
> Before we get to that, a brief note on /how/ I think that org should
> be specified. I think that org should be specified in terms of an
> /environment/ that defines the properties, etc. that can be used in a
> document. For instance, the org standard would say something to the
> effect of "An environment may specify block bounding keywords that may
> be used like #+<kwd_0>\n...#+<kwd_1>. and the environment would specify
> "begin_src and end_src are a pair of block bounding keyword that
> indicates a source code block." This is for two reasons. First, this
> allows for development of org tool features independent of the
> standard. Second, this separates the individual features of org mode
> from the overall structure.
>
> Org DOM:
> The first thing to specify is the org DOM. (Maybe a different name
> should be used to avoid confusion with the HTML DOM) This is the
> structure of an org-mode document, without the textual
> representation. Many org-related tools operate on org documents
> without needing to use the textual representation. Specifying the DOM
> separately would (a) create a separation of concerns and (b) allow for
> better libraries built around org mode.
>
> Org Syntax:
> This would be specifying the mapping between the DOM and the textual
> representation, specified in terms of an environment.
>
> Org Standard Environments:
> This is how I would specify elements such as #+begin_src..#+end_src
> would be specified, as standardized elements of the environment. This
> would be structured as a number of individual standard environments,
> such as "Source Blocks" or "Standard Header Properties" (specifying
> #+title, #+author, etc.)
>
> I would appreciate thoughts on these ideas about how to develop and
> org specification.
>
> Thanks for reading,
> Asa Zeren


--
Tim Cross