Re: Formal description of Org files

Re: Formal description of Org files

[Rustom Mody]

[-- Attachment #1: Type: text/plain, Size: 1803 bytes --]

>
> Hi,
>
> I have a question.
>
> At FOSDEM, someone asked me if there was a formal description of the
> structure of Org files, in some language
> that would be the input for a parser (or parser generator?) so that Org
> file could be easily parsed.
>
> Unfortunately I did not catch the name of the format description language
> that could be
> used for something, not did I catch the name of the person who talked to
> me.
>
> Can anyone help out here?  Let me know what language to use, and maybe help
> work on such a formal description?  I
> think it would be useful to have....
>

Something like yacc (bison, antlr etc) are all 'executable BNF' languages.
 When they work they can make the code an order of magnitude smaller and
development/programming correspondingly easier.

That said I see a couple of hitches.

1. Grammatical handling of languages is based on the assumption of a clear
and well defined set of tokens/lexemes.  I expect this would be harder in
org than the typical programming languages for which yacc etc are used.  For
example in most 'normal' languages there are comments and strings.  These
involve some non trivial handling which is entirely hidden from the grammar
by being pushed into the lexer.

2. Parsing a program is done for the full program as a rule (IDEs are the
exception to the rule). Sensible parsing of program fragments, where the
fragmenting could be quite arbitrary, is a bit of a research problem

3. As I see it, the main declarative tool (somewhat akin to grammars) that
org uses is regular expressions.  IOW org is written with re-s strung
together with programming logic ie vanilla elisp.  An alternative that stays
within the regular framework (not using the heavy guns of context free
parsing) may be ragel: http://www.complang.org/ragel/

Rusi

[-- Attachment #2: Type: text/html, Size: 2058 bytes --]

Re: Formal description of Org files

[MidLifeXis at PerlMonks]

[-- Attachment #1: Type: text/plain, Size: 3157 bytes --]

This is something I would like to see adopted by the org community as well.  One 
of my main reasons is to allow other languages to be able to at least, given the 
proper configuration information (for TODO keywords, etc), to be able to deal 
with org files.  I am not quite sure where the lines should be drawn, but I am 
envisioning something like:

* The ORG format definition and actions

The format of the .org file (ORG) itself and a set of actions that are well 
behaved (add node, move node up, node id, property value, etc).  A core set of 
validation tests could be created to exercise any compliant org library.

* The configuration for a set of ORG files

The configuration for a given org (note the small "org") application, defining 
particular tweaks that an application can use to make sense of a properly 
formatted ORG file, such as valid TODO keywords, etc.  This area is muddy in my 
mind.

* The org application itself - how the org files are used in a particular 
application, or the ideas and data that set of ORG files represents, or how an 
exporter handles the data thrown at it, or....

But that is just /my/ dream.






________________________________
From: Rustom Mody <rustompmody@gmail.com>
To: emacs-orgmode <emacs-orgmode@gnu.org>
Sent: Fri, April 15, 2011 12:13:45 PM
Subject: Re: [O] Formal description of Org files


Hi,
>
>I have a question.
>
>At FOSDEM, someone asked me if there was a formal description of the structure 
>of Org files, in some language
>that would be the input for a parser (or parser generator?) so that Org file 
>could be easily parsed.
>
>Unfortunately I did not catch the name of the format description language that 
>could be
>used for something, not did I catch the name of the person who talked to me.
>
>Can anyone help out here?  Let me know what language to use, and maybe help work 
>on such a formal description?  I
>think it would be useful to have....
>
Something like yacc (bison, antlr etc) are all 'executable BNF' languages.  When 
they work they can make the code an order of magnitude smaller and 
development/programming correspondingly easier.

That said I see a couple of hitches.

1. Grammatical handling of languages is based on the assumption of a clear and 
well defined set of tokens/lexemes.  I expect this would be harder in org than 
the typical programming languages for which yacc etc are used.  For example in 
most 'normal' languages there are comments and strings.  These involve some non 
trivial handling which is entirely hidden from the grammar by being pushed into 
the lexer.

2. Parsing a program is done for the full program as a rule (IDEs are the 
exception to the rule). Sensible parsing of program fragments, where the 
fragmenting could be quite arbitrary, is a bit of a research problem

3. As I see it, the main declarative tool (somewhat akin to grammars) that org 
uses is regular expressions.  IOW org is written with re-s strung together with 
programming logic ie vanilla elisp.  An alternative that stays within the 
regular framework (not using the heavy guns of context free parsing) may be 
ragel: http://www.complang.org/ragel/

Rusi

[-- Attachment #2: Type: text/html, Size: 4206 bytes --]

Re: Formal description of Org files

[Gustav Wikström]

Hi again list.

There was a discussion many years back about a formal description of
Org files [1]. In some way that might be achieved now in org-elements,
but that still is heavily bound to elisp.

So my question is; have there been more discussions of constructing
such a formal grammar? Maybe in EBNF form. I suspect getting every
aspect of Org mode into such a description would be difficult. But
imagine the possibilities. Tools such as ANTLR and similar would
straight away have a way of parsing Org mode. Org mode source files
could be parsed in many other languages, simplifying the process of
expanding Org mode to other platforms ( Android & iOS, web ).

Further rant; To me Org mode should be less about Emacs and more about
the source file format. Let Emacs be the main carrier, but let Org
mode also expand into other domains. One way to simplify this might be
to provide the formal description so that other tools might be easier
to develop based on this grammar.

(For me, the biggest limitation of Org mode is lacking tools to
utilize it on the run. The aim of this is thus to feed thoughts on how
to simplify processes that can expand Org mode into those "more
mobile" domains).

Best regards
Gustav

[1] http://lists.gnu.org/archive/html/emacs-orgmode/2011-07/msg01282.html

> Hi Jambunathan,
>
> Jambunathan K <address@hidden> writes:
>
> > A simple worg page that "collects" the low hanging fruits to begin
> > with will be a good but firm first step in this direction.
>
> Indeed.
>
> It's on my todo list, but not with a high priority for now.
>
> If you feel like starting it, please go ahead!
>
> --
>  Bastien

Re: Formal description of Org files

[Nicolas Goaziou]

Hello,

Gustav Wikström <gustav.erik@gmail.com> writes:

> There was a discussion many years back about a formal description of
> Org files [1]. In some way that might be achieved now in org-elements,
> but that still is heavily bound to elisp.
>
> So my question is; have there been more discussions of constructing
> such a formal grammar? Maybe in EBNF form. I suspect getting every
> aspect of Org mode into such a description would be difficult. But
> imagine the possibilities. Tools such as ANTLR and similar would
> straight away have a way of parsing Org mode. Org mode source files
> could be parsed in many other languages, simplifying the process of
> expanding Org mode to other platforms ( Android & iOS, web ).
>
> Further rant; To me Org mode should be less about Emacs and more about
> the source file format. Let Emacs be the main carrier, but let Org
> mode also expand into other domains. One way to simplify this might be
> to provide the formal description so that other tools might be easier
> to develop based on this grammar.
>
> (For me, the biggest limitation of Org mode is lacking tools to
> utilize it on the run. The aim of this is thus to feed thoughts on how
> to simplify processes that can expand Org mode into those "more
> mobile" domains).

See

  (info "(org) Org syntax")


Regards,

-- 
Nicolas Goaziou

Re: Formal description of Org files

[Samuel Loury]

[-- Attachment #1: Type: text/plain, Size: 1098 bytes --]


Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Gustav Wikström <gustav.erik@gmail.com> writes:
>>
>> So my question is; have there been more discussions of constructing
>> such a formal grammar? Maybe in EBNF form. I suspect getting every
>> aspect of Org mode into such a description would be difficult. But
>> imagine the possibilities. Tools such as ANTLR and similar would
>> straight away have a way of parsing Org mode. 
>> Org mode source files could be parsed in many other languages,
>> simplifying the process of expanding Org mode to other platforms (
>> Android & iOS, web ).
>>

[...]

>
> See
>
>   (info "(org) Org syntax")
I just read the document on worg¹. It is really good! I imagine we are
not far from a formal grammar. I suppose we could write one in EBNF
form.

Nicolas, I can see on the worg version that it is in draft status. Could
you say what is left to do to consider it mature?

¹  http://orgmode.org/worg/dev/org-syntax.html
-- 
Konubinix
GPG Key    : 7439106A
Fingerprint: 5993 BE7A DA65 E2D9 06CE  5C36 75D2 3CED 7439 106A

[-- Attachment #2: Type: application/pgp-signature, Size: 472 bytes --]

Re: Formal description of Org files

[Nicolas Goaziou]

Hello,

Samuel Loury <konubinix@gmail.com> writes:

> Nicolas, I can see on the worg version that it is in draft status. Could
> you say what is left to do to consider it mature?

I have plans to change properties drawers syntax soon. Also differences
between export blocks and special blocks should be discussed on the ML
(I started a thread about this) and finalized.

Moreover, emphasis marker syntax is not satisfying, nor is the
description of $...$ constructs, but it is more a long-term plan.

Eventually, I think the description is not always clear. EBNF could help
here.


Regards,

-- 
Nicolas Goaziou                                                0x80A93738

Re: Formal description of Org files

[Gustav Wikström]

Ah, yes. I've seen that and I really really admire your work. :-) What
I'm aiming at is something more standardised. Something that can be
used to generate language lexers and parsers in other programming
languages.. But, as I stated a moment ago in another mail to the list
- I don't really know the power of such a standardised grammar and of
what can be done if it was there.. I only envision the possibilities.
;-)

Best regards
Gustav

On Sun, Sep 21, 2014 at 2:43 PM, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:
> Hello,
>
> Gustav Wikström <gustav.erik@gmail.com> writes:
>
>> There was a discussion many years back about a formal description of
>> Org files [1]. In some way that might be achieved now in org-elements,
>> but that still is heavily bound to elisp.
>>
>> So my question is; have there been more discussions of constructing
>> such a formal grammar? Maybe in EBNF form. I suspect getting every
>> aspect of Org mode into such a description would be difficult. But
>> imagine the possibilities. Tools such as ANTLR and similar would
>> straight away have a way of parsing Org mode. Org mode source files
>> could be parsed in many other languages, simplifying the process of
>> expanding Org mode to other platforms ( Android & iOS, web ).
>>
>> Further rant; To me Org mode should be less about Emacs and more about
>> the source file format. Let Emacs be the main carrier, but let Org
>> mode also expand into other domains. One way to simplify this might be
>> to provide the formal description so that other tools might be easier
>> to develop based on this grammar.
>>
>> (For me, the biggest limitation of Org mode is lacking tools to
>> utilize it on the run. The aim of this is thus to feed thoughts on how
>> to simplify processes that can expand Org mode into those "more
>> mobile" domains).
>
> See
>
>   (info "(org) Org syntax")
>
>
> Regards,
>
> --
> Nicolas Goaziou

Re: Formal description of Org files

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 668 bytes --]

On Sunday, 21 Sep 2014 at 14:10, Gustav Wikström wrote:

[...]

> (For me, the biggest limitation of Org mode is lacking tools to
> utilize it on the run. The aim of this is thus to feed thoughts on how
> to simplify processes that can expand Org mode into those "more
> mobile" domains).

Just curious: what is it you wish to do in a mobile environment.  I have
everything I need with MobileOrg and running full emacs + org on an
OpenPandora.  Obviously, your needs may be different than mine.

(email composed on train offline on my OpenPandora in Emacs with gnus ;-)

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 24.3.1, Org release_8.3beta-372-gdd70cf

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 180 bytes --]

Re: Formal description of Org files

[Grant Rettke]

Eric do you use org in Emacs on the Pandora?

Just curious also to know where Emacs on Pandora users hang out?

On Fri, Sep 26, 2014 at 3:12 AM, Eric S Fraga <e.fraga@ucl.ac.uk> wrote:
> On Sunday, 21 Sep 2014 at 14:10, Gustav Wikström wrote:
>
> [...]
>
>> (For me, the biggest limitation of Org mode is lacking tools to
>> utilize it on the run. The aim of this is thus to feed thoughts on how
>> to simplify processes that can expand Org mode into those "more
>> mobile" domains).
>
> Just curious: what is it you wish to do in a mobile environment.  I have
> everything I need with MobileOrg and running full emacs + org on an
> OpenPandora.  Obviously, your needs may be different than mine.
>
> (email composed on train offline on my OpenPandora in Emacs with gnus ;-)
>
> --
> : Eric S Fraga (0xFFFCF67D), Emacs 24.3.1, Org release_8.3beta-372-gdd70cf



-- 
Grant Rettke
gcr@wisdomandwonder.com | http://www.wisdomandwonder.com/
“Wisdom begins in wonder.” --Socrates
((λ (x) (x x)) (λ (x) (x x)))
“Life has become immeasurably better since I have been forced to stop
taking it seriously.” --Thompson

Re: Formal description of Org files

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 752 bytes --]

On Friday, 26 Sep 2014 at 07:53, Grant Rettke wrote:
> Eric do you use org in Emacs on the Pandora?

Yes.  I use Pandian: Debian distribution on the Pandora.  It comes with
Emacs 24.3.  I install org (and gnus and others) from git.

Emacs is also available on the stock OS for the OpenPandora.  I run
Debian because nobody has ported unison (a key tool in my day to day use
of all of my computers) to the OpenPandora OS (SuperSaxxon), as far as I
can tell.

> Just curious also to know where Emacs on Pandora users hang out?

Not sure there are many?  If they hand out anywhere, it would be on the
OpenPandora fora at http://boards.openpandora.org/


-- 
: Eric S Fraga (0xFFFCF67D), Emacs 24.4.50.1, Org release_8.3beta-366-gb2fca7

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 180 bytes --]

Re: Formal description of Org files

[Gustav Wikström]

Hi!

> Just curious: what is it you wish to do in a mobile environment.  I have
> everything I need with MobileOrg and running full emacs + org on an
> OpenPandora.  Obviously, your needs may be different than mine.
>
> (email composed on train offline on my OpenPandora in Emacs with gnus ;-)
>
> --
> : Eric S Fraga (0xFFFCF67D), Emacs 24.3.1, Org release_8.3beta-372-gdd70cf

My wish is to be able to do mostly everything related to
task-tracking, scheduling, working with references and so on. General
GTD stuff. OpenPandora sounds neat, except it's not quite the standard
tool available out there ;-) And a general idea of Emacs in your
pocket is nice, except it's not really possible without a different UI
than the keyboard. So; what way forward with GTD in Emacs Org-mode
then, while still having the wish of it to be more accessible?

Thus the idea of a more formal (and parsable) grammar in a
standardized format. My hope is that it will make Org-mode more
general than it already is. As I said already; Org-mode to me is more
than Emacs. To think BIG, Org-mode grammar could be a standard for PIM
or GTD related software. To think a bit less big, it might help
developers create software with non-Emacs tools. By using the formal
grammar to work with Org-mode source documents in more accessible
ways. Maybe with UI's available through mobile phones or tablets (and
without the use of special schedules of read/write in Emacs to keep
the mobile system synced; MobileOrg).

I have to admit I might be overstating the importance of Org-mode
formal grammar with this goal in mind. I haven't used EBNF-parsers and
don't really know how powerful they can be. Reading about those things
give high hopes though.

Best regards
Gustav

Re: Formal description of Org files

[Samuel Loury]

[-- Attachment #1: Type: text/plain, Size: 2762 bytes --]

Hi,
Gustav Wikström <gustav.erik@gmail.com> writes:

>> Just curious: what is it you wish to do in a mobile environment.  I have
>> everything I need with MobileOrg and running full emacs + org on an
>> OpenPandora.  Obviously, your needs may be different than mine.
>>
>> (email composed on train offline on my OpenPandora in Emacs with gnus ;-)
>>
>> --
>> : Eric S Fraga (0xFFFCF67D), Emacs 24.3.1, Org release_8.3beta-372-gdd70cf
>
> My wish is to be able to do mostly everything related to
> task-tracking, scheduling, working with references and so on. General
> GTD stuff. OpenPandora sounds neat, except it's not quite the standard
> tool available out there ;-) And a general idea of Emacs in your
> pocket is nice, except it's not really possible without a different UI
> than the keyboard. So; what way forward with GTD in Emacs Org-mode
> then, while still having the wish of it to be more accessible?
>
> Thus the idea of a more formal (and parsable) grammar in a
> standardized format. My hope is that it will make Org-mode more
> general than it already is. As I said already; Org-mode to me is more
> than Emacs. To think BIG, Org-mode grammar could be a standard for PIM
> or GTD related software. To think a bit less big, it might help
> developers create software with non-Emacs tools. By using the formal
> grammar to work with Org-mode source documents in more accessible
> ways. Maybe with UI's available through mobile phones or tablets (and
> without the use of special schedules of read/write in Emacs to keep
> the mobile system synced; MobileOrg).
This sounds like a good idea. And since there are a lot of parsers in
other languages recognizing a subset of org mode syntax, other people
like this idea too.

I think a difficulty with that solution is that it won't capture the
heavy customization the user can make on org-mode using hooks or
customizing variables.

Having a real emacs in the pocket would provide org-mode along with the
customization. Nevertheless, as you stated, emacs is not really user
friendly when you don't have a keyboard at hand.

For that reason, I think that as well as embedding emacs into the phone,
we could provide a UI communicating with it using an IPC mechanism.

This way, we would have the solution with more to gain (a functional
org-mode as well as the customization) and less to write (only the
needed UI code). I already tried using epc to perform a two way
communication between a python shell and emacs. I talked about this in
another thread¹.

What do you think of this approach?

¹ http://mid.gmane.org/87iokv4659.fsf@gmail.com
-- 
Konubinix
GPG Key    : 7439106A
Fingerprint: 5993 BE7A DA65 E2D9 06CE  5C36 75D2 3CED 7439 106A

[-- Attachment #2: Type: application/pgp-signature, Size: 472 bytes --]

Formal description of Org files

[Carsten Dominik]

Hi,

I have a question.

At FOSDEM, someone asked me if there was a formal description of the structure of Org files, in some language that would be the input for a parser (or parser generator?) so that Org file could be easily parsed.

Unfortunately I did not catch the name of the format description language that could be
used for something, not did I catch the name of the person who talked to me.

Can anyone help out here?  Let me know what language to use, and maybe help work on such a formal description?  I think it would be useful to have....

- Carsten

Re: Formal description of Org files

[Rainer Stengele]

Am 15.04.2011 09:58, schrieb Carsten Dominik:
> Hi,
> 
> I have a question.
> 
> At FOSDEM, someone asked me if there was a formal description of the structure of Org files, in some language that would be the input for a parser (or parser generator?) so that Org file could be easily parsed.
> 
> Unfortunately I did not catch the name of the format description language that could be
> used for something, not did I catch the name of the person who talked to me.
> 
> Can anyone help out here?  Let me know what language to use, and maybe help work on such a formal description?  I think it would be useful to have....
> 
> - Carsten
> 
Hi,

maybe Backus-Naur was meant?

- Rainer

Re: Formal description of Org files

[Carsten Dominik]

On Apr 15, 2011, at 10:54 AM, Rainer Stengele wrote:

> Am 15.04.2011 09:58, schrieb Carsten Dominik:
>> Hi,
>> 
>> I have a question.
>> 
>> At FOSDEM, someone asked me if there was a formal description of the structure of Org files, in some language that would be the input for a parser (or parser generator?) so that Org file could be easily parsed.
>> 
>> Unfortunately I did not catch the name of the format description language that could be
>> used for something, not did I catch the name of the person who talked to me.
>> 
>> Can anyone help out here?  Let me know what language to use, and maybe help work on such a formal description?  I think it would be useful to have....
>> 
>> - Carsten
>> 
> Hi,
> 
> maybe Backus-Naur was meant?


That is very well possible.  Sounds like a word I would not have recognized....
So would on Org-mode grammar start like this?

-------------------------------------------------------------------------------

<headline> ::= "*"+ <opt-todo-keyword>
                    <opt-priority-cookie>
                    <title>
                    <opt-tags>

<opt-todo-keyword> ::= <whitespace> <todo-keyword> | ""

<opt-priority-cookie> ::= <whitespace> <priority-cookie> | ""

<opt-tags> ::= <whitespace> <tags> | ""



<todo-keyword> ::= "TODO" | "DONE"

<priority-cookie> ::= "[" "#" <priority> "]"

<priority> ::= "A" | "B" | "C"                   

<tags> ::= ( ":" <tag> )+ ":"

<whitespace> ::= " " <opt-whitespace> | "\t" <opt-whitespace>

<opt-whitespace> ::= <whitespace> | ""

-------------------------------------------------------------------------------

Re: Formal description of Org files

[Peter Frings]

On 15 Apr 2011, at 12:57, Carsten Dominik wrote:

> On Apr 15, 2011, at 10:54 AM, Rainer Stengele wrote:
> 
>> Am 15.04.2011 09:58, schrieb Carsten Dominik:
>>> Hi,
>>> 
>>> I have a question.
>>> 
>>> At FOSDEM, someone asked me if there was a formal description of the structure of Org files, in some language that would be the input for a parser (or parser generator?) so that Org file could be easily parsed.

<snip>

>> Hi,
>> 
>> maybe Backus-Naur was meant?
> 
> That is very well possible.  Sounds like a word I would not have recognized....
> So would on Org-mode grammar start like this?

> headline> ::= "*"+ <opt-todo-keyword>
>                    <opt-priority-cookie>
>                    <title>
>                    <opt-tags>
> 
> <opt-todo-keyword> ::= <whitespace> <todo-keyword> | “”
<snip>

Yes, that seems like extended BN. [1]
Good luck with it! You’ll need it... :-)

[1] http://en.wikipedia.org/wiki/Extended_Backus–Naur_Form


Cheers,
Peter.
-- 
c++;         // this makes c bigger but returns the old value

Re: Formal description of Org files

[Carsten Dominik]

On Apr 15, 2011, at 1:50 PM, Peter Frings wrote:

> 
> On 15 Apr 2011, at 12:57, Carsten Dominik wrote:
> 
>> On Apr 15, 2011, at 10:54 AM, Rainer Stengele wrote:
>> 
>>> Am 15.04.2011 09:58, schrieb Carsten Dominik:
>>>> Hi,
>>>> 
>>>> I have a question.
>>>> 
>>>> At FOSDEM, someone asked me if there was a formal description of the structure of Org files, in some language that would be the input for a parser (or parser generator?) so that Org file could be easily parsed.
> 
> <snip>
> 
>>> Hi,
>>> 
>>> maybe Backus-Naur was meant?
>> 
>> That is very well possible.  Sounds like a word I would not have recognized....
>> So would on Org-mode grammar start like this?
> 
>> headline> ::= "*"+ <opt-todo-keyword>
>>                   <opt-priority-cookie>
>>                   <title>
>>                   <opt-tags>
>> 
>> <opt-todo-keyword> ::= <whitespace> <todo-keyword> | “”
> <snip>
> 
> Yes, that seems like extended BN. [1]
> Good luck with it! You’ll need it... :-)

Well, if tis is complicated, I am not going to do this.
But maybe there is a volunteer out there - I'd be happy to
support him or her.

- Carsten

Re: Formal description of Org files

[Jambunathan K]

Carsten Dominik <carsten.dominik@gmail.com> writes:

> On Apr 15, 2011, at 1:50 PM, Peter Frings wrote:
>
>> 
>> On 15 Apr 2011, at 12:57, Carsten Dominik wrote:
>> 
>>> On Apr 15, 2011, at 10:54 AM, Rainer Stengele wrote:
>>> 
>>>> Am 15.04.2011 09:58, schrieb Carsten Dominik:
>>>>> Hi,
>>>>> 
>>>>> I have a question.
>>>>> 
>>>>> At FOSDEM, someone asked me if there was a formal description of
>>>>> the structure of Org files, in some language that would be the
>>>>> input for a parser (or parser generator?) so that Org file could
>>>>> be easily parsed.
>> 
>> <snip>
>> 
>>>> Hi,
>>>> 
>>>> maybe Backus-Naur was meant?
>>> 
>>> That is very well possible.  Sounds like a word I would not have recognized....
>>> So would on Org-mode grammar start like this?
>> 
>>> headline> ::= "*"+ <opt-todo-keyword>
>>>                   <opt-priority-cookie>
>>>                   <title>
>>>                   <opt-tags>
>>> 
>>> <opt-todo-keyword> ::= <whitespace> <todo-keyword> | “”
>> <snip>
>> 
>> Yes, that seems like extended BN. [1]
>> Good luck with it! You’ll need it... :-)
>
> Well, if tis is complicated, I am not going to do this.
> But maybe there is a volunteer out there - I'd be happy to
> support him or her.

IF there is a need for a lexer and parser may be one should start with
the parser that is part of cedet.

http://www.emacswiki.org/emacs/HowToSetUpSemanticBovinatorForANewLanguage

Jambunathan K.

> - Carsten

-- 

Re: Formal description of Org files

[Olaf.Hamann]

If I remember well, the thing with BNF and similar approaches was called
'context free grammar' - and I'm not sure, if this requirement is
fullfilled by the org syntax. As I mean to remember is e.g. that Perl
could not be declared fully with a context free grammar.

As if 'parse an org-file completely into a token stream' is not required
by syntax highlighting tools etc,
it might be enough to give some regexps or so. Perhaps for use with etags.
Perhaps for modifying ecb to extract main elements of an org-file into
it's subwindows.

Greetings, Olaf

Re: Formal description of Org files

[Christian Egli]

Carsten Dominik <carsten.dominik@gmail.com> writes:

> At FOSDEM, someone asked me if there was a formal description of the
> structure of Org files, in some language that would be the input for a
> parser (or parser generator?) so that Org file could be easily parsed.

Maybe the person was talking about antlr[1], "ANother Tool for Language
Recognition, a language tool that provides a framework for constructing
recognizers, interpreters, compilers, and translators from grammatical
descriptions containing actions in a variety of target languages".

It even seems to have preliminary support for generating an elisp
parser[2][3]

There is also an emacs mode for editing antlr files[4].

Sounds like an interesting project.

Thanks
Christian

Footnotes: 
[1]  http://www.antlr.org/
[2]  http://www.antlr.org/wiki/display/ANTLR3/Code+Generation+Targets
[3]  https://github.com/olabini/antlr-elisp
[4]  http://antlr-mode.sourceforge.net/
-- 
Christian Egli
Swiss Library for the Blind, Visually Impaired and Print Disabled
Grubenstrasse 12, CH-8045 Zürich, Switzerland

Re: Formal description of Org files

[Peter Frings]

On 15 Apr 2011, at 14:58, Christian Egli wrote:

> Carsten Dominik <carsten.dominik@gmail.com> writes:
> 
>> At FOSDEM, someone asked me if there was a formal description of the
>> structure of Org files, in some language that would be the input for a
>> parser (or parser generator?) so that Org file could be easily parsed.
> 
> Maybe the person was talking about antlr[1], "ANother Tool for Language
> Recognition, a language tool that provides a framework for constructing
> recognizers, interpreters, compilers, and translators from grammatical
> descriptions containing actions in a variety of target languages”.
<snip>

> Sounds like an interesting project.

Wow, if that thing can export syntax diagrams in PNG or PDF I’d be really happy. Looks very interesting — albeit serious overkill for what I’d use it :-).

thanks,
Peter.
-- 
c++;         // this makes c bigger but returns the old value

Re: Formal description of Org files

[Filippo A. Salustri]

[-- Attachment #1: Type: text/plain, Size: 2023 bytes --]

Antlr is just another in a long line of lexical parsers.  I still remember
the original lex (for lexical analysis), which in combination with yacc (for
parsing and grammar) could make pretty much any conventional programming
language.
Then GNU came up with Flex (fast lex) and Bison (instead of yacc...get it?
:)
Then IDEs really started to take off and much of the ugly parts of writing
languages disappeared, which led to all kinds of new tools like antlr.

But they all basically do the same thing: let one describe the syntax and
grammar of (quasi) formal (programming) languages.

I don't know if any of them produce diagrams, but I wouldn't be surprised if
at least some did.

Cheers.
Fil

On 15 April 2011 09:42, Peter Frings <peter.frings@agfa.com> wrote:

>
> On 15 Apr 2011, at 14:58, Christian Egli wrote:
>
> > Carsten Dominik <carsten.dominik@gmail.com> writes:
> >
> >> At FOSDEM, someone asked me if there was a formal description of the
> >> structure of Org files, in some language that would be the input for a
> >> parser (or parser generator?) so that Org file could be easily parsed.
> >
> > Maybe the person was talking about antlr[1], "ANother Tool for Language
> > Recognition, a language tool that provides a framework for constructing
> > recognizers, interpreters, compilers, and translators from grammatical
> > descriptions containing actions in a variety of target languages”.
> <snip>
>
> > Sounds like an interesting project.
>
> Wow, if that thing can export syntax diagrams in PNG or PDF I’d be really
> happy. Looks very interesting — albeit serious overkill for what I’d use it
> :-).
>
> thanks,
> Peter.
> --
> c++;         // this makes c bigger but returns the old value
>
>
>


-- 
Filippo A. Salustri, Ph.D., P.Eng.
Mechanical and Industrial Engineering
Ryerson University
350 Victoria St, Toronto, ON
M5B 2K3, Canada
Tel: 416/979-5000 ext 7749
Fax: 416/979-5265
Email: salustri@ryerson.ca
http://deseng.ryerson.ca/~fil/

[-- Attachment #2: Type: text/html, Size: 2671 bytes --]

Re: Formal description of Org files

[Wes Hardaker]

>>>>> On Fri, 15 Apr 2011 09:58:09 +0200, Carsten Dominik <carsten.dominik@gmail.com> said:

CD> Unfortunately I did not catch the name of the format description
CD> language that could be used for something, not did I catch the name
CD> of the person who talked to me.

Another option, besides those mentioned, is probably ABNF.

-- 
Wes Hardaker                                     
My Pictures:  http://capturedonearth.com/
My Thoughts:  http://pontifications.hardakers.net/

Re: Formal description of Org files

[Eric Schulte]

If one goal of such a formal description of Org-mode would be to parse
text Org-mode files into an abstract syntax tree (which is reminiscent
of [1] and [2]) then perhaps we should look at parsers which have
already been applied to other document languages (e.g. tex, html etc...)
or at least support xml output.  If such parsers exist and have been
widely applied, then maybe they could support the exportation of
Org-mode files, or even automate the writing of code for such exporters?

Best -- Eric

Wes Hardaker <wjhns209@hardakers.net> writes:

>>>>>> On Fri, 15 Apr 2011 09:58:09 +0200, Carsten Dominik
> <carsten.dominik@gmail.com> said:
>
> CD> Unfortunately I did not catch the name of the format description
> CD> language that could be used for something, not did I catch the name
> CD> of the person who talked to me.
>
> Another option, besides those mentioned, is probably ABNF.


Footnotes: 
[1]  the effort to unify exporting behind a single parser of Org-mode
     files into an abstract elisp tree

[2]  http://johnmacfarlane.net/pandoc/, which as I recall has an
     Org-mode backend

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Formal description of Org files

[Nick Dokos]

Eric Schulte <schulte.eric@gmail.com> wrote:

> If one goal of such a formal description of Org-mode would be to parse
> text Org-mode files into an abstract syntax tree ...
> 

I think this should be the starting point: what are the goals for all this?
Providing a formal description in EBNF is one thing. Preparing an attribute
grammar for input into a specific tool is another (and probably an order of
magnitude - or two - harder) - what would the resulting parser(s) be used for?

Clear(er) answers to these questions should go a long way towards figuring out
what specific tool(s) should be used - or whether it's at all necessary to
worry about that.

Nick

Re: Formal description of Org files

[Olivier Schwander]

Le 15 Apr 2011 14:31, Nick Dokos a écrit:
> Eric Schulte <schulte.eric@gmail.com> wrote:
> 
> > If one goal of such a formal description of Org-mode would be to parse
> > text Org-mode files into an abstract syntax tree ...
> > 
> 
> I think this should be the starting point: what are the goals for all this?
> Providing a formal description in EBNF is one thing. Preparing an attribute
> grammar for input into a specific tool is another (and probably an order of
> magnitude - or two - harder) - what would the resulting parser(s) be used for?
> 
> Clear(er) answers to these questions should go a long way towards figuring out
> what specific tool(s) should be used - or whether it's at all necessary to
> worry about that.

The primary goal I see for such a formal description is to provide a
specification that third party parsers are supposed to respect. Writing
a real parser may be too much project specific and difficult to
generalize in a way usable by the community.

During the development of neo[1], I was confronted to the need of
defining what is an org file (actually, what is an headline, a todo
keyword, a tag, a drawer, a timestamp, etc) and determining what is the
expected output of a parser.

Maybe the most appropriate format for such a description would be free
text, letting parser developers choosing between context-free grammars,
regexps or whatever they want ( with a bunch of example org files for
reference and tests).

Regards,

Olivier

[1] I am just discovering this thread

Re: Formal description of Org files

[Marcelo de Moraes Serpa]

[-- Attachment #1: Type: text/plain, Size: 2221 bytes --]

Hi guys,

I was going to create a new thread, but this one seems to fit exactly what
I'm looking for.

I'm creating a web app that interacts with orgmode files and allows you to
edit orgmode files on the browser. The edit part is not done. I'm quite good
at Javascript, and I wouldn't mind hacking something akin to orgmode elisp
code and this will be what I'll do if everything else fails, but wouldn't
using a grammar be a cleaner and more elegant solution?

Thanks,

Marcelo.

On Wed, Apr 20, 2011 at 7:37 AM, Olivier Schwander <
olivier.schwander@chadok.info> wrote:

> Le 15 Apr 2011 14:31, Nick Dokos a écrit:
> > Eric Schulte <schulte.eric@gmail.com> wrote:
> >
> > > If one goal of such a formal description of Org-mode would be to parse
> > > text Org-mode files into an abstract syntax tree ...
> > >
> >
> > I think this should be the starting point: what are the goals for all
> this?
> > Providing a formal description in EBNF is one thing. Preparing an
> attribute
> > grammar for input into a specific tool is another (and probably an order
> of
> > magnitude - or two - harder) - what would the resulting parser(s) be used
> for?
> >
> > Clear(er) answers to these questions should go a long way towards
> figuring out
> > what specific tool(s) should be used - or whether it's at all necessary
> to
> > worry about that.
>
> The primary goal I see for such a formal description is to provide a
> specification that third party parsers are supposed to respect. Writing
> a real parser may be too much project specific and difficult to
> generalize in a way usable by the community.
>
> During the development of neo[1], I was confronted to the need of
> defining what is an org file (actually, what is an headline, a todo
> keyword, a tag, a drawer, a timestamp, etc) and determining what is the
> expected output of a parser.
>
> Maybe the most appropriate format for such a description would be free
> text, letting parser developers choosing between context-free grammars,
> regexps or whatever they want ( with a bunch of example org files for
> reference and tests).
>
> Regards,
>
> Olivier
>
> [1] I am just discovering this thread
>
>

[-- Attachment #2: Type: text/html, Size: 2794 bytes --]

Re: Formal description of Org files

[Bastien]

Hi Marcelo,

Marcelo de Moraes Serpa <celoserpa@gmail.com> writes:

> I'm creating a web app that interacts with orgmode files and allows
> you to edit orgmode files on the browser. The edit part is not done.

Wow, this would be a really useful tool.  Can't wait to test this!

> I'm quite good at Javascript, and I wouldn't mind hacking something
> akin to orgmode elisp code and this will be what I'll do if
> everything else fails, but wouldn't using a grammar be a cleaner and
> more elegant solution?

Can you tell more about what you mean by "grammar"?

Back in february, at FOSDEM, someone asked for a description of the
org-mode format specification.  This is still something that needs to be
done.  Any stab at this (on Worg) would be really nice.  You can start
anywhere (headlines, TODO keywords, etc.)

If the "grammar" needs to be described in a specific format (more than
just a formal description of the various syntactic elements of an Org
file), let us know.

Thanks,

-- 
 Bastien

Re: Formal description of Org files

[Nick Dokos]

Bastien <bzg@altern.org> wrote:

> Hi Marcelo,
> 
> Marcelo de Moraes Serpa <celoserpa@gmail.com> writes:
> 
> > I'm creating a web app that interacts with orgmode files and allows
> > you to edit orgmode files on the browser. The edit part is not done.
> 
> Wow, this would be a really useful tool.  Can't wait to test this!
> 
> > I'm quite good at Javascript, and I wouldn't mind hacking something
> > akin to orgmode elisp code and this will be what I'll do if
> > everything else fails, but wouldn't using a grammar be a cleaner and
> > more elegant solution?
> 
> Can you tell more about what you mean by "grammar"?
> 

A formal grammar of some sort I presume: BNF or EBNF.

> Back in february, at FOSDEM, someone asked for a description of the
> org-mode format specification.  This is still something that needs to be
> done.  Any stab at this (on Worg) would be really nice.  You can start
> anywhere (headlines, TODO keywords, etc.)
> 
> If the "grammar" needs to be described in a specific format (more than
> just a formal description of the various syntactic elements of an Org
> file), let us know.
> 

There was a discussion about this on the list a few months ago, but I
can't get to search.gmane.org to find the thread atm, although I can get
to news.gmane.org: is that me or is gmane's search on the fritz?

Nick

Re: Formal description of Org files

[Bastien]

Nick Dokos <nicholas.dokos@hp.com> writes:

> There was a discussion about this on the list a few months ago, but I
> can't get to search.gmane.org to find the thread atm, although I can get
> to news.gmane.org: is that me or is gmane's search on the fritz?

Looks like gmane search is down, I can't use it either.

-- 
 Bastien

Re: Formal description of Org files

[suvayu ali]

On Sat, Jul 16, 2011 at 7:09 PM, Nick Dokos <nicholas.dokos@hp.com> wrote:
> There was a discussion about this on the list a few months ago, but I
> can't get to search.gmane.org to find the thread atm, although I can get
> to news.gmane.org: is that me or is gmane's search on the fritz?
>

I think you are replying to that same thread.

And about Gmane search, maybe try something like this Google trick:

"Formal description of Org files"
site:http://lists.gnu.org/archive/html/emacs-orgmode/

> Nick
>

-- 
Suvayu

Open source is the future. It sets us free.

Re: Formal description of Org files

[Nick Dokos]

suvayu ali <fatkasuvayu+linux@gmail.com> wrote:

> On Sat, Jul 16, 2011 at 7:09 PM, Nick Dokos <nicholas.dokos@hp.com> wrote:
> > There was a discussion about this on the list a few months ago, but I
> > can't get to search.gmane.org to find the thread atm, although I can get
> > to news.gmane.org: is that me or is gmane's search on the fritz?
> >
> 
> I think you are replying to that same thread.
> 

Right you are! I blitzed on the fact that Marcelo was replying in that thread:
I thought it was a new one.

Sorry about the noise,
Nick

Re: Formal description of Org files

[Eric Schulte]

Bastien <bzg@altern.org> writes:

> Hi Marcelo,
>
> Marcelo de Moraes Serpa <celoserpa@gmail.com> writes:
>
>> I'm creating a web app that interacts with orgmode files and allows
>> you to edit orgmode files on the browser. The edit part is not done.
>
> Wow, this would be a really useful tool.  Can't wait to test this!
>

I agree this could be very cool.  In browser editing could make Org-mode
backed collaboration with non-emacs users much more feasible, and could
greatly enhance current Org-mode/git backed wikis.  My only fear is that
you could end up implementing much of Emacs in JS.

Another option for serving Org-mode files could be an Emacs-based
web-server, for example elnode [1], which I just ran across today.

>
>> I'm quite good at Javascript, and I wouldn't mind hacking something
>> akin to orgmode elisp code and this will be what I'll do if
>> everything else fails, but wouldn't using a grammar be a cleaner and
>> more elegant solution?
>
> Can you tell more about what you mean by "grammar"?
>
> Back in february, at FOSDEM, someone asked for a description of the
> org-mode format specification.  This is still something that needs to be
> done.  Any stab at this (on Worg) would be really nice.  You can start
> anywhere (headlines, TODO keywords, etc.)
>
> If the "grammar" needs to be described in a specific format (more than
> just a formal description of the various syntactic elements of an Org
> file), let us know.
>

As I mentioned earlier in this thread, I think any formal expression
would be more useful if could be fed to existing parser-generation tools
to automatically write Org-mode parsers, or perhaps automatically
convert between Org-mode and other document formats.  I'm not sure
however to what degree that is just wishful thinking..

Cheers -- Eric

Footnotes: 
[1]  https://github.com/nicferrier/elnode

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Formal description of Org files

[Christopher Witte]

[-- Attachment #1: Type: text/plain, Size: 1882 bytes --]

On 18 July 2011 01:14, Eric Schulte <schulte.eric@gmail.com> wrote:

>
> As I mentioned earlier in this thread, I think any formal expression
> would be more useful if could be fed to existing parser-generation tools
> to automatically write Org-mode parsers, or perhaps automatically
> convert between Org-mode and other document formats.  I'm not sure
> however to what degree that is just wishful thinking..
>
>
Perhaps Pandoc (http://johnmacfarlane.net/pandoc/) would meet those
requirements.  It uses a great parsing library with a syntax that greatly
resembles a BNF grammar.  An output module is already written for org-mode,
so writing an input module would allow us to convert from org-mode to any
other of the existing Pandoc output formats (including "plain text,
markdown<http://daringfireball.net/projects/markdown/>,
reStructuredText<http://docutils.sourceforge.net/docs/ref/rst/introduction.html>,
HTML <http://www.w3.org/TR/html40/>, LaTeX <http://www.latex-project.org/>,
ConTeXt <http://www.pragma-ade.nl/>,
PDF<http://en.wikipedia.org/wiki/Portable_Document_Format>,
RTF <http://en.wikipedia.org/wiki/Rich_Text_Format>, DocBook
XML<http://www.docbook.org/>,
OpenDocument XML <http://opendocument.xml.org/>,
ODT<http://en.wikipedia.org/wiki/OpenDocument>,
GNU Texinfo <http://www.gnu.org/software/texinfo/>, MediaWiki
markup<http://www.mediawiki.org/wiki/Help:Formatting>,
textile <http://redcloth.org/textile>, groff
man<http://developer.apple.com/DOCUMENTATION/Darwin/Reference/ManPa>pages,
Emacs
org-mode <http://orgmode.org/>, EPUB
<http://en.wikipedia.org/wiki/EPUB>ebooks, and
S5 <http://meyerweb.com/eric/tools/s5/> and
Slidy<http://www.w3.org/Talks/Tools/Slidy>HTML slide shows").  I've
been thinking of doing this for sometime now, as
an exercise in learning Haskell, but realistically I just don't have the
time.  does anyone else have the time?

Chris.

[-- Attachment #2: Type: text/html, Size: 2470 bytes --]

Re: Formal description of Org files

[Bastien]

Hi Eric,

Eric Schulte <schulte.eric@gmail.com> writes:

> Another option for serving Org-mode files could be an Emacs-based
> web-server, for example elnode [1], which I just ran across today.

Elnode looks nice.

I've just tested it for 10 seconds and displayed the "Hello!" in
http://localhost:8010 -- already exciting :)

Still I don't really see how it could help collaboration on Org
files.  

Did you have any particular idea in mind?

-- 
 Bastien

Re: Formal description of Org files

[Marcelo de Moraes Serpa]

[-- Attachment #1: Type: text/plain, Size: 1863 bytes --]

Hey Bastien!

>Can you tell more about what you mean by "grammar"?

I think Nick pretty much nailed down the description of what a grammar would
be. I'm not well-versed in compiler-theory and my real world experience with
parsers are limited - I made some pretty hackish parsers in the past but
none used a grammar or parser-generator, though.

If having a grammer is so hard, then I think I will just use the elisp
regexp-based parsing implementation as a reference :)

@Eric: I would only need some basic syntax highlighting and tab / space
handling, as well as folding. I don't mean to implement an online version of
the org, since the best place to use org will always be emacs ... or not.
Let's see how it goes, I will keep you guys posted.

On Sat, Jul 16, 2011 at 11:56 AM, Bastien <bzg@altern.org> wrote:

> Hi Marcelo,
>
> Marcelo de Moraes Serpa <celoserpa@gmail.com> writes:
>
> > I'm creating a web app that interacts with orgmode files and allows
> > you to edit orgmode files on the browser. The edit part is not done.
>
> Wow, this would be a really useful tool.  Can't wait to test this!
>
> > I'm quite good at Javascript, and I wouldn't mind hacking something
> > akin to orgmode elisp code and this will be what I'll do if
> > everything else fails, but wouldn't using a grammar be a cleaner and
> > more elegant solution?
>
> Can you tell more about what you mean by "grammar"?
>
> Back in february, at FOSDEM, someone asked for a description of the
> org-mode format specification.  This is still something that needs to be
> done.  Any stab at this (on Worg) would be really nice.  You can start
> anywhere (headlines, TODO keywords, etc.)
>
> If the "grammar" needs to be described in a specific format (more than
> just a formal description of the various syntactic elements of an Org
> file), let us know.
>
> Thanks,
>
> --
>  Bastien
>

[-- Attachment #2: Type: text/html, Size: 2525 bytes --]

Re: Formal description of Org files

[Marcelo de Moraes Serpa]

[-- Attachment #1: Type: text/plain, Size: 2265 bytes --]

>
> If having a grammer is so hard, then I think I will just use the elisp
> regexp-based parsing implementation as a reference :)


By the way, that's what this guy did in this Ruby gem:

https://github.com/bdewey/org-ruby

Cheers,

- Marcelo.

On Mon, Jul 18, 2011 at 4:02 PM, Marcelo de Moraes Serpa <
celoserpa@gmail.com> wrote:

> Hey Bastien!
>
> >Can you tell more about what you mean by "grammar"?
>
> I think Nick pretty much nailed down the description of what a grammar
> would be. I'm not well-versed in compiler-theory and my real world
> experience with parsers are limited - I made some pretty hackish parsers in
> the past but none used a grammar or parser-generator, though.
>
> If having a grammer is so hard, then I think I will just use the elisp
> regexp-based parsing implementation as a reference :)
>
> @Eric: I would only need some basic syntax highlighting and tab / space
> handling, as well as folding. I don't mean to implement an online version of
> the org, since the best place to use org will always be emacs ... or not.
> Let's see how it goes, I will keep you guys posted.
>
>
> On Sat, Jul 16, 2011 at 11:56 AM, Bastien <bzg@altern.org> wrote:
>
>> Hi Marcelo,
>>
>> Marcelo de Moraes Serpa <celoserpa@gmail.com> writes:
>>
>> > I'm creating a web app that interacts with orgmode files and allows
>> > you to edit orgmode files on the browser. The edit part is not done.
>>
>> Wow, this would be a really useful tool.  Can't wait to test this!
>>
>> > I'm quite good at Javascript, and I wouldn't mind hacking something
>> > akin to orgmode elisp code and this will be what I'll do if
>> > everything else fails, but wouldn't using a grammar be a cleaner and
>> > more elegant solution?
>>
>> Can you tell more about what you mean by "grammar"?
>>
>> Back in february, at FOSDEM, someone asked for a description of the
>> org-mode format specification.  This is still something that needs to be
>> done.  Any stab at this (on Worg) would be really nice.  You can start
>> anywhere (headlines, TODO keywords, etc.)
>>
>> If the "grammar" needs to be described in a specific format (more than
>> just a formal description of the various syntactic elements of an Org
>> file), let us know.
>>
>> Thanks,
>>
>> --
>>  Bastien
>>
>
>

[-- Attachment #2: Type: text/html, Size: 3525 bytes --]

Re: Formal description of Org files

[Nicolas Goaziou]

Hello,

Marcelo de Moraes Serpa <celoserpa@gmail.com> writes:

> I'm creating a web app that interacts with orgmode files and allows you to
> edit orgmode files on the browser. The edit part is not done. I'm quite good
> at Javascript, and I wouldn't mind hacking something akin to orgmode elisp
> code and this will be what I'll do if everything else fails, but wouldn't
> using a grammar be a cleaner and more elegant solution?

I didn't follow closely the thread, but I think having a formal
description of Org files is unrealistic at the moment.

Org "format" is heavily dependant on user-configurable variables deeply
installed within emacs.  Some of these variables can drastically change
the grammar of the Org file. Example:

#+begin_src org
- item 1

- item a
#+end_src

Now, if org-empty-line-terminates-plain-lists is a non-nil value, any
parser should detect two lists of one item each, while keeping it to nil
will imply there is one list of two items.

Another example: (setq org-deadline-string "ZORGLUB:"). I let you
imagine what confusion it might introduce if the external tool you want
to use/implement isn't aware of this change.

Sure, it's all plain text, so Org files are accessible from outside
emacs. But, for now, only the plain-text structure is accessible from
the wild, not the Org one.

I can see three options here:

  - We remove every configurability concerning Org structure: we enforce
    DEADLINE to stay DEADLINE, lists items to be separated by a blank
    line, etc. But this isn't serious, is it?

  - We consider that Org "format" is a package containing the body _and_
    every configuration variable. But it isn't plain-text nor a format
    anymore.

  - We implement an intermediary real format (maybe in plain text, but
    I'd favor a nested-lists construct) that would serve as the common
    denominator for every exporter _and_ importer out there. I already
    suggested something like this in a recent thread about
    exporters. Obviously, I think this one is the only viable (and,
    icing on the cake, this would also be very useful for Org
    development).

Regards,

-- 
Nicolas Goaziou

Re: Formal description of Org files

[Bastien]

Hi Nicolas,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> I didn't follow closely the thread, but I think having a formal
> description of Org files is unrealistic at the moment.

Still it would be useful to *try* to describe Org's various elements as
formally as possible.

> Org "format" is heavily dependant on user-configurable variables deeply
> installed within emacs.  Some of these variables can drastically change
> the grammar of the Org file. Example:
>
> #+begin_src org
> - item 1
>
> - item a
> #+end_src
>
> Now, if org-empty-line-terminates-plain-lists is a non-nil value, any
> parser should detect two lists of one item each, while keeping it to nil
> will imply there is one list of two items.
>
> Another example: (setq org-deadline-string "ZORGLUB:"). I let you
> imagine what confusion it might introduce if the external tool you want
> to use/implement isn't aware of this change.
>
> Sure, it's all plain text, so Org files are accessible from outside
> emacs. But, for now, only the plain-text structure is accessible from
> the wild, not the Org one.
>
> I can see three options here:
>
>   - We remove every configurability concerning Org structure: we enforce
>     DEADLINE to stay DEADLINE, lists items to be separated by a blank
>     line, etc. But this isn't serious, is it?

Nope :)

>   - We consider that Org "format" is a package containing the body _and_
>     every configuration variable. But it isn't plain-text nor a format
>     anymore.

Not really useful -- the purpose of better defining the format is to
make it parsable outside Emacs.

>   - We implement an intermediary real format (maybe in plain text, but
>     I'd favor a nested-lists construct) that would serve as the common
>     denominator for every exporter _and_ importer out there. I already
>     suggested something like this in a recent thread about
>     exporters. Obviously, I think this one is the only viable (and,
>     icing on the cake, this would also be very useful for Org
>     development).

I see yet another option:

Describe a plain text format with the _default_ config options.  
Meaning DEADLINE will be DEADLINE -- and nothing else.  

-- 
 Bastien

Re: Formal description of Org files

[Jambunathan K]

http://www.mediawiki.org/wiki/Markup_spec should be a good starting
point this.

Specifically one could adopt an approach similar to what the "Parser
outline" section says and do some reverse literate programming
(elisp--->Natual Language)

Let me put on my didactic hat and remind the participants in this
list/thread the "GTD approach" to Getting Things Done.

Collect->Process->Organize->Review->Do

I see (and I could be wrong) an intermingling of all of the above steps
in a single step in this thread. A simple worg page that "collects" the
low hanging fruits to begin with will be a good but firm first step in
this direction. It would also enable consolidation of notes in a single
place (rather than it being fragmented in a bunch of mboxy thingies) and
also enable community contribution.

Let's all step back from the mailing list (or this thread) for a while
and contirbute to "growing" a spec from within a Worg page. I think it
would help if we keep our focus away from lexers/parsers and bnf/ebnf
etc and return to it when we refine the model.

Formal Descriptions only adds precision and (possibly refinement) to a
model. They are generally of limited help while growing the model
itself. So a informal spec in plain English (with accompanying regexes)
should be good enough. Mankind has made good progress with just pen and
paper and lately with plain text files :-).

ps: I definitely see Nicolas "at it". Others seem more like
cheerleaders. Both have their roles to play and their respective
contributions are equally helpful.

Jambunathan K.

Re: Formal description of Org files

[Bastien]

Hi Jambunathan,

Jambunathan K <kjambunathan@gmail.com> writes:

> A simple worg page that "collects" the low hanging fruits to begin
> with will be a good but firm first step in this direction.

Indeed.

It's on my todo list, but not with a high priority for now.

If you feel like starting it, please go ahead!  

-- 
 Bastien