Formal description of Org files

Formal description of Org files

[Carsten Dominik]

Hi,

I have a question.

At FOSDEM, someone asked me if there was a formal description of the structure of Org files, in some language that would be the input for a parser (or parser generator?) so that Org file could be easily parsed.

Unfortunately I did not catch the name of the format description language that could be
used for something, not did I catch the name of the person who talked to me.

Can anyone help out here?  Let me know what language to use, and maybe help work on such a formal description?  I think it would be useful to have....

- Carsten

Re: Formal description of Org files

[Rainer Stengele]

Am 15.04.2011 09:58, schrieb Carsten Dominik:
> Hi,
> 
> I have a question.
> 
> At FOSDEM, someone asked me if there was a formal description of the structure of Org files, in some language that would be the input for a parser (or parser generator?) so that Org file could be easily parsed.
> 
> Unfortunately I did not catch the name of the format description language that could be
> used for something, not did I catch the name of the person who talked to me.
> 
> Can anyone help out here?  Let me know what language to use, and maybe help work on such a formal description?  I think it would be useful to have....
> 
> - Carsten
> 
Hi,

maybe Backus-Naur was meant?

- Rainer

Re: Formal description of Org files

[Carsten Dominik]

On Apr 15, 2011, at 10:54 AM, Rainer Stengele wrote:

> Am 15.04.2011 09:58, schrieb Carsten Dominik:
>> Hi,
>> 
>> I have a question.
>> 
>> At FOSDEM, someone asked me if there was a formal description of the structure of Org files, in some language that would be the input for a parser (or parser generator?) so that Org file could be easily parsed.
>> 
>> Unfortunately I did not catch the name of the format description language that could be
>> used for something, not did I catch the name of the person who talked to me.
>> 
>> Can anyone help out here?  Let me know what language to use, and maybe help work on such a formal description?  I think it would be useful to have....
>> 
>> - Carsten
>> 
> Hi,
> 
> maybe Backus-Naur was meant?


That is very well possible.  Sounds like a word I would not have recognized....
So would on Org-mode grammar start like this?

-------------------------------------------------------------------------------

<headline> ::= "*"+ <opt-todo-keyword>
                    <opt-priority-cookie>
                    <title>
                    <opt-tags>

<opt-todo-keyword> ::= <whitespace> <todo-keyword> | ""

<opt-priority-cookie> ::= <whitespace> <priority-cookie> | ""

<opt-tags> ::= <whitespace> <tags> | ""



<todo-keyword> ::= "TODO" | "DONE"

<priority-cookie> ::= "[" "#" <priority> "]"

<priority> ::= "A" | "B" | "C"                   

<tags> ::= ( ":" <tag> )+ ":"

<whitespace> ::= " " <opt-whitespace> | "\t" <opt-whitespace>

<opt-whitespace> ::= <whitespace> | ""

-------------------------------------------------------------------------------

Re: Formal description of Org files

[Peter Frings]

On 15 Apr 2011, at 12:57, Carsten Dominik wrote:

> On Apr 15, 2011, at 10:54 AM, Rainer Stengele wrote:
> 
>> Am 15.04.2011 09:58, schrieb Carsten Dominik:
>>> Hi,
>>> 
>>> I have a question.
>>> 
>>> At FOSDEM, someone asked me if there was a formal description of the structure of Org files, in some language that would be the input for a parser (or parser generator?) so that Org file could be easily parsed.

<snip>

>> Hi,
>> 
>> maybe Backus-Naur was meant?
> 
> That is very well possible.  Sounds like a word I would not have recognized....
> So would on Org-mode grammar start like this?

> headline> ::= "*"+ <opt-todo-keyword>
>                    <opt-priority-cookie>
>                    <title>
>                    <opt-tags>
> 
> <opt-todo-keyword> ::= <whitespace> <todo-keyword> | “”
<snip>

Yes, that seems like extended BN. [1]
Good luck with it! You’ll need it... :-)

[1] http://en.wikipedia.org/wiki/Extended_Backus–Naur_Form


Cheers,
Peter.
-- 
c++;         // this makes c bigger but returns the old value

Re: Formal description of Org files

[Carsten Dominik]

On Apr 15, 2011, at 1:50 PM, Peter Frings wrote:

> 
> On 15 Apr 2011, at 12:57, Carsten Dominik wrote:
> 
>> On Apr 15, 2011, at 10:54 AM, Rainer Stengele wrote:
>> 
>>> Am 15.04.2011 09:58, schrieb Carsten Dominik:
>>>> Hi,
>>>> 
>>>> I have a question.
>>>> 
>>>> At FOSDEM, someone asked me if there was a formal description of the structure of Org files, in some language that would be the input for a parser (or parser generator?) so that Org file could be easily parsed.
> 
> <snip>
> 
>>> Hi,
>>> 
>>> maybe Backus-Naur was meant?
>> 
>> That is very well possible.  Sounds like a word I would not have recognized....
>> So would on Org-mode grammar start like this?
> 
>> headline> ::= "*"+ <opt-todo-keyword>
>>                   <opt-priority-cookie>
>>                   <title>
>>                   <opt-tags>
>> 
>> <opt-todo-keyword> ::= <whitespace> <todo-keyword> | “”
> <snip>
> 
> Yes, that seems like extended BN. [1]
> Good luck with it! You’ll need it... :-)

Well, if tis is complicated, I am not going to do this.
But maybe there is a volunteer out there - I'd be happy to
support him or her.

- Carsten

Re: Formal description of Org files

[Jambunathan K]

Carsten Dominik <carsten.dominik@gmail.com> writes:

> On Apr 15, 2011, at 1:50 PM, Peter Frings wrote:
>
>> 
>> On 15 Apr 2011, at 12:57, Carsten Dominik wrote:
>> 
>>> On Apr 15, 2011, at 10:54 AM, Rainer Stengele wrote:
>>> 
>>>> Am 15.04.2011 09:58, schrieb Carsten Dominik:
>>>>> Hi,
>>>>> 
>>>>> I have a question.
>>>>> 
>>>>> At FOSDEM, someone asked me if there was a formal description of
>>>>> the structure of Org files, in some language that would be the
>>>>> input for a parser (or parser generator?) so that Org file could
>>>>> be easily parsed.
>> 
>> <snip>
>> 
>>>> Hi,
>>>> 
>>>> maybe Backus-Naur was meant?
>>> 
>>> That is very well possible.  Sounds like a word I would not have recognized....
>>> So would on Org-mode grammar start like this?
>> 
>>> headline> ::= "*"+ <opt-todo-keyword>
>>>                   <opt-priority-cookie>
>>>                   <title>
>>>                   <opt-tags>
>>> 
>>> <opt-todo-keyword> ::= <whitespace> <todo-keyword> | “”
>> <snip>
>> 
>> Yes, that seems like extended BN. [1]
>> Good luck with it! You’ll need it... :-)
>
> Well, if tis is complicated, I am not going to do this.
> But maybe there is a volunteer out there - I'd be happy to
> support him or her.

IF there is a need for a lexer and parser may be one should start with
the parser that is part of cedet.

http://www.emacswiki.org/emacs/HowToSetUpSemanticBovinatorForANewLanguage

Jambunathan K.

> - Carsten

-- 

Re: Formal description of Org files

[Olaf.Hamann]

If I remember well, the thing with BNF and similar approaches was called
'context free grammar' - and I'm not sure, if this requirement is
fullfilled by the org syntax. As I mean to remember is e.g. that Perl
could not be declared fully with a context free grammar.

As if 'parse an org-file completely into a token stream' is not required
by syntax highlighting tools etc,
it might be enough to give some regexps or so. Perhaps for use with etags.
Perhaps for modifying ecb to extract main elements of an org-file into
it's subwindows.

Greetings, Olaf

Re: Formal description of Org files

[Christian Egli]

Carsten Dominik <carsten.dominik@gmail.com> writes:

> At FOSDEM, someone asked me if there was a formal description of the
> structure of Org files, in some language that would be the input for a
> parser (or parser generator?) so that Org file could be easily parsed.

Maybe the person was talking about antlr[1], "ANother Tool for Language
Recognition, a language tool that provides a framework for constructing
recognizers, interpreters, compilers, and translators from grammatical
descriptions containing actions in a variety of target languages".

It even seems to have preliminary support for generating an elisp
parser[2][3]

There is also an emacs mode for editing antlr files[4].

Sounds like an interesting project.

Thanks
Christian

Footnotes: 
[1]  http://www.antlr.org/
[2]  http://www.antlr.org/wiki/display/ANTLR3/Code+Generation+Targets
[3]  https://github.com/olabini/antlr-elisp
[4]  http://antlr-mode.sourceforge.net/
-- 
Christian Egli
Swiss Library for the Blind, Visually Impaired and Print Disabled
Grubenstrasse 12, CH-8045 Zürich, Switzerland

Re: Formal description of Org files

[Peter Frings]

On 15 Apr 2011, at 14:58, Christian Egli wrote:

> Carsten Dominik <carsten.dominik@gmail.com> writes:
> 
>> At FOSDEM, someone asked me if there was a formal description of the
>> structure of Org files, in some language that would be the input for a
>> parser (or parser generator?) so that Org file could be easily parsed.
> 
> Maybe the person was talking about antlr[1], "ANother Tool for Language
> Recognition, a language tool that provides a framework for constructing
> recognizers, interpreters, compilers, and translators from grammatical
> descriptions containing actions in a variety of target languages”.
<snip>

> Sounds like an interesting project.

Wow, if that thing can export syntax diagrams in PNG or PDF I’d be really happy. Looks very interesting — albeit serious overkill for what I’d use it :-).

thanks,
Peter.
-- 
c++;         // this makes c bigger but returns the old value

Re: Formal description of Org files

[Filippo A. Salustri]

[-- Attachment #1: Type: text/plain, Size: 2023 bytes --]

Antlr is just another in a long line of lexical parsers.  I still remember
the original lex (for lexical analysis), which in combination with yacc (for
parsing and grammar) could make pretty much any conventional programming
language.
Then GNU came up with Flex (fast lex) and Bison (instead of yacc...get it?
:)
Then IDEs really started to take off and much of the ugly parts of writing
languages disappeared, which led to all kinds of new tools like antlr.

But they all basically do the same thing: let one describe the syntax and
grammar of (quasi) formal (programming) languages.

I don't know if any of them produce diagrams, but I wouldn't be surprised if
at least some did.

Cheers.
Fil

On 15 April 2011 09:42, Peter Frings <peter.frings@agfa.com> wrote:

>
> On 15 Apr 2011, at 14:58, Christian Egli wrote:
>
> > Carsten Dominik <carsten.dominik@gmail.com> writes:
> >
> >> At FOSDEM, someone asked me if there was a formal description of the
> >> structure of Org files, in some language that would be the input for a
> >> parser (or parser generator?) so that Org file could be easily parsed.
> >
> > Maybe the person was talking about antlr[1], "ANother Tool for Language
> > Recognition, a language tool that provides a framework for constructing
> > recognizers, interpreters, compilers, and translators from grammatical
> > descriptions containing actions in a variety of target languages”.
> <snip>
>
> > Sounds like an interesting project.
>
> Wow, if that thing can export syntax diagrams in PNG or PDF I’d be really
> happy. Looks very interesting — albeit serious overkill for what I’d use it
> :-).
>
> thanks,
> Peter.
> --
> c++;         // this makes c bigger but returns the old value
>
>
>


-- 
Filippo A. Salustri, Ph.D., P.Eng.
Mechanical and Industrial Engineering
Ryerson University
350 Victoria St, Toronto, ON
M5B 2K3, Canada
Tel: 416/979-5000 ext 7749
Fax: 416/979-5265
Email: salustri@ryerson.ca
http://deseng.ryerson.ca/~fil/

[-- Attachment #2: Type: text/html, Size: 2671 bytes --]

Re: Formal description of Org files

[Wes Hardaker]

>>>>> On Fri, 15 Apr 2011 09:58:09 +0200, Carsten Dominik <carsten.dominik@gmail.com> said:

CD> Unfortunately I did not catch the name of the format description
CD> language that could be used for something, not did I catch the name
CD> of the person who talked to me.

Another option, besides those mentioned, is probably ABNF.

-- 
Wes Hardaker                                     
My Pictures:  http://capturedonearth.com/
My Thoughts:  http://pontifications.hardakers.net/

Re: Formal description of Org files

[Eric Schulte]

If one goal of such a formal description of Org-mode would be to parse
text Org-mode files into an abstract syntax tree (which is reminiscent
of [1] and [2]) then perhaps we should look at parsers which have
already been applied to other document languages (e.g. tex, html etc...)
or at least support xml output.  If such parsers exist and have been
widely applied, then maybe they could support the exportation of
Org-mode files, or even automate the writing of code for such exporters?

Best -- Eric

Wes Hardaker <wjhns209@hardakers.net> writes:

>>>>>> On Fri, 15 Apr 2011 09:58:09 +0200, Carsten Dominik
> <carsten.dominik@gmail.com> said:
>
> CD> Unfortunately I did not catch the name of the format description
> CD> language that could be used for something, not did I catch the name
> CD> of the person who talked to me.
>
> Another option, besides those mentioned, is probably ABNF.


Footnotes: 
[1]  the effort to unify exporting behind a single parser of Org-mode
     files into an abstract elisp tree

[2]  http://johnmacfarlane.net/pandoc/, which as I recall has an
     Org-mode backend

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Formal description of Org files

[Nick Dokos]

Eric Schulte <schulte.eric@gmail.com> wrote:

> If one goal of such a formal description of Org-mode would be to parse
> text Org-mode files into an abstract syntax tree ...
> 

I think this should be the starting point: what are the goals for all this?
Providing a formal description in EBNF is one thing. Preparing an attribute
grammar for input into a specific tool is another (and probably an order of
magnitude - or two - harder) - what would the resulting parser(s) be used for?

Clear(er) answers to these questions should go a long way towards figuring out
what specific tool(s) should be used - or whether it's at all necessary to
worry about that.

Nick

Re: Formal description of Org files

[Olivier Schwander]

Le 15 Apr 2011 14:31, Nick Dokos a écrit:
> Eric Schulte <schulte.eric@gmail.com> wrote:
> 
> > If one goal of such a formal description of Org-mode would be to parse
> > text Org-mode files into an abstract syntax tree ...
> > 
> 
> I think this should be the starting point: what are the goals for all this?
> Providing a formal description in EBNF is one thing. Preparing an attribute
> grammar for input into a specific tool is another (and probably an order of
> magnitude - or two - harder) - what would the resulting parser(s) be used for?
> 
> Clear(er) answers to these questions should go a long way towards figuring out
> what specific tool(s) should be used - or whether it's at all necessary to
> worry about that.

The primary goal I see for such a formal description is to provide a
specification that third party parsers are supposed to respect. Writing
a real parser may be too much project specific and difficult to
generalize in a way usable by the community.

During the development of neo[1], I was confronted to the need of
defining what is an org file (actually, what is an headline, a todo
keyword, a tag, a drawer, a timestamp, etc) and determining what is the
expected output of a parser.

Maybe the most appropriate format for such a description would be free
text, letting parser developers choosing between context-free grammars,
regexps or whatever they want ( with a bunch of example org files for
reference and tests).

Regards,

Olivier

[1] I am just discovering this thread

Re: Formal description of Org files

[Marcelo de Moraes Serpa]

[-- Attachment #1: Type: text/plain, Size: 2221 bytes --]

Hi guys,

I was going to create a new thread, but this one seems to fit exactly what
I'm looking for.

I'm creating a web app that interacts with orgmode files and allows you to
edit orgmode files on the browser. The edit part is not done. I'm quite good
at Javascript, and I wouldn't mind hacking something akin to orgmode elisp
code and this will be what I'll do if everything else fails, but wouldn't
using a grammar be a cleaner and more elegant solution?

Thanks,

Marcelo.

On Wed, Apr 20, 2011 at 7:37 AM, Olivier Schwander <
olivier.schwander@chadok.info> wrote:

> Le 15 Apr 2011 14:31, Nick Dokos a écrit:
> > Eric Schulte <schulte.eric@gmail.com> wrote:
> >
> > > If one goal of such a formal description of Org-mode would be to parse
> > > text Org-mode files into an abstract syntax tree ...
> > >
> >
> > I think this should be the starting point: what are the goals for all
> this?
> > Providing a formal description in EBNF is one thing. Preparing an
> attribute
> > grammar for input into a specific tool is another (and probably an order
> of
> > magnitude - or two - harder) - what would the resulting parser(s) be used
> for?
> >
> > Clear(er) answers to these questions should go a long way towards
> figuring out
> > what specific tool(s) should be used - or whether it's at all necessary
> to
> > worry about that.
>
> The primary goal I see for such a formal description is to provide a
> specification that third party parsers are supposed to respect. Writing
> a real parser may be too much project specific and difficult to
> generalize in a way usable by the community.
>
> During the development of neo[1], I was confronted to the need of
> defining what is an org file (actually, what is an headline, a todo
> keyword, a tag, a drawer, a timestamp, etc) and determining what is the
> expected output of a parser.
>
> Maybe the most appropriate format for such a description would be free
> text, letting parser developers choosing between context-free grammars,
> regexps or whatever they want ( with a bunch of example org files for
> reference and tests).
>
> Regards,
>
> Olivier
>
> [1] I am just discovering this thread
>
>

[-- Attachment #2: Type: text/html, Size: 2794 bytes --]

Re: Formal description of Org files

[Bastien]

Hi Marcelo,

Marcelo de Moraes Serpa <celoserpa@gmail.com> writes:

> I'm creating a web app that interacts with orgmode files and allows
> you to edit orgmode files on the browser. The edit part is not done.

Wow, this would be a really useful tool.  Can't wait to test this!

> I'm quite good at Javascript, and I wouldn't mind hacking something
> akin to orgmode elisp code and this will be what I'll do if
> everything else fails, but wouldn't using a grammar be a cleaner and
> more elegant solution?

Can you tell more about what you mean by "grammar"?

Back in february, at FOSDEM, someone asked for a description of the
org-mode format specification.  This is still something that needs to be
done.  Any stab at this (on Worg) would be really nice.  You can start
anywhere (headlines, TODO keywords, etc.)

If the "grammar" needs to be described in a specific format (more than
just a formal description of the various syntactic elements of an Org
file), let us know.

Thanks,

-- 
 Bastien

Re: Formal description of Org files

[Nick Dokos]

Bastien <bzg@altern.org> wrote:

> Hi Marcelo,
> 
> Marcelo de Moraes Serpa <celoserpa@gmail.com> writes:
> 
> > I'm creating a web app that interacts with orgmode files and allows
> > you to edit orgmode files on the browser. The edit part is not done.
> 
> Wow, this would be a really useful tool.  Can't wait to test this!
> 
> > I'm quite good at Javascript, and I wouldn't mind hacking something
> > akin to orgmode elisp code and this will be what I'll do if
> > everything else fails, but wouldn't using a grammar be a cleaner and
> > more elegant solution?
> 
> Can you tell more about what you mean by "grammar"?
> 

A formal grammar of some sort I presume: BNF or EBNF.

> Back in february, at FOSDEM, someone asked for a description of the
> org-mode format specification.  This is still something that needs to be
> done.  Any stab at this (on Worg) would be really nice.  You can start
> anywhere (headlines, TODO keywords, etc.)
> 
> If the "grammar" needs to be described in a specific format (more than
> just a formal description of the various syntactic elements of an Org
> file), let us know.
> 

There was a discussion about this on the list a few months ago, but I
can't get to search.gmane.org to find the thread atm, although I can get
to news.gmane.org: is that me or is gmane's search on the fritz?

Nick

Re: Formal description of Org files

[Bastien]

Nick Dokos <nicholas.dokos@hp.com> writes:

> There was a discussion about this on the list a few months ago, but I
> can't get to search.gmane.org to find the thread atm, although I can get
> to news.gmane.org: is that me or is gmane's search on the fritz?

Looks like gmane search is down, I can't use it either.

-- 
 Bastien

Searching the org list (was: Formal description of Org files)

[Memnon Anon]

Nick Dokos <nicholas.dokos@hp.com> writes:

> There was a discussion about this on the list a few months ago, but I
> can't get to search.gmane.org to find the thread atm, although I can get
> to news.gmane.org: is that me or is gmane's search on the fritz?

I had some trouble with gmane search for days, too.

Alternatives:

http://www.mail-archive.com/emacs-orgmode@gnu.org/
http://lists.gnu.org/archive/html/emacs-orgmode/

Memnon

Re: Formal description of Org files

[suvayu ali]

On Sat, Jul 16, 2011 at 7:09 PM, Nick Dokos <nicholas.dokos@hp.com> wrote:
> There was a discussion about this on the list a few months ago, but I
> can't get to search.gmane.org to find the thread atm, although I can get
> to news.gmane.org: is that me or is gmane's search on the fritz?
>

I think you are replying to that same thread.

And about Gmane search, maybe try something like this Google trick:

"Formal description of Org files"
site:http://lists.gnu.org/archive/html/emacs-orgmode/

> Nick
>

-- 
Suvayu

Open source is the future. It sets us free.

Re: Formal description of Org files

[Nick Dokos]

suvayu ali <fatkasuvayu+linux@gmail.com> wrote:

> On Sat, Jul 16, 2011 at 7:09 PM, Nick Dokos <nicholas.dokos@hp.com> wrote:
> > There was a discussion about this on the list a few months ago, but I
> > can't get to search.gmane.org to find the thread atm, although I can get
> > to news.gmane.org: is that me or is gmane's search on the fritz?
> >
> 
> I think you are replying to that same thread.
> 

Right you are! I blitzed on the fact that Marcelo was replying in that thread:
I thought it was a new one.

Sorry about the noise,
Nick

Re: Formal description of Org files

[Eric Schulte]

Bastien <bzg@altern.org> writes:

> Hi Marcelo,
>
> Marcelo de Moraes Serpa <celoserpa@gmail.com> writes:
>
>> I'm creating a web app that interacts with orgmode files and allows
>> you to edit orgmode files on the browser. The edit part is not done.
>
> Wow, this would be a really useful tool.  Can't wait to test this!
>

I agree this could be very cool.  In browser editing could make Org-mode
backed collaboration with non-emacs users much more feasible, and could
greatly enhance current Org-mode/git backed wikis.  My only fear is that
you could end up implementing much of Emacs in JS.

Another option for serving Org-mode files could be an Emacs-based
web-server, for example elnode [1], which I just ran across today.

>
>> I'm quite good at Javascript, and I wouldn't mind hacking something
>> akin to orgmode elisp code and this will be what I'll do if
>> everything else fails, but wouldn't using a grammar be a cleaner and
>> more elegant solution?
>
> Can you tell more about what you mean by "grammar"?
>
> Back in february, at FOSDEM, someone asked for a description of the
> org-mode format specification.  This is still something that needs to be
> done.  Any stab at this (on Worg) would be really nice.  You can start
> anywhere (headlines, TODO keywords, etc.)
>
> If the "grammar" needs to be described in a specific format (more than
> just a formal description of the various syntactic elements of an Org
> file), let us know.
>

As I mentioned earlier in this thread, I think any formal expression
would be more useful if could be fed to existing parser-generation tools
to automatically write Org-mode parsers, or perhaps automatically
convert between Org-mode and other document formats.  I'm not sure
however to what degree that is just wishful thinking..

Cheers -- Eric

Footnotes: 
[1]  https://github.com/nicferrier/elnode

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Formal description of Org files

[Christopher Witte]

[-- Attachment #1: Type: text/plain, Size: 1882 bytes --]

On 18 July 2011 01:14, Eric Schulte <schulte.eric@gmail.com> wrote:

>
> As I mentioned earlier in this thread, I think any formal expression
> would be more useful if could be fed to existing parser-generation tools
> to automatically write Org-mode parsers, or perhaps automatically
> convert between Org-mode and other document formats.  I'm not sure
> however to what degree that is just wishful thinking..
>
>
Perhaps Pandoc (http://johnmacfarlane.net/pandoc/) would meet those
requirements.  It uses a great parsing library with a syntax that greatly
resembles a BNF grammar.  An output module is already written for org-mode,
so writing an input module would allow us to convert from org-mode to any
other of the existing Pandoc output formats (including "plain text,
markdown<http://daringfireball.net/projects/markdown/>,
reStructuredText<http://docutils.sourceforge.net/docs/ref/rst/introduction.html>,
HTML <http://www.w3.org/TR/html40/>, LaTeX <http://www.latex-project.org/>,
ConTeXt <http://www.pragma-ade.nl/>,
PDF<http://en.wikipedia.org/wiki/Portable_Document_Format>,
RTF <http://en.wikipedia.org/wiki/Rich_Text_Format>, DocBook
XML<http://www.docbook.org/>,
OpenDocument XML <http://opendocument.xml.org/>,
ODT<http://en.wikipedia.org/wiki/OpenDocument>,
GNU Texinfo <http://www.gnu.org/software/texinfo/>, MediaWiki
markup<http://www.mediawiki.org/wiki/Help:Formatting>,
textile <http://redcloth.org/textile>, groff
man<http://developer.apple.com/DOCUMENTATION/Darwin/Reference/ManPa>pages,
Emacs
org-mode <http://orgmode.org/>, EPUB
<http://en.wikipedia.org/wiki/EPUB>ebooks, and
S5 <http://meyerweb.com/eric/tools/s5/> and
Slidy<http://www.w3.org/Talks/Tools/Slidy>HTML slide shows").  I've
been thinking of doing this for sometime now, as
an exercise in learning Haskell, but realistically I just don't have the
time.  does anyone else have the time?

Chris.

[-- Attachment #2: Type: text/html, Size: 2470 bytes --]

Re: Formal description of Org files

[Bastien]

Hi Eric,

Eric Schulte <schulte.eric@gmail.com> writes:

> Another option for serving Org-mode files could be an Emacs-based
> web-server, for example elnode [1], which I just ran across today.

Elnode looks nice.

I've just tested it for 10 seconds and displayed the "Hello!" in
http://localhost:8010 -- already exciting :)

Still I don't really see how it could help collaboration on Org
files.  

Did you have any particular idea in mind?

-- 
 Bastien

Re: Formal description of Org files

[Marcelo de Moraes Serpa]

[-- Attachment #1: Type: text/plain, Size: 1863 bytes --]

Hey Bastien!

>Can you tell more about what you mean by "grammar"?

I think Nick pretty much nailed down the description of what a grammar would
be. I'm not well-versed in compiler-theory and my real world experience with
parsers are limited - I made some pretty hackish parsers in the past but
none used a grammar or parser-generator, though.

If having a grammer is so hard, then I think I will just use the elisp
regexp-based parsing implementation as a reference :)

@Eric: I would only need some basic syntax highlighting and tab / space
handling, as well as folding. I don't mean to implement an online version of
the org, since the best place to use org will always be emacs ... or not.
Let's see how it goes, I will keep you guys posted.

On Sat, Jul 16, 2011 at 11:56 AM, Bastien <bzg@altern.org> wrote:

> Hi Marcelo,
>
> Marcelo de Moraes Serpa <celoserpa@gmail.com> writes:
>
> > I'm creating a web app that interacts with orgmode files and allows
> > you to edit orgmode files on the browser. The edit part is not done.
>
> Wow, this would be a really useful tool.  Can't wait to test this!
>
> > I'm quite good at Javascript, and I wouldn't mind hacking something
> > akin to orgmode elisp code and this will be what I'll do if
> > everything else fails, but wouldn't using a grammar be a cleaner and
> > more elegant solution?
>
> Can you tell more about what you mean by "grammar"?
>
> Back in february, at FOSDEM, someone asked for a description of the
> org-mode format specification.  This is still something that needs to be
> done.  Any stab at this (on Worg) would be really nice.  You can start
> anywhere (headlines, TODO keywords, etc.)
>
> If the "grammar" needs to be described in a specific format (more than
> just a formal description of the various syntactic elements of an Org
> file), let us know.
>
> Thanks,
>
> --
>  Bastien
>

[-- Attachment #2: Type: text/html, Size: 2525 bytes --]

Re: Formal description of Org files

[Marcelo de Moraes Serpa]

[-- Attachment #1: Type: text/plain, Size: 2265 bytes --]

>
> If having a grammer is so hard, then I think I will just use the elisp
> regexp-based parsing implementation as a reference :)


By the way, that's what this guy did in this Ruby gem:

https://github.com/bdewey/org-ruby

Cheers,

- Marcelo.

On Mon, Jul 18, 2011 at 4:02 PM, Marcelo de Moraes Serpa <
celoserpa@gmail.com> wrote:

> Hey Bastien!
>
> >Can you tell more about what you mean by "grammar"?
>
> I think Nick pretty much nailed down the description of what a grammar
> would be. I'm not well-versed in compiler-theory and my real world
> experience with parsers are limited - I made some pretty hackish parsers in
> the past but none used a grammar or parser-generator, though.
>
> If having a grammer is so hard, then I think I will just use the elisp
> regexp-based parsing implementation as a reference :)
>
> @Eric: I would only need some basic syntax highlighting and tab / space
> handling, as well as folding. I don't mean to implement an online version of
> the org, since the best place to use org will always be emacs ... or not.
> Let's see how it goes, I will keep you guys posted.
>
>
> On Sat, Jul 16, 2011 at 11:56 AM, Bastien <bzg@altern.org> wrote:
>
>> Hi Marcelo,
>>
>> Marcelo de Moraes Serpa <celoserpa@gmail.com> writes:
>>
>> > I'm creating a web app that interacts with orgmode files and allows
>> > you to edit orgmode files on the browser. The edit part is not done.
>>
>> Wow, this would be a really useful tool.  Can't wait to test this!
>>
>> > I'm quite good at Javascript, and I wouldn't mind hacking something
>> > akin to orgmode elisp code and this will be what I'll do if
>> > everything else fails, but wouldn't using a grammar be a cleaner and
>> > more elegant solution?
>>
>> Can you tell more about what you mean by "grammar"?
>>
>> Back in february, at FOSDEM, someone asked for a description of the
>> org-mode format specification.  This is still something that needs to be
>> done.  Any stab at this (on Worg) would be really nice.  You can start
>> anywhere (headlines, TODO keywords, etc.)
>>
>> If the "grammar" needs to be described in a specific format (more than
>> just a formal description of the various syntactic elements of an Org
>> file), let us know.
>>
>> Thanks,
>>
>> --
>>  Bastien
>>
>
>

[-- Attachment #2: Type: text/html, Size: 3525 bytes --]

Re: Formal description of Org files

[Nicolas Goaziou]

Hello,

Marcelo de Moraes Serpa <celoserpa@gmail.com> writes:

> I'm creating a web app that interacts with orgmode files and allows you to
> edit orgmode files on the browser. The edit part is not done. I'm quite good
> at Javascript, and I wouldn't mind hacking something akin to orgmode elisp
> code and this will be what I'll do if everything else fails, but wouldn't
> using a grammar be a cleaner and more elegant solution?

I didn't follow closely the thread, but I think having a formal
description of Org files is unrealistic at the moment.

Org "format" is heavily dependant on user-configurable variables deeply
installed within emacs.  Some of these variables can drastically change
the grammar of the Org file. Example:

#+begin_src org
- item 1

- item a
#+end_src

Now, if org-empty-line-terminates-plain-lists is a non-nil value, any
parser should detect two lists of one item each, while keeping it to nil
will imply there is one list of two items.

Another example: (setq org-deadline-string "ZORGLUB:"). I let you
imagine what confusion it might introduce if the external tool you want
to use/implement isn't aware of this change.

Sure, it's all plain text, so Org files are accessible from outside
emacs. But, for now, only the plain-text structure is accessible from
the wild, not the Org one.

I can see three options here:

  - We remove every configurability concerning Org structure: we enforce
    DEADLINE to stay DEADLINE, lists items to be separated by a blank
    line, etc. But this isn't serious, is it?

  - We consider that Org "format" is a package containing the body _and_
    every configuration variable. But it isn't plain-text nor a format
    anymore.

  - We implement an intermediary real format (maybe in plain text, but
    I'd favor a nested-lists construct) that would serve as the common
    denominator for every exporter _and_ importer out there. I already
    suggested something like this in a recent thread about
    exporters. Obviously, I think this one is the only viable (and,
    icing on the cake, this would also be very useful for Org
    development).

Regards,

-- 
Nicolas Goaziou

Re: Formal description of Org files

[Bastien]

Hi Nicolas,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> I didn't follow closely the thread, but I think having a formal
> description of Org files is unrealistic at the moment.

Still it would be useful to *try* to describe Org's various elements as
formally as possible.

> Org "format" is heavily dependant on user-configurable variables deeply
> installed within emacs.  Some of these variables can drastically change
> the grammar of the Org file. Example:
>
> #+begin_src org
> - item 1
>
> - item a
> #+end_src
>
> Now, if org-empty-line-terminates-plain-lists is a non-nil value, any
> parser should detect two lists of one item each, while keeping it to nil
> will imply there is one list of two items.
>
> Another example: (setq org-deadline-string "ZORGLUB:"). I let you
> imagine what confusion it might introduce if the external tool you want
> to use/implement isn't aware of this change.
>
> Sure, it's all plain text, so Org files are accessible from outside
> emacs. But, for now, only the plain-text structure is accessible from
> the wild, not the Org one.
>
> I can see three options here:
>
>   - We remove every configurability concerning Org structure: we enforce
>     DEADLINE to stay DEADLINE, lists items to be separated by a blank
>     line, etc. But this isn't serious, is it?

Nope :)

>   - We consider that Org "format" is a package containing the body _and_
>     every configuration variable. But it isn't plain-text nor a format
>     anymore.

Not really useful -- the purpose of better defining the format is to
make it parsable outside Emacs.

>   - We implement an intermediary real format (maybe in plain text, but
>     I'd favor a nested-lists construct) that would serve as the common
>     denominator for every exporter _and_ importer out there. I already
>     suggested something like this in a recent thread about
>     exporters. Obviously, I think this one is the only viable (and,
>     icing on the cake, this would also be very useful for Org
>     development).

I see yet another option:

Describe a plain text format with the _default_ config options.  
Meaning DEADLINE will be DEADLINE -- and nothing else.  

-- 
 Bastien

Re: Formal description of Org files

[Jambunathan K]

http://www.mediawiki.org/wiki/Markup_spec should be a good starting
point this.

Specifically one could adopt an approach similar to what the "Parser
outline" section says and do some reverse literate programming
(elisp--->Natual Language)

Let me put on my didactic hat and remind the participants in this
list/thread the "GTD approach" to Getting Things Done.

Collect->Process->Organize->Review->Do

I see (and I could be wrong) an intermingling of all of the above steps
in a single step in this thread. A simple worg page that "collects" the
low hanging fruits to begin with will be a good but firm first step in
this direction. It would also enable consolidation of notes in a single
place (rather than it being fragmented in a bunch of mboxy thingies) and
also enable community contribution.

Let's all step back from the mailing list (or this thread) for a while
and contirbute to "growing" a spec from within a Worg page. I think it
would help if we keep our focus away from lexers/parsers and bnf/ebnf
etc and return to it when we refine the model.

Formal Descriptions only adds precision and (possibly refinement) to a
model. They are generally of limited help while growing the model
itself. So a informal spec in plain English (with accompanying regexes)
should be good enough. Mankind has made good progress with just pen and
paper and lately with plain text files :-).

ps: I definitely see Nicolas "at it". Others seem more like
cheerleaders. Both have their roles to play and their respective
contributions are equally helpful.

Jambunathan K.

Re: Formal description of Org files

[Bastien]

Hi Jambunathan,

Jambunathan K <kjambunathan@gmail.com> writes:

> A simple worg page that "collects" the low hanging fruits to begin
> with will be a good but firm first step in this direction.

Indeed.

It's on my todo list, but not with a high priority for now.

If you feel like starting it, please go ahead!  

-- 
 Bastien