Using Org-mode file format for storing configuration data

Using Org-mode file format for storing configuration data

[Karl Voit]

Hi!

I plan to implement a new weblog system that parses Org-mode files
and generates (static) HTML output. Yes, I am aware that there are
other solutions out there but I do not like them for various
reasons.[1]

So for my new system, I am thinking of using Org-mode files for
writing (and parsing) the user-defined preferences.

- why not using elisp-methods like "setq" or "defvar" or "defcustom"?
  - this weblog will be implemented in Python
  - Sorry, I do not have any elisp knowledge and it seems to be the
    case that this is not going to change within the next months.
  - In Python I have to parse a basic sub-set of Org-mode format
    anyhow. An additional parser would be more work to do.
  - Why not using Org-mode for end-user configuration as well?
    - It's more user-friendly for non elisp users like me.

- Possible methods to store configuration/settings of a weblog system
  that scans Org-mode files to generate HTML:
  - in drawers: see below
  - in tables: see below
  - in tags: see below
  - other possibilities?

What do you think of this?
Can you imagine a better way of storing key-value-pairs in Org-mode?

My focus is user friendly maintenance and overview including in-line
documentation of the preferences.


* Configuring Paths (in Drawers)
:PROPERTIES:
:COLUMNS:  %25ITEM %PATH
:END:

- advantages
  - usage of column view
  - easy to add comments for each setting
- disadvantages
  - Properties have to be the same
    - not that flexible in naming Properties

- possible name for property "VALUE" which makes it more general.

** Image-Path
:PROPERTIES:
:PATH: ~/images
:END:

In this folder, the Web log image files are placed after being
processed (resizing, ...).

** HTML-Path
:PROPERTIES:
:PATH: ~/blog/html
:END:



* Configuring Paths (in Tables)

| *Variable*      | *Value*       | *Comments*                                          |
|-----------------+---------------+-----------------------------------------------------|
| Image Path      | ~/images      | The path where the images should be placed in.      |
| HTML Path       | ~/blog/html   | Destination path for the HTML file folder structure |
| Another Setting | another value | This has to be very short.                          |

- advantages
  - easy to write and maintain
  - quick overview
- disadvantages
  - hard to add (long) comments for each setting



* Configuring Paths (with Tags)

- advantages
  - same settings can be shared through multiple tags on same value
  - easy to add comments for each setting
- disadvantages
  - seems strange at first glance; not sure if this really makes any
    sense at all
  - multiple word preference name has to be written in CamelCase or
    separated with dashes

** ~/images                                          :ImagePath:

** ~/blog/html                                        :HtmlPath:



  1. http://article.gmane.org/gmane.emacs.orgmode/49747/
-- 
Karl Voit

Re: Using Org-mode file format for storing configuration data

[Bastien]

Hi Karl,

Karl Voit <devnull@Karl-Voit.at> writes:

> I plan to implement a new weblog system that parses Org-mode files
> and generates (static) HTML output. Yes, I am aware that there are
> other solutions out there but I do not like them for various
> reasons.[1]

It's always good to have new implementations -- looking forward to
see this one happen!

I have a plan to add a org-e-rss.el backend that would allow converting
an Org file into a RSS feed.  Together with a more clever org-publish.el,
this could be used as a simple blog by updated the RSS feed depending on
what new headlines have been published.  This would suits most of my own
needs, and it's not that much work...  Just to let you know.

Best,

-- 
 Bastien

Org to RSS (was: Using Org-mode file format for storing configuration data)

[Karl Voit]

* Bastien <bzg@altern.org> wrote:
> Hi Karl,

Hi!

> Karl Voit <devnull@Karl-Voit.at> writes:
>
>> I plan to implement a new weblog system that parses Org-mode files
>> and generates (static) HTML output. Yes, I am aware that there are
>> other solutions out there but I do not like them for various
>> reasons.[1]
>
> It's always good to have new implementations -- looking forward to
> see this one happen!

Me too :-)

> I have a plan to add a org-e-rss.el backend that would allow converting
> an Org file into a RSS feed.  Together with a more clever org-publish.el,
> this could be used as a simple blog by updated the RSS feed depending on
> what new headlines have been published.  This would suits most of my own
> needs, and it's not that much work...  Just to let you know.

Great news!

Although in my particular case, it will not be part of my solution -
sadly :-(

-- 
Karl Voit

Re: Using Org-mode file format for storing configuration data

[Daniel Clemente]

El Sun, 30 Dec 2012 19:04:25 +0100 Karl Voit va escriure:
> 
> I plan to implement a new weblog system that parses Org-mode files
> and generates (static) HTML output. Yes, I am aware that there are
> other solutions out there but I do not like them for various
> reasons.[1]

  Nice! I also don't like existing solutions and I was thinking on writing some Python to do the export. But the complexities of exporting are so well resolved in elisp that it's much easier to invoke elisp code than to write your own in Python.


> 
> So for my new system, I am thinking of using Org-mode files for
> writing (and parsing) the user-defined preferences.

  I happened to be thinking the same two days ago. I wanted a place for scripts to store data as key→value pairs: e.g. configurations, or observations (like: file count on ~). Normally I would use lots of small hidden files for that. Now I wanted a central registry.
  I thought of using some cache server, or a small NoSQL database, or a system monitor for observations (e.g. collectd), and of course, org-mode. In the end, I think I will stay with the many small files; it's much easier!
  

>   - In Python I have to parse a basic sub-set of Org-mode format
>     anyhow. An additional parser would be more work to do.

  Don't do it from scratch; there are already some parsers which work. I tried: https://github.com/bjonnh/PyOrgMode


> - Possible methods to store configuration/settings of a weblog system
>   that scans Org-mode files to generate HTML:
>   - in drawers: see below
>   - in tables: see below
>   - in tags: see below
>   - other possibilities?
> 
   From the ones you say, I prefer property drawers. It's the most DB-like and it's analogous to storing data (well, strings).
   You don't need all the table benefits (reordering, exporting, formatting, formulae, …).
   Nor the tags benefits (search, multiple tags, …)


> What do you think of this?
> Can you imagine a better way of storing key-value-pairs in Org-mode?
> 
> My focus is user friendly maintenance and overview including in-line
> documentation of the preferences.

  Of course, storing configuration in .org is very utopic (being all .org), but I would prefer *not* to do it. I would use a simple ~/.file.conf with some variables in the usual style:

# a comment
path=~/web/
# where to export images
images=~/web/images


  I think this wins for usability and „friendly maintenance“, since people know it and it works. And it allows you to define many projects (e.g. check the configuration file for the program unison).

  There are even libraries to parse these simple files: http://docs.python.org/2/library/configparser.html

  And if you want to design an equivalent grammar for configuration files (beware! that's another adventure!), check http://augeas.net/ . It can manipulate configuration files in a format-agnostic way.


  But I think it's more important to center efforts in developing a good exporter web publisher. As you said, the current ones are not powerful enough.

  Good luck.

Re: Using Org-mode file format for storing configuration data

[Karl Voit]

* Daniel Clemente <n142857@gmail.com> wrote:
> El Sun, 30 Dec 2012 19:04:25 +0100 Karl Voit va escriure:
>> 
>> I plan to implement a new weblog system that parses Org-mode
>> files and generates (static) HTML output. Yes, I am aware that
>> there are other solutions out there but I do not like them for
>> various reasons.[1]
>
>   Nice! I also don't like existing solutions and I was thinking on
>   writing some Python to do the export. But the complexities of
>   exporting are so well resolved in elisp that it's much easier to
>   invoke elisp code than to write your own in Python.

Sorry, I can not use the fabulous elisp because it would take me to
develop my blog SW for years - I'd have to learn elisp from scratch.
Surely, an elisp primer should not try to write something like a web
log as his first elisp project.

>> So for my new system, I am thinking of using Org-mode files for
>> writing (and parsing) the user-defined preferences.
>
>   I happened to be thinking the same two days ago.

>>   - In Python I have to parse a basic sub-set of Org-mode format
>>   anyhow. An additional parser would be more work to do.
>
>   Don't do it from scratch; there are already some parsers which
>   work. I tried: https://github.com/bjonnh/PyOrgMode

I did not start to evaluate current Org-parsers in Python but I am
very sceptic that I will be able to use such a parser. I plan to do
many "intelligent" transformations on the parser level such as
"id:heading42" ->
"http://my.bog.com/YYYY/MM/DD/articleofheading42.html" and so forth.

But we'll see.

>> - Possible methods to store configuration/settings of a weblog system
>>   that scans Org-mode files to generate HTML:
>>   - in drawers: see below
>>   - in tables: see below
>>   - in tags: see below
>>   - other possibilities?

>    From the ones you say, I prefer property drawers. It's the most
>    DB-like and it's analogous to storing data (well, strings).
>    You don't need all the table benefits (reordering, exporting,
>    formatting, formulae, …).  Nor the tags benefits (search,
>    multiple tags, …)

Ack.

>> My focus is user friendly maintenance and overview including
>> in-line documentation of the preferences.
>
>   Of course, storing configuration in .org is very utopic (being
>   all .org), but I would prefer *not* to do it. I would use a
>   simple ~/.file.conf with some variables in the usual style:

> # a comment
> path=~/web/
> # where to export images
> images=~/web/images

OK, the usual INI format. I've done this a couple of times already.

>   I think this wins for usability and „friendly maintenance“,
>   since people know it and it works. And it allows you to define
>   many projects (e.g. check the configuration file for the program
>   unison).

I wanted to check, whether there is something in the idea of using
the same Org format for configuration as well. So far I tend to use
INI once more.

>   But I think it's more important to center efforts in developing
>   a good exporter web publisher. As you said, the current ones are
>   not powerful enough.

Sorry, there seems to be a misunderstanding.

My exporter will be a *very basic one*. At least for the first
versions.  The issues that drove me to plan my own new blog system
are related to completely other areas.

If you want to have a "good Org to HTML engine", you definitely
don't want to use my future blogging system.

-- 
Karl Voit