Minimal overhead Org-mode blogging system

Minimal overhead Org-mode blogging system

[Karl Voit]

Hi!

I got a nice idea on how a very easy to use Org-mode blog system
should look like.

Currently, I am using Serendipidy with web-based editor to write
HTML. Org-mode enabled me to write blog entries and export it to
HTML. Then I paste the HTML and have to modify minor things (images,
...) a bit. I guess the time from finishing the Org-mode entry to
the final blog entry is approximately ten to twenty minutes.

Overall, I do not want to do this process when I just want to
quickly write a view paragraphs within a couple of minutes. I need
a workflow with much less annoying overhead.

Therefore I sat down and thought about a workflow that should be
enough for writing simple weblog entries:

  - create an Org-mode heading (anywhere!)
  - make sure that there is an (uniq) :ID: property
  - add the tag :blog: to heading
  - <write content, subheadings, ...>
  - change state of top-heading to DONE
    - this enables blog entries «in the queue»
  - (manually) invoke generation-script

This enables me quick blogging with a list of advantages:

  - a blog entry can be located anywhere in all of my Orgmode files
  - no extra formatting steps
  - very small (almost non-existent) overhead to create a blog entry
  - no duplicate information
    - updates only in Orgmode, not HTML or any in-between format
  - static (fast) pages
  - self-hosting without any fancy services behind like RDBS

What do you think of my ideas so far?


Of course, I looked into existing solutions and found those:

  - http://orgmode.org/worg/org-blog-wiki.html
    - cool overview page for various solutions
  - http://orgmode.org/worg/blorgit.html
    - pretty complex set up :-(
    - I do not need a web-interface to edit Org-mode files
    - seem to have «different» use cases
  - http://orgmode.org/worg/org-tutorials/org-jekyll.html
    - uses HTML as in-between format; seems to provide many error
      possibilities(?)
    - converting whole files only (not desired)
    - have to try it someday
  - http://emacs-fu.blogspot.com/2009/05/writing-and-blogging-with-org-mode.html
    - uses only HTML export
  - http://blog.herraiz.org/archives/241
    - uses only HTML export
  - https://github.com/chrismgray/ikiwiki-org-plugin
    - promising but only one part of a possible solution

So nothing offers the features and small footprint as my idea above
:-(


With some prerequisites, it should not be that hard to even
implement it by myself:

  - usage of only very basic markup
    - paragraphs (p)
    - headings (h1..n)
    - http-references (a href)
    - lists (ul)
    - images (img)
    - quote (verbatim)

Still there are some open issues:

  - comments
    - simplest form: generate unique Email link and add at bottom
      - very easy to be done for catch-all MTAs
      - automatically derive whitelist for MTA to avoid old spam
    - simple HTML form
      - POST to script, adding comment to my inbox.org (containing
        link to ID)
    - disqus: I do not want to outsource comment hosting :-(
  - how to include and format graphics?
    - sometimes, I e.g. want to have an image aligned right with text
      flowing around it
  - probably: usage of in-between format like ikiwiki
    - Orgmode syntax -> ikiwiki markup (markdown?) -> usual
      ikiwiki-workflow
      - should be not much effort since prerequisites limit to few
        markup things
    - benefits from not having to re-implement many things
    - in-between-format HTML (like Manoj uses) is way too complicated
      causing misc potential error sources

The basic script workflow is not that complicated:

  - find all headings with state DONE and tag :blog:
    - optionally: add all other tags starting with "blog-" as blog tags
  - one entry starts at such a heading until EOF OR same or less level
    heading is found
  - compare raw text and IDs with last run
    - known ID, raw text unchanged: ignore, no change
    - new ID
      - generate new blog entry
        - extract YYYY-MM-DD from LOGBOOK-drawer (first *->DONE transformation)
        - generate YYYY/MM/DD-folder structure in blog accordingly
        - generate sanitized blog title as file name
    - known ID, raw text differs
      - generate update of existing entry
        - add "(Update n)" (with n is the n-th update) to entry title
          - optionally: add this also to URL
            - disadvantage: broken old URLs
            - advantage: URL reflects update state
  - on any activity:
    - re-write RSS feed for last n entries
    - optionally: generate overview page for last n entries
    - optionally: generate calendar archive page(s)
    - optionally: generate tag overview page(s)


-- 
Karl Voit

Re: Minimal overhead Org-mode blogging system

[Steinar Bang]

There is also this approach:
 http://comments.gmane.org/gmane.emacs.orgmode/45360

I don't know if anything more has happened with this for the last 15
weeks, though.

Re: Minimal overhead Org-mode blogging system

[Puneeth Chaganti]

On 12/4/11, Karl Voit <devnull@karl-voit.at> wrote:
> Hi!
>
> I got a nice idea on how a very easy to use Org-mode blog system
> should look like.
>
> Currently, I am using Serendipidy with web-based editor to write
> HTML. Org-mode enabled me to write blog entries and export it to
> HTML. Then I paste the HTML and have to modify minor things (images,
> ...) a bit. I guess the time from finishing the Org-mode entry to
> the final blog entry is approximately ten to twenty minutes.
>
> Overall, I do not want to do this process when I just want to
> quickly write a view paragraphs within a couple of minutes. I need
> a workflow with much less annoying overhead.
>
> Therefore I sat down and thought about a workflow that should be
> enough for writing simple weblog entries:
>
>   - create an Org-mode heading (anywhere!)
>   - make sure that there is an (uniq) :ID: property
>   - add the tag :blog: to heading
>   - <write content, subheadings, ...>
>   - change state of top-heading to DONE
>     - this enables blog entries «in the queue»
>   - (manually) invoke generation-script
>
> This enables me quick blogging with a list of advantages:
>
>   - a blog entry can be located anywhere in all of my Orgmode files
>   - no extra formatting steps
>   - very small (almost non-existent) overhead to create a blog entry
>   - no duplicate information
>     - updates only in Orgmode, not HTML or any in-between format
>   - static (fast) pages
>   - self-hosting without any fancy services behind like RDBS

I have a system, that does most of what you are looking for.

https://github.com/punchagan/blog-files

Though it seems to be a little more complicated than it needs to be,
it works for me and I haven't had the time and motivation to simplify
it.

--
Puneeth

Re: Minimal overhead Org-mode blogging system

[Eric Schulte]

Puneeth Chaganti <punchagan@gmail.com> writes:

> On 12/4/11, Karl Voit <devnull@karl-voit.at> wrote:
>> Hi!
>>
>> I got a nice idea on how a very easy to use Org-mode blog system
>> should look like.
>>
>> Currently, I am using Serendipidy with web-based editor to write
>> HTML. Org-mode enabled me to write blog entries and export it to
>> HTML. Then I paste the HTML and have to modify minor things (images,
>> ...) a bit. I guess the time from finishing the Org-mode entry to
>> the final blog entry is approximately ten to twenty minutes.
>>
>> Overall, I do not want to do this process when I just want to
>> quickly write a view paragraphs within a couple of minutes. I need
>> a workflow with much less annoying overhead.
>>
>> Therefore I sat down and thought about a workflow that should be
>> enough for writing simple weblog entries:
>>
>>   - create an Org-mode heading (anywhere!)
>>   - make sure that there is an (uniq) :ID: property
>>   - add the tag :blog: to heading
>>   - <write content, subheadings, ...>
>>   - change state of top-heading to DONE
>>     - this enables blog entries «in the queue»
>>   - (manually) invoke generation-script
>>
>> This enables me quick blogging with a list of advantages:
>>
>>   - a blog entry can be located anywhere in all of my Orgmode files
>>   - no extra formatting steps
>>   - very small (almost non-existent) overhead to create a blog entry
>>   - no duplicate information
>>     - updates only in Orgmode, not HTML or any in-between format
>>   - static (fast) pages
>>   - self-hosting without any fancy services behind like RDBS
>
> I have a system, that does most of what you are looking for.
>
> https://github.com/punchagan/blog-files
>
> Though it seems to be a little more complicated than it needs to be,
> it works for me and I haven't had the time and motivation to simplify
> it.
>

It would be great to collect these links on the relevant Worg page [1].
As this topic seems to re-surface every couple of months.  Perhaps from
there we could begin to build consensus on what features would be most
desirable for any potential future Org-mode blogging extension.

Cheers,

Footnotes: 
[1]  http://orgmode.org/worg/org-blog-wiki.html

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Minimal overhead Org-mode blogging system

[Bastien]

Hi Eric,

Eric Schulte <eric.schulte@gmx.com> writes:

> It would be great to collect these links on the relevant Worg page
> [1].

Done.  Thanks all for the useful input!

-- 
 Bastien

Re: Minimal overhead Org-mode blogging system

[Karl Voit]

* Puneeth Chaganti <punchagan@gmail.com> wrote:
> On 12/4/11, Karl Voit <devnull@karl-voit.at> wrote:
>>
>> Therefore I sat down and thought about a workflow that should be
>> enough for writing simple weblog entries:
>>
>>   - create an Org-mode heading (anywhere!)
>>   - make sure that there is an (uniq) :ID: property
>>   - add the tag :blog: to heading
>>   - <write content, subheadings, ...>
>>   - change state of top-heading to DONE
>>     - this enables blog entries «in the queue»
>>   - (manually) invoke generation-script
>>
>> This enables me quick blogging with a list of advantages:
>>
>>   - a blog entry can be located anywhere in all of my Orgmode files
>>   - no extra formatting steps
>>   - very small (almost non-existent) overhead to create a blog entry
>>   - no duplicate information
>>     - updates only in Orgmode, not HTML or any in-between format
>>   - static (fast) pages
>>   - self-hosting without any fancy services behind like RDBS
>
> I have a system, that does most of what you are looking for.
>
> https://github.com/punchagan/blog-files

... I really do like your blog (and found several very interesting
entries *g*) but I (can not and) do not want to use Wordpress.

Quickly overlooking your system, so far I could not copy most of my
steps of my wishful workflow in your solution :-(

Maybe I find time to implement the system described by me (in Python
and probably using ikiwiki as in-between-format) next summer or so.

-- 
Karl Voit

Re: Minimal overhead Org-mode blogging system

[Puneeth Chaganti]

Hi Kurt,

On Thu, Dec 8, 2011 at 6:01 AM, Karl Voit <devnull@karl-voit.at> wrote:
> * Puneeth Chaganti <punchagan@gmail.com> wrote:
>> On 12/4/11, Karl Voit <devnull@karl-voit.at> wrote:
>>>
>>> Therefore I sat down and thought about a workflow that should be
>>> enough for writing simple weblog entries:
>>>
>>>   - create an Org-mode heading (anywhere!)
>>>   - make sure that there is an (uniq) :ID: property
>>>   - add the tag :blog: to heading
>>>   - <write content, subheadings, ...>
>>>   - change state of top-heading to DONE
>>>     - this enables blog entries «in the queue»
>>>   - (manually) invoke generation-script
>>>
>>> This enables me quick blogging with a list of advantages:
>>>
>>>   - a blog entry can be located anywhere in all of my Orgmode files
>>>   - no extra formatting steps
>>>   - very small (almost non-existent) overhead to create a blog entry
>>>   - no duplicate information
>>>     - updates only in Orgmode, not HTML or any in-between format
>>>   - static (fast) pages
>>>   - self-hosting without any fancy services behind like RDBS
>>
>> I have a system, that does most of what you are looking for.
>>
>> https://github.com/punchagan/blog-files
>
> ... I really do like your blog (and found several very interesting
> entries *g*) but I (can not and) do not want to use Wordpress.

I'm sorry that the repository doesn't have a README, but this solution
doesn't use Wordpress.  It basically uses the publishing mechanism of
org-mode and is based on ideas (and code) from org-jekyll and
reprise.py (https://github.com/uggedal/reprise).

-- Puneeth

Re: Minimal overhead Org-mode blogging system

[Karl Voit]

* Puneeth Chaganti <punchagan@gmail.com> wrote:
> Hi Kurt,

Almost :-)

> On Thu, Dec 8, 2011 at 6:01 AM, Karl Voit <devnull@karl-voit.at> wrote:
>> * Puneeth Chaganti <punchagan@gmail.com> wrote:
>>>
>>> https://github.com/punchagan/blog-files
>>
>> ... I really do like your blog (and found several very interesting
>> entries *g*) but I (can not and) do not want to use Wordpress.
>
> I'm sorry that the repository doesn't have a README, 

You have to add a README.org to the project - it's definitely worth
it :-)

I'd wish for a short description of:
* software requirements
* blog workflow
* limits

It's part of my letter to Santa Claus already *g*

> but this solution doesn't use Wordpress. 

Oh. Sorry, I read something about Wordpress in the workflow in one
of your blog posts. Must have been an older article then.

> It basically uses the publishing mechanism of org-mode and is
> based on ideas (and code) from org-jekyll and reprise.py
> (https://github.com/uggedal/reprise).

So I guess I have to try it by myself in order to estimate its value
to my requirements.

Thanks anyway!

-- 
Karl Voit

Re: Minimal overhead Org-mode blogging system

[Puneeth Chaganti]

On Thu, Dec 8, 2011 at 7:49 PM, Karl Voit <devnull@karl-voit.at> wrote:
> * Puneeth Chaganti <punchagan@gmail.com> wrote:
>> Hi Kurt,
>
> Almost :-)

Aarrgggh! Really sorry about that! My sleep deprivation showing it's effect. :-)

>
>> On Thu, Dec 8, 2011 at 6:01 AM, Karl Voit <devnull@karl-voit.at> wrote:
>>> * Puneeth Chaganti <punchagan@gmail.com> wrote:
>>>>
>>>> https://github.com/punchagan/blog-files
>>>
>>> ... I really do like your blog (and found several very interesting
>>> entries *g*) but I (can not and) do not want to use Wordpress.
>>
>> I'm sorry that the repository doesn't have a README,
>
> You have to add a README.org to the project - it's definitely worth
> it :-)
>
> I'd wish for a short description of:
> * software requirements
> * blog workflow
> * limits
>
> It's part of my letter to Santa Claus already *g*

I've added a short README that tries to include the things that you
are looking for.  But, it's something I wrote up quickly.  Feel free
to ask any specific questions.

Hope that helps,
Puneeth

Re: Minimal overhead Org-mode blogging system

[Karl Voit]

* Puneeth Chaganti <punchagan@gmail.com> wrote:
> On Thu, Dec 8, 2011 at 7:49 PM, Karl Voit <devnull@karl-voit.at> wrote:
>> * Puneeth Chaganti <punchagan@gmail.com> wrote:
>>> Hi Kurt,
>>
>> Almost :-)
>
> Aarrgggh! Really sorry about that! My sleep deprivation showing it's effect. :-)

*g* No worries. I by myself mix up names all the time.

>>>> * Puneeth Chaganti <punchagan@gmail.com> wrote:
>>>>>
>>>>> https://github.com/punchagan/blog-files
>>>>
>> You have to add a README.org to the project - it's definitely worth
>> it :-)
>
> I've added a short README that tries to include the things that you
> are looking for.  But, it's something I wrote up quickly.  Feel free
> to ask any specific questions.

Quickly reading your README.org already gives me *way* more ideas on
how your system works! Thank you *very* much for that. 

Now I know that I definitely have to set up a test installation of
your system since it sound really close to my imaginary optimum
solution.

-- 
Karl Voit

Re: Minimal overhead Org-mode blogging system

[Steinar Bang]

>>>>> Puneeth Chaganti <punchagan@gmail.com>:
[snip!]
>>>>> https://github.com/punchagan/blog-files

> I've added a short README that tries to include the things that you
> are looking for.  But, it's something I wrote up quickly.  Feel free
> to ask any specific questions.

I've been trying to use it, but I keep running into issues with
reprise.pay.

The latest one, is:

Traceback (most recent call last):
  File "reprise.py", line 258, in <module>
    env.get_template('cloud.html'))
  File "reprise.py", line 140, in generate_tag_cloud
    maxFreq = max(t['freq'] for t in tag_freq)
ValueError: max() arg is an empty sequence

The code for this, is:

def generate_tag_cloud(entries, template):
    tags = sum([e['tags'] for e in entries], [])
    tag_freq = [{'tag': tag, 'freq': tags.count(tag)} for tag in set(tags)
                if tags.count(tag) > 3]
    maxFreq = max(t['freq'] for t in tag_freq)
    ...

I've debugged this with pdb (`M-x pdb'), and tag_freq is empty.

If I'm interpreting the python code correctly (I don't know python...)
only those tags that have more than 3 occurrences will be in the
tag_freq collection...?

My problem is that I have only one article, my first, so obviously the
tag_freq collection will stay emtpy, and reprise.py will crash...

I tried removing the test, ie.

def generate_tag_cloud(entries, template):
    tags = sum([e['tags'] for e in entries], [])
    tag_freq = [{'tag': tag, 'freq': tags.count(tag)} for tag in set(tags)]
    maxFreq = max(t['freq'] for t in tag_freq)
    ...

but that just gave me a different error (division by zero):

  File "reprise.py", line 257, in <module>
    env.get_template('cloud.html'))
  File "reprise.py", line 147, in generate_tag_cloud
    'freq': t['freq']} for t in tag_freq]
  File "reprise.py", line 144, in normalize
    return min_r + (val - min_f) * (max_r - min_r) / float (max_f - min_f)

Any ideas of how to proceed?

Thanks!


- Steinar

Re: Minimal overhead Org-mode blogging system

[Puneeth Chaganti]

On Sun, Jan 15, 2012 at 12:08 PM, Steinar Bang <sb@dod.no> wrote:
>>>>>> Puneeth Chaganti <punchagan@gmail.com>:
> [snip!]
>>>>>> https://github.com/punchagan/blog-files
>
>> I've added a short README that tries to include the things that you
>> are looking for.  But, it's something I wrote up quickly.  Feel free
>> to ask any specific questions.
>
> I've been trying to use it, but I keep running into issues with
> reprise.pay.
>
> The latest one, is:
>
> Traceback (most recent call last):
>  File "reprise.py", line 258, in <module>
>    env.get_template('cloud.html'))
>  File "reprise.py", line 140, in generate_tag_cloud
>    maxFreq = max(t['freq'] for t in tag_freq)
> ValueError: max() arg is an empty sequence
>
> The code for this, is:
>
> def generate_tag_cloud(entries, template):
>    tags = sum([e['tags'] for e in entries], [])
>    tag_freq = [{'tag': tag, 'freq': tags.count(tag)} for tag in set(tags)
>                if tags.count(tag) > 3]
>    maxFreq = max(t['freq'] for t in tag_freq)
>    ...
>
> I've debugged this with pdb (`M-x pdb'), and tag_freq is empty.
>
> If I'm interpreting the python code correctly (I don't know python...)
> only those tags that have more than 3 occurrences will be in the
> tag_freq collection...?
>
> My problem is that I have only one article, my first, so obviously the
> tag_freq collection will stay emtpy, and reprise.py will crash...
>
> I tried removing the test, ie.
>
> def generate_tag_cloud(entries, template):
>    tags = sum([e['tags'] for e in entries], [])
>    tag_freq = [{'tag': tag, 'freq': tags.count(tag)} for tag in set(tags)]
>    maxFreq = max(t['freq'] for t in tag_freq)
>    ...
>
> but that just gave me a different error (division by zero):
>
>  File "reprise.py", line 257, in <module>
>    env.get_template('cloud.html'))
>  File "reprise.py", line 147, in generate_tag_cloud
>    'freq': t['freq']} for t in tag_freq]
>  File "reprise.py", line 144, in normalize
>    return min_r + (val - min_f) * (max_r - min_r) / float (max_f - min_f)
>
> Any ideas of how to proceed?

This is totally home brew stuff.  Sorry for the trouble you are
having, in getting it to run.

Try commenting out the lines 257, 258 in reprise.py and see if that
works for you.  I'll take a look later, and try to see if this can be
improved.

Thanks,
Puneeth

Re: Minimal overhead Org-mode blogging system

[Steinar Bang]

>>>>> Puneeth Chaganti <punchagan@gmail.com>:

> This is totally home brew stuff.  

Well, thank you for sharing this home brew stuff.
I've been trying to use the other org based blog solutions, but they
have all failed for some reason or other, and they have also seemed
abandoned. 

> Sorry for the trouble you are having, in getting it to run.

No worries! :-)

> Try commenting out the lines 257, 258 in reprise.py and see if that
> works for you.  I'll take a look later, and try to see if this can be
> improved.

Yep, commenting out the tag cloud generation made the reprise.py script
run witout failing.

Thanks!

Re: Minimal overhead Org-mode blogging system

[Chris Gray]

On Thu, 19 Jan 2012 23:15:16 +0100, Steinar Bang <sb@dod.no> wrote:
> >>>>> Puneeth Chaganti <punchagan@gmail.com>:
> 
> > This is totally home brew stuff.  
> 
> Well, thank you for sharing this home brew stuff.
> I've been trying to use the other org based blog solutions, but they
> have all failed for some reason or other, and they have also seemed
> abandoned.

Have another look at ikiwiki.  Either plugin for org works (I am the
author of one of them), and the nice thing is that it is a really
complete system.  It has plugins [1] for just about anything you could
want to do with a blog or wiki, and is generally well-engineered.

If you have any trouble with my plugin, please open an issue on github.
I promise it's not abandoned. :)

Cheers,
Chris

[1] http://ikiwiki.info/plugins/

Re: Minimal overhead Org-mode blogging system

[Scott Randby]

On 01/20/2012 01:19 PM, Chris Gray wrote:
> On Thu, 19 Jan 2012 23:15:16 +0100, Steinar Bang <sb@dod.no> wrote:
>>>>>>> Puneeth Chaganti <punchagan@gmail.com>:
>>
>>> This is totally home brew stuff.  
>>
>> Well, thank you for sharing this home brew stuff.
>> I've been trying to use the other org based blog solutions, but they
>> have all failed for some reason or other, and they have also seemed
>> abandoned.
> 
> Have another look at ikiwiki.  Either plugin for org works (I am the
> author of one of them), and the nice thing is that it is a really
> complete system.  It has plugins [1] for just about anything you could
> want to do with a blog or wiki, and is generally well-engineered.
> 
> If you have any trouble with my plugin, please open an issue on github.
> I promise it's not abandoned. :)
> 
> Cheers,
> Chris
> 
> [1] http://ikiwiki.info/plugins/

Unless this plugin does not have "org" in its name, then it cannot be
found on this page. What plugin are you talking about?

Scott Randby

> 
> 

Re: Minimal overhead Org-mode blogging system

[Nick Dokos]

Scott Randby <srandby@gmail.com> wrote:

> On 01/20/2012 01:19 PM, Chris Gray wrote:
> > On Thu, 19 Jan 2012 23:15:16 +0100, Steinar Bang <sb@dod.no> wrote:
> >>>>>>> Puneeth Chaganti <punchagan@gmail.com>:
> >>
> >>> This is totally home brew stuff.  
> >>
> >> Well, thank you for sharing this home brew stuff.
> >> I've been trying to use the other org based blog solutions, but they
> >> have all failed for some reason or other, and they have also seemed
> >> abandoned.
> > 
> > Have another look at ikiwiki.  Either plugin for org works (I am the
> > author of one of them), and the nice thing is that it is a really
> > complete system.  It has plugins [1] for just about anything you could
> > want to do with a blog or wiki, and is generally well-engineered.
> > 
> > If you have any trouble with my plugin, please open an issue on github.
> > I promise it's not abandoned. :)
> > 
> > Cheers,
> > Chris
> > 
> > [1] http://ikiwiki.info/plugins/
> 
> Unless this plugin does not have "org" in its name, then it cannot be
> found on this page. What plugin are you talking about?
> 
> Scott Randby
> 

Chris's plugin is on github. You can clone it with 

   git clone git://github.com/chrismgray/ikiwiki-org-plugin.git


Nick


	

Re: Minimal overhead Org-mode blogging system

[Chris Gray]

On Sat, 21 Jan 2012 00:53:33 -0500, Nick Dokos <nicholas.dokos@hp.com> wrote:
> Scott Randby <srandby@gmail.com> wrote:
> 
> > On 01/20/2012 01:19 PM, Chris Gray wrote:
> > > On Thu, 19 Jan 2012 23:15:16 +0100, Steinar Bang <sb@dod.no> wrote:
> > >>>>>>> Puneeth Chaganti <punchagan@gmail.com>:
> > >>
> > >>> This is totally home brew stuff.  
> > >>
> > >> Well, thank you for sharing this home brew stuff.
> > >> I've been trying to use the other org based blog solutions, but they
> > >> have all failed for some reason or other, and they have also seemed
> > >> abandoned.
> > > 
> > > Have another look at ikiwiki.  Either plugin for org works (I am the
> > > author of one of them), and the nice thing is that it is a really
> > > complete system.  It has plugins [1] for just about anything you could
> > > want to do with a blog or wiki, and is generally well-engineered.
> > > 
> > > If you have any trouble with my plugin, please open an issue on github.
> > > I promise it's not abandoned. :)
> > > 
> > > Cheers,
> > > Chris
> > > 
> > > [1] http://ikiwiki.info/plugins/
> > 
> > Unless this plugin does not have "org" in its name, then it cannot be
> > found on this page. What plugin are you talking about?
> > 
> > Scott Randby
> > 
> 
> Chris's plugin is on github. You can clone it with 
> 
>    git clone git://github.com/chrismgray/ikiwiki-org-plugin.git
> 

Thanks, I should have been more clear.  Both the plugins are documented
here: http://ikiwiki.info/todo/org_mode/

Cheers,
Chris

Re: Minimal overhead Org-mode blogging system

[tychoish]

[-- Attachment #1: Type: text/plain, Size: 1447 bytes --]

On Wed, Dec 07, 2011 at 10:50:52PM +0530, Puneeth Chaganti wrote:
> I have a system, that does most of what you are looking for.
>
> https://github.com/punchagan/blog-files
>
> Though it seems to be a little more complicated than it needs to be,
> it works for me and I haven't had the time and motivation to simplify
> it.


This is a commentary on the entire thread rather than on this specific
suggestion (though it's applicable here.)

All of these "take a git repo with text files in a lightweight markup
language (e.g. markdown, org, rst, etc.) and build a blog/website" tools
have this major flaw and there's no good solution:

They rebuild all pages in the site every time you update the site. Which
doesn't matter at all when you have 10 posts, but when you have a
hundred posts you notice the rebuild process, and by the time you have
1000-1500 posts, its totally unusable. Every time you fix a comma it
takes 1-3 minutes and nearly OOMs a VPS system to fix.

So what's the solution?

- Incremental builds
- Cached build elements.
- make-style dependency checking.
- indexes (for tags, archives, etc.) that are

The truth is that the part of the pipe that handles the filtering of the
text is important, but is not particularly central or crucial in the
grand scheme of the usability of this kind of application.

Cheers,
sam

--
tycho(ish) @
garen@tychoish.com
http://tychoish.com/
"don't get it right, get it written" -- james thurber

[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]

Re: Minimal overhead Org-mode blogging system

[Eric Schulte]

tychoish <garen@tychoish.com> writes:

> On Wed, Dec 07, 2011 at 10:50:52PM +0530, Puneeth Chaganti wrote:
>> I have a system, that does most of what you are looking for.
>>
>> https://github.com/punchagan/blog-files
>>
>> Though it seems to be a little more complicated than it needs to be,
>> it works for me and I haven't had the time and motivation to simplify
>> it.
>
>
> This is a commentary on the entire thread rather than on this specific
> suggestion (though it's applicable here.)
>
> All of these "take a git repo with text files in a lightweight markup
> language (e.g. markdown, org, rst, etc.) and build a blog/website" tools
> have this major flaw and there's no good solution:
>
> They rebuild all pages in the site every time you update the site. Which
> doesn't matter at all when you have 10 posts, but when you have a
> hundred posts you notice the rebuild process, and by the time you have
> 1000-1500 posts, its totally unusable. Every time you fix a comma it
> takes 1-3 minutes and nearly OOMs a VPS system to fix.
>
> So what's the solution?
>
> - Incremental builds
> - Cached build elements.
> - make-style dependency checking.
> - indexes (for tags, archives, etc.) that are
>
> The truth is that the part of the pipe that handles the filtering of the
> text is important, but is not particularly central or crucial in the
> grand scheme of the usability of this kind of application.
>
> Cheers,
> sam
>

Note that regular Org-mode projects [1] do *not* re-publish every single
page after an update, but rather only publish pages which have changed
since the previous publish.  Thus a git repository with a pos-update
hook which runs `org-publish' in a batch Emacs session does a good job
of publishing updates without having to re-publish the entire site.

This is the approach taken for my lab's wiki [2], which is just a git
repository [3] with a post-update hook and a couple of helper scripts
[4] which re-publish updated pages after every commit.

Best,

Footnotes: 
[1]  http://orgmode.org/manual/Publishing.html

[2]  http://wiki.adaptive.cs.unm.edu

[3]  http://gitweb.adaptive.cs.unm.edu/wiki.git

[4]  http://gitweb.adaptive.cs.unm.edu/wiki.git/tree/HEAD:/data

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Minimal overhead Org-mode blogging system

[Bastien]

Eric Schulte <eric.schulte@gmx.com> writes:

> Note that regular Org-mode projects [1] do *not* re-publish every single
> page after an update, but rather only publish pages which have changed
> since the previous publish.  Thus a git repository with a pos-update
> hook which runs `org-publish' in a batch Emacs session does a good job
> of publishing updates without having to re-publish the entire site.
>
> This is the approach taken for my lab's wiki [2], which is just a git
> repository [3] with a post-update hook and a couple of helper scripts
> [4] which re-publish updated pages after every commit.

FWIW, this is also the way Worg is published.

-- 
 Bastien

Re: Minimal overhead Org-mode blogging system

[François Pinard]

tychoish <garen@tychoish.com> writes:

> On Wed, Dec 07, 2011 at 10:50:52PM +0530, Puneeth Chaganti wrote:

> [...] and nearly OOMs a VPS system to fix.

It took a bit of doing to get a meaning out of this sentence! :-)
Remember to be kind to your non-English audience...

François

Re: Minimal overhead Org-mode blogging system

[Nathan Neff]

On Sun, Dec 4, 2011 at 10:51 AM, Karl Voit <devnull@karl-voit.at> wrote:
> Hi!
>
> I got a nice idea on how a very easy to use Org-mode blog system
> should look like.
>
> Currently, I am using Serendipidy with web-based editor to write
> HTML. Org-mode enabled me to write blog entries and export it to
> HTML. Then I paste the HTML and have to modify minor things (images,
> ...) a bit. I guess the time from finishing the Org-mode entry to
> the final blog entry is approximately ten to twenty minutes.
>
> Overall, I do not want to do this process when I just want to
> quickly write a view paragraphs within a couple of minutes. I need
> a workflow with much less annoying overhead.

Hello,

I agree with you about the "Much less annoying overhead".

I love the org-mode format, but I do not like the existing solutions
that others are using to blog with org-mode.  The main reasons is that
it seems like a lot of overhead to create the look & feel of the site
and how blog posts are shown, etc.

So, I've been using Jekyll, and have found it to be okay, once you have it
set up the way you want.  Now that I have it set up the way I want, I want to
just use org-mode formatted files, instead of markdown.

After some googling, I found this project, which looks promising:
https://github.com/abhiyerra/jekyll

It's just a small fork of Jekyll, that uses org-ruby to find *.org
files in your project
and convert it to HTML.

The main advantages that I can see are:

  - Edit my blog posts in Emacs or Vim, using Org-mode format, which I
like better than Markdown.
  - Jekyll has an outstanding feature set, including auto-preview.  So
I can just run jekyll --watch (or something like that) in my blog's
directory and it will automatically detect changes and re-render the
HTML.
  - Jekyll has lots of meta-data tags (I think they're called YAML
front matter and liquid templates). and I'll make sure that org-ruby
can parse these tags as well.

I will see if it works for me, and then report back to the mailing list :-)

--Nate

Re: Minimal overhead Org-mode blogging system

[Neil Smithline]

Karl Voit <devnull <at> Karl-Voit.at> writes: 
> Therefore I sat down and thought about a workflow that should be
> enough for writing simple weblog entries:
> 
>   - create an Org-mode heading (anywhere!)
>   - make sure that there is an (uniq) :ID: property
>   - add the tag :blog: to heading
>   - <write content, subheadings, ...>
>   - change state of top-heading to DONE
>     - this enables blog entries «in the queue»
>   - (manually) invoke generation-script
> 
> This enables me quick blogging with a list of advantages:
> 
>   - a blog entry can be located anywhere in all of my Orgmode files
>   - no extra formatting steps
>   - very small (almost non-existent) overhead to create a blog entry
>   - no duplicate information
>     - updates only in Orgmode, not HTML or any in-between format
>   - static (fast) pages
>   - self-hosting without any fancy services behind like RDBS
> Karl,

I'm wondering if you've played around with this at all? I happen to really like
the idea but I wonder about its performance.

Unless I'm mistaken, and I very likely may be, won't you have to scan all of
your .org files to look for the special tags/properties/todo states/whatever? 

If not, I'd love to have a pointer to how you can accomplish this without
scanning every .org file. That would be cool. 

Re: Minimal overhead Org-mode blogging system

[Jude DaShiell]

Understand, I use update here in the sense of some file modification 
that subsequently gets saved.  If files to be modified get archived into 
org-mode's revision control system, the blog tag and associated done tag 
could be searched for within the save process and an org database could 
build with file name and then tripplets of date stamp, line number for 
blog tag, line number for done tag and each tripplet would hold another 
blog entry in that unique file which is the first field in the data 
base.  So you want to find a blog entry?  Search the org-generated data 
base for a date stamp and you come up with the file and the range of 
line numbers holding that blog entry.  Search one file and go to 
specific location in second file.  That if it's done or gets done will 
keep file searching to a nice minimum permanently.

On Sun, 13 May 2012, Neil Smithline wrote:

> 
> Karl Voit <devnull <at> Karl-Voit.at> writes: 
> > Therefore I sat down and thought about a workflow that should be
> > enough for writing simple weblog entries:
> > 
> >   - create an Org-mode heading (anywhere!)
> >   - make sure that there is an (uniq) :ID: property
> >   - add the tag :blog: to heading
> >   - <write content, subheadings, ...>
> >   - change state of top-heading to DONE
> >     - this enables blog entries ?in the queue?
> >   - (manually) invoke generation-script
> > 
> > This enables me quick blogging with a list of advantages:
> > 
> >   - a blog entry can be located anywhere in all of my Orgmode files
> >   - no extra formatting steps
> >   - very small (almost non-existent) overhead to create a blog entry
> >   - no duplicate information
> >     - updates only in Orgmode, not HTML or any in-between format
> >   - static (fast) pages
> >   - self-hosting without any fancy services behind like RDBS
> > Karl,
> 
> I'm wondering if you've played around with this at all? I happen to really like
> the idea but I wonder about its performance.
> 
> Unless I'm mistaken, and I very likely may be, won't you have to scan all of
> your .org files to look for the special tags/properties/todo states/whatever? 
> 
> If not, I'd love to have a pointer to how you can accomplish this without
> scanning every .org file. That would be cool. 
> 
> 
> 

----------------------------------------------------------------
Jude <jdashiel-at-shellworld-dot-net>
<http://www.shellworld.net/~jdashiel/nj.html>

Re: Minimal overhead Org-mode blogging system

[Neil Smithline]

I like your indexing idea. I use a less-complex system involving 
symbolic links for my agenda files. Yours sounds better.

This is what I use for my agendas:

(setq org-agenda-files
   (list (expand-file-name "~/Documents/+OrgAgendas")))

(defun org-add-agenda-file ()
   (interactive)
   (make-symbolic-link (buffer-file-name) "~/Documents/+OrgAgendas"))

It is just a quick-and-dirty solution. If I remove or move a file, I get 
errors. Also, if I stop using a file for agenda items I must manually 
unlink the symlink.

Have you implemented your indexing system Jude or just designed it? I'd 
love to see it if you have something working. I imagine it could be used 
for todos, cross-referencing tags, properties, etc...

And to prevent Carsten from yelling at me :-D, I would insist that, by 
default, Emacs would not create the cross-referencing database. You'd 
have to explicitly enable it.

Neil

On Mon May 14 22:24:08 2012, Jude DaShiell wrote:
> Understand, I use update here in the sense of some file modification
> that subsequently gets saved.  If files to be modified get archived into
> org-mode's revision control system, the blog tag and associated done tag
> could be searched for within the save process and an org database could
> build with file name and then tripplets of date stamp, line number for
> blog tag, line number for done tag and each tripplet would hold another
> blog entry in that unique file which is the first field in the data
> base.  So you want to find a blog entry?  Search the org-generated data
> base for a date stamp and you come up with the file and the range of
> line numbers holding that blog entry.  Search one file and go to
> specific location in second file.  That if it's done or gets done will
> keep file searching to a nice minimum permanently.
>
> On Sun, 13 May 2012, Neil Smithline wrote:
>
>>
>> Karl Voit <devnull <at> Karl-Voit.at> writes:
>>> Therefore I sat down and thought about a workflow that should be
>>> enough for writing simple weblog entries:
>>>
>>>    - create an Org-mode heading (anywhere!)
>>>    - make sure that there is an (uniq) :ID: property
>>>    - add the tag :blog: to heading
>>>    - <write content, subheadings, ...>
>>>    - change state of top-heading to DONE
>>>      - this enables blog entries ?in the queue?
>>>    - (manually) invoke generation-script
>>>
>>> This enables me quick blogging with a list of advantages:
>>>
>>>    - a blog entry can be located anywhere in all of my Orgmode files
>>>    - no extra formatting steps
>>>    - very small (almost non-existent) overhead to create a blog entry
>>>    - no duplicate information
>>>      - updates only in Orgmode, not HTML or any in-between format
>>>    - static (fast) pages
>>>    - self-hosting without any fancy services behind like RDBS
>>> Karl,
>>
>> I'm wondering if you've played around with this at all? I happen to really like
>> the idea but I wonder about its performance.
>>
>> Unless I'm mistaken, and I very likely may be, won't you have to scan all of
>> your .org files to look for the special tags/properties/todo states/whatever?
>>
>> If not, I'd love to have a pointer to how you can accomplish this without
>> scanning every .org file. That would be cool.
>>
>>
>>
>
> ----------------------------------------------------------------
> Jude <jdashiel-at-shellworld-dot-net>
> <http://www.shellworld.net/~jdashiel/nj.html>
>

Re: Minimal overhead Org-mode blogging system

[Jude DaShiell]

I don't know enough lisp to implement this indexing system.  On Tue, 15 
May 2012, Neil Smithline wrote:

> I like your indexing idea. I use a less-complex system involving symbolic
> links for my agenda files. Yours sounds better.
> 
> This is what I use for my agendas:
> 
> (setq org-agenda-files
>   (list (expand-file-name "~/Documents/+OrgAgendas")))
> 
> (defun org-add-agenda-file ()
>   (interactive)
>   (make-symbolic-link (buffer-file-name) "~/Documents/+OrgAgendas"))
> 
> It is just a quick-and-dirty solution. If I remove or move a file, I get
> errors. Also, if I stop using a file for agenda items I must manually unlink
> the symlink.
> 
> Have you implemented your indexing system Jude or just designed it? I'd love
> to see it if you have something working. I imagine it could be used for todos,
> cross-referencing tags, properties, etc...
> 
> And to prevent Carsten from yelling at me :-D, I would insist that, by
> default, Emacs would not create the cross-referencing database. You'd have to
> explicitly enable it.
> 
> Neil
> 
> On Mon May 14 22:24:08 2012, Jude DaShiell wrote:
> > Understand, I use update here in the sense of some file modification
> > that subsequently gets saved.  If files to be modified get archived into
> > org-mode's revision control system, the blog tag and associated done tag
> > could be searched for within the save process and an org database could
> > build with file name and then tripplets of date stamp, line number for
> > blog tag, line number for done tag and each tripplet would hold another
> > blog entry in that unique file which is the first field in the data
> > base.  So you want to find a blog entry?  Search the org-generated data
> > base for a date stamp and you come up with the file and the range of
> > line numbers holding that blog entry.  Search one file and go to
> > specific location in second file.  That if it's done or gets done will
> > keep file searching to a nice minimum permanently.
> >
> > On Sun, 13 May 2012, Neil Smithline wrote:
> >
> > >
> > > Karl Voit <devnull <at> Karl-Voit.at> writes:
> > > > Therefore I sat down and thought about a workflow that should be
> > > > enough for writing simple weblog entries:
> > > >
> > > >    - create an Org-mode heading (anywhere!)
> > > >    - make sure that there is an (uniq) :ID: property
> > > >    - add the tag :blog: to heading
> > > >    - <write content, subheadings, ...>
> > > >    - change state of top-heading to DONE
> > > >      - this enables blog entries ?in the queue?
> > > >    - (manually) invoke generation-script
> > > >
> > > > This enables me quick blogging with a list of advantages:
> > > >
> > > >    - a blog entry can be located anywhere in all of my Orgmode files
> > > >    - no extra formatting steps
> > > >    - very small (almost non-existent) overhead to create a blog entry
> > > >    - no duplicate information
> > > >      - updates only in Orgmode, not HTML or any in-between format
> > > >    - static (fast) pages
> > > >    - self-hosting without any fancy services behind like RDBS
> > > > Karl,
> > >
> > > I'm wondering if you've played around with this at all? I happen to really
> > > like
> > > the idea but I wonder about its performance.
> > >
> > > Unless I'm mistaken, and I very likely may be, won't you have to scan all
> > > of
> > > your .org files to look for the special tags/properties/todo
> > > states/whatever?
> > >
> > > If not, I'd love to have a pointer to how you can accomplish this without
> > > scanning every .org file. That would be cool.
> > >
> > >
> > >
> >
> > ----------------------------------------------------------------
> > Jude <jdashiel-at-shellworld-dot-net>
> > <http://www.shellworld.net/~jdashiel/nj.html>
> >
> 
> 

----------------------------------------------------------------
Jude <jdashiel-at-shellworld-dot-net>
<http://www.shellworld.net/~jdashiel/nj.html>

Re: Minimal overhead Org-mode blogging system

[Neil Smithline]

First, the only way to learn is to do. That said, I do understand that 
RL gets in the way of fun.

Regarding the implementation, if you skip the index system, which is 
definitely an elegant solution, and modify my symbolic link solution 
for agendas to keep track of files with blog entries. The symlink 
solution is only a pseudo index and is not automated, but it is trivial 
to use.

Just replace every use of the word "agenda" in my code to "blog" and 
then bind `org-add-blog-file' to some convenient key sequence. Then use 
selective exporting ([[info:elisp#Char%20Classes][info:elisp#Char 
Classes]]) when publishing your blog. You would assign 
`org-export-select-tags' to a tag like :blog:.

Then you just need to define a custom export function to change the 
value of `org-export-select-tags'. I just cobbled it together (that's 
secret code for "haven't tested") the custom export function at 
https://gist.github.com/2759661. I think that should get you most of 
the way.

Neil Smithline
http://www.neilsmithline.com
Proud GNU Emacs user since 1986, v. 18.24.

On Wed May 16 05:51:33 2012, Jude DaShiell wrote:
> I don't know enough lisp to implement this indexing system.  On Tue, 15
> May 2012, Neil Smithline wrote:
>
>> I like your indexing idea. I use a less-complex system involving symbolic
>> links for my agenda files. Yours sounds better.
>>
>> This is what I use for my agendas:
>>
>> (setq org-agenda-files
>>    (list (expand-file-name "~/Documents/+OrgAgendas")))
>>
>> (defun org-add-agenda-file ()
>>    (interactive)
>>    (make-symbolic-link (buffer-file-name) "~/Documents/+OrgAgendas"))
>>
>> It is just a quick-and-dirty solution. If I remove or move a file, I get
>> errors. Also, if I stop using a file for agenda items I must manually unlink
>> the symlink.
>>
>> Have you implemented your indexing system Jude or just designed it? I'd love
>> to see it if you have something working. I imagine it could be used for todos,
>> cross-referencing tags, properties, etc...
>>
>> And to prevent Carsten from yelling at me :-D, I would insist that, by
>> default, Emacs would not create the cross-referencing database. You'd have to
>> explicitly enable it.
>>
>> Neil
>>
>> On Mon May 14 22:24:08 2012, Jude DaShiell wrote:
>>> Understand, I use update here in the sense of some file modification
>>> that subsequently gets saved.  If files to be modified get archived into
>>> org-mode's revision control system, the blog tag and associated done tag
>>> could be searched for within the save process and an org database could
>>> build with file name and then tripplets of date stamp, line number for
>>> blog tag, line number for done tag and each tripplet would hold another
>>> blog entry in that unique file which is the first field in the data
>>> base.  So you want to find a blog entry?  Search the org-generated data
>>> base for a date stamp and you come up with the file and the range of
>>> line numbers holding that blog entry.  Search one file and go to
>>> specific location in second file.  That if it's done or gets done will
>>> keep file searching to a nice minimum permanently.
>>>
>>> On Sun, 13 May 2012, Neil Smithline wrote:
>>>
>>>>
>>>> Karl Voit <devnull <at> Karl-Voit.at> writes:
>>>>> Therefore I sat down and thought about a workflow that should be
>>>>> enough for writing simple weblog entries:
>>>>>
>>>>>     - create an Org-mode heading (anywhere!)
>>>>>     - make sure that there is an (uniq) :ID: property
>>>>>     - add the tag :blog: to heading
>>>>>     - <write content, subheadings, ...>
>>>>>     - change state of top-heading to DONE
>>>>>       - this enables blog entries ?in the queue?
>>>>>     - (manually) invoke generation-script
>>>>>
>>>>> This enables me quick blogging with a list of advantages:
>>>>>
>>>>>     - a blog entry can be located anywhere in all of my Orgmode files
>>>>>     - no extra formatting steps
>>>>>     - very small (almost non-existent) overhead to create a blog entry
>>>>>     - no duplicate information
>>>>>       - updates only in Orgmode, not HTML or any in-between format
>>>>>     - static (fast) pages
>>>>>     - self-hosting without any fancy services behind like RDBS
>>>>> Karl,
>>>>
>>>> I'm wondering if you've played around with this at all? I happen to really
>>>> like
>>>> the idea but I wonder about its performance.
>>>>
>>>> Unless I'm mistaken, and I very likely may be, won't you have to scan all
>>>> of
>>>> your .org files to look for the special tags/properties/todo
>>>> states/whatever?
>>>>
>>>> If not, I'd love to have a pointer to how you can accomplish this without
>>>> scanning every .org file. That would be cool.
>>>>
>>>>
>>>>
>>>
>>> ----------------------------------------------------------------
>>> Jude <jdashiel-at-shellworld-dot-net>
>>> <http://www.shellworld.net/~jdashiel/nj.html>
>>>
>>
>>
>
> ----------------------------------------------------------------
> Jude <jdashiel-at-shellworld-dot-net>
> <http://www.shellworld.net/~jdashiel/nj.html>
>

Re: Minimal overhead Org-mode blogging system

[Karl Voit]

Sorry for the late reply. I was off the mailing list for one and a
half month. Therefore I include a full quote.

* Neil Smithline <emacs-orgmode@neilsmithline.com> wrote:
>
> Karl Voit <devnull <at> Karl-Voit.at> writes: 
>> Therefore I sat down and thought about a workflow that should be
>> enough for writing simple weblog entries:
>> 
>>   - create an Org-mode heading (anywhere!)
>>   - make sure that there is an (uniq) :ID: property
>>   - add the tag :blog: to heading
>>   - <write content, subheadings, ...>
>>   - change state of top-heading to DONE
>>     - this enables blog entries «in the queue»
>>   - (manually) invoke generation-script
>> 
>> This enables me quick blogging with a list of advantages:
>> 
>>   - a blog entry can be located anywhere in all of my Orgmode files
>>   - no extra formatting steps
>>   - very small (almost non-existent) overhead to create a blog entry
>>   - no duplicate information
>>     - updates only in Orgmode, not HTML or any in-between format
>>   - static (fast) pages
>>   - self-hosting without any fancy services behind like RDBS
>> Karl,
>
> I'm wondering if you've played around with this at all? I happen to really like
> the idea but I wonder about its performance.

I am writing my PhD thesis these days and therefore I do not have
time for this blogging thing until November or January :-(

Performance is no issue at all on *my* side. I plan to re-generate
the whole blog every night using a cron job.

But I do understand your concern when you need something which is
updaten instantly. (Not my personal requirement.)

> Unless I'm mistaken, and I very likely may be, won't you have to scan all of
> your .org files to look for the special tags/properties/todo states/whatever? 

Yes. I plan to scan each (or a defined subset) of my Org-mode files.

> If not, I'd love to have a pointer to how you can accomplish this without
> scanning every .org file. That would be cool. 

Sorry.

But anyhow: whatever I hack, it will be on github [1] and everybody
is free to fork and add a better performing real-time update
mechanism. I also have a cool name for my project - be curious *g*

  1. https://github.com/novoid/
-- 
Karl Voit