Is it possible to keep /all/ the heading properties in one place?

Is it possible to keep /all/ the heading properties in one place?

[Oleh Krehel]

Hi all,

I've recently gotten back to reorganizing my Org setup. There are a few
things I find lacking, one of which is having all the auxiliary
information for a heading stored in a single place, that I can further
make less noticeable with `prettify-symbols-mode'.

    (add-to-list 'prettify-symbols-alist
                   '(":PROPERTIES:" . ":"))
    (prettify-symbols-mode)

Here's a list of the diverse things that I'd like to hide into
:PROPERTIES:

- Tags - there are always a pain to re-align, they show up as diffs in
  git commits
- The clock entries - they are stored in a :LOGBOOK:, separately from
  :PROPERTIES:
- The :SCHEDULED: and :DEADLINE: time stamps
- I also have the "Added: ..." line from a recommended capture setup, I
  think it would be better placed in :PROPERTIES:

Is it possible to set up what I described above? Any pointers? Ideas to
do it better?

thanks,
Oleh

Re: Is it possible to keep /all/ the heading properties in one place?

[Nicolas Goaziou]

Hello,

Oleh Krehel <ohwoeowho@gmail.com> writes:

> I've recently gotten back to reorganizing my Org setup. There are a few
> things I find lacking, one of which is having all the auxiliary
> information for a heading stored in a single place, that I can further
> make less noticeable with `prettify-symbols-mode'.
>
>     (add-to-list 'prettify-symbols-alist
>                    '(":PROPERTIES:" . ":"))
>     (prettify-symbols-mode)
>
> Here's a list of the diverse things that I'd like to hide into
> :PROPERTIES:
>
> - Tags - there are always a pain to re-align, they show up as diffs in
>   git commits
> - The clock entries - they are stored in a :LOGBOOK:, separately from
>   :PROPERTIES:
> - The :SCHEDULED: and :DEADLINE: time stamps

It is not possible to move these things into the properties drawer.

> - I also have the "Added: ..." line from a recommended capture setup, I
>   think it would be better placed in :PROPERTIES:

Not sure what it is, so I cannot tell.

Regards,

-- 
Nicolas Goaziou

Re: Is it possible to keep /all/ the heading properties in one place?

[Oleh Krehel]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

>> Here's a list of the diverse things that I'd like to hide into
>> :PROPERTIES:
>>
>> - Tags - there are always a pain to re-align, they show up as diffs in
>>   git commits
>> - The clock entries - they are stored in a :LOGBOOK:, separately from
>>   :PROPERTIES:
>> - The :SCHEDULED: and :DEADLINE: time stamps
>
> It is not possible to move these things into the properties drawer.

Why not? What's the barrier to doing this? I think it would not only
make the text files cleaner, but also the org.el code.

Re: Is it possible to keep /all/ the heading properties in one place?

[Nicolas Goaziou]

Oleh Krehel <ohwoeowho@gmail.com> writes:

> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>
>>> Here's a list of the diverse things that I'd like to hide into
>>> :PROPERTIES:
>>>
>>> - Tags - there are always a pain to re-align, they show up as diffs in
>>>   git commits
>>> - The clock entries - they are stored in a :LOGBOOK:, separately from
>>>   :PROPERTIES:
>>> - The :SCHEDULED: and :DEADLINE: time stamps
>>
>> It is not possible to move these things into the properties drawer.
>
> Why not? What's the barrier to doing this? I think it would not only
> make the text files cleaner, but also the org.el code.

Because not everything is a node property.

TODO keywords, tags and properties are all different, and blurring the
distinction between them would not make Org easier. Note that Org
doesn't force you to use any of them.

CLOCK cannot be located within PROPERTIES drawer because it not
a key-value association. You can have multiple clocks with different
values.

SCHEDULED and DEADLINE could have been moved within PROPERTIES drawer.
It was even discussed a couple of times on this ML. However, Carsten
decided to keep them separated, mainly because such an important
information should not be hidden in the document, in particular for
newcomers. I still agree with him.


Regards,

-- 
Nicolas

Re: Is it possible to keep /all/ the heading properties in one place?

[Oleh Krehel]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Because not everything is a node property.

This shouldn't prevent us from keeping things that /are/ node properties
all in one place.

> TODO keywords, tags and properties are all different, and blurring the
> distinction between them would not make Org easier. Note that Org
> doesn't force you to use any of them.

I would most definitely make Org easier in some respects. Suppose a
person wants to parse an Org file's content, just for displaying it on a
website (like Github does right now). If all properties were in a single
place, they could be trivially skipped with a regex, resulting in an
almost ASCII-like export of the Org file.

> CLOCK cannot be located within PROPERTIES drawer because it not
> a key-value association. You can have multiple clocks with different
> values.

:LOGBOOK: could be the key, and all of its contents would be the value.
Same thing with putting :TAGS: into :PROPERTIES:.

> SCHEDULED and DEADLINE could have been moved within PROPERTIES drawer.
> It was even discussed a couple of times on this ML. However, Carsten
> decided to keep them separated, mainly because such an important
> information should not be hidden in the document, in particular for
> newcomers. I still agree with him.

Could we have an option to customize this? Just declare a standard
getter and setter interface for :SCHEDULED: and :DEADLINE:. I'll
customize them in my config to put them in the :PROPERTIES: drawer.

Here's another idea for the :PROPERTIES: drawer that might make things a
lot less hairy - make it fully in Lisp:

    (properties
     (scheduled [2016-02-25 Thu])
     (id ca23d969-d189-4d38-aee3-aa21feb5b305)
     (logbook
      (clock [2016-02-25 Thu 15:03])
      (clock [2016-02-25 Thu 14:33] [2016-02-25 Thu 14:58])
      (clock [2016-02-25 Thu 13:24] [2016-02-25 Thu 13:49]))
     (added [2016-02-25 Thu 11:24]))

I think it would greatly enhance the parsing of Org files, and simplify
many functions in org.el. With this, a simple `read' will suffice to
parse all the stuff.

Re: Is it possible to keep /all/ the heading properties in one place?

[Nicolas Goaziou]

Oleh Krehel <ohwoeowho@gmail.com> writes:

> I would most definitely make Org easier in some respects. Suppose a
> person wants to parse an Org file's content, just for displaying it on a
> website (like Github does right now). If all properties were in a single
> place, they could be trivially skipped with a regex, resulting in an
> almost ASCII-like export of the Org file.

(Parsing - regexp - trivially: pick only two)

There already is a parser, and a syntax definition.

>> CLOCK cannot be located within PROPERTIES drawer because it not
>> a key-value association. You can have multiple clocks with different
>> values.
>
> :LOGBOOK: could be the key, and all of its contents would be the value.
> Same thing with putting :TAGS: into :PROPERTIES:.

Contents of :LOGBOOK: can be almost anything. You need a parser
understanding the whole Org syntax to parse it. I don't see how it would
make anything easier. Also, you cannot parse Org syntax only using
regexps.

>> SCHEDULED and DEADLINE could have been moved within PROPERTIES drawer.
>> It was even discussed a couple of times on this ML. However, Carsten
>> decided to keep them separated, mainly because such an important
>> information should not be hidden in the document, in particular for
>> newcomers. I still agree with him.
>
> Could we have an option to customize this? Just declare a standard
> getter and setter interface for :SCHEDULED: and :DEADLINE:. I'll
> customize them in my config to put them in the :PROPERTIES: drawer.

I'm against making syntax customizable. I'd rather have a clear
definition of a single syntax anyone could implement.

However, I agree we could allow to customize how meta-data is
/displayed/. Since planning lines and properties drawers are required to
be next to each other, it is easy to make them disappear under the same
overlay. See, for example, `org-end-of-meta-data'.

> Here's another idea for the :PROPERTIES: drawer that might make things a
> lot less hairy - make it fully in Lisp:
>
>     (properties
>      (scheduled [2016-02-25 Thu])
>      (id ca23d969-d189-4d38-aee3-aa21feb5b305)
>      (logbook
>       (clock [2016-02-25 Thu 15:03])
>       (clock [2016-02-25 Thu 14:33] [2016-02-25 Thu 14:58])
>       (clock [2016-02-25 Thu 13:24] [2016-02-25 Thu 13:49]))
>      (added [2016-02-25 Thu 11:24]))
>
> I think it would greatly enhance the parsing of Org files, and simplify
> many functions in org.el. With this, a simple `read' will suffice to
> parse all the stuff.

I do not feel like asking users to write directly the AST for their
plain text documents, really. Parsing an Org document is a solved
problem. I do not pretend the solution cannot be improved, but at least,
it is complete.

I'm not sure about your motivations. If they are about reducing visual
clutter, you can work it out at the display level. I'm pretty sure this
improvement would be appreciated.

OTOH, if they are about making it easier to implement external parsers,
I think you should try first to implement the current syntax. It is
quite documented and not really hard to grok.


Regards,

-- 
Nicolas

Re: Is it possible to keep /all/ the heading properties in one place?

[Eric S Fraga]

On Thursday, 25 Feb 2016 at 17:52, Nicolas Goaziou wrote:
> If they are about reducing visual clutter, you can work it out at the
> display level. I'm pretty sure this improvement would be appreciated.

And the OP's use of prettify-symbols-mode brings a number of
possibilities to mind!  Can anybody suggest some pretty unicode symbols
suitable for SCHEDULED and DEADLINE?  :-)

Oh, and I'm with you on being clear about the distinction between
parsing and displaying.  Very different use cases and hence different
requirements.

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.0.91.1, Org release_8.3.3-619-gca1fb8

Re: Is it possible to keep /all/ the heading properties in one place?

[Rasmus]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> On Thursday, 25 Feb 2016 at 17:52, Nicolas Goaziou wrote:
>> If they are about reducing visual clutter, you can work it out at the
>> display level. I'm pretty sure this improvement would be appreciated.
>
> And the OP's use of prettify-symbols-mode brings a number of
> possibilities to mind!  Can anybody suggest some pretty unicode symbols
> suitable for
> SCHEDULED

A clock: 🕗.  You could even try to be fancy and choose the right one based
on the scheduled time (don’t know if regexp are supported by p-s-m).

Maybe an unchecked box.

A board holding a piece of paper.

> and DEADLINE?  :-)

💀!

something with a flag?

BLACK FLAG (⚑) CHEQUERED FLAG (🏁) WAVING BLACK FLAG (🏴) WAVING WHITE FLAG
(🏳) WHITE FLAG (⚐) WHITE FLAG WITH HORIZONTAL MIDDLE BLACK STRIPE (⛿)


Probably Evolution or Kmail uses some symbols that could be interesting.


Rasmus

-- 
Don't slow down Johnny, leave the Cadillac runnin'

Re: Is it possible to keep /all/ the heading properties in one place?

[Eric S Fraga]

Rasmus,

thanks for the suggestions.  However, your email has highlighted the
fact that the font(s) I use do not have proper coverage of unicode
characters!  :-(  I wonder which font you use?  I typically use DejaVu
and Terminus but also Lucida.  None of these can display most of the
characters you suggested unfortunately.

eric
-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.0.91.1, Org release_8.3.3-619-gca1fb8

Re: Is it possible to keep /all/ the heading properties in one place?

[Rasmus]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> Rasmus,
>
> thanks for the suggestions.  However, your email has highlighted the
> fact that the font(s) I use do not have proper coverage of unicode
> characters!  :-(  I wonder which font you use?  I typically use DejaVu
> and Terminus but also Lucida.  None of these can display most of the
> characters you suggested unfortunately.

Emacs should pick the right font.  Most likely, you need to install
Symbola,

Nombre                    : ttf-symbola
Descripción               : Font for unicode symbols (part of Unicode Fonts for
                            Ancient Scripts).
URL                       : http://users.teilar.gr/~g1951d/


The only fonts I’ve got configured are Fira as the main font, DejaVu as
fallback and XITS for math.

Hope it helps,
Rasmus

-- 
And I faced endless streams of vendor-approved Ikea furniture. . .

Re: Is it possible to keep /all/ the heading properties in one place?

[Eric S Fraga]

On Sunday, 28 Feb 2016 at 12:32, Rasmus wrote:
> Emacs should pick the right font.  Most likely, you need to install
> Symbola,

Indeed!  Thanks for the pointer.  Installing symbola did the job in
terms of being able to see those symbols.

Unfortunately, I cannot get org to prettify scheduled and deadline
entries without causing org to have problems fontifying the rest of the
file.  But I'm not sure why that is happening so will post later when/if
I narrow this down.

> The only fonts I’ve got configured are Fira as the main font, DejaVu as
> fallback and XITS for math.

I know this is OT for this list but can you show me how you configure
fallback fonts?

Thanks,
eric
-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.0.91.1, Org release_8.3.4-626-gb62d55

Re: Is it possible to keep /all/ the heading properties in one place?

[Rasmus]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> On Sunday, 28 Feb 2016 at 12:32, Rasmus wrote:
>> Emacs should pick the right font.  Most likely, you need to install
>> Symbola,
>
> Indeed!  Thanks for the pointer.  Installing symbola did the job in
> terms of being able to see those symbols.
>
> Unfortunately, I cannot get org to prettify scheduled and deadline
> entries without causing org to have problems fontifying the rest of the
> file.  But I'm not sure why that is happening so will post later when/if
> I narrow this down.

Did you try prettify-symbol-mode?  In at least the master version of Emacs
you should be able to use prettify-symbols-alist.  Though org will do some
fortification already, which might cause the error you are seeing.

>> The only fonts I’ve got configured are Fira as the main font, DejaVu as
>> fallback and XITS for math.
>
> I know this is OT for this list but can you show me how you configure
> fallback fonts?

Fallback might be too strong, but you could use the prepend argument to
set-fontset-font.  Here’s what I used to use.  In Emacs-25 it seems I
don’t really mess with this anymore, though.

(mapc (lambda (x)
          (set-fontset-font x 'mathematical
                            (font-spec :family "XITS Math") nil 'append)
          (set-fontset-font x 'symbol
                            (font-spec :family "DejaVu Sans Mono") nil 'prepend)
          (set-fontset-font x 'greek
                            (font-spec :family "Fira Mono") nil 'prepend)
          (set-fontset-font x '(#x1f601 . #x1f567) "Symbola"))
        '("fontset-startup" "fontset-default"))

Rasmus

-- 
Got mashed potatoes. Ain't got no T-Bone. No T-Bone

Re: Is it possible to keep /all/ the heading properties in one place?

[Eric S Fraga]

On Sunday, 28 Feb 2016 at 17:46, Rasmus wrote:
> Did you try prettify-symbol-mode?

That's exactly what I am using and tried:

#+begin_src emacs-lisp
  (defun esf/setup-org-prettify-symbols ()
    (setq prettify-symbols-alist '(("\\to" . →)
                                   ("SCHEDULED:". 🕗)
                                   ("DEADLINE:" . 🏁)
                                   )))
  (add-hook 'org-mode-hook 'esf/setup-org-prettify-symbols)
#+end_src

The \to is there for testing and this bit seems to work without
problems.  It's the two new lines that cause difficulties.

> Though org will do some fortification already, which might cause the
> error you are seeing.

There's some clash somewhere because enabling the above causes
fontification to fail partway through my org file.

> Fallback might be too strong, but you could use the prepend argument to
> set-fontset-font.  Here’s what I used to use.  In Emacs-25 it seems I
> don’t really mess with this anymore, though.

Thanks for this.  But it does seem like it's not necessary in emacs
25.  I only use pre-25 emacs on my Nokia but I don't need this kind of
font support there so I'll leave things be.

Thanks again,
eric
-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.0.91.1, Org release_8.3.4-626-gb62d55

Re: Is it possible to keep /all/ the heading properties in one place?

[Oleh Krehel]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> I do not feel like asking users to write directly the AST for their
> plain text documents, really.

It's not an AST though. It's simply nested lists. Like JSON or XML but
better. And the idea is to both have it automatic and manual. For
example, `org-set-property' would work exactly as it does right now -
interactively. But on the programming level it would use `read',
`delete-sexp', `plist-put' and `prin1'. Isn't it much better to defer
all the heavy lifting to the Elisp reader?

Additionally, LISP readers are readily available outside of Emacs.  It
would ease other projects' Org-mode integration. Like mobile apps,
taskwarrior, github, whatever - the people would be able to parse and
modify Org with simply:

    import lisp_reader

instead of grokking the full Org property syntax and all if its oddities
and idiosyncrasies. Because the basic Org heading structure is genius
simple. It's all the extra "stuff" than drags it down in terms of
simplicity.

> Parsing an Org document is a solved problem. I do not pretend the
> solution cannot be improved, but at least, it is complete.

Just like reading LISP is a solved problem. And not just in Emacs.

> I'm not sure about your motivations. If they are about reducing visual
> clutter, you can work it out at the display level. I'm pretty sure this
> improvement would be appreciated.

The motivation is to have Org look simpler by virtue of /being/ simpler.
Compare (require 'org-element) and hours of grokking it and looking up
docs to simply:

    (defun all-props ()
      (save-excursion
        (goto-char (point-min))
        (let (props)
          (while (re-search-forward "^(properties" nil t)
            (goto-char (match-beginning 0))
            (push (read (current-buffer)) props))
          (nreverse props))))
    (mapcar (lambda (p)
              (assoc 'deadline (cdr p)))
            (all-props))
    ;; =>
    ;; ((deadline "<2016-02-26 Fri 17:00 +1w>") nil)

Re: Is it possible to keep /all/ the heading properties in one place?

[Nicolas Goaziou]

Oleh Krehel <ohwoeowho@gmail.com> writes:

> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

>> I do not feel like asking users to write directly the AST for their
>> plain text documents, really.
>
> It's not an AST though. It's simply nested lists.

So is Lisp.

> Like JSON or XML but better. And the idea is to both have it automatic
> and manual. For example, `org-set-property' would work exactly as it
> does right now - interactively. But on the programming level it would
> use `read', `delete-sexp', `plist-put' and `prin1'. Isn't it much
> better to defer all the heavy lifting to the Elisp reader?

No it isn't, IMO.

It boils down to ask users to write Lisp by hand at some point. Not
everyone wants to use interactive tools. Unfortunately, writing Lisp is
not fun in a basic text editing environment.

> Additionally, LISP readers are readily available outside of Emacs.  It
> would ease other projects' Org-mode integration. Like mobile apps,
> taskwarrior, github, whatever - the people would be able to parse and
> modify Org with simply:
>
>     import lisp_reader
>
> instead of grokking the full Org property syntax

Here is the full Org property syntax:

--8<---------------cut here---------------start------------->8---
3.7 Property Drawers
────────────────────

  Property drawers are a special type of drawer containing properties
  attached to a headline.  They are located right after a [headline] and
  its [planning] information.

  ┌────
  │ HEADLINE
  │ PROPERTYDRAWER
  │ 
  │ HEADLINE
  │ PLANNING
  │ PROPERTYDRAWER
  └────

  PROPERTYDRAWER follows the pattern

  ┌────
  │ :PROPERTIES:
  │ CONTENTS
  │ :END:
  └────

  where CONTENTS consists of zero or more [node properties].


4.9 Node Properties
───────────────────

  Node properties can only exist in [property drawers].  Their pattern
  is any of the following

  ┌────
  │ :NAME: VALUE
  │ 
  │ :NAME+: VALUE
  │ 
  │ :NAME:
  │ 
  │ :NAME+:
  └────

  NAME can contain any non-whitespace character but cannot end with
  a plus sign.  It cannot be the empty string.

  VALUE can contain anything but a newline character.

--8<---------------cut here---------------end--------------->8---

I don't think this is very impressive nor particularly difficult to
implement.

> and all if its oddities and idiosyncrasies. Because the basic Org
> heading structure is genius simple. It's all the extra "stuff" than
> drags it down in terms of simplicity.

You may want to have a look at "outline.el", which is Org without all
the extra "stuff".

> The motivation is to have Org look simpler by virtue of /being/ simpler.
> Compare (require 'org-element) and hours of grokking it and looking up
> docs to simply:

You don't have to grok "org-element.el" which is but an implementation
of Org syntax. See <http://orgmode.org/worg/dev/org-syntax.html>
instead.

>     (defun all-props ()
>       (save-excursion
>         (goto-char (point-min))
>         (let (props)
>           (while (re-search-forward "^(properties" nil t)
>             (goto-char (match-beginning 0))
>             (push (read (current-buffer)) props))
>           (nreverse props))))
>     (mapcar (lambda (p)
>               (assoc 'deadline (cdr p)))
>             (all-props))
>     ;; =>
>     ;; ((deadline "<2016-02-26 Fri 17:00 +1w>") nil)

Alas, the Devil is in the detail:

  (example
   ...
   (properties ...))

Anyway, at this point I don't know what to add. You want to improve Org
and this is fine. However simplistic examples do not help understanding
the full picture, at least for me. So, implement your idea, test the
syntax, ask for feedback here. In the end, if it happens to be superior,
users will naturally switch to it, for the benefit of everyone.

You may also want to have a look at Skribilo
(http://www.nongnu.org/skribilo/), if you don't know it already.


Regards,

-- 
Nicolas

Re: Is it possible to keep /all/ the heading properties in one place?

[Oleh Krehel]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> So is Lisp.

I like Lisp.

> It boils down to ask users to write Lisp by hand at some point. Not
> everyone wants to use interactive tools.

I claim that a user can get by without having to write a single line of
Lisp manually. M-x `customize' has been doing it successfully for years.

> Unfortunately, writing Lisp is not fun in a basic text editing
> environment.

Right, not in a basic one. But it's great in Emacs. And for others, they
could use `read' followed by a graphical tool like a TreeView widget,
followed by `prin1'. This Lisp is supposed to be so simple that a
`read'->`delete-sexp'->`prin1' is an `identity' operation.

> Here is the full Org property syntax:
>
> 3.7 Property Drawers
> ────────────────────
>
>   Property drawers are a special type of drawer containing properties
>   attached to a headline.  They are located right after a [headline] and
>   its [planning] information.
>
>   ┌────
>   │ HEADLINE
>   │ PROPERTYDRAWER
>   │ 
>   │ HEADLINE
>   │ PLANNING
>   │ PROPERTYDRAWER
>   └────
>
>   PROPERTYDRAWER follows the pattern
>
>   ┌────
>   │ :PROPERTIES:
>   │ CONTENTS
>   │ :END:
>   └────

Missed out the logging drawer and the tags. That's two things more out
of a single place than I'd like.

> I don't think this is very impressive nor particularly difficult to
> implement.

I get a headache just trying to imagine how I could hide the properties
/and/ the logging /and/ the tags into a single fold-able thing.

> You may want to have a look at "outline.el", which is Org without all
> the extra "stuff".

Already have been using it for years. I actually prefer outline.el to
org-babel for most things now. But not for GTD - for that I need Org.

> Alas, the Devil is in the detail:
>
>   (example
>    ...
>    (properties ...))

Not really. You see, the `beginning-of-defun' trick is: a defun is a
thing that starts in the first column (that's why we escape all parens
in the first column in Elisp). This trick has been working successfully
for decades. It's both fast and simple.

Since all Lisp properties are written to programmatically by Org, they
are always perfectly indented. And if a person wants to edit it
manually, he'll have to simply remember to `indent-sexp'.

> Anyway, at this point I don't know what to add.

Suggestion, and I've said it before, Org needs a standard simple inline
kbd markup. Just like Markdown's <kbd></kbd> and Texinfo's @kbd{}.
Since Org is tied into Emacs having an easy (and unambiguous) way to
denote keyboard shortcuts would be very convenient.

Adding that would also progress the direction of having Org be the
language for Emacs manuals. Right now, Org's own manual is in Texinfo,
which is a shame since no one likes Texinfo and few people understand
enough of it to get by.

> You want to improve Org and this is fine. However simplistic examples
> do not help understanding the full picture, at least for me. So,
> implement your idea, test the syntax, ask for feedback here. In the
> end, if it happens to be superior, users will naturally switch to it,
> for the benefit of everyone.

Thanks, I'll be trying that out. In the last week I've been archiving
most of my gtd.org. So the switch should be easier now.

Do we have a way in Org to archive a heading form anywhere into a memoir
format like this:

    * 2014
    ** 2014-01 January
    *** 2014-01-03 Thursday
    **** CANCELLED Foobar
    CLOSED: [2014-01-03 Fri 09:42] SCHEDULED: <2013-12-25 Wed>
    Added: [2013-08-08 Thu 17:38]
    **** DONE Baz
    CLOSED: [2014-01-03 Fri 17:12]

Each heading is archived into level 4, on the day it was closed.  I had
it working a few years ago, relying on org-archive.el. But when I tried
this week, it stopped working, so I had to rewrite it.

> You may also want to have a look at Skribilo
> (http://www.nongnu.org/skribilo/), if you don't know it already.

Thanks, I didn't know about it. It looks very similar to Racket's
Scribble, and my own ElTeX.

Re: Is it possible to keep /all/ the heading properties in one place?

[Nicolas Goaziou]

Hello,

Oleh Krehel <ohwoeowho@gmail.com> writes:

> Already have been using it for years. I actually prefer outline.el to
> org-babel for most things now. But not for GTD - for that I need Org.
>
>> Alas, the Devil is in the detail:
>>
>>   (example
>>    ...
>>    (properties ...))
>
> Not really. You see, the `beginning-of-defun' trick is: a defun is a
> thing that starts in the first column (that's why we escape all parens
> in the first column in Elisp). This trick has been working successfully
> for decades. It's both fast and simple.

It reminds me one mistake I made a couple years ago, when I changed
"key=value" to ":key value" in export attributes, for consistency with
Babel parameters.

When looking at

  #+ATTR_LATEX: :key1 value1 :key2 value2

I thought it would be easy to parse it using `read', and get
a ready-to-use plist.

Unfortunately, users were required to tell the difference between
a symbol and a string (e.g., :key #value is invalid, so you need to
write :key "#value"), and to know the characters to quote in the strings
they would like to use.

Some users complained, and told me that "Org is not a programming
language". Fair enough. Org has, indeed, to strike a balance between
structure, required for efficiency and maintainability, and sloppiness,
needed when you want to quickly jot down notes in a document, or simply
let your ideas flow as you type, without hindrance.

In my experience, I think your suggestion would just repeat the very
same mistake. Yet, again, I may be wrong, so I guess you will have to
try and take the requirements for Org into consideration.

> Suggestion, and I've said it before, Org needs a standard simple inline
> kbd markup. Just like Markdown's <kbd></kbd> and Texinfo's @kbd{}.
> Since Org is tied into Emacs having an easy (and unambiguous) way to
> denote keyboard shortcuts would be very convenient.

This is export back-end specific. Org has no use for a "kbd" markup.

> Adding that would also progress the direction of having Org be the
> language for Emacs manuals. Right now, Org's own manual is in Texinfo,
> which is a shame since no one likes Texinfo and few people understand
> enough of it to get by.

Org can export to Texinfo. It can be helpful for those not understanding
well enough the latter.

> Do we have a way in Org to archive a heading form anywhere into a memoir
> format like this:
>
>     * 2014
>     ** 2014-01 January
>     *** 2014-01-03 Thursday
>     **** CANCELLED Foobar
>     CLOSED: [2014-01-03 Fri 09:42] SCHEDULED: <2013-12-25 Wed>
>     Added: [2013-08-08 Thu 17:38]
>     **** DONE Baz
>     CLOSED: [2014-01-03 Fri 17:12]
>
> Each heading is archived into level 4, on the day it was closed.  I had
> it working a few years ago, relying on org-archive.el. But when I tried
> this week, it stopped working, so I had to rewrite it.

I suggest to start a bug report, if possible with an ECM demonstrating
the issue, in another thread.

> Thanks, I didn't know about it. It looks very similar to Racket's
> Scribble, and my own ElTeX.

Indeed, and they are all focused towards exporting. Export is but one
part of Org.


Regards,

-- 
Nicolas Goaziou

Re: Is it possible to keep /all/ the heading properties in one place?

[Oleh Krehel]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Some users complained, and told me that "Org is not a programming
> language". Fair enough. Org has, indeed, to strike a balance between
> structure, required for efficiency and maintainability, and sloppiness,
> needed when you want to quickly jot down notes in a document, or simply
> let your ideas flow as you type, without hindrance.

Simply the headline structure is enough for all of this. The rest, like
properties, tags and logging can be made more rigorous.  For instance,
the current tags implementation implies the illusion that they can be
manipulated by hand. However, in practice it's frustrating to align
them.

>> Suggestion, and I've said it before, Org needs a standard simple inline
>> kbd markup. Just like Markdown's <kbd></kbd> and Texinfo's @kbd{}.
>> Since Org is tied into Emacs having an easy (and unambiguous) way to
>> denote keyboard shortcuts would be very convenient.
>
> This is export back-end specific. Org has no use for a "kbd" markup.

I really disagree here. The very first thing I did used Org for was
jotting down the Emacs key bindings that I wanted to remember.  Not
being able to distinguish the key bindings from symbols removes from the
markup's expressiveness.

For example, some people write like this:

    ~M-x forward-char~ or ~C-f~

while I write like this:

    ~M-x~ =forward-char= or ~C-f~

As long as only I read my own documents, my notation isn't a
problem. But if I share it with someone else, they will have to get used
to my non-standard notation.


>> Adding that would also progress the direction of having Org be the
>> language for Emacs manuals. Right now, Org's own manual is in Texinfo,
>> which is a shame since no one likes Texinfo and few people understand
>> enough of it to get by.

> Org can export to Texinfo. It can be helpful for those not understanding
> well enough the latter.

Org can export to Texinfo. But not well enough to be able to write Org
manual in Org.

That was my point. To be able to write a manual as rich as any existing
one, with only Org's built-in markup, i.e. without a tonne of export
specific #+TEXINFO or #+HTML includes.

I made some progress towards this goal with some custom code for
ivy.org -> ivy.html + ivy.texi export:
https://github.com/abo-abo/swiper/blob/master/doc/ivy-ox.el.

I made use of my own kbd markup that I wrote once in Org, but exported
the same both to Texinfo and HTML (see e.g. http://oremacs.com/swiper/#key-bindings).

regards,
Oleh

Re: Is it possible to keep /all/ the heading properties in one place?

[Thomas S. Dye]

Aloha Oleh,

Oleh Krehel writes:

>
>> Org can export to Texinfo. It can be helpful for those not understanding
>> well enough the latter.
>
> Org can export to Texinfo. But not well enough to be able to write Org
> manual in Org.

I worked on a project to port the Org manual to Org about three years
ago.  At the time, I thought the Texinfo exporter supported the project
quite well.

The project had a mixed reception and ultimately wasn't adopted, but I
don't remember Texinfo exporter deficiencies figuring into the decision.
My impression at the time was that the programmers who looked at the
project thought the Texinfo exporter was doing its job well.

hth,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Is it possible to keep /all/ the heading properties in one place?

[Nicolas Goaziou]

Hello,

Thomas S. Dye <tsd@tsdye.com> writes:

> I worked on a project to port the Org manual to Org about three years
> ago.  At the time, I thought the Texinfo exporter supported the project
> quite well.
>
> The project had a mixed reception and ultimately wasn't adopted, but I
> don't remember Texinfo exporter deficiencies figuring into the
> decision.

AFAIR, the problem is that Emacs developers fix spelling issues right
into the ".texi" document, so we would need to backport each of their
changes into the Org source. Duplicated efforts.

However, I still think it would be nice to have the document somewhere
in the Org tree, perhaps in contrib/ as a start, then possibly one day
in doc/.

As a side note, another interesting project would be an "info" export
back-end. Currently, producing an info file is very slow because we go
through the "texinfo" format first.


Regards,

-- 
Nicolas Goaziou

Re: Is it possible to keep /all/ the heading properties in one place?

[Thomas S. Dye]

Nicolas Goaziou writes:

> However, I still think it would be nice to have the document somewhere
> in the Org tree, perhaps in contrib/ as a start, then possibly one day
> in doc/.

Would it serve a purpose?

All the best,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Is it possible to keep /all/ the heading properties in one place?

[Nicolas Goaziou]

Hello,

Thomas S. Dye <tsd@tsdye.com> writes:

> Nicolas Goaziou writes:
>
>> However, I still think it would be nice to have the document somewhere
>> in the Org tree, perhaps in contrib/ as a start, then possibly one day
>> in doc/.
>
> Would it serve a purpose?

Mainly for demonstration. It would also give us some feedback on the
texinfo back-end.

Regards,

--
Nicolas

Re: Is it possible to keep /all/ the heading properties in one place?

[Marcin Borkowski]

On 2016-02-29, at 11:05, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:

> Hello,
>
> Thomas S. Dye <tsd@tsdye.com> writes:
>
>> I worked on a project to port the Org manual to Org about three years
>> ago.  At the time, I thought the Texinfo exporter supported the project
>> quite well.
>>
>> The project had a mixed reception and ultimately wasn't adopted, but I
>> don't remember Texinfo exporter deficiencies figuring into the
>> decision.
>
> AFAIR, the problem is that Emacs developers fix spelling issues right
> into the ".texi" document, so we would need to backport each of their
> changes into the Org source. Duplicated efforts.

I know next to nothing about texinfo, but would a converter from texinfo
to Org be possible?  (Or from info to Org, for that matter?)

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: Is it possible to keep /all/ the heading properties in one place?

[Nicolas Goaziou]

Hello,

Marcin Borkowski <mbork@mbork.pl> writes:

> I know next to nothing about texinfo, but would a converter from texinfo
> to Org be possible?  (Or from info to Org, for that matter?)

Possible. 

You need a texinfo parser. Also, since Texinfo and Org do not provide
exactly the same features, there are some design choices involved at
some point.

Anyway, a parser is not enough. `libxml-parse-html-region' is included
in Emacs, but we have no converter from HTML to Org yet.

Regards,

-- 
Nicolas Goaziou

Revisiting moving manual to Org (was: Is it possible to keep /all/ the heading properties in one place?)

[Kyle Meyer]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Thomas S. Dye <tsd@tsdye.com> writes:
>
>> I worked on a project to port the Org manual to Org about three years
>> ago.  At the time, I thought the Texinfo exporter supported the project
>> quite well.
>>
>> The project had a mixed reception and ultimately wasn't adopted, but I
>> don't remember Texinfo exporter deficiencies figuring into the
>> decision.
>
> AFAIR, the problem is that Emacs developers fix spelling issues right
> into the ".texi" document, so we would need to backport each of their
> changes into the Org source. Duplicated efforts.

It is duplicated effort, but it's not too much beyond a normal backport.
If this is the only issue with moving to Org for the manual, I'd be
happy to take care of it when I backport changes.

--
Kyle

Re: Is it possible to keep /all/ the heading properties in one place?

[Achim Gratz]

Thomas S. Dye writes:
> The project had a mixed reception and ultimately wasn't adopted, but I
> don't remember Texinfo exporter deficiencies figuring into the
> decision.

Well, maybe except that the texinfo exporter is even slower than
texinfo5.  I haven't tried it in the last few months, but I expect that
the makefile integration still wokrs, even though the manual is now out
of date… nope, there's an error now:

--8<---------------cut here---------------start------------->8---
Invalid search bound (wrong side of point)
Makefile:39: recipe for target 'orgmanual.html' failed
--8<---------------cut here---------------end--------------->8---

So it needs some love and at least one bugfix.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: Is it possible to keep /all/ the heading properties in one place?

[Thomas S. Dye]

Aloha Achim,

Achim Gratz writes:

> Thomas S. Dye writes:
>> The project had a mixed reception and ultimately wasn't adopted, but I
>> don't remember Texinfo exporter deficiencies figuring into the
>> decision.
>
> Well, maybe except that the texinfo exporter is even slower than
> texinfo5.  I haven't tried it in the last few months, but I expect that
> the makefile integration still wokrs, even though the manual is now out
> of date… nope, there's an error now:
>
> --8<---------------cut here---------------start------------->8---
> Invalid search bound (wrong side of point)
> Makefile:39: recipe for target 'orgmanual.html' failed
> --8<---------------cut here---------------end--------------->8---
>
> So it needs some love and at least one bugfix.

Sorry to hear that.  At the time, I enjoyed working with the texinfo
exporter and didn't mind that it worked slowly.

I guess Oleh Krehel's estimation of the texinfo exporter was right.

Thanks for checking.

All the best,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Is it possible to keep /all/ the heading properties in one place?

[Nicolas Goaziou]

Hello,

Thomas S. Dye <tsd@tsdye.com> writes:

>> So it needs some love and at least one bugfix.
>
> Sorry to hear that.  At the time, I enjoyed working with the texinfo
> exporter and didn't mind that it worked slowly.

The bug is not related to Texinfo back-end. It was fixed a few days ago.

There are a couple things to fix however,

- #+comment: foo should simply be # foo
- texinfo back-end provides #+CINDEX:, #+VINDEX: ... keywords, so you
  don't need to create macros for that.
- #+SUBTITLE: is parsed, so @value{VERSION} should be
  @@info:@value{VERSION}@@.
- The back-end already takes care of adding @documentencoding
- Mistakes reported by Org Lint should be fixed (in particular, commas
  within macros should be escaped with a single \).

Once done, the document properly exports.

Regards,

-- 
Nicolas Goaziou

Re: Is it possible to keep /all/ the heading properties in one place?

[Michael Brand]

Hi Oleh

On Thu, Feb 25, 2016 at 11:59 AM, Oleh Krehel <ohwoeowho@gmail.com> wrote:

> - Tags - there are always a pain to re-align, they show up as diffs in
>   git commits

Did you consider to set org-tags-column to 0?

Or put the tags into your own property :MyTag: in the property drawer
instead of using Org's tags?

Michael