Timestamps in properties not exported by icalendar exporter

Timestamps in properties not exported by icalendar exporter

[David Engster]

I have the problem that a certain kind Org entries is not exported by
the icalendar exporter, namely those created by the gnus-icalendar
package.

This package creates Org entries from calendar invites in the following
way (I've omitted some of the properties, but you get the idea);

** Some appointment (location)
   :PROPERTIES:
   :ICAL_EVENT: t
   :ID:       040000008200E00074C5B7101A82E00800
   :DT:       <2013-12-09 Mon 13:00-13:30>
   :END:

   Body text


These entries show up in the agenda just fine, but the icalendar
exporter does not export it because the timestamp is in the properties
(the gnus-icalendar package puts it there so that you can easily change
it if the organizer decides to change the event and sends a change
request).

The exporter could of course simply take the first timestamp it finds in
the properties, but it could be the wrong one; for instance, there could
be a LOGBOOK timestamp before the one in DT. Hence I think it would be
nice if you could tell the exporter which property to use as timestamp
during export.

-David

Re: Timestamps in properties not exported by icalendar exporter

[Nicolas Goaziou]

Hello,

David Engster <deng@randomsample.de> writes:

> I have the problem that a certain kind Org entries is not exported by
> the icalendar exporter, namely those created by the gnus-icalendar
> package.
>
> This package creates Org entries from calendar invites in the following
> way (I've omitted some of the properties, but you get the idea);
>
> ** Some appointment (location)
>    :PROPERTIES:
>    :ICAL_EVENT: t
>    :ID:       040000008200E00074C5B7101A82E00800
>    :DT:       <2013-12-09 Mon 13:00-13:30>
>    :END:
>
>    Body text
>
>
> These entries show up in the agenda just fine, but the icalendar
> exporter does not export it because the timestamp is in the properties
> (the gnus-icalendar package puts it there so that you can easily change
> it if the organizer decides to change the event and sends a change
> request).
>
> The exporter could of course simply take the first timestamp it finds in
> the properties, but it could be the wrong one; for instance, there could
> be a LOGBOOK timestamp before the one in DT. Hence I think it would be
> nice if you could tell the exporter which property to use as timestamp
> during export.

Can't you just walk the buffer and turn such properties into plain
time-stamps (or scheduled, deadline...), in a hook?


Regards,

-- 
Nicolas Goaziou

Re: Timestamps in properties not exported by icalendar exporter

[David Engster]

Nicolas Goaziou writes:
> David Engster <deng@randomsample.de> writes:
>> These entries show up in the agenda just fine, but the icalendar
>> exporter does not export it because the timestamp is in the properties
>> (the gnus-icalendar package puts it there so that you can easily change
>> it if the organizer decides to change the event and sends a change
>> request).
>>
>> The exporter could of course simply take the first timestamp it finds in
>> the properties, but it could be the wrong one; for instance, there could
>> be a LOGBOOK timestamp before the one in DT. Hence I think it would be
>> nice if you could tell the exporter which property to use as timestamp
>> during export.
>
> Can't you just walk the buffer and turn such properties into plain
> time-stamps (or scheduled, deadline...), in a hook?

Yes, I could do that for my specific setup. But it would be nice if this
stuff could "just work", so that things like Outlook calendar invites
can be directly exported to .ics.

I mean, those entries show up in the agenda, so I found it rather
surprising that they are completely ignored by the exporter. I think it
would make sense if the exporter also looked for time-stamps in the
properties. Maybe it could just take the first one it finds (which I
guess is what the agenda does?), and also give the user the ability to
configure which properties to look at (or maybe an exclude option).

-David

Re: Timestamps in properties not exported by icalendar exporter

[Nicolas Goaziou]

David Engster <deng@randomsample.de> writes:

> Yes, I could do that for my specific setup. But it would be nice if this
> stuff could "just work", so that things like Outlook calendar invites
> can be directly exported to .ics.

AFAIU, we're talking about a third-party package which implements its
own UI. We cannot support every UI in the wild.

> I mean, those entries show up in the agenda, so I found it rather
> surprising that they are completely ignored by the exporter.

This is an agenda bug, which probably use a regexp to find timestamps.
But timestamps in properties are not valid Org timestamps, per Org
syntax.

> I think it would make sense if the exporter also looked for
> time-stamps in the properties.

There are already plenty of locations to use timestamps. We have
scheduled, deadline, plain timestamps... I don't think we need more of
them.

Also, a hook is easy enough to implement in this situation.


Regards,

-- 
Nicolas Goaziou

Re: Timestamps in properties not exported by icalendar exporter

[David Engster]

Nicolas Goaziou writes:
> David Engster <deng@randomsample.de> writes:
>> I mean, those entries show up in the agenda, so I found it rather
>> surprising that they are completely ignored by the exporter.
>
> This is an agenda bug, which probably use a regexp to find timestamps.
> But timestamps in properties are not valid Org timestamps, per Org
> syntax.

OK.

>> I think it would make sense if the exporter also looked for
>> time-stamps in the properties.
>
> There are already plenty of locations to use timestamps. We have
> scheduled, deadline, plain timestamps... I don't think we need more of
> them.

It's these multitude of timestamp locations which makes changing the
timestamp of an existing entry through Elisp so tedious (I had to deal
with that in org-caldav), so using a property for it seemed like a good
idea. Anyway, I understand your reasoning, and I guess we should change
how gnus-icalendar generates its Org entries.

-David

Re: Timestamps in properties not exported by icalendar exporter

[Nicolas Goaziou]

David Engster <deng@randomsample.de> writes:

> It's these multitude of timestamp locations which makes changing the
> timestamp of an existing entry through Elisp so tedious (I had to deal
> with that in org-caldav)

The number of locations may be daunting but all of them make sense
actually. Also, I don't think it is really difficult to change
a timestamp through Elisp, due to specialized functions (e.g.
`org-schedule').

Anyway, if you think this area can be improved, feel free to make
suggestions.

> so using a property for it seemed like a good idea.

This has been discussed a few times on this ML already.

Carsten's argument is that new users shouldn't have to deal with hidden
data, like property drawers, for such basic features.

IMO, it also have an advantage from an implementation POV: important
timestamps are located right below the headline, whereas a property
drawer can be found anywhere in the whole section.

> Anyway, I understand your reasoning, and I guess we should change how
> gnus-icalendar generates its Org entries.

I think so, too.


Regards,

-- 
Nicolas Goaziou

Re: Timestamps in properties not exported by icalendar exporter

[David Engster]

Nicolas Goaziou writes:
> David Engster <deng@randomsample.de> writes:
>
>> It's these multitude of timestamp locations which makes changing the
>> timestamp of an existing entry through Elisp so tedious (I had to deal
>> with that in org-caldav)
>
> The number of locations may be daunting but all of them make sense
> actually. Also, I don't think it is really difficult to change
> a timestamp through Elisp, due to specialized functions (e.g.
> `org-schedule').

Not difficult, but tedious. The problem is that an entry might have
several timestamps, and you have to find the correct one to change.

> Anyway, if you think this area can be improved, feel free to make
> suggestions.

I'm actually not sure what org-element is capable of nowadays. What I'd
like to have is a way to parse entries into a structure which lets me
access certain elements of the entry, like headline, timestamps,
properties, body text, etc., and also a way to convert this structure
*back* to a plain Org entry. This way, I could change certain elements
of the entry in a robust way. It might be that the *appearance* of the
entry changes a bit during this procedure, but its elements should be
preserved.

(BTW: This is similar to what we have in CEDET, where the Semantic
parser generates 'tags' which describes things like variable or function
declarations, and you can define SRecode templates which can generate
textual output from these tags in pretty much any way you like.)

For example, in org-caldav I had to write a function which changes the
headline of an entry, but preserves any timestamps which might be in
it. Again: not difficult to do, especially since there is
org-complex-heading-regexp, but it's still tedious since Org entries are
so flexible in their appearance.

>> so using a property for it seemed like a good idea.
>
> This has been discussed a few times on this ML already.
>
> Carsten's argument is that new users shouldn't have to deal with hidden
> data, like property drawers, for such basic features.

Oh no, the user shouldn't have to do this, I agree
completely. Properties are nice for *code*, since they can be easily
parsed.

-David

Re: Timestamps in properties not exported by icalendar exporter

[Nicolas Goaziou]

David Engster <deng@randomsample.de> writes:

> I'm actually not sure what org-element is capable of nowadays. What I'd
> like to have is a way to parse entries into a structure

See `org-element-parse-buffer'.

> which lets me access certain elements of the entry, like headline,
> timestamps, properties, body text, etc.,

See `org-element-map'.

> and also a way to convert this structure *back* to a plain Org entry.

See `org-element-interpret-data'.


Regards,

-- 
Nicolas Goaziou

Re: Timestamps in properties not exported by icalendar exporter

[David Engster]

Nicolas Goaziou writes:
> David Engster <deng@randomsample.de> writes:
>
>> I'm actually not sure what org-element is capable of nowadays. What I'd
>> like to have is a way to parse entries into a structure
>
> See `org-element-parse-buffer'.
>
>> which lets me access certain elements of the entry, like headline,
>> timestamps, properties, body text, etc.,
>
> See `org-element-map'.
>
>> and also a way to convert this structure *back* to a plain Org entry.
>
> See `org-element-interpret-data'.

I had a hunch you already implemented all this. :-)

I'll look into it.

Thanks,
-David

Re: Timestamps in properties not exported by icalendar exporter

[Nick Dokos]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> David Engster <deng@randomsample.de> writes:
>
>> Yes, I could do that for my specific setup. But it would be nice if this
>> stuff could "just work", so that things like Outlook calendar invites
>> can be directly exported to .ics.
>
> AFAIU, we're talking about a third-party package which implements its
> own UI. We cannot support every UI in the wild.
>

I was confused at first because David has (had?) his own gnus-icalendar
package, but I believe he means the gnus-icalendar that is part of
gnus. That's probably a bit more official than a third-party package in
the wild, but I appreciate Nicolas's desire to control the chaos.

It might make more sense to advertise an API/convention/whatever that
such packages can use and provide a patch to modify gnus-icalendar
accordingly. After all, I presume gnus-icalendar can only be used with
gnus, so other mailers will have to have their own icalendar->org. If
each picks its own convention, chaos reigns.

We just need to come up with the "standard" and advertise it
appropriately. Maybe something like this:

"provide the starting time/duration information as a scheduled
timestamp, and optionally provide everything in properties.
The names of the properties should be the names that gnus-icalendar
currently uses (explicitly listed out, of course)."

>> I mean, those entries show up in the agenda, so I found it rather
>> surprising that they are completely ignored by the exporter.
>
> This is an agenda bug, which probably use a regexp to find timestamps.
> But timestamps in properties are not valid Org timestamps, per Org
> syntax.
>

If gnus-icalendar is modified as above, then when the agenda is
rewritten using org-element, things are not going to break.

>> I think it would make sense if the exporter also looked for
>> time-stamps in the properties.
>
> There are already plenty of locations to use timestamps. We have
> scheduled, deadline, plain timestamps... I don't think we need more of
> them.
>
> Also, a hook is easy enough to implement in this situation.
>

-- 
Nick