RE: Timestamp parsing inside node properties and other contexts out of org-element-object-restrictions (was: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)])

["Tom Davey" <tom@tomdavey.com>]

Hi Tim, 

Thanks for these thoughtful comments. I agree that the Org developers (to
whom I, as a mere user, owe enormous thanks) must be wary before making
changes to how timestamps are handled. 

This argues, I would say, for keeping what I believe was the status quo
since at least Org version 9.4.4: Agenda views would display entries with
active timestamps in property drawers. That has been my historical
experience. 

Tim, has your historical experience been different? In the invoicing example
you give, were the timestamps in the properties drawer active, or inactive? 

I have just verified with a simple test that Org version 9.4.4, which was
shipped with Emacs 27.2 I believe, does display entries with an active
timestamp as the value of a property in the ordinary :PROPERTIES: drawer.
That's the situation I'm calling the "status quo." I'm wondering if my
experience coincides with the experience of others. 

Here's the simple entry that will be shown on the Week/Day Agenda view in
9.4.4: 

* TODO Test of active timestamps
   :PROPERTIES:
   :Created:  <2022-03-22 Tue 18:30>
   :END:

And note this: adding a second active timestamp to the test entry, e.g., to
accompany a SCHEDULED: keyword, results in the entry appearing on the Agenda
twice, as would be expected: 

* TODO Test of active timestamps
   SCHEDULED: <2022-03-22 Tue 18:30>
   :PROPERTIES:
   :Created:  <2022-03-22 Tue 18:30>
   :END:

This second example shows why the variable
org-agenda-skip-additional-timestamps-same-entry is valuable. I rarely want
an entry to display twice on the same day. 

Tom Davey 

--
Tom Davey
tom@tomdavey.com
New York NY USA

-----Original Message-----
From: Emacs-orgmode <emacs-orgmode-bounces+tom=tomdavey.com@gnu.org> On
Behalf Of Tim Cross
Sent: Tuesday, March 22, 2022 5:10 PM
To: Ihor Radchenko <yantar92@gmail.com>
Cc: Ignacio Casso <ignaciocasso@hotmail.com>; emacs-orgmode@gnu.org;
tom@tomdavey.com; Nicolas Goaziou <mail@nicolasgoaziou.fr>
Subject: Re: Timestamp parsing inside node properties and other contexts out
of org-element-object-restrictions (was: [BUG] Agenda no longer works for
timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @
/home/ignacio/repos/emacs/lisp/org/)])


Ihor Radchenko <yantar92@gmail.com> writes:

> Ihor Radchenko <yantar92@gmail.com> writes:
>
>> After further reading the source code, I figured that agenda is, in 
>> fact, supposed to handle timestamps inside property drawers. Optional 
>> arguments for org-at-timestamp-p imply that, in agenda specifically, 
>> timestamps inside node properties are considered timestamps despite 
>> they are not being parsed as timestamps by org-element.
>
> Even though I fixed the reported issue with agenda not showing 
> headings with matching timestamps inside property drawers, this 
> situation is revealing a big inconsistency in Org mode's handling of
timestamps.
>
> org-at-timestamp-p usage implies that Org syntax for timestamps is not 
> only context-dependent, but also depends on current command!
>
> org-at-timestamp-p is called with non-nil argument in a number of 
> functions in Org:
> - org-clock-timestamps-change
> - org-mouse-delete-timestamp
> - org-mouse-context-menu
> - org-follow-timestamp-link
> - org-get-repeat
> - org-auto-repeat-maybe
> - org-time-stamp
> - ... many more in org.el
>
> So, depending on the current command, Org may on may not treat objects 
> matching org-ts-regexp-both as timestamps.
>
> This situation complicates syntax and makes org-element unreliable 
> when dealing with Org buffers.
>
> Should we just simply allow timestamps to be a part of node property 
> values? Should we _not_ treat timestamp-looking text outside their 
> allowed contexts (like quotes, source blocks, etc) as timestamps?
>

I think we have to be very wary here. I can see any changes here causing
lots of breakage for people. I know for my own use case, I use timestamps a
lot in property draws and various source blocks. I never want any of them
showing up in my agenda.

As an example, I was recently working for a company which required that you
put a timestamp in both a file header and in comments. The format they used
was pretty much the same as an org-mode active timestamp. I use org mode to
tangle the source files I write (as well as manage my client data, such as
todos, invoicing, contacts etc), so these files are searched for agenda
items, but I do not want any of those timestamps causing lines in my agenda
views. These timestamps are most commonly found in source and example
blocks.

I think the only time an org timestamp should be recognised in a source
block is when that source block is an org-mode source block. I don't think
they should ever be 'recognised' in example blocks.  

IN addition, my invoicing solution, which is based on org, uses timestamps
to track invoice periods etc. None of these should ever appear in the
agenda. This information is typically tracked in property draws.

Unfortunately, I think org timestamps is a bit of a mess and we need to be
very careful about further complicating matters. The inability to handle
timezones is a major limitation IMO. The inability to handle daylight
savings transitions in a consistent manner (particularly for calculation of
periods, duration, etc) is often a source of errors and if you are
unfortunate enough to regularly travel across different timezones, forget
about using org mode to manage your schedule.

I have done several deep dives into org-mode's timestamp stuff, but usually
come back up gasping for air. Managing data and time data is often far more
complicated than it may appear on the surface. I think we need to be
extremely conservative when considering changes in this area (it is the main
reason I've never managed to find a way to add time zone data - every
solution I could think of was either really high on the level of breakage
and frustration it would create for users or it adversely impacted the
convenience/usability of org timestamps).