Re: Timestamp parsing inside node properties and other contexts out of org-element-object-restrictions (was: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)])

[Tim Cross <theophilusx@gmail.com>]

"Tom Davey" <tom@tomdavey.com> writes:

> Hi Tim, 
>
> Thanks for these thoughtful comments. I agree that the Org developers (to
> whom I, as a mere user, owe enormous thanks) must be wary before making
> changes to how timestamps are handled. 
>
> This argues, I would say, for keeping what I believe was the status quo
> since at least Org version 9.4.4: Agenda views would display entries with
> active timestamps in property drawers. That has been my historical
> experience. 
>
> Tim, has your historical experience been different? In the invoicing example
> you give, were the timestamps in the properties drawer active, or inactive? 
>
> I have just verified with a simple test that Org version 9.4.4, which was
> shipped with Emacs 27.2 I believe, does display entries with an active
> timestamp as the value of a property in the ordinary :PROPERTIES: drawer.
> That's the situation I'm calling the "status quo." I'm wondering if my
> experience coincides with the experience of others. 
>
> Here's the simple entry that will be shown on the Week/Day Agenda view in
> 9.4.4: 
>
> * TODO Test of active timestamps
>    :PROPERTIES:
>    :Created:  <2022-03-22 Tue 18:30>
>    :END:
>
> And note this: adding a second active timestamp to the test entry, e.g., to
> accompany a SCHEDULED: keyword, results in the entry appearing on the Agenda
> twice, as would be expected: 
>
> * TODO Test of active timestamps
>    SCHEDULED: <2022-03-22 Tue 18:30>
>    :PROPERTIES:
>    :Created:  <2022-03-22 Tue 18:30>
>    :END:
>
> This second example shows why the variable
> org-agenda-skip-additional-timestamps-same-entry is valuable. I rarely want
> an entry to display twice on the same day. 
>

For my invoicing system, the timestamps are in property draws, but not
under TODO headings, so I guess they wouldn't show up. I have used
timestamps in TODO entry property draws, but don't recall seeing them in
the agenda when I do. However, I also use a couple of custom agenda
views more often than the default agenda view, so I just may not have
noticed.

Personally, I have no use case for TODOs being displayed in the agenda
based on their created date. I do like to log when I created the todo,
but would not want to see that cluttering my agenda where I only want to
see scheduled and deadline tasks. I use a property draw for this type of
information as I consider it metadata about the entry. If I really want
ot see the item in the agenda based on a timestamp, then I explicitly
add that timestamp to the entry (not in  a property draw).

In my invoicing module, property draws are used to track invoicing
metadata i.e. last invoice number, last invoice date, invoice creation
date and invoice period for each invoice, invoice due date, paid datge
etc. Some of the timestamps are 'inactive' (they never change) and some
are active (will get modified). Perhaps that was a mistake on my part
and all of them should be inactive, but at the time, I thought of the
distinction as not being purely about whether they show in the agenda,
but more about whether they were immutable or not. 

The need to have another variable to limit the scope of timestamps when
you do allow timestamps in property draws makes things feel very
'kludgy'. I get very concerned about all these 'enhancements' we keep
adding which then also require additional complexity with multiple
variables interacting at different levels. I find the number of
variables and complexity associated with the agenda compared to when I
first started using org mode to be mind boggling. I often wonder how the
maintainers are able to keep on top of the increasing complexity of org
mode. 

What I definitely would NOT want is for TODO tasks in source blocks
which have a timestamp associated with them showing up in the agenda.
The possible exception might be when the source block is an org source
block. What I want in source blocks is for them to behave like a
microcosm of the original code environment - I want them to behave
exactly as they would when I just edit that code in a code dedicated
buffer with the addition of a minimal set of babel/noweb features to
allow me to work with code spread over multiple source code blocks etc.
I definitely don't want the additional processing overhead of org mode
search through my source blocks looking for things which look like they
might be timestamps or worse, looking for such things and then trying to
do something clever with them (like adding to the agenda).

With regards to timestamps in property draws, I'm not sure that arguing
the status quo is to keep that behaviour. I'm not sure that behaviour
was supposed to work or that it is well defined. It may have worked
previously 'by accident' rather than design (something which tends to
happen more as complexity increases). For example, what is the situation
with timestamps in property draws and inheritance? What is the situation
with timestamps and property draws for items which are not TODOs (i.e.
normal headings with a property draw)? What happens if I have multiple
properties with timestamps as their value?

I would agree with Ihor's observation things seem to be inconsistent and
we probably need to work at getting things to be more consistent.
However, I would lean towards a solution which simplifies the situation
in the general case, even if that makes some specialised uses cases more
difficult. Given that property draws can have arbitrary property names
and some of these could be active timestamps and given there could be
multiple properties which have timestamp values in the same property
draw and there is supposed to be an inheritance component to properties,
I would be inclined to not support adding properties with timestamps to
the agenda. If it turns out a lot of people want this functionality,
then I would suggest a specific property name be reserved for this i.e.
:Agenda-ts: or :Agenda-entry:, rather than allowing any property name with a timestamp. 
I wouldn't use :Created: as that would be overloading the property to
perform two roles (track when an entry was created and add it to the
agenda).