Re: [FEATURE REQUEST] Timezone support in org-mode datestamps and org-agenda

[Tim Cross <theophilusx@gmail.com>]

Max Nikulin <manikulin@gmail.com> writes:

> On 14/01/2023 02:06, Jean Louis wrote:
>> This is good for review as related to PostgreSQL database:
>
> I agree that PostgreSQL is an example of good implementation of time-related calculations.
>
>> https://wiki.postgresql.org/wiki/Don%27t_Do_This#Don.27t_use_timestamp_.28without_time_zone.29
>
> I do not find this section enlightening in the context of this discussion. It is more
> related to storage formats of DB than to higher level aspects suitable for applications.
>
>> It would mean that timestamps become time zone aware. Something like this:
>> <2023-01-15 Sun +1>
>
> Welcome to another set of issues.
>
> First of all, you have just date without specific time point. It may mean particular
> calendar day irrelevant to time zone. ICQ had a bug when they decide to store birth dates
> as timestamps. Significant fraction of users received automatic greetings on wrong day, a
> kind of off by one error. So dates must stored as dates omitting time and zone.
>
> Let's assume <2023-01-15 Sun 09:00 +1>
>
> It may be suitable for timestamps in the past, but future is more tricky. There is no
> problem if you are going to watch Lunar eclipse. However if your plan is to attend a local
> event there is a chance that you will arrive at wrong time. Sometimes offset of timezones
> is changed and it may happen between the moment when you added a scheduled time and the
> moment of the event.
>
> Notice that Org timestamps already associated with a timezone, the one is set in libc
> (system timezone, or the one set for particular process). So daylight saving time and
> administrative time transitions should be taken into account. So Org timestamps may be
> ambiguous (mostly) 1 hour per year around backward time transition, e.g. for timezones
> having DST. This case explicitly specifying time offset helps to avoid uncertainty.

If org was to add TZ capabilities to timestamps, the underlying format
would have to be UTC. My previous post where I suggested there was two
'layers', the underlying storage layer (used for calculations like
duration or comparison etc) and a presentation layer (what the human
sees, which is often in whatever their local TZ is). While overly
simplified, this is basically the approach used by postgres. If org mode
was to adopt TZ support for timestamps, I am quite certain that the only
sane way to do this will be to store the timestamps as UTC. Anything
else will just cause massive complications for anyone who moves between
different timezones as well as complicating calculations which cross
over DST transition points. 

I think most people are aware of the challenges of dealing with daylight
savings and the fact that the transition date+time into and out of DST
can change based on various criteria, including political whims
(e.g. Australia eastern DST transition date was changed in 2000 because
Sydney hosted the Olympics that year).  This means there is also an
historical context when calculating and formatting timestamps - it isn't
as easy as looking up to see if a location has DST and when the
transition date/time is - you have to determine that as of the date of
the timestamp.

THen we have the further complication of calendar changes and different
calendars. For example, many people don't realise that Russia didn't
adopt the Gregorian calendar until 1918. For example, the reference to
Napoleon earlier is more complicated when considered from a Russian
locale.

Daryl mentioned elsewhere in this thread that how hard this feature
would be depends largely on the available libraries for elisp with
respect to working with date/time values. Sadly, the available elisp
libraries are inadequate in this area. While many of the functions can
be called with an optional argument defining time zone information,
there is no library which can retrieve this information in a consistent
manner and which supports historical data lookup e.g. what were the DST
transition dates for NSW Australia in 1970 (trick questions, NSW didn't
adopt DST until 1971). The existing library functions are focused more
on simpler time calculations where time zone information is less
relevant i.e. running timers, timing command execution, comparing file
modification times etc. Sometimes, it is easy to forget that while Elisp
is powerful, it isn't really a general purpose programming language like
C or Rust or even CL and Scheme. It is primarily a programming language
focused  on a specific domain, the editor. While it has some great
libraries  it also has  anumber of areas where it lacks the sort of
sophistication we have grown to expect with more general purpose
languages.

A further complication is that accessing date and time related data is
very system dependent and there is no high level library which abstracts
this so that you  can easily get consistent results across all the
platforms Emacs runs on. Even the underlying data type can vary greatly
(i.e. some platforms use a 64 bit value while others only use a 32 bit one).

I therefore suspect that in order to consider adding this feature to org
it will first be necessary to add a more sophisticated date+time library
to Emacs - probably one which is able to interface more closely with OS
APIs - most likely, a C library with an Elisp API which is able to query
OS native TZ database information and which implements basic time
calculations and conversions between different TZ values. It is likely
that such a library already exists and just needs to be added to
Emacs. However, in addition to finding an existing library, it would of
course also have to be GPLv3 licensed in order to be added to Emacs

Another aspect of this feature request which hasn't yet been considered
is what to do with respect to exporting data. Will we also need some way
to define/set the TZ for exported documents? Are there issues to be
considered with respect to code tangling?