From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id wDn6AcVLwmNPLwEAbAwnHQ (envelope-from ) for ; Sat, 14 Jan 2023 07:29:25 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id IIn/AcVLwmMiCwAA9RJhRA (envelope-from ) for ; Sat, 14 Jan 2023 07:29:25 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 9E5321E9B5 for ; Sat, 14 Jan 2023 07:29:23 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pGa1k-0005Oq-ER; Sat, 14 Jan 2023 01:28:28 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pGa1i-0005OF-ME for emacs-orgmode@gnu.org; Sat, 14 Jan 2023 01:28:26 -0500 Received: from mail-wm1-x336.google.com ([2a00:1450:4864:20::336]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pGa1f-0005bL-SS for emacs-orgmode@gnu.org; Sat, 14 Jan 2023 01:28:26 -0500 Received: by mail-wm1-x336.google.com with SMTP id m8-20020a05600c3b0800b003d96f801c48so20069347wms.0 for ; Fri, 13 Jan 2023 22:28:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wakatara.com; s=google; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=9FXwuy3jcY2AbeHq+1AUOMKDmXPYJPHc731GHmtfFuk=; b=k5AIT6CMA3OP39RDRdA8AFrYw2PpAgopfXhQ2XxHNoBE0uXzipp305tZ+Dz55zB1aM fJfhBwJ/PkC82bmY0ZPEwvJeiFLYiOI7K5eIsdjJDqN17rF1M2ZOrnKWqEt0Q7WvnazY pIoGtunZQj31uUWo3+2Fl0cYw97MWwnEQHDdH88X/LQaM1eToQM06YXHydap6LFAqLzw wramXk25QPXtiKwoEBgMuWlsG/vcmAe0gZ6WNRdgc+EXrp9POxt3fzXLYXF3i0WN8lXq WJV/bJZTIjptEUHUaNkJnc8KzsHiwJH3x2FfS5hoC81/9oZhpeuP8kAxyyIQBJPsYLvg OrXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9FXwuy3jcY2AbeHq+1AUOMKDmXPYJPHc731GHmtfFuk=; b=ZvkSVJV+xMFpAE8PQQhW2Xebg+Qd7chJbBJp623Yr0rFcI4dESR/j6S0PiNnhyXlYP xqWCC84PuUvVfu/XV5kf1LddrZLH/Bn+OY3WdY28Nv5QQUjmd6eEDHgvvBX/M4fVIqR7 zl0v+v0TcCEn79jQp4cF1CXCgeeYezi10iwJSaSKNJnRC2EwDAq4AQlugaJrWdrUiXE3 ODIxDehcgwPEyHZqPgS4wdfpTdLXq04Kcj6OVmroirwmxQ8eZ9B9mCeJUmBG8QSfRU4j b0GF69dnBUgNAQapz7CrMfy7BQN3OEk4i0VeTs8qv7tHHEqFV4ehXK72lrzWjjNJ3k49 CftQ== X-Gm-Message-State: AFqh2krVrWolqbroi0dXhFySr8wu+7pciVjWe40wgjN60/V+iiYaeUcF ZkYGVfjq5dLjdePVAKxE9Uh7VGhOVMY2xanPmHi2SpiRLcGNyR1Y X-Google-Smtp-Source: AMrXdXtbcCs42MnAPu9Lt9T+j7kvedLzN0nLtlViUsXhEUU3GvAsxdAMtBtWWzk9xnj7RzK61NoQaZvgYoOWkyyz7Yk= X-Received: by 2002:a05:600c:b99:b0:3da:74c:2ef5 with SMTP id fl25-20020a05600c0b9900b003da074c2ef5mr628053wmb.113.1673677700849; Fri, 13 Jan 2023 22:28:20 -0800 (PST) MIME-Version: 1.0 References: <86zgamtv6o.fsf@gmail.com> In-Reply-To: <86zgamtv6o.fsf@gmail.com> From: Daryl Manning Date: Sat, 14 Jan 2023 13:27:44 +0700 Message-ID: Subject: Re: [FEATURE REQUEST] Timezone support in org-mode datestamps and org-agenda To: Tim Cross Cc: emacs-orgmode@gnu.org Content-Type: multipart/alternative; boundary="000000000000d40e3205f23374c5" Received-SPF: pass client-ip=2a00:1450:4864:20::336; envelope-from=daryl@wakatara.com; helo=mail-wm1-x336.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=wakatara.com header.s=google header.b=k5AIT6CM; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=none ARC-Seal: i=1; s=key1; d=yhetil.org; t=1673677763; a=rsa-sha256; cv=none; b=nZbMY8FyQkWhNSW47zjH9nyrTNIO7Zio4tIagWjoOhY5X+B8x1b4ctUPk+JzA/Z6lWesrd fWwdCXMg1vleuPokVJ7ebR+rbnFnSwUIjPEmF9R1VDyKPPiit73PhMhyZYh6i0ymenP4uP JXLQWMRpoVZUORLH1AVY3oiTmpRmXYL9gB5OQX6EphoEVAsfmS8ailZu+KNuLusKz7EZYt gWn6BaGqJ5jPmKcPPXzYB6+KYS6Ui7UuHwQ+9UhynpyXs4k0w7L3RfyIULgM+gqULiQRf7 MiHRrJLwtpIY44o4g9AJlzMa7q7RJg/bsPZypm+lhheU87sn8G3MBiKGhx9KnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1673677763; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=9FXwuy3jcY2AbeHq+1AUOMKDmXPYJPHc731GHmtfFuk=; b=iKn2gK6m+STGjbCq5lai32GgUDzMAQ5rAjDsq3u10mOA9ayqWCaxpXdRPNy00bvInmcp4I 6qBqaQMNdIkJEpMcVlJ8fRqodGP4EC0CYyGtRQNJYxyGRVCDeltAJ9Kg2WmjwrQl4f+n5I miGMGHSlS9hWqQJZ8Xz1MJVDQQxTjrlmxtRheyVp43/EozVk+k/FuCg6q4vsDCpo6cERV6 BhfCKCqH9OWeps6u5MI/M0pM4FhiNUZu+wzwQ+BBZ0mSskdiVuO54XZYxAAZ7hFt+6hpbW 3C8J0vAc6kU4MZN4c9fmGU2K7igE5UQr7g1MKCa921RzxSurP30cEWh3dW6pmA== X-Migadu-Queue-Id: 9E5321E9B5 X-Migadu-Scanner: scn0.migadu.com Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=wakatara.com header.s=google header.b=k5AIT6CM; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=none X-Migadu-Spam-Score: -4.91 X-Spam-Score: -4.91 X-TUID: v8sfB+WOZAvi --000000000000d40e3205f23374c5 Content-Type: text/plain; charset="UTF-8" I agree this is perhaps not a trivial undertaking. It really depends on the quality of date and time libraries that exist in the lisp ecosystem. For example, in Rust I found my timezone woes in one of my own apps were trivial due to the chronos package, but likewise the reliance of Go depending on its underlying time and date type actually introduced a weird timezone and DST bug in my own app. Is/are there lisp packages which handle the complexity involved in time and date difference calculations and DST changes? That would perhaps be the first question. It makes things tremendously easier and would reduce much of the work (perhaps to just the argument of the format and parsing the datetime stamp and making org-agenda aware of which timezone it's in.). To Jean Louis' point: using timestamps without timezones is a don't in much of computing these days and perhaps hearkens back to the simpler age emacs and org originated in. =] (kidding!). My lips-fu is not adequate to taking this on myself so the issue of who would want ot work on it over other features is perhaps the bigger question. Could someone scope out the work and approach at least. I imagine it is tricky and non-trivial, but perhaps less complex if good libraries exist as above. I'd simply make the timezone format a slowly to be evolved to standard over time with exceptions as noted in my first note. not having tz in timestamps was a "assume local" assumption and under-specification (or considered "hidden".). Daryl. On Fri, Jan 13, 2023 at 8:34 PM Tim Cross wrote: > > Daryl Manning writes: > > > Following on from thread at > https://www.reddit.com/r/orgmode/comments/zrppqw/ > > > > [First off, I just wanted to say thank you to everyone that works on > org-mode. It is a wonder.] > > > > While I realize a few kicks at this can may have been taken, I wanted to > (re-)propose Timezone support in org-mode. The > > world is much less local these days and we're all more remote and > coordinating globally these days. > > > > *Background* > > > > 1. org-time-stamp-formats TZ currently only affects display and exports > > 2. org-agenda itself is not TZ aware > > 3. Several discussions on this have taken place over time > > 4. Concerns raise included breaking backwards compatibility > > > > *Proposal* > > > > 1. org-mode sets an optional variable (org-timezone-aware t) which > enables TZ > > 2. org-agenda needs a way to determine which timezone it is in > > 3. Once enabled, any timestamp not exhibiting a TZ in it is considered > "local time" wherever that is (I do not think UTC > > would work for this) > > 4. org-agenda can calc local based on TZ differences > > > > I understand this is by no means trivial and quite gnarly with DST and > such to figure out but I do believe libs exists to > > deal with that heavy lifting. Currently, it does feel like a hole in > org-mode as a 21st century organizer (disclaimer: > > digital nomading so might feel it more keenly). Also, just interested in > making org-mode a more awesome tool for > > everybody. > > > > I'd love an understanding of the alluded to reservations raised in the > reddit thread and in the mailing list threads > > mentioned that the format change might break things (I was unsure if > that was referring to say, how time ranges were > > handled, or how say date ranges got dealt with (for example, say a > flight from Singapore to Vancouver which takes off in > > one time zone and lands in another - something that is often in my > cal.). > > > > I agree this would be a great feature to add. However, after having > looked at it in some detail, I realise that not only is it not a trivial > task, it is actually a very large and complex task and will require > extensive work which will almost certainly result in breakage with > regards to backwards compatibility. > > At the risk of over simplifying the matter, I would suggest the big > challenge here is that there are two somewhat competing (and > conflicting) use cases. On one hand, you want a high level abstraction > which makes working with dates and times easy and clear. TO some extent, > that is what we have now. On the other hand, we need something far more > complex which is able to handle multiple time zones. This means being > able to handle both base timezone offsets as well as offset adjustments > for things like daylight savings time. There is a lot of hidden > complexity here. You have to handle the fact that daylight saving > chang-over dates/times are dynamic i.e. not necessarily the same every > year. This adds the additional complexity of having to somehow track > historical information regarding such changes, which isn't as readily > accessible in a consistent manner on all platforms. > > Then there is the other 'messy' stuff. For example, when calculating > time ranges and number of days, hours/ minutes between two dates you > need to remember to add/remove the hour if the range crosses over a > daylight savings period. Similarly you need to ensure you make the > correct adjustment when changing timezones (consider emacs on a laptop > for someone who travels a lot between different time zones). Not only do > you need to take into account the different timezone offset, you also > need to look at the date and then adjust according to the timezone > offset for the current timezone at the time of the timestamp rather than > just considering the current time offset. > > I expect what is needed is an 'internal' UTC based representation which > is used for all calculations and an 'interface' layer, which converts > the UTC representation into the local display representation for > consumption by humans. The problem with this is that it is likely to > break the core feature of org files i.e. that they are in plain text. I > guess we could possibly do somehting clever with overlays such that UTC > date/times are rendered/presented in local time format. However, we > would then also require some type of interface for users to enter > dates/times (to ensure they are converted to the correct UTC internal > format). > > However, the real challenge here is that this is a very large piece of > work and it needs someone who is prepared to take it on. I suspect until > someone who needs to scratch this itch sufficiently comes along, this is > a feature which will be unlikely to make it to the top of the todo > list. There are simply far too many existing feature improvements and > additions people will prefer to work on before taking on this > one. Things are made more difficult because it isn't the sort of task > which can be achieved with small increments over time. This is more > likely to be something which needs to be developed in a feature branch, > which once it reaches a sufficient level of completeness can be used and > verified working for a wide variety of environments before then working > out how to fold it back into the core system and handle whatever change > management will be necessary. > --000000000000d40e3205f23374c5 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I agree this is perhaps not a trivial undertaking. It= really depends on the quality of date and time libraries that exist in the= lisp ecosystem. For example, in Rust I found my timezone woes in one of my= own apps were trivial due to the chronos package, but likewise the relianc= e of Go depending on its underlying time and date type actually introduced = a weird timezone and DST bug in my own app.

Is/are= there lisp packages which handle the complexity involved in time and date = difference calculations and DST changes? That would perhaps be the first qu= estion. It makes things tremendously easier and would reduce much of the wo= rk (perhaps to just the argument of the format and parsing the datetime sta= mp and making org-agenda aware of which timezone it's in.).
<= div>
To Jean Louis' point: using timestamps without timez= ones is a don't in much of computing these days and perhaps hearkens ba= ck to the simpler age emacs and org originated in.=C2=A0 =3D]=C2=A0 (kiddin= g!).

My lips-fu is not adequate to taking this on = myself so the issue of who would want ot work on it over other features is = perhaps the bigger question. Could someone scope out the work and approach = at least. I imagine it is tricky and non-trivial, but perhaps less complex = if good libraries exist as above.

I'd simply m= ake the timezone format a slowly to be evolved to standard over time with e= xceptions as noted in my first note. not having tz in timestamps was a &quo= t;assume local" assumption and under-specification (or considered &quo= t;hidden".).

Daryl.

=
= On Fri, Jan 13, 2023 at 8:34 PM Tim Cross <theophilusx@gmail.com> wrote:

Daryl Manning <d= aryl@wakatara.com> writes:

> Following on from thread at https://www.reddit= .com/r/orgmode/comments/zrppqw/
>
> [First off, I just wanted to say thank you to everyone that works on o= rg-mode. It is a wonder.]
>
> While I realize a few kicks at this can may have been taken, I wanted = to (re-)propose Timezone support in org-mode. The
> world is much less local these days and we're all more remote and = coordinating globally these days.
>
> *Background*
>
> 1. org-time-stamp-formats TZ currently only affects display and export= s
> 2. org-agenda itself is not TZ aware
> 3. Several discussions on this have taken place over time
> 4. Concerns raise included breaking backwards compatibility
>
> *Proposal*
>
> 1. org-mode sets an optional variable (org-timezone-aware t) which ena= bles TZ
> 2. org-agenda needs a way to determine which timezone it is in
> 3. Once enabled, any timestamp not exhibiting a TZ in it is considered= "local time" wherever that is (I do not think UTC
> would work for this)
> 4. org-agenda can calc local based on TZ differences
>
> I understand this is by no means trivial and quite gnarly with DST and= such to figure out but I do believe libs exists to
> deal with that heavy lifting. Currently, it does feel like a hole in o= rg-mode as a 21st century organizer (disclaimer:
> digital nomading so might feel it more keenly). Also, just interested = in making org-mode a more awesome tool for
> everybody.
>
> I'd love an understanding of the alluded to reservations raised in= the reddit thread and in the mailing list threads
> mentioned that the format change might break things (I was unsure if t= hat was referring to say, how time ranges were
> handled, or how say date ranges got dealt with (for example, say a fli= ght from Singapore to Vancouver which takes off in
> one time zone and lands in another - something that is often in my cal= .).
>

I agree this would be a great feature to add. However, after having
looked at it in some detail, I realise that not only is it not a trivial task, it is actually a very large and complex task and will require
extensive work which will almost certainly result in breakage with
regards to backwards compatibility.

At the risk of over simplifying the matter, I would suggest the big
challenge here is that there are two somewhat competing (and
conflicting) use cases. On one hand, you want a high level abstraction
which makes working with dates and times easy and clear. TO some extent, that is what we have now. On the other hand, we need something far more
complex which is able to handle multiple time zones. This means being
able to handle both base timezone offsets as well as offset adjustments
for things like daylight savings time. There is a lot of hidden
complexity here. You have to handle the fact that daylight saving
chang-over dates/times are dynamic i.e. not necessarily the same every
year. This adds the additional complexity of having to somehow track
historical information regarding such changes, which isn't as readily accessible in a consistent manner on all platforms.

Then there is the other 'messy' stuff. For example, when calculatin= g
time ranges and number of days, hours/ minutes between two dates you
need to remember to add/remove the hour if the range crosses over a
daylight savings period. Similarly you need to ensure you make the
correct adjustment when changing timezones (consider emacs on a laptop
for someone who travels a lot between different time zones). Not only do you need to take into account the different timezone offset, you also
need to look at the date and then adjust according to the timezone
offset for the current timezone at the time of the timestamp rather than just considering the current time offset.=C2=A0

I expect what is needed is an 'internal' UTC based representation w= hich
is used for all calculations and an 'interface' layer, which conver= ts
the UTC representation into the local display representation for
consumption by humans. The problem with this is that it is likely to
break the core feature of org files i.e. that they are in plain text. I
guess we could possibly do somehting clever with overlays such that UTC
date/times are rendered/presented in local time format. However, we
would then also require some type of interface for users to enter
dates/times (to ensure they are converted to the correct UTC internal
format).

However, the real challenge here is that this is a very large piece of
work and it needs someone who is prepared to take it on. I suspect until someone who needs to scratch this itch sufficiently comes along, this is a feature which will be unlikely to make it to the top of the todo
list. There are simply far too many existing feature improvements and
additions people will prefer to work on before taking on this
one. Things are made more difficult because it isn't the sort of task which can be achieved with small increments over time. This is more
likely to be something which needs to be developed in a feature branch,
which once it reaches a sufficient level of completeness can be used and verified working for a wide variety of environments before then working
out how to fold it back into the core system and handle whatever change
management will be necessary.
--000000000000d40e3205f23374c5--