From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id OFpJJI2412P5AgEAbAwnHQ (envelope-from ) for ; Mon, 30 Jan 2023 13:31:09 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id KG5aI42412OkuQAAG6o9tA (envelope-from ) for ; Mon, 30 Jan 2023 13:31:09 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 8448D2FADA for ; Mon, 30 Jan 2023 13:31:08 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pMTIv-0002yp-QK; Mon, 30 Jan 2023 07:30:33 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pMTIu-0002yh-21 for emacs-orgmode@gnu.org; Mon, 30 Jan 2023 07:30:32 -0500 Received: from mout02.posteo.de ([185.67.36.66]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pMTIr-0004k5-0x for emacs-orgmode@gnu.org; Mon, 30 Jan 2023 07:30:31 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id C821D24022B for ; Mon, 30 Jan 2023 13:30:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1675081826; bh=oXOCK0w32ikPYaf2k39wc53JdICbKWDVHOH42IqXhDM=; h=From:To:Cc:Subject:Date:From; b=GXY/A19bVRcveiW/PLvFqoYenX+LXE1MNvTbht8tG3Vr1OW2Qlm2MIZQRKDYQCo2h Qcg8tPPVp6At0aFBGlUxXoxLUsjlLpSkedmGWjMGCN2hFx6HSkXOZKvMDL/MMd11Kx 54MpQAvtYXgHHSFmc9Ieo9Ux4o9MVR0e1QfWfsTdUbpgsH8p+XSoDK7+hhYah1S/rA iMGA8VLtJ4NevY0BHlCd4eDmgkhg5c6F/UeWpS7DO1cKvU+/6Y4kqAN3rvAUk0mLtj S2ywtiV6eDlk21NVURYvEj0Qy4DTue3bh4NMnFZl4Rf92qQkzfCWUFJXTDLPL8m4o5 YSFgPDFZSk4kg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4P56vw1kL4z6tmS; Mon, 30 Jan 2023 13:30:19 +0100 (CET) From: Ihor Radchenko To: Sterling Hooten Cc: "Thomas S. Dye" , Tim Cross , Jean Louis , Daryl Manning , rjhorn@alum.mit.edu, emacs-orgmode@gnu.org Subject: Re: [FEATURE REQUEST] Timezone support in org-mode datestamps and org-agenda In-Reply-To: <3035CDD5-41DD-4516-9E4E-9E0DF16BE2E0@gmail.com> References: <63c66048.630a0220.427bf.a5f6@mx.google.com> <87r0vtiks0.fsf@localhost> <63c671c0.a70a0220.61aa5.56b8@mx.google.com> <87fsc88aq9.fsf@localhost> <63c7dd3d.170a0220.6b4d6.f84f@mx.google.com> <877cxk6oeu.fsf@localhost> <63c86454.170a0220.80970.652d@mx.google.com> <63c8f5a6.170a0220.ea8cf.7f96@mx.google.com> <63c9b654.170a0220.d82d2.4254@mx.google.com> <87mt6e86sr.fsf@tsdye.online> <63c9d976.620a0220.a7d40.113b@mx.google.com> <87tu0mjb24.fsf@tsdye.online> <63ca1283.170a0220.5bc81.0fdd@mx.google.com> <87pmb9k8oi.fsf@tsdye.online> <3035CDD5-41DD-4516-9E4E-9E0DF16BE2E0@gmail.com> Date: Mon, 30 Jan 2023 12:30:49 +0000 Message-ID: <874js85hmu.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=185.67.36.66; envelope-from=yantar92@posteo.net; helo=mout02.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Seal: i=1; s=key1; d=yhetil.org; t=1675081869; a=rsa-sha256; cv=none; b=lV+tQ2m8uIWyIS6vEfTmAoU1EzNmA7ysPFuLHhRMF1Q0yr0Oorvnrzv8Spw+YpLOag13ye E3epMh2nY9MGcPAfURqj7eFVRE2++FgksitfEFz3XoHqOuz7XUlQ9DGgqDi7Qe9LYgSwC0 CsFcww9flg5NfknQsoil7eNlxpPYviVpxjfmUbHr70yuxI3/LF9eXaf7zNSYyHrHwROGO7 OQYxN3DqjqUku64J1ivFrqf5W1XgtWeT+LhLo4I0sTzQabF+jflZYcRZv2QximTFe6TgZ5 dABGSJ8XNSu7nomhvwPgHAqJc+HYMsYbYRQMB3eni/9nc+c5U9CZDv042ntl8g== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b="GXY/A19b"; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1675081869; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=WMkWxT12rZlap41AoGJvseq8TjXwon/P1hgV0uW1of4=; b=ZnnoUToZWnk1XKhEjWeLfeo8mPomJYoEcLHkUZdV0oOh6lfczdVzX9K6FvhG9xfr6MLUNL +bHOc74T6d6Mf335JIA7+0257q2bL6thrGLs+fdQirjZx5L59HlBq5ZQ62geTTOAgnRZJZ jtYU5MR3yV2SvSwJTt6hltC0RwuI/86jcxUrHmSWeMEcpmxb3lPJbEqmWaFEwFDDzSJlID oL9rmNUwtq7Pf02UUyOtEBh9ybFrZnsTA5paxRTYxCS7Uw1heWceyx8uNwJJFGrapUWNiQ SZ0HM03kBeFxv8kYtq0hwlX4efQl7cNCBxTxAji3sAPRCbWjt3V/H+39K+3i9A== Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b="GXY/A19b"; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.net X-Migadu-Scanner: scn1.migadu.com X-Migadu-Spam-Score: -4.54 X-Spam-Score: -4.54 X-Migadu-Queue-Id: 8448D2FADA X-TUID: Pi2eWMIvegiE Sterling Hooten writes: > This is an initial glossary compiled from various standards and sources; > it's incomplete, probably incorrect, and open to critique, but is useful > in articulating a possible road map forward. Do I understand correctly that the terms are simply taken from ISO (https:/= /www.iso.org/obp/ui/#iso:std:iso:8601:-1:ed-1:v1:en)?=20 > What does format does Org have now? > > =E2=80=A2 The format currently in use for timestamps is floating, field-b= ased, > and has a resolution/precision of minutes. > > What kinds of representations would a calendar system capable of > handling timezones require? > > =E2=80=A2 Instant (fixed) > =E2=80=A2 This is referring to an unambiguous moment in time > =E2=80=A2 e.g., 2007-02-03T05:00:00.000Z > =E2=80=A2 Offset (fixed) > =E2=80=A2 This captures the idea of "when did it happen for the person = who > made the observation" > =E2=80=A2 e.g., 2007-02-03T04:00:00.000+01:00 > =E2=80=A2 Instant with explicit offset and zone (fixed) > =E2=80=A2 e.g., 2007-01-01T02:00:00.000+01:00[America/Chicago] > =E2=80=A2 Zoned local date time (floating) > =E2=80=A2 Tricky, requires decisions about how to interpret timestamps = after > political changes. > =E2=80=A2 e.g., 2007-01-01T01:00:00.000[America/Chicago] Note that representing the time zone does not require second/sub-second precision. Let's not complicate the matters. Also, [time zone] is not compatible with Org's delimiting syntax. We need to come up with something else. > What would a roadmap be? > > =E2=80=A2 Design and implement an absolute and offset timestamp system > =E2=80=A2 Decide on a time scale UTC. Because we are bound by capabilities of Emacs time API. > =E2=80=A2 Decide on a format and syntax Sure. > =E2=80=A2 Implement instant timestamps > =E2=80=A2 Implement offeset timestamps This is all available via Emacs time API as implemented in glibc. Time db is supported. Time zone names are supported. Fixed offsets are also supported via TZ POSIX standard (https://www.gnu.org/software/libc/manual/html_node/TZ-Variable.html). > =E2=80=A2 Design and implement the time zone aware calendar system This i= s a > separate project. Org support for time zones is exactly the calendar system you are talking about. This will be most of the work TBD. > What format and syntax should Org use? > > A heretical suggestion: We should abandon the day of week abbreviation > and use a new format. As you saw from a number of replies, people do find week abbreviation useful. > The current format generates a three leter abbreviation of the day of > the week [2023-01-25 Wed 12:12]. I suggest supporting this as a > legacy/simple format but switch to a format/encoding like > [2023-01-25T15:13:42Z] for the new system. Specifically I'm advocating > for an extended ISO 8601 format, compatible with expanded dates and > Level 2 of the EDTF, with some (bracket?) notation surrounding it such > that Org can parse the syntax as a timestamp. I advocate further for the > use of durations and repeating intervals to follow the same standard > format. Thus instead of a range being formatted as: > > [2023-01-25 Wed 13:57]=E2=80=93[2023-01-26 Thu 13:57] > > it would be: > > [2023-01-25T16:57:42Z/2023-01-26T16:57:42Z]. As I (and others) replied, we will not do it. We aim for human-readable representation. Hence, while we can draw ideas from EDTF and ISO, we will not follow them precisely. > If the square bracket delimiter syntax is insufficient or too difficult > to parse unambiguously, we could just encapsulate the ISO format in a > sub-syntax (e.g., [&&(ISO format)] similar to the [%%(diary sexp)] > technique). This is ugly, but perhaps a stepping stone during > development to separate syntax parsing concerns from calculating etc. This would be a breaking change that will force all the libraries adapt. If we need supporting raw ISO syntax at any point, it can be simply made a subset of [%%(diary sexp)]. At the end, diary sexps are nothing but Elisp functions. We don't need to invent yet another syntax for ISO. > What are the advantages of switching to a standard format for the new > system? > ... > =E2=80=A2 We have a way of distinguishing new timestamps from legacy/simp= le ones > By making a change in syntax we reduce (or eliminate?) the possibility > of ambiguity between "which version" of a timestamp is being parsed. A > legacy timestamp can be treated as such, and new timestamps are easily > identified by the 'T' present instead of spaces, or in the delimiters > wrapping the representation. We should not introduce this problem to start with. The versions should be mutually compatible if at all possible. Note, for example, that <2023-01-30 Mon 14:00 @Europe/Berlin> does not even break the existing Org versions, except that time zone part is ignored. I am strongly against introducing a brand-new timestamp syntax and abandoning (or maintaining infinitely) the current one. Proliferation of multiple parallel syntax variants is something we really want to avoid maintenance-wise. > =E2=80=A2 We free ourselves from the constraints of the legacy timestamp = format > Trying to engineer a new syntax which also parses as an extension of > the legacy one is more complex and embeds things like "day of the > week" and the use of spaces as separators into this new system. Easier > to have two side by side. This should only be the last resort. We are not quite there. > =E2=80=A2 We can defer to existing parsing and calculating systems There = are > already programs written which support the ISO standard and EDTF. No. We can only defer the existing parsing to Elisp time API. Which does not support EDTF. And supporting approximate dates from EDTF would be hard to implement anyway - do we really need those? > =E2=80=A2 We can directly (or nearly directly) import the regular expre= ssions > and parsing mechanisms already written. It's not like these regular expressions are particularly complex. And we are not going to make them complex either way. > =E2=80=A2 These enable decent testing suites as we build the system, as= we can > check against existing packages to see if our parsing and > calculations agree. Or we can write a simple converter from Org syntax to ISO and do the same. Though I do not know any existing testing suite we could utilize here. > =E2=80=A2 Users who wish to use external libraries (irrespective of lan= guage > or license) can extract the new timestamp and parse or calculate > externally. I would not be so dramatic here. We do provide syntax description in https://orgmode.org/worg/org-syntax.html#Timestamps It is not a big deal to follow it. > =E2=80=A2 Org is part of a standard > =E2=80=A2 We are able to defer to experts and 35 years of knowledge rat= her > than debate among ourselves How so? > =E2=80=A2 Interfacing with other programs is simplified as the area ins= ide the > delimiter notation can be passed as a string without parsing. No. Org is not context-free. Parsing will be required anyway. > =E2=80=A2 New users and collaborators can be onboarded faster without n= eeding > to learn a new system Is it really so much decisive factor? I doubt so. > =E2=80=A2 Org documentation can refer to the standard instead of bearin= g the > burden of exposition. That's a plus. But not decisive one. > =E2=80=A2 The move to include time zones in the format is simplified > =E2=80=A2 The ISO standard has recently adopted a format for time zones= from > RFC3339 and JAVAZDT, we can adhere to 8601 and keep things > consistent. We have to follow POSIX TZ anyway. Maybe with minor additions. > What other perspectives should the new format support? > > In addition to the representations necessary for a timezone aware > calendar system, I suggest the new format be compatible with two other > representations: finer/ arbitrary resolution for scientific work, and > Level 2 of the Extended Date/Time Format for bibliographic and metadata > systems. Please frame it as a separate proposal. We have enough complexities with time zones to add this on top. > The Extended Data/Time Format (EDTF) was designed by the Library of > Congress to address limitations of the ISO standard for metadata and > archival purposes. A draft specification was created in 2012 and EDTF > functionality has now been integrated into ISO 8601-2019. Of great > interest is the ability to express the concepts of uncertainty and > approximation. Archival work includes scenarios where the precise date > may be unknown, so a format was created with qualifiers capable of > handling these situations. In the EDTF format '1984?' expresses possibly > the year 1984, but not definitely, while '2004-06~' expresses year-month > approximate. This format has been implemented by multiple library > systems and in 2021 Wikibase added an extension to support EDTF. That's all nice but what a headache will it be to implement. What will 2004-06~ mean for agenda, for example? In any case, it is also out of scope of this specific feature request. > The initial technical or code burden to support these perspectives is > minimal. Both can be parsed and calculated with by existing libraries, > and the functionality to actually calculate with them can be delayed. > The important thing is selecting a format which won't exclude them. No. They cannot. Not by the existing Elisp libraries, available in Emacs core. --=20 Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at