From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id kLm7HTU35F8nUAAA0tVLHw (envelope-from ) for ; Thu, 24 Dec 2020 06:37:41 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id uOZ+GTU35F9dFgAAbx9fmQ (envelope-from ) for ; Thu, 24 Dec 2020 06:37:41 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 0B118940276 for ; Thu, 24 Dec 2020 06:37:41 +0000 (UTC) Received: from localhost ([::1]:56680 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ksKFn-00067F-WD for larch@yhetil.org; Thu, 24 Dec 2020 01:37:40 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45402) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ksKE7-0005R3-Mo for emacs-orgmode@gnu.org; Thu, 24 Dec 2020 01:35:55 -0500 Received: from mail-wm1-x32f.google.com ([2a00:1450:4864:20::32f]:53629) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ksKE5-0000HC-Lo for emacs-orgmode@gnu.org; Thu, 24 Dec 2020 01:35:55 -0500 Received: by mail-wm1-x32f.google.com with SMTP id k10so783380wmi.3 for ; Wed, 23 Dec 2020 22:35:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=hif/5HWC9GFvfrlUUYbTTpLtnDh43KUxioNh0wlhx50=; b=NO+vETavonPK6c9EDrxiIh3BzNDOzWf9L5pZ4lP/vui9ZkMeMKOZgc8xd7rJegRaEj R4E8d+a6lExKxfGUoYbfZULQmRio5yt7GcpqX/Kx1QdJbA1EwIwu2JOrwxOWUKMPUyld qxSfTHhT1ouNAnnrGtJZof9zThN72ZO/FINiOpiSlKt+xAJCdn6WqokGQFR31RMOhyGp TKpYkA4IzAzAafjoSpFZImEQacVs4mbrT8nQsXyTQTisGaQGL2nNQy6LGpRSAiaQ/EfD 8eE5pZF3eks7+lhKRS7GFiKgFvYsJRVQbKX1qZ4LtU1nMdMPmtNXB17hWYVslTr+h3zb GiKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=hif/5HWC9GFvfrlUUYbTTpLtnDh43KUxioNh0wlhx50=; b=DiLIJFj/v24AYj6b/y56IYAOEPo3h2KpdZyo167eGADPk8f+s6bFqf0zTQoPjxfaXY qbFLH+5pRsgZxYxVH9sVs3pyhwOcbEaYP1aU9HGrrodgmLq/XhrP5cb73LOtIEzLoNeW BjKf/H1LEVdwN1BOIXOCFbTZfqJGh1rEGHYc8NRjgcNF7PUs9bM0PoBdluUuId8DC4hy c9Ktu4ykEqACuIgcdRTugsO7DbXntzko+jQCSK5QXFpw2DSsa3BftnLGbyNT7CZkBSl5 fZ1qiuFQ6Js1Qxy8bVHvtKd1Ie55HykaOeAo2pS1s9ZwB/qAi3Mft9jZc2e49NaAKyUL etow== X-Gm-Message-State: AOAM531kiub6542MnzjClk2flUPyr9RnRV0sC1T30V5v6Yr85ImmG+8m ozmXvj3THcoc4YH+gEYwrfyCTiozEqbRkaDLGtM= X-Google-Smtp-Source: ABdhPJwHajFW6yZ1MVto9hzZlfuZ3QTae940NIKU35VUmg1nIh2HL1zj8w/qkdHtP9gs6b2wW8oZEWaj48MEa9bR4h4= X-Received: by 2002:a7b:c088:: with SMTP id r8mr2877802wmh.45.1608791751818; Wed, 23 Dec 2020 22:35:51 -0800 (PST) MIME-Version: 1.0 References: <505431.1608780887@apollo2.minshall.org> In-Reply-To: <505431.1608780887@apollo2.minshall.org> From: Tom Gillespie Date: Thu, 24 Dec 2020 01:35:40 -0500 Message-ID: Subject: Re: did behaviour of RET change again? To: Greg Minshall Content-Type: text/plain; charset="UTF-8" Received-SPF: pass client-ip=2a00:1450:4864:20::32f; envelope-from=tgbugs@gmail.com; helo=mail-wm1-x32f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Emacs Org mode mailing list Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -0.53 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=NO+vETav; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Queue-Id: 0B118940276 X-Spam-Score: -0.53 X-Migadu-Scanner: scn0.migadu.com X-TUID: JKVVPi+zNGHX > possibly i'm misunderstanding, but my sense is that the value of org > adapt indentation doesn't change what you might actually find ("in a > .org file in the wild"). so, whatever its value, your grammar would > have to deal with all cases? Yep, we can't magically change all the files out in the wild. Until I wrote this email out I agreed about the grammar too, but as it turns out I think there might be a compromise, which is that a grammar could be allowed to interpret certain types of lines with leading whitespace as if they were paragraph lines instead of as say, the start of an org-babel block. That way an interoperable org spec could be made simpler without preventing e.g. the elisp implementation from going above and beyond the spect to provide support for leading whitespace. My interest in the org-adapt-indentation setting is to try to align what most users are doing out in the wild with a representation for their org files that is less ambiguous and more performant (among other things). If I had to guess I would say that most new org files have leading whitespace precisely because org-adapt-indentation is set to t by default. Getting users to transition their files requires an incentive, and some users may find that they use org in such a way that they don't need to. While writing this email I thought of a reasonable incentive, which is that only files without leading significant whitespace (i.e. that look like those that are authored with org-adapt-indentation nil) have specified behavior for things like org-babel blocks. This would allow best effort by the elisp org-mode implementation to allow users who don't care about interoperability to continue as they have been doing, while also providing clear guidance for what users must do if they want consistent behavior on other tools that consume org formatted files. This is critical for keeping the org spec from becoming overly complex. The first step would thus be to reduce the rate at which new org files are created with leading whitespace by changing org-adapt-indentation to be nil by default. The second step would be to give users a clear way forward to safely normalizing their files. Org has a habit of reindenting subsets of files from time to time, but in general doing a complete switch to have no significant whitespace can be risky. The third step would be to let the incentives and needs of users play out over time, but users would generally probably be happier because by default they would be in the "my files are portable and generally interpretable without me having to do any additional work" state instead of the "why did no one tell me the defaults weren't portable" state. > or, and maybe this would make sense, you'd define an "interoperability" > form of .org, that all "wild" .org files could be (programmatically) > converted into, without losing any of their semantics? Yep exactly. For many use cases the cost of stripping the whitespace that corresponds to heading level is not unreasonable, e.g. since it would only have to be done once. However, if you are writing another org implementation, then every single time you parse a heading and its accompanying section you have to strip the whitespace, and that will happen every single time a user modifies the file, which starts to get expensive. Alternately you could implement it so that everything is stripped once and then you keep a version in memory that the user edits which doesn't have leading whitespace, but then every time you save you have to splice it all back in to preserve the roundtrip. This also doesn't even begin to deal with the fact that users can create malformed org files that can have lines that are less than the expected significant indentation. This becomes a complete nightmare when trying to come up with some rational way of dealing with org babel blocks for languages like python where there is significant whitespace. Consider the horror if trying to specify the correct behavior in a situation like this. Better just to declare it a malformed babel block and move on. Unfortunately you can't always detect that such things are malformed during the first pass of parsing because you have to count the number of spaces. #+begin_src org # -*- org-adapt-indentation: t -*- ,*** Oh No ,#+begin_src python class What: have = 'you' done = 'now' ,#+end_src #+end_src In order to ensure that org files can be reliably interpreted this either means that the specification for handling cases like this becomes extremely complex, or the spec can say "you can have leading whitespace, but nasal demons may come for you, there is only specified behavior if you do not have leading whitespace." Having unspecified behavior in cases like that would mean that an implementation could be fully compliant if it were to treat any #+begin_src lang line that started with whitespace as if it were a paragraph instead of a babel block. I suspect that this will be the best way forward. > (anyway, good luck with that, even with any significant subset of that!) Thanks, and thanks for the inspiration! Tom