From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id kCyfMPpWuF6aawAA0tVLHw (envelope-from ) for ; Sun, 10 May 2020 19:33:14 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id 2HFUGwhXuF47cAAAbx9fmQ (envelope-from ) for ; Sun, 10 May 2020 19:33:28 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 25323940C73 for ; Sun, 10 May 2020 19:33:26 +0000 (UTC) Received: from localhost ([::1]:39040 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jXrhW-000539-JY for larch@yhetil.org; Sun, 10 May 2020 15:33:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51172) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jXrh5-00051N-SB for emacs-orgmode@gnu.org; Sun, 10 May 2020 15:32:59 -0400 Received: from relay10.mail.gandi.net ([217.70.178.230]:36051) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jXrh3-000365-St for emacs-orgmode@gnu.org; Sun, 10 May 2020 15:32:59 -0400 Received: from localhost (40-67.ipv4.commingeshautdebit.fr [185.131.40.67]) (Authenticated sender: admin@nicolasgoaziou.fr) by relay10.mail.gandi.net (Postfix) with ESMTPSA id E4FB4240008; Sun, 10 May 2020 19:32:53 +0000 (UTC) From: Nicolas Goaziou To: Ihor Radchenko Subject: Re: [patch suggestion] Mitigating the poor Emacs performance on huge org files: Do not use overlays for PROPERTY and LOGBOOK drawers References: <87h7x9e5jo.fsf@localhost> <875zdpia5i.fsf@nicolasgoaziou.fr> <87y2qi8c8w.fsf@localhost> <87r1vu5qmc.fsf@nicolasgoaziou.fr> <87imh5w1zt.fsf@localhost> <87blmxjckl.fsf@localhost> <87y2q13tgs.fsf@nicolasgoaziou.fr> <878si1j83x.fsf@localhost> <87d07bzvhd.fsf@nicolasgoaziou.fr> <87imh34usq.fsf@localhost> Mail-Followup-To: Ihor Radchenko , emacs-orgmode@gnu.org Date: Sun, 10 May 2020 21:32:53 +0200 In-Reply-To: <87imh34usq.fsf@localhost> (Ihor Radchenko's message of "Mon, 11 May 2020 00:30:13 +0800") Message-ID: <87pnbby49m.fsf@nicolasgoaziou.fr> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=217.70.178.230; envelope-from=mail@nicolasgoaziou.fr; helo=relay10.mail.gandi.net X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/10 14:41:44 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: emacs-orgmode@gnu.org Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Scanner: scn0 X-Spam-Score: -1.01 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Scan-Result: default: False [-1.01 / 13.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; GENERIC_REPUTATION(0.00)[-0.53905972596421]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.51.188.0/24:c]; IP_REPUTATION_HAM(0.00)[asn: 22989(0.08), country: US(-0.00), ip: 209.51.188.17(-0.54)]; DWL_DNSWL_FAIL(0.00)[209.51.188.17:server fail]; MX_GOOD(-0.50)[cached: eggs.gnu.org]; RCPT_COUNT_TWO(0.00)[2]; MAILLIST(-0.20)[mailman]; FREEMAIL_TO(0.00)[gmail.com]; FORGED_RECIPIENTS_MAILLIST(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:22989, ipnet:209.51.188.0/24, country:US]; TAGGED_FROM(0.00)[larch=yhetil.org]; FROM_NEQ_ENVFROM(0.00)[mail@nicolasgoaziou.fr,emacs-orgmode-bounces@gnu.org]; ARC_NA(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[nicolasgoaziou.fr]; HAS_LIST_UNSUB(-0.01)[]; DNSWL_BLOCKED(0.00)[209.51.188.17:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.51.188.17:from]; FORGED_SENDER_MAILLIST(0.00)[] X-TUID: cKpgP3a2U4I7 Ihor Radchenko writes: > This should be better: > https://gist.github.com/yantar92/e37c2830d3bb6db8678b14424286c930 Thank you. > This might get tricky in the following case: > > :PROPERTIES: > :CREATED: [2020-04-13 Mon 22:31] > > :SHOWFROMDATE: 2020-05-11 > :ID: e05e3b33-71ba-4bbc-abba-8a92c565ad34 > :END: > > > > :PROPERTIES: > :CREATED: [2020-04-27 Mon 13:50] > > :ID: b2eef49f-1c5c-4ff1-8e10-80423c8d8532 > :ATTACH_DIR_INHERIT: t > :END: > > If the text in the region is replaced by something else, in between> should not be fully hidden. We cannot simply look at the > 'invisible property before and after the changed region. Be careful: "replaced by something else" is ambiguous. "Replacing" is an unlikely change: you would need to do (setf (buffer-substring x y) "foo") We can safely assume this will not happen. If it does, we can accept the subsequent glitch. Anyway it is less confusing to think in terms of deletion and insertion. In the case above, you probably mean "the region is deleted then something else is inserted", or the other way. So there are two actions going on, i.e., `after-change-functions' are called twice. In particular the situation you foresee /cannot happen/ with an insertion. Text is inserted at a single point. Let's assume this is in the first drawer. Once inserted, both text before and after the new text were part of the same drawer. Insertion introduces other problems, though. More on this below. It is true the deletion can produce the situation above. But in this case, there is nothing to do, you just merged two drawers into a single one, which stays invisible. Problem solved. IOW, big changes like the one you describe are not an issue. I think the "check if previous and next parts match" trick gives us roughly the same functionality, and the same glitches, as overlays. However, I think we can do better than that, and also fix the glitches from overlays. Here are two of them. Write the following drawer: :FOO: bar :END: fold it and delete the ":f". The overlay is still there, and you cannot remove it with TAB any more. Or, with the same initial drawer, from beginning of buffer, evaluate: (progn (re-search-forward ":END:") (replace-match "")) This is no longer a drawer: you just removed its closing line. Yet, the overlay is still there, and TAB is ineffective. Here's an idea to develop that would make folding more robust, and still fast. Each syntactical element has a "sensitive part", a particular area that can change the nature of the element when it is altered. Luckily drawers (and blocks) are sturdy. For a drawer, there are three things to check: 1. the opening line must match org-drawer-regexp 2. the closing line must match org-property-end-re (case ignored) 3. between those, you must not insert text match org-property-end-re or org-outline-regexp-bol Obviously, point 3 needs not be checked during deletion. Instead of `after-change-functions', we may use `modification-hooks' for deletions, and `insert-behind-hooks' for insertions. For example, we might add modification-hooks property to both opening and closing line, and `insert-behind-hooks' on all the drawer. If any of the 3 points above is verified, we remove all properties. Note that if we can implement something robust with text properties, we might use them for headlines too, for another significant speed-up. WDYT? > I think that using fontification (something similar to > org-fontify-drawers) instead of after-change-functions should be > faster. I don't think it would be faster. With `after-change-functions', `modification-hooks' or `insert-behind-hook', we know exactly where the change happened. Fontification is fuzzier. It is not instantaneous either. It is an option only if we cannot do something fast and accurate with `after-change-functions', IMO.