From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id UBbMCejz2V5EWwAA0tVLHw (envelope-from ) for ; Fri, 05 Jun 2020 07:27:36 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id 0OeNBejz2V42JgAAbx9fmQ (envelope-from ) for ; Fri, 05 Jun 2020 07:27:36 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 79DEE94051C for ; Fri, 5 Jun 2020 07:27:35 +0000 (UTC) Received: from localhost ([::1]:35548 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jh6lJ-0003F9-4w for larch@yhetil.org; Fri, 05 Jun 2020 03:27:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53648) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jh6kR-0003Ew-FD for emacs-orgmode@gnu.org; Fri, 05 Jun 2020 03:26:39 -0400 Received: from relay10.mail.gandi.net ([217.70.178.230]:43415) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jh6kP-0002Ui-OO for emacs-orgmode@gnu.org; Fri, 05 Jun 2020 03:26:38 -0400 Received: from localhost (40-67.ipv4.commingeshautdebit.fr [185.131.40.67]) (Authenticated sender: admin@nicolasgoaziou.fr) by relay10.mail.gandi.net (Postfix) with ESMTPSA id C918024000C; Fri, 5 Jun 2020 07:26:32 +0000 (UTC) From: Nicolas Goaziou To: Ihor Radchenko Subject: Re: [patch suggestion] Mitigating the poor Emacs performance on huge org files: Do not use overlays for PROPERTY and LOGBOOK drawers References: <87h7x9e5jo.fsf@localhost> <875zdpia5i.fsf@nicolasgoaziou.fr> <87y2qi8c8w.fsf@localhost> <87r1vu5qmc.fsf@nicolasgoaziou.fr> <87imh5w1zt.fsf@localhost> <87blmxjckl.fsf@localhost> <87y2q13tgs.fsf@nicolasgoaziou.fr> <878si1j83x.fsf@localhost> <87d07bzvhd.fsf@nicolasgoaziou.fr> <87imh34usq.fsf@localhost> <87pnbby49m.fsf@nicolasgoaziou.fr> <87tv0efvyd.fsf@localhost> <874kse1seu.fsf@localhost> <87r1vhqpja.fsf@nicolasgoaziou.fr> <87tv0d2nk7.fsf@localhost> <87o8qkhy3g.fsf@nicolasgoaziou.fr> <87sgfqu5av.fsf@localhost> <87sgfn6qpc.fsf@nicolasgoaziou.fr> <87367d4ydc.fsf@localhost> Mail-Followup-To: Ihor Radchenko , emacs-orgmode@gnu.org Date: Fri, 05 Jun 2020 09:26:31 +0200 In-Reply-To: <87367d4ydc.fsf@localhost> (Ihor Radchenko's message of "Tue, 02 Jun 2020 17:21:03 +0800") Message-ID: <87r1uuotw8.fsf@nicolasgoaziou.fr> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=217.70.178.230; envelope-from=mail@nicolasgoaziou.fr; helo=relay10.mail.gandi.net X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/05 03:26:33 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: emacs-orgmode@gnu.org Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Spam-Score: -1.01 X-TUID: vBpPsLEY+ah4 Hello, Ihor Radchenko writes: > [The patch itself will be provided in the following email] Thank you. > I have found char-property-alias-alist variable that controls how Emacs > calculates text property value if the property is not set. This variable > can be buffer-local, which allows independent 'invisible states in > different buffers. Great. I didn't know about this variable! > All the implementation stays in > org--get-buffer-local-text-property-symbol, which takes care about > generating unique property name and mapping it to 'invisible (or any > other) text property. See also `gensym'. Do we really need to use it for something else than `invisible'? If not, the tool doesn't need to be generic. > I simplified the code as suggested, without using pairs of before- and > after-change-functions. Great! > Handling text inserted into folded/invisible region is handled by a > simple after-change function. After testing, it turned out that simple > re-hiding text based on 'invisible property of the text before/after the > inserted region works pretty well. OK, but this may not be sufficient if we want to do slightly better than overlays in that area. This is not mandatory, though. > Modifications to BEGIN/END line of the drawers and blocks is handled via > 'modification-hooks + 'insert-behind-hooks text properties (there is no > after-change-functions analogue for text properties in Emacs). The > property is applied during folding and the modification-hook function is > made aware about the drawer/block boundaries (via apply-partially > passing element containing :begin :end markers for the current > drawer/block). Passing the element boundary is important because the > 'modification-hook will not directly know where it belongs to. Only the > modified region (which can be larger than the drawer) is passed to the > function. In the worst case, the region can be the whole buffer (if one > runs revert-buffer). As discussed before, I don't think you need to use `modification-hooks' or `insert-behind-hooks' if you already use `after-change-functions'. `after-change-functions' are also triggered upon text properties changes. So, what is the use case for the other hooks? > It turned out that adding 'modification-hook text property takes a > significant cpu time (partially, because we need to take care about > possible existing 'modification-hook value, see > org--add-to-list-text-property). For now, I decided to not clear the > modification hooks during unfolding because of poor performance. > However, this approach would lead to partial unfolding in the following > case: > > :asd: > :drawer: > lksjdfksdfjl > sdfsdfsdf > :end: > > If :asd: was inserted in front of folded :drawer:, changes in :drawer: > line of the new folded :asd: drawer would reveal the text between > :drawer: and :end:. > > Let me know what you think on this. I have first to understand the use case for `modification-hook'. But I think unfolding is the right thing to do in this situation, isn't it? > My simplified implementation of element boundary parser > (org--get-element-region-at-point) appears to be much faster and also > uses much less memory in comparison with org-element-at-point. > Moreover, not all the places where org-element-at-point is called > actually need the full parsed element. For example, org-hide-drawer-all, > org-hide-drawer-toggle, org-hide-block-toggle, and > org--hide-wrapper-toggle only need element type and some information > about the element boundaries - the information we can get from > org--get-element-region-at-point. [...] > What do you think about the idea of making use of > org--get-element-region-at-point in org code base? `org--get-element-region-at-point' is certainly faster, but it is also wrong, unfortunately. Org syntax is not context-free grammar. If you try to parse it locally, starting from anywhere, it will fail at some point. For example, your function would choke in the following case: [fn:1] Def1 #+begin_something [fn:2] Def2 #+end_something AFAIK, the only proper way to parse it is to start from a known position in the buffer. If you have no information about the buffer, the headline above is the position you want. With cache could help to start below. Anyway, in this particular case, you should not use `org--get-element-region-at-point'. Hopefully, we don't need to parse anything. In an earlier message, I suggested a few checks to make on the modified text in order to decide if something should be unfolded, or not. I suggest to start from there, and fix any shortcomings we might encounter. We're replacing overlays: low-level is good in this area. WDYT? Regards, -- Nicolas Goaziou