From: Ihor Radchenko <email@example.com> To: Nicolas Goaziou <firstname.lastname@example.org> Cc: email@example.com Subject: Re: [patch suggestion] Mitigating the poor Emacs performance on huge org files: Do not use overlays for PROPERTY and LOGBOOK drawers Date: Mon, 27 Apr 2020 00:04:15 +0800 [thread overview] Message-ID: <87y2qi8c8w.fsf@localhost> (raw) In-Reply-To: <firstname.lastname@example.org> > You cannot. You may however mimic it with `cursor-sensor-functions' text > property. These assume Cursor Sensor minor mode is active, tho. > I haven't tested it, but I assume it would slow down text properties > a bit, too, but hopefully not as much as overlays. Unfortunately, isearch sets inhibit-point-motion-hooks to non-nil internally. Anyway, I came up with some workaround, which seems to work (see below). Though it would be better if isearch supported hidden text in addition to overlays. > Missing `isearch-open-invisible' is a deal breaker, IMO. It may be worth > experimenting with `cursor-sensor-functions'. So far, I came up with the following partial solution searching and showing hidden text. ;; Unfortunately isearch, sets inhibit-point-motion-hooks and we ;; cannot even use cursor-sensor-functions as a workaround ;; I used a less ideas approach with advice to isearch-search-string as ;; a workaround (defun org-find-text-property-region (pos prop) "Find a region containing PROP text property around point POS." (require 'org-macs) ;; org-with-point-at (org-with-point-at pos (let* ((beg (and (get-text-property pos prop) pos)) (end beg)) (when beg (setq beg (or (previous-single-property-change pos prop) beg)) (setq end (or (next-single-property-change pos prop) end)) (unless (equal beg end) (cons beg end)))))) ;; :FIXME: re-hide properties when point moves away (define-advice isearch-search-string (:after (&rest _) put-overlay) "Reveal hidden text at point." (when-let ((region (org-find-text-property-region (point) 'invisible))) (with-silent-modifications (put-text-property (car region) (cdr region) 'org-invisible (get-text-property (point) 'invisible))) (remove-text-properties (car region) (cdr region) '(invisible nil)))) ;; this seems to be unstable, but I cannot figure out why (defun org-restore-invisibility-specs (&rest _) "" (let ((pos (point-min))) (while (< (setq pos (next-single-property-change pos 'org-invisible nil (point-max))) (point-max)) (when-let ((region (org-find-text-property-region pos 'org-invisible))) (with-silent-modifications (put-text-property (car region) (cdr region) 'invisible (get-text-property pos 'org-invisible)) (remove-text-properties (car region) (cdr region) '(org-invisible nil))))))) (add-hook 'post-command-hook #'org-restore-invisibility-specs) (defun org-flag-region (from to flag spec) "Hide or show lines from FROM to TO, according to FLAG. SPEC is the invisibility spec, as a symbol." (pcase spec ('outline (remove-overlays from to 'invisible spec) ;; Use `front-advance' since text right before to the beginning of ;; the overlay belongs to the visible line than to the contents. (when flag (let ((o (make-overlay from to nil 'front-advance))) (overlay-put o 'evaporate t) (overlay-put o 'invisible spec) (overlay-put o 'isearch-open-invisible #'delete-overlay)))) (_ (with-silent-modifications (remove-text-properties from to '(invisible nil)) (when flag (put-text-property from to 'invisible spec) ))))) ;; This normally deletes invisible text property. We do not want this now. (defun org-unfontify-region (beg end &optional _maybe_loudly) "Remove fontification and activation overlays from links." (font-lock-default-unfontify-region beg end) (let* ((buffer-undo-list t) (inhibit-read-only t) (inhibit-point-motion-hooks t) (inhibit-modification-hooks t) deactivate-mark buffer-file-name buffer-file-truename) (decompose-region beg end) (remove-text-properties beg end '(mouse-face t keymap t org-linked-text t ;; Do not remove invisible during fontification ;; invisible t intangible t org-emphasis t)) (org-remove-font-lock-display-properties beg end))) > Anyway, the real fix should come from Emacs itself. There are ways to > make overlays faster. These ways have already been discussed on the > Emacs devel mailing list, but no one implemented them. It is a bit sad > that we have to find workarounds for that. I guess that it is a very old story starting from the times when XEmacs was a thing . I recently heard about binary tree implementation of overlays (there should be a branch in emacs git repo) , but there was no update on that branch for a while. So, I do not have much hope on Emacs implementing efficient overlay access in the near future. (And I have problems with huge org files already).  https://www.reddit.com/r/planetemacs/comments/e9lgwn/history_of_lucid_emacs_fsf_emacs_schism/  https://lists.gnu.org/archive/html/emacs-devel/2019-12/msg00323.html Nicolas Goaziou <email@example.com> writes: > Hello, > > Ihor Radchenko <firstname.lastname@example.org> writes: > >> To my surprise, the patch did not break org to unusable state and >> the performance on the sample org file  improved drastically. You can >> try by yourself! > > It is not a surprise, really. Text properties are much faster than > overlays, and very close to them features-wise. They are a bit more > complex to handle, however. > >> However, this did introduce some visual glitches with drawer display. >> Though drawers can still be folded/unfolded with <tab>, they are not >> folded on org-mode startup for some reason (can be fixed by running >> (org-cycle-hide-drawers 'all)). Also, some drawers (or parts of drawers) >> are unfolded for no apparent reason sometimes. A blind guess is that it >> is something to do with lack of 'isearch-open-invisible, which I am not >> sure how to set via text properties. > > You cannot. You may however mimic it with `cursor-sensor-functions' text > property. These assume Cursor Sensor minor mode is active, tho. > I haven't tested it, but I assume it would slow down text properties > a bit, too, but hopefully not as much as overlays. > > Note there are clear advantages using text properties. For example, when > you move contents around, text properties are preserved. So there's no > more need for the `org-cycle-hide-drawer' dance, i.e., it is not > necessary anymore to re-hide drawers. > >> Any thoughts about the use of text properties or about the patch >> suggestion are welcome. > > Missing `isearch-open-invisible' is a deal breaker, IMO. It may be worth > experimenting with `cursor-sensor-functions'. > > We could also use text properties for property drawers, and overlays for > regular ones. This might give us a reasonable speed-up with an > acceptable feature trade-off. > > Anyway, the real fix should come from Emacs itself. There are ways to > make overlays faster. These ways have already been discussed on the > Emacs devel mailing list, but no one implemented them. It is a bit sad > that we have to find workarounds for that. > > Regards, > > -- > Nicolas Goaziou -- Ihor Radchenko, PhD, Center for Advancing Materials Performance from the Nanoscale (CAMP-nano) State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong University, Xi'an, China Email: email@example.com, firstname.lastname@example.org
next prev parent reply other threads:[~2020-04-26 16:08 UTC|newest] Thread overview: 88+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-04-24 6:55 Ihor Radchenko 2020-04-24 8:02 ` Nicolas Goaziou 2020-04-25 0:29 ` stardiviner 2020-04-26 16:04 ` Ihor Radchenko [this message] 2020-05-04 16:56 ` Karl Voit 2020-05-07 7:18 ` Karl Voit 2020-05-09 15:43 ` Ihor Radchenko 2020-05-07 11:04 ` Christian Heinrich 2020-05-09 15:46 ` Ihor Radchenko 2020-05-08 16:38 ` Nicolas Goaziou 2020-05-09 13:58 ` Nicolas Goaziou 2020-05-09 16:22 ` Ihor Radchenko 2020-05-09 17:21 ` Nicolas Goaziou 2020-05-10 5:25 ` Ihor Radchenko 2020-05-10 9:47 ` Nicolas Goaziou 2020-05-10 13:29 ` Ihor Radchenko 2020-05-10 14:46 ` Nicolas Goaziou 2020-05-10 16:21 ` Ihor Radchenko 2020-05-10 16:38 ` Nicolas Goaziou 2020-05-10 17:08 ` Ihor Radchenko 2020-05-10 19:38 ` Nicolas Goaziou 2020-05-09 15:40 ` Ihor Radchenko 2020-05-09 16:30 ` Ihor Radchenko 2020-05-09 17:32 ` Nicolas Goaziou 2020-05-09 18:06 ` Ihor Radchenko 2020-05-10 14:59 ` Nicolas Goaziou 2020-05-10 15:15 ` Kyle Meyer 2020-05-10 16:30 ` Ihor Radchenko 2020-05-10 19:32 ` Nicolas Goaziou 2020-05-12 10:03 ` Nicolas Goaziou 2020-05-17 15:00 ` Ihor Radchenko 2020-05-17 15:40 ` Ihor Radchenko 2020-05-18 14:35 ` Nicolas Goaziou 2020-05-18 16:52 ` Ihor Radchenko 2020-05-19 13:07 ` Nicolas Goaziou 2020-05-23 13:52 ` Ihor Radchenko 2020-05-23 13:53 ` Ihor Radchenko 2020-05-23 15:26 ` Ihor Radchenko 2020-05-26 8:33 ` Nicolas Goaziou 2020-06-02 9:21 ` Ihor Radchenko 2020-06-02 9:23 ` Ihor Radchenko 2020-06-02 12:10 ` Bastien 2020-06-02 13:12 ` Ihor Radchenko 2020-06-02 13:23 ` Bastien 2020-06-02 13:30 ` Ihor Radchenko 2020-06-02 9:25 ` Ihor Radchenko 2020-06-05 7:26 ` Nicolas Goaziou 2020-06-05 8:18 ` Ihor Radchenko 2020-06-05 13:50 ` Nicolas Goaziou 2020-06-08 5:05 ` Ihor Radchenko 2020-06-08 5:06 ` Ihor Radchenko 2020-06-08 5:08 ` Ihor Radchenko 2020-06-10 17:14 ` Nicolas Goaziou 2020-06-21 9:52 ` Ihor Radchenko 2020-06-21 15:01 ` Nicolas Goaziou 2020-08-11 6:45 ` Ihor Radchenko 2020-08-11 23:07 ` Kyle Meyer 2020-08-12 6:29 ` Ihor Radchenko 2020-09-20 5:53 ` Ihor Radchenko 2020-09-20 11:45 ` Kévin Le Gouguec 2020-09-22 9:05 ` Ihor Radchenko 2020-09-22 10:00 ` Ihor Radchenko 2020-09-23 6:16 ` Kévin Le Gouguec 2020-09-23 6:48 ` Ihor Radchenko 2020-09-23 7:09 ` Bastien 2020-09-23 7:30 ` Ihor Radchenko 2020-09-24 18:07 ` Kévin Le Gouguec 2020-09-25 2:16 ` Ihor Radchenko 2020-12-15 17:38 ` [9.4] Fixing logbook visibility during isearch Kévin Le Gouguec 2020-12-16 3:15 ` Ihor Radchenko 2020-12-16 18:05 ` Kévin Le Gouguec 2020-12-17 3:18 ` Ihor Radchenko 2020-12-17 14:50 ` Kévin Le Gouguec 2020-12-18 2:23 ` Ihor Radchenko 2020-12-24 23:37 ` Kévin Le Gouguec 2020-12-25 2:51 ` Ihor Radchenko 2020-12-25 10:59 ` Kévin Le Gouguec 2020-12-25 12:32 ` Ihor Radchenko 2020-12-25 21:35 ` Kévin Le Gouguec 2020-12-26 4:14 ` Ihor Radchenko 2020-12-26 11:44 ` Kévin Le Gouguec 2020-12-26 12:22 ` Ihor Radchenko 2020-12-04 5:58 ` [patch suggestion] Mitigating the poor Emacs performance on huge org files: Do not use overlays for PROPERTY and LOGBOOK drawers Ihor Radchenko 2021-03-21 9:09 ` Ihor Radchenko 2021-05-03 17:28 ` Bastien 2021-09-21 13:32 ` Timothy 2021-10-26 17:25 ` Matt Price 2021-10-27 6:27 ` Ihor Radchenko
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: https://www.orgmode.org/ * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=87y2qi8c8w.fsf@localhost \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --subject='Re: [patch suggestion] Mitigating the poor Emacs performance on huge org files: Do not use overlays for PROPERTY and LOGBOOK drawers' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Code repositories for project(s) associated with this inbox: https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).