From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id YCa5M9FKsF6nRAAA0tVLHw (envelope-from ) for ; Mon, 04 May 2020 17:03:13 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id WE4eBN1KsF7qIgAAbx9fmQ (envelope-from ) for ; Mon, 04 May 2020 17:03:25 +0000 Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:470:142::17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 2D0FB944728 for ; Mon, 4 May 2020 16:57:10 +0000 (UTC) Received: from localhost ([::1]:45722 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jVeP0-0000Wb-9O for larch@yhetil.org; Mon, 04 May 2020 12:57:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50592) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jVeOc-0000WD-5Z for emacs-orgmode@gnu.org; Mon, 04 May 2020 12:56:46 -0400 Received: from ciao.gmane.io ([159.69.161.202]:42510) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jVeOa-0002th-Ng for emacs-orgmode@gnu.org; Mon, 04 May 2020 12:56:45 -0400 Received: from list by ciao.gmane.io with local (Exim 4.92) (envelope-from ) id 1jVeOV-000EYc-I8 for emacs-orgmode@gnu.org; Mon, 04 May 2020 18:56:39 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: emacs-orgmode@gnu.org From: Karl Voit Subject: Re: [patch suggestion] Mitigating the poor Emacs performance on huge org files: Do not use overlays for PROPERTY and LOGBOOK drawers Date: Mon, 4 May 2020 18:56:21 +0200 Organization: www.karl-voit.at Message-ID: <2020-05-04T18-47-10@devnull.Karl-Voit.at> References: <87h7x9e5jo.fsf@localhost> <875zdpia5i.fsf@nicolasgoaziou.fr> <87y2qi8c8w.fsf@localhost> User-Agent: slrn/pre1.0.0-18 (Linux) X-GPG-Key: http://www.Karl-Voit.at/Karl_Voit_GnuPG_public_key.gpg X-Registered-Linux-User: 224337 X-Confession: Pastafarian http://www.venganza.org/ Received-SPF: pass client-ip=159.69.161.202; envelope-from=geo-emacs-orgmode@m.gmane-mx.org; helo=ciao.gmane.io X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/04 11:11:53 X-ACL-Warn: Detected OS = Linux 3.11 and newer [fuzzy] X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.249, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Karl Voit Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Scanner: scn0 X-Spam-Score: -1.01 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 2001:470:142::17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Scan-Result: default: False [-1.01 / 13.00]; HAS_REPLYTO(0.00)[news1142@Karl-Voit.at]; GENERIC_REPUTATION(0.00)[-0.49497229700269]; DWL_DNSWL_BLOCKED(0.00)[2001:470:142::17:from]; R_SPF_ALLOW(-0.20)[+ip6:2001:470:142::/48:c]; REPLYTO_DN_EQ_FROM_DN(0.00)[]; TO_DN_NONE(0.00)[]; IP_REPUTATION_HAM(0.00)[asn: 22989(0.13), country: US(-0.00), ip: 2001:470:142::17(-0.49)]; HAS_ORG_HEADER(0.00)[]; MX_GOOD(-0.50)[cached: eggs.gnu.org]; MAILLIST(-0.20)[mailman]; FORGED_RECIPIENTS_MAILLIST(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:22989, ipnet:2001:470:142::/48, country:US]; MIME_TRACE(0.00)[0:+]; TAGGED_FROM(0.00)[larch=yhetil.org]; ARC_NA(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; FROM_NEQ_ENVFROM(0.00)[devnull@Karl-Voit.at,emacs-orgmode-bounces@gnu.org]; FROM_HAS_DN(0.00)[]; URIBL_BLOCKED(0.00)[karl-voit.at:url,reddit.com:url,gnu.org:url]; MIME_GOOD(-0.10)[text/plain]; REPLYTO_DOM_EQ_FROM_DOM(0.00)[]; DMARC_NA(0.00)[Karl-Voit.at]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; DNSWL_BLOCKED(0.00)[2001:470:142::17:from]; FORGED_SENDER_MAILLIST(0.00)[] X-TUID: Z87tsegu2hNH Hi Ihor, * Ihor Radchenko wrote: > > So far, I came up with the following partial solution searching and > showing hidden text. > > (defun org-find-text-property-region (pos prop) > (define-advice isearch-search-string (:after (&rest _) put-overlay) > (defun org-restore-invisibility-specs (&rest _) > (add-hook 'post-command-hook #'org-restore-invisibility-specs) > (defun org-flag-region (from to flag spec) > (defun org-unfontify-region (beg end &optional _maybe_loudly) After a couple of hours working with these patches, my feedback is very positive. Besides some visual glitches when creating a new heading with org-expiry-insinuate activated (which automatically adds :CREATED: properties), I could not detect any side-effect so far (will keep testing). The visual glitch looks like that: :PROPERTIES:X:CREATED: [2020-05-04 Mon 18>54] X ... with "X" being my character that symbolizes collapsed content. The way it looked without the patch was a simple collapsed property drawer. To me, this is acceptable considering the huge performance gain I got. THANK YOU VERY MUCH! I can't remember where I had this way of working within my large Org files[3] since ages. >> Anyway, the real fix should come from Emacs itself. There are ways to >> make overlays faster. These ways have already been discussed on the >> Emacs devel mailing list, but no one implemented them. It is a bit sad >> that we have to find workarounds for that. > > I guess that it is a very old story starting from the times when XEmacs > was a thing [1]. I recently heard about binary tree implementation of > overlays (there should be a branch in emacs git repo) [2], but there was > no update on that branch for a while. So, I do not have much hope on > Emacs implementing efficient overlay access in the near future. (And I > have problems with huge org files already). I can not express how this also reflects my personal situation. > [1] https://www.reddit.com/r/planetemacs/comments/e9lgwn/history_of_lucid_emacs_fsf_emacs_schism/ > [2] https://lists.gnu.org/archive/html/emacs-devel/2019-12/msg00323.html [3] https://karl-voit.at/2020/05/03/current-org-files -- get mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML into Org-mode: > get Memacs from https://github.com/novoid/Memacs < Personal Information Management > http://Karl-Voit.at/tags/pim/ Emacs-related > http://Karl-Voit.at/tags/emacs/