From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id MIypOrvQtl6hBgAA0tVLHw (envelope-from ) for ; Sat, 09 May 2020 15:48:11 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id 6Ri8BsnQtl7BcgAA1q6Kng (envelope-from ) for ; Sat, 09 May 2020 15:48:25 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id DE018940997 for ; Sat, 9 May 2020 15:48:22 +0000 (UTC) Received: from localhost ([::1]:36746 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jXRiA-0008F3-J3 for larch@yhetil.org; Sat, 09 May 2020 11:48:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46166) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jXRhm-0008Ee-W0 for emacs-orgmode@gnu.org; Sat, 09 May 2020 11:47:59 -0400 Received: from mail-pg1-x52f.google.com ([2607:f8b0:4864:20::52f]:35878) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jXRhk-0007Zn-RC for emacs-orgmode@gnu.org; Sat, 09 May 2020 11:47:58 -0400 Received: by mail-pg1-x52f.google.com with SMTP id d22so2344159pgk.3 for ; Sat, 09 May 2020 08:47:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:in-reply-to:references:date:message-id:mime-version; bh=oqV+kcYplo8RLPpfGePdPAWF1leKSkSftNRMnTi5YP8=; b=u2Tvi2lBgPqpOohJwcd3KkqeoPOqG8fmfj39OFjMfjlknHOlGcuirQrEZP4ArU3drA fMmcUMvVdgWgwKoBScxlYG+LBn+OiBH305IaL3mdN1EmCDK1sIJ7oWEVCrs+3oqF1JyK EUH1bBnSTNpTuj7ciNDzFAzK0VDeNEDVvVV7/YpTRKlCzd3yGWM3rzhooWf582JCL1x6 ernzNQ55Qfatigm2vzRjrx4WHPnFNluTjzejLqOX6rLK/Qb9n15E2kgXtFFS7FedE0sF IHCD0UyUnRSDq5F8LcXeCqfjgftuuJEC3k8He4Nce2u0bvpQ5yY81lLfkCuihjbyoK/3 1yOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:in-reply-to:references:date :message-id:mime-version; bh=oqV+kcYplo8RLPpfGePdPAWF1leKSkSftNRMnTi5YP8=; b=OAW8daUnIDC17VseDVW/k+H9tLB9kZYbrjMHeGBd9WealwKN5ojRUzGG2GjVC7D4Le daCYbCI+BSADEpXTmphD6Q1hvG3I5581HJo8aFnjL32xME0hsJ/3icqEHqarwMj05SMS Cwk0p0iTQ9NRX4usAIwRvo7lmlBeZFgMCGY3ZahFWsVxWx57AI3gUSisPCTzc7vdHj8a avYb/X79oYXYmU9gCouEBLyz+aodYNJi9pErc88s8HUJ9nZqgteckqUEpp2pCyZ3N/Du /+eHlo2EhrHXViBJNzyogY/MCmyfVhYawV2uKalXBUlZJY0WAeTGNjtzefwkeinuvi4/ Vh9g== X-Gm-Message-State: AGi0PuaWUHimGpGDwlm5hVtZJJxoNlBaKKa9JXxl+uoJPRtjwQra9zVZ 21sHHxfA5C491t5WAbUvuavncI36sniXdA== X-Google-Smtp-Source: APiQypI6K475ngYly6m12YHAno5DAPx/o9TEceGIHnfbB3AreNPZh1CNh4g/ulP0HuF6cbStv3Uvkg== X-Received: by 2002:a62:764b:: with SMTP id r72mr8407310pfc.207.1589039274384; Sat, 09 May 2020 08:47:54 -0700 (PDT) Received: from localhost ([210.3.160.222]) by smtp.gmail.com with ESMTPSA id f30sm5204486pje.29.2020.05.09.08.47.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 09 May 2020 08:47:53 -0700 (PDT) From: Ihor Radchenko To: Karl Voit , emacs-orgmode@gnu.org Subject: Re: [patch suggestion] Mitigating the poor Emacs performance on huge org files: Do not use overlays for PROPERTY and LOGBOOK drawers In-Reply-To: <2020-05-04T18-47-10@devnull.Karl-Voit.at> References: <87h7x9e5jo.fsf@localhost> <875zdpia5i.fsf@nicolasgoaziou.fr> <87y2qi8c8w.fsf@localhost> <2020-05-04T18-47-10@devnull.Karl-Voit.at> Date: Sat, 09 May 2020 23:43:36 +0800 Message-ID: <87ftc9w1uf.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=2607:f8b0:4864:20::52f; envelope-from=yantar92@gmail.com; helo=mail-pg1-x52f.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Scanner: scn0 X-Spam-Score: -1.21 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=u2Tvi2lB; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Scan-Result: default: False [-1.21 / 13.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; GENERIC_REPUTATION(0.00)[-0.54020104588272]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.51.188.0/24:c]; FREEMAIL_FROM(0.00)[gmail.com]; DWL_DNSWL_BLOCKED(0.00)[209.51.188.17:from,gmail.com:dkim]; IP_REPUTATION_HAM(0.00)[asn: 22989(0.09), country: US(-0.00), ip: 209.51.188.17(-0.54)]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.50)[cached: eggs.gnu.org]; MAILLIST(-0.20)[mailman]; FORGED_RECIPIENTS_MAILLIST(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:22989, ipnet:209.51.188.0/24, country:US]; TAGGED_FROM(0.00)[larch=yhetil.org]; FROM_NEQ_ENVFROM(0.00)[yantar92@gmail.com,emacs-orgmode-bounces@gnu.org]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RCVD_COUNT_FIVE(0.00)[6]; FROM_HAS_DN(0.00)[]; URIBL_BLOCKED(0.00)[sutd.edu.sg:email,reddit.com:url,karl-voit.at:email,gnu.org:url]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[emacs-orgmode@gnu.org]; HAS_LIST_UNSUB(-0.01)[]; DNSWL_BLOCKED(0.00)[209.51.188.17:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.51.188.17:from]; MID_RHS_NOT_FQDN(0.50)[]; FORGED_SENDER_MAILLIST(0.00)[] X-TUID: kNL4BCfa72hZ > The visual glitch looks like that: > > :PROPERTIES:X:CREATED: [2020-05-04 Mon 18>54] > X Should be partially fixed in the latest patch I just sent. OLD <<< :PROPERTIES:X:CREATED: [2020-05-04 Mon 18>54] NEW >>> :PROPERTIES:X X Best, Ihor Karl Voit writes: > Hi Ihor, > > * Ihor Radchenko wrote: >> >> So far, I came up with the following partial solution searching and >> showing hidden text. >> >> (defun org-find-text-property-region (pos prop) >> (define-advice isearch-search-string (:after (&rest _) put-overlay) >> (defun org-restore-invisibility-specs (&rest _) >> (add-hook 'post-command-hook #'org-restore-invisibility-specs) >> (defun org-flag-region (from to flag spec) >> (defun org-unfontify-region (beg end &optional _maybe_loudly) > > After a couple of hours working with these patches, my feedback is > very positive. Besides some visual glitches when creating a new > heading with org-expiry-insinuate activated (which automatically > adds :CREATED: properties), I could not detect any side-effect so > far (will keep testing). > > The visual glitch looks like that: > > :PROPERTIES:X:CREATED: [2020-05-04 Mon 18>54] > X > > ... with "X" being my character that symbolizes collapsed content. > The way it looked without the patch was a simple collapsed property > drawer. > > To me, this is acceptable considering the huge performance gain I > got. > > THANK YOU VERY MUCH! I can't remember where I had this way of > working within my large Org files[3] since ages. > >>> Anyway, the real fix should come from Emacs itself. There are ways to >>> make overlays faster. These ways have already been discussed on the >>> Emacs devel mailing list, but no one implemented them. It is a bit sad >>> that we have to find workarounds for that. >> >> I guess that it is a very old story starting from the times when XEmacs >> was a thing [1]. I recently heard about binary tree implementation of >> overlays (there should be a branch in emacs git repo) [2], but there was >> no update on that branch for a while. So, I do not have much hope on >> Emacs implementing efficient overlay access in the near future. (And I >> have problems with huge org files already). > > I can not express how this also reflects my personal situation. > >> [1] https://www.reddit.com/r/planetemacs/comments/e9lgwn/history_of_lucid_emacs_fsf_emacs_schism/ >> [2] https://lists.gnu.org/archive/html/emacs-devel/2019-12/msg00323.html > > [3] https://karl-voit.at/2020/05/03/current-org-files > > -- > get mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML into Org-mode: > > get Memacs from https://github.com/novoid/Memacs < > Personal Information Management > http://Karl-Voit.at/tags/pim/ > Emacs-related > http://Karl-Voit.at/tags/emacs/ > > -- Ihor Radchenko, PhD, Center for Advancing Materials Performance from the Nanoscale (CAMP-nano) State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong University, Xi'an, China Email: yantar92@gmail.com, ihor_radchenko@alumni.sutd.edu.sg