From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id qBIULMIhpWN5NAAAbAwnHQ (envelope-from ) for ; Fri, 23 Dec 2022 04:34:26 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id MATwK8IhpWPhHgAAauVa8A (envelope-from ) for ; Fri, 23 Dec 2022 04:34:26 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 6BB1E22800 for ; Fri, 23 Dec 2022 04:34:25 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p8YoH-0001sW-LD; Thu, 22 Dec 2022 22:33:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p8YoG-0001s2-VU for emacs-orgmode@gnu.org; Thu, 22 Dec 2022 22:33:24 -0500 Received: from mail-ot1-x32a.google.com ([2607:f8b0:4864:20::32a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p8YoE-0008WM-Pn for emacs-orgmode@gnu.org; Thu, 22 Dec 2022 22:33:24 -0500 Received: by mail-ot1-x32a.google.com with SMTP id p17-20020a9d6951000000b00678306ceb94so2212400oto.5 for ; Thu, 22 Dec 2022 19:33:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=T3fJ+mJ93ROim+uH5SjXiPv0PMiCQhuO0gUUOVe0hkU=; b=PH8WkeML9Q7VNDng9jy8TjXKiZVBRLndAZ5wqzDhPCA1tHTcgpU1zEt+JCXHOWmkBs nvZv9dY1uVYJUyRH4Tc7Swwme+wJ5+MxUrT3jB1f4gfNEOYsqNlUfEhy4GMl1t6rLpaX hxiI4zNBUAyAuAtHz+dkCPlaGGtP/BiKQYCvozl3j9Qrnh0qndFfwy66iADVmj8al3Cp GOSFZNertnG7M8Xf+ota2bZc2/Y16/hGbDAdwz4VvkPeo3Hv+IIK/t3PmkMoeUDFCucC bnxt5gPI2I1kvhnzSsH9yJt9My+tJnnYM7lx94A7JrHGWoa1hAlomQNZHfQPsIYTRPXa 8RfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=T3fJ+mJ93ROim+uH5SjXiPv0PMiCQhuO0gUUOVe0hkU=; b=RqqFmDNf/ERVDhQf0JUcwY+Vfv+vDT+ilCyv0LxIQsiSZxsz9jgdFA7PqgEd9TNFbT 8FU6bRkNhN01IDlZIay+kwEy2JfrkcLLfesKMlU/t335LSUzigrIt0Wmb9w5LcJwJsJY zlKutUgeDQWG98tvFQ0sub6eIX2uyd4LrwrT3ZPFC+2lnYrvks8vU9WvXP9Sv63mJrxx dCoMYzzaL2bysOFYWPw8gtJO/JuqpT2Tz2C0yb3DYQrcQj1o5JnyXnOX73v5yw607fsN Ur4EajEAteM3LBQYqyf3tDcJbVGt2ryYNRJVGKnkUzXmU2J00MyCkFCwTkAGSKzOVjhB qM8w== X-Gm-Message-State: AFqh2komV55gMztQl958xaZL8OakTNzuq8FYTWsCkNKV0tkoUzPG+GGH XWdYoplJxBXtgOnPwAeoT1LiPPwN+d7bN0tH X-Google-Smtp-Source: AMrXdXtcydNlCYVOaULJTtqJpMRPULwd5+wSDFe/DgYIawkSOCJxhtnegMZ9gsOYpVw2WgulsssiBQ== X-Received: by 2002:a05:6830:22d1:b0:670:8cc8:3a02 with SMTP id q17-20020a05683022d100b006708cc83a02mr3926863otc.12.1671766399343; Thu, 22 Dec 2022 19:33:19 -0800 (PST) Received: from smtpclient.apple ([2804:14c:101:8547:8ced:79d3:8734:b22c]) by smtp.gmail.com with ESMTPSA id x6-20020a056830244600b006705829996fsm1116978otr.56.2022.12.22.19.33.17 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Dec 2022 19:33:18 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Subject: Re: Completely hide properties drawer in 9.6 From: Sterling Hooten In-Reply-To: <87tu1xx20u.fsf@localhost> Date: Fri, 23 Dec 2022 00:33:15 -0300 Cc: emacs-orgmode@gnu.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <4CC6A1F4-0B08-44E0-AE4D-60CA11636663@gmail.com> <87k02uqrbw.fsf@localhost> <3B566A00-1AAB-4CFC-96C2-C25D05AD5855@gmail.com> <87bko6t3tb.fsf@localhost> <25EC8C19-CFD9-4973-8F5B-896AF45C7002@gmail.com> <87tu1xx20u.fsf@localhost> To: Ihor Radchenko X-Mailer: Apple Mail (2.3696.120.41.1.1) Received-SPF: pass client-ip=2607:f8b0:4864:20::32a; envelope-from=hooten@gmail.com; helo=mail-ot1-x32a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1671766465; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=T3fJ+mJ93ROim+uH5SjXiPv0PMiCQhuO0gUUOVe0hkU=; b=hq4EJyFwtNwJv0aU8+S0ytnhu92taSJQW3W93+KxqzgwDOg7FngUBP6EN4oj3v1de0lyNi cJB9uwYsfezmMLwPfNRZJoT+A5RUzIvSraVobQdeX1eRgo6HzIFiMllEtsdkk9cL8cyYYg mv7YxQoMp5xPsqj0T/xshXQBEMcqwUGdB+8f8qessu2Vscvm7zfQCDZhBjffgvnT70rbHg yqiBqgMi/IeMWRlYd9Cbl72fxHUeOJIlpRN7uTePM3+m6uwA4kNrcPbXkaEQtAiqt/yYXc sQgav7ICJdirXoENm4YHZWk8KUBR9k/2fDaWR2yeGjTgbFN+n7yP5GF7LABeIQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=PH8WkeML; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=key1; d=yhetil.org; t=1671766465; a=rsa-sha256; cv=none; b=g1sISGC1wza/LZ/v0S2OtAhgwcgphTLRRBv/mT+jqusM5kpir3wZtSVI3khHdtuL2esAcP U48m92u2FeATXzG1udZ6J41AozMizuHP1cR0VbRYa8jJ3tb84G1lw/1jcyuUeEN3+Jkh/k yDKBvtgMXMbXGfC0QhYJKICHu3cVuvXPIopGkxZHj3pmKAN/CxQRV/Qgv3VN4gmm7fAtXZ jKQFAISmoarILSAhsmSJxh6Ufiu8zEcod8DrJnX/WOLjuZu10mlvhd1TbkgF4swvGc7V6+ apGYKzjO6xtLjwqPYDoF69Z+PZceEm7Ij6HtYCsXcBg6meEruaZR1Mlh+3jYpA== X-Spam-Score: -5.55 X-Migadu-Queue-Id: 6BB1E22800 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=PH8WkeML; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gmail.com X-Migadu-Scanner: scn1.migadu.com X-Migadu-Spam-Score: -5.55 X-TUID: UbTSD7JuNcPh So I've got a solution somewhat working but I'd like some help = simplifying it. The good news is that it's significantly faster than the previous Org implementation I was using (the one you wrote on stack exchange), and can now handle folding a file with 30k lines in around .065s for hiding and .005s for showing. It does seem to be dependent on garbage collection settings though. I'm just going to walk through my solution and then ask some specific questions about how to fix it. Most of the work is actually in dealing with invisible edits and then handling =3Dorg-return=3D when called in a heading with a hidden = property drawer. Starting with a basic minor mode: #+begin_src emacs-lisp (define-minor-mode org-hide-properties-mode "Toggle `org-hide-properties-mode'. Hide all property drawers in = buffer and any further created." :init-value nil :lighter (" HP") (if org-hide-properties-mode (progn=20 (org-fold-add-folding-spec 'property-drawer-hidden-edges '((:ellipsis . nil) (:isearch-open . t) (:front-sticky . nil) (:rear-sticky . nil)) nil 'append) (org-fold-add-folding-spec 'property-drawer-hidden-interior '((:ellipsis . nil) (:isearch-open . t) (:front-sticky . t) (:rear-sticky . t)) nil 'append) (advice-add 'org-insert-property-drawer :after = #'org-fold-hide-property-drawer) (org-fold-hide-property-drawer-all)) (org-fold-show-property-drawer-all) (advice-remove 'org-insert-property-drawer = #'org-fold-hide-property-drawer))) #+end_src In order to both have newly added properties automatically adopt the invisibility text-property the interior characters of the properties box needs to be sticky. But this conflicted with being able to type with the point before the hidden text. To satisfy both these requirements I applied =3Dproperty-drawer-hidden-interior=3D as a = folding spec which was sticky to all but the first two and last two characters of the property drawer. Then I applied =3Dproperty-drawer-hidden-edges=3D to these two remaining chunks. In this way you can add in new properties and they'll be invisible, but typing at the edges appears visibly. Is there an alternative way of doing this without two separate folding specs? Maybe an additional key for ":interior-sticky" which is sticky on all characters sharing the same spec, but not sticky at the front and rear? I couldn't quite figure out what the :fragile setting was. #+begin_src emacs-lisp (defun org-fold--hide-property-drawers (begin end) "Hide all property drawers betweeen BEGIN and END." (org-with-wide-buffer (goto-char begin) (while (and (< (point) end) (re-search-forward org-property-drawer-re end t)) (with-silent-modifications ;; Hide interior of drawer to be sticky so org-entry-put = will respect invisibility (org-fold-region (+ (match-beginning 0) 1) (- (match-end 0) = 2) t 'property-drawer-hidden-interior) ;; Hide first two and last two characters without sticky (org-fold-region (- (match-beginning 0) 1) (+ = (match-beginning 0) 1) t 'property-drawer-hidden-edges) (org-fold-region (- (match-end 0) 2) (match-end 0) t = 'property-drawer-hidden-edges))))) (defun org-fold--show-property-drawers (begin end) "Unhide all property drawers in buffer between BEG and END" (org-fold-region begin end nil 'property-drawer-hidden-edges) ;; SWH 2022-12-15 HACK because I'm using two diferent folding = specs (org-fold-region begin end nil 'property-drawer-hidden-interior)) (defun org-fold-hide-property-drawer-all () =20 "Hide all property drawers in visible part of current buffer." (org-fold--hide-property-drawers (point-min) (point-max))) =20 (defun org-fold-show-property-drawer-all () "Unhide all property drawers visible part of current buffer" (org-fold--show-property-drawers (point-min) (point-max))) (defun org-fold-toggle-property-drawer-all () "Show or hide all property drawers in buffer." (interactive) (require 'org-macs) (if org-properties-hidden-p (progn=20 (setq org-properties-hidden-p nil) (org-fold-show-property-drawer-all)) (setq org-properties-hidden-p t) (org-fold-hide-property-drawer-all))) #+end_src Folding headings or subtrees: #+begin_src emacs-lisp (defun org-fold-show-property-drawer (&optional arg) "Unhide property drawer in heading at point. With non-nil ARG show the entire subtree." (org-with-wide-buffer=20 (org-back-to-heading) (let* ((beg (point)) (end (if arg (org-element-property :end = (org-element-at-point)) (org-next-visible-heading 1) (1+ (point))))) (org-fold-region beg end nil 'property-drawer-hidden-edges) (org-fold-region beg end nil 'property-drawer-hidden-interior)))) (defun org-fold-hide-property-drawer (&optional arg) "Completely hide the property drawer in heading at point. With non-nil ARG hide the entire subtree." (org-with-wide-buffer=20 (org-back-to-heading) (let* ((beg (point)) (end (if arg (org-element-property :end = (org-element-at-point)) (org-next-visible-heading 1) (1+ (point))))) (org-fold--hide-property-drawers beg end)))) #+end_src Is there a faster or better way of getting the boundary points for a heading without the subheadings? I've wanted this feature in a few other situations but don't know a clean way to fetch it. I suppose you could also just have that the regex stop searching for the beginning of the property box after 2 lines past the heading, since the properties box needs to be in the first two lines. These next sections are more opinionated in how to handle the edge cases when dealing with hidden text that shouldn't be moved during normal editing (e.g., if the property drawer has an empty line between it and the headline, it isn't recognized as such). My intention was to match default behavior as much as possible without breaking the parsing of the property drawer or needing to reveal the invisible text. The default =3Dorg-end-of-meta-data=3D will go past any blank lines and only stop at a non-empty line. I would prefer for hitting on a headline to act as if the planning line and properties box were immutable, and just pass directly after them. For this I wrote a version that will stop after the logbooks or property drawer. Is there a better way to do this? I tried using the match-data but it seemed unreliable (if there's nothing but blank lines after the meta data it matches on the next heading). #+begin_src emacs-lisp (defun swh-org-end-of-meta-data (&optional full) "Skip planning line and properties drawer in current entry. Differs from `org-end-of-meta-data' in that it stops after the first = logbook. When optional argument FULL is t, also skip planning information, clocking lines and any kind of drawer. When FULL is non-nil but not t, skip planning information, properties, clocking lines and logbook drawers." (org-back-to-heading t) (forward-line) ;; Skip planning information. (when (looking-at-p org-planning-line-re) (forward-line)) ;; Skip property drawer. (when (looking-at org-property-drawer-re) (goto-char (match-end 0)) (forward-line)) ;; When FULL is not nil, skip more. (when (and full (not (org-at-heading-p))) (catch 'exit (let ((end (save-excursion (outline-next-heading) (point))) (re (concat "[ \t]*$" "\\|" org-clock-line-re))) (while (not (eobp)) (cond ;; Skip clock lines. ((looking-at-p re) (forward-line)) ;; Skip logbook drawer. ((looking-at-p org-logbook-drawer-re) (if (re-search-forward "^[ \t]*:END:[ \t]*$" end t) ;; SWH 2022-12-19 Exit after finding the /first/ = logbook (progn (forward-line) (throw 'exit t)) (throw 'exit t))) ;; When FULL is t, skip regular drawer too. ((and (eq full t) (looking-at-p org-drawer-regexp)) (if (re-search-forward "^[ \t]*:END:[ \t]*$" end t) (forward-line) (throw 'exit t))) (t (throw 'exit t)))))))) #+end_src This tries to deal with hitting on a headline in a natural way: #+begin_src emacs-lisp (defun org-return-newline-at-heading-skip-drawers (fun &rest args) "If at heading do nothing, if on heading skip the planning line, property drawer, logbook, and insert a newline after the last drawer or right before the first non-empty line." ;; FIXME Should also handle the case when on planning line (if (not (org-at-heading-p)) (apply fun args) ;; Store point so undo will put it back in heading (push (point) buffer-undo-list) ;; HACK to get entry to unfold when hidden from org-cycle (org-fold-show-entry 'hide-drawers) (swh-org-end-of-meta-data 'logbook) (if (not (org-at-heading-p)) (if (eobp) (org-return) (beginning-of-line) (newline) (forward-line -1)) (beginning-of-line) (newline) (forward-line -1)))) (advice-add 'org-return :around = #'org-return-newline-at-heading-skip-drawers) #+end_src I wanted the editing to be natural and the hidden text unobtrusive. I added a new =3Dorg-catch-invisible-edits=3D flag of =3D'swh=3D and = overwrote =3Dorg-fold-check-before-invisible-edit=3D to accomodate this. To allow for insertion before and after the hidden text I increased the lookback for invisible characters from one to two. This allows for having a trailing newline folded after the :END: and thus you can type without appending to the end of property drawer (and messing up the syntax). Deletion is still not allowed after the hidden text. This is ugly but it seems to be working at the moment. #+begin_src emacs-lisp (defun org-fold-check-before-invisible-edit (kind) "Check if editing KIND is dangerous with invisible text around. The detailed reaction depends on the user option `org-fold-catch-invisible-edits'." ;; First, try to get out of here as quickly as possible, to reduce = overhead (when (and org-fold-catch-invisible-edits (or (not (boundp 'visible-mode)) (not visible-mode)) (or (org-invisible-p) (org-invisible-p (max (point-min) (1- (point)))) ;; SWH 2022-12-19 Searching 2 characters back to allow = for newlines in hidden property drawers (org-invisible-p (max (point-min) (- (point) 2))))) ;; OK, we need to take a closer look. Only consider invisibility ;; caused by folding of headlines, drawers, and blocks. Edits ;; inside links will be handled by font-lock. =20 ;; SWH 2022-12-14 Add property-drawer-hidden-edges to allow for = completely hiding properties (let* ((invisible-at-point (org-fold-folded-p (point) '(headline = drawer block property-drawer-hidden-edges))) (invisible-before-point (and (not (bobp)) (org-fold-folded-p (1- (point)) '(headline drawer block = property-drawer-hidden-edges)))) (invisible-before-point-2 (and (not (bobp)) ;; SWH 2022-12-19 counting invisble things 2 characters = away to allow for newlines in hidden property drawers (org-fold-folded-p (- (point) 2) '(headline drawer = block property-drawer-hidden-edges)))) (border-and-ok-direction (or ;; Check if we are acting predictably before invisible ;; text. (and invisible-at-point (not invisible-before-point) (memq kind '(insert delete-backward))) ;; Check if we are acting predictably after invisible text ;; This works not well, and I have turned it off. It seems ;; better to always show and stop after invisible text. ;; (and (not invisible-at-point) invisible-before-point ;; (memq kind '(insert delete))) ))) (when (or invisible-at-point invisible-before-point = invisible-before-point-2) (when (and invisible-before-point-2 (memq kind = '(delete-backward))) (user-error "Can't delete after invisible areas is prohibited, = make them visible first")) ;; (when (and invisible-before-point-2 (not = invisible-before-point) (memq kind '(insert))) ;; (message "SWH allowing modifying two places after invisible = point editing")) (when (and (eq org-fold-catch-invisible-edits 'swh) invisible-at-point (memq kind '(delete))) (user-error "Show hidden properties before editing = invisible")) (when (or (eq org-fold-catch-invisible-edits 'error) (and invisible-before-point (eq = org-fold-catch-invisible-edits 'swh))) (user-error "Editing in invisible areas is prohibited, make = them visible first")) (if (and org-custom-properties-overlays (y-or-n-p "Display invisible properties in this buffer? = ")) (org-toggle-custom-properties-visibility) ;; Make the area visible (if (and (eq org-fold-catch-invisible-edits 'swh) border-and-ok-direction) (message "Allowing modifying before invisible point = editing") (save-excursion (org-fold-show-set-visibility 'local)) (when invisible-before-point (org-with-point-at (1- (point)) = (org-fold-show-set-visibility 'local))) (cond ((eq org-fold-catch-invisible-edits 'show) ;; That's it, we do the edit after showing (message "Unfolding invisible region around point before editing") (sit-for 1)) ((and (eq org-fold-catch-invisible-edits 'smart) border-and-ok-direction) (message "Unfolding invisible region around point before = editing")) ((and invisible-before-point-2 (not invisible-before-point) = (memq kind '(insert delete))) (message "Allowing modifying two places after invisible = point editing")) (t ;; Don't do the edit, make the user repeat it in full = visibility (user-error "Edit in invisible region aborted, repeat to = confirm with text visible"))))))))) #+end_src I am also experiencing a weird (overlay?) problem with i-search. After implementing when I search for things it will make all of the windows greyed out, and then only show the matching characters. I can see the text again if I =3Dmark-whole-buffer=3D. But I don't know what's causing this. Thanks again for your help, this is trickier than I initially thought but I think it's coming together.