From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id wrCPE6fI3V4aOQAA0tVLHw (envelope-from ) for ; Mon, 08 Jun 2020 05:12:07 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id YicTD6fI3V5JMwAA1q6Kng (envelope-from ) for ; Mon, 08 Jun 2020 05:12:07 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 982939403EC for ; Mon, 8 Jun 2020 05:12:06 +0000 (UTC) Received: from localhost ([::1]:60274 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jiA4r-0006Ag-KL for larch@yhetil.org; Mon, 08 Jun 2020 01:12:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55580) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jiA4C-0006AO-EA for emacs-orgmode@gnu.org; Mon, 08 Jun 2020 01:11:24 -0400 Received: from mail-pf1-x42a.google.com ([2607:f8b0:4864:20::42a]:36807) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jiA48-00031z-Fx for emacs-orgmode@gnu.org; Mon, 08 Jun 2020 01:11:24 -0400 Received: by mail-pf1-x42a.google.com with SMTP id x22so8014449pfn.3 for ; Sun, 07 Jun 2020 22:11:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version; bh=fmZQtqW4pweQLYavQHiHFKIu7TSnrJYUQJZsZSQsQqo=; b=SKWVYPX3nA1TuqmmwgKFUlFqHYONOkXfxPJNpsgqGKYeCVbiMhbK76X263Vzy9CP0x 9k8mNKmrX1Hn/pd2lo+AeR8X/f4DQ8stTg8kgwiJaAa/ffTJ7Ilb4DeltgWKJdS1+nmC Moa5NktrXxZHEiyzBbhnBSIVuqmOT/QpPIV8SbLbjw4aSQogc9d5mSztbJGBdVfWETE6 4SsV0X9qwMXM8rRwCiSmV196Ojy9RHn3gxKIGcv49BASv7l9TnepJie3eydx2NQJnAPe FdsQb6dezeRmKk3hPFsMz3J1Cx0hFBmZ7y9LmKgHk8wveUJPEn4ZX2knZebsN1DzIlnL RgsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=fmZQtqW4pweQLYavQHiHFKIu7TSnrJYUQJZsZSQsQqo=; b=q7cx1Z9TN934jzq0/YYsgL5m8jDRXiU0H8mpqibBPl9WlCJn81UuoKi2Vf5unstH5z glRj50faphHXmZN7tuha/mOm7RyQAM+ZRWf8YxWe2rsMq5RrJY7c1+MkJqHcKIDiDv/8 4Y1MdNGiZD+B8FFooQ4buKwSNQcjapp9j4ZI0qIMvYqEPfY95vOAWoEBMAnwdjAzeamP XwLzUeO/at1YwQN75ghz+8VNBUebRc2NnzYz6gwMHA1QEt6U5Ag9qOraZXR9iEta8KYu RoNfzw09GG2wdLs9RxMMQAoQ6NAK4H/RxqjlCWbhI0S+YmO6cOsxy82BWvE/hikELe3L 5YyA== X-Gm-Message-State: AOAM532vP49NVk88uz3z0bjBeKFyFYgY+bqMzZOg051PM6gyH5duo0Yj mnvIcfYswysnVmxFKvszgfi/WJzyjwM= X-Google-Smtp-Source: ABdhPJwQ9/1dsdCxOgCAkoqdY6q5+P0CDUru6VJ43y37wRMlFEqTskjXRAQAQuBwEK3honzKHzSNAA== X-Received: by 2002:a65:4904:: with SMTP id p4mr6393603pgs.333.1591593078857; Sun, 07 Jun 2020 22:11:18 -0700 (PDT) Received: from localhost ([210.3.160.230]) by smtp.gmail.com with ESMTPSA id y9sm5825994pfr.184.2020.06.07.22.11.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 07 Jun 2020 22:11:18 -0700 (PDT) From: Ihor Radchenko To: Nicolas Goaziou Subject: Re: [patch suggestion] Mitigating the poor Emacs performance on huge org files: Do not use overlays for PROPERTY and LOGBOOK drawers In-Reply-To: <875zc2du63.fsf@localhost> References: <87h7x9e5jo.fsf@localhost> <87r1vu5qmc.fsf@nicolasgoaziou.fr> <87imh5w1zt.fsf@localhost> <87blmxjckl.fsf@localhost> <87y2q13tgs.fsf@nicolasgoaziou.fr> <878si1j83x.fsf@localhost> <87d07bzvhd.fsf@nicolasgoaziou.fr> <87imh34usq.fsf@localhost> <87pnbby49m.fsf@nicolasgoaziou.fr> <87tv0efvyd.fsf@localhost> <874kse1seu.fsf@localhost> <87r1vhqpja.fsf@nicolasgoaziou.fr> <87tv0d2nk7.fsf@localhost> <87o8qkhy3g.fsf@nicolasgoaziou.fr> <87sgfqu5av.fsf@localhost> <87sgfn6qpc.fsf@nicolasgoaziou.fr> <87367d4ydc.fsf@localhost> <87r1uuotw8.fsf@nicolasgoaziou.fr> <87mu5iq618.fsf@localhost> <87ftb9pqop.fsf@nicolasgoaziou.fr> <875zc2du63.fsf@localhost> Date: Mon, 08 Jun 2020 13:06:28 +0800 Message-ID: <873676du3v.fsf@localhost> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=2607:f8b0:4864:20::42a; envelope-from=yantar92@gmail.com; helo=mail-pf1-x42a.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: emacs-orgmode@gnu.org Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (body hash did not verify) header.d=gmail.com header.s=20161025 header.b=SKWVYPX3; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Spam-Score: 0.59 X-TUID: MGt6iB17R7Ae --=-=-= Content-Type: text/plain The patch (against 1aa095ccf) is attached. --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=featuredrawertextprop-20200608.patch diff --git a/contrib/lisp/org-notify.el b/contrib/lisp/org-notify.el index 9f8677871..ab470ea9b 100644 --- a/contrib/lisp/org-notify.el +++ b/contrib/lisp/org-notify.el @@ -246,7 +246,7 @@ seconds. The default value for SECS is 20." (switch-to-buffer (find-file-noselect file)) (org-with-wide-buffer (goto-char begin) - (outline-show-entry)) + (org-show-entry)) (goto-char begin) (search-forward "DEADLINE: <") (search-forward ":") diff --git a/contrib/lisp/org-velocity.el b/contrib/lisp/org-velocity.el index bfc4d6c3e..2312b235c 100644 --- a/contrib/lisp/org-velocity.el +++ b/contrib/lisp/org-velocity.el @@ -325,7 +325,7 @@ use it." (save-excursion (when narrow (org-narrow-to-subtree)) - (outline-show-all))) + (org-show-all))) (defun org-velocity-edit-entry/inline (heading) "Edit entry at HEADING in the original buffer." diff --git a/doc/org-manual.org b/doc/org-manual.org index efad195e1..c6f167eac 100644 --- a/doc/org-manual.org +++ b/doc/org-manual.org @@ -509,11 +509,11 @@ Org uses just two commands, bound to {{{kbd(TAB)}}} and Switch back to the startup visibility of the buffer (see [[*Initial visibility]]). -- {{{kbd(C-u C-u C-u TAB)}}} (~outline-show-all~) :: +- {{{kbd(C-u C-u C-u TAB)}}} (~org-show-all~) :: #+cindex: show all, command #+kindex: C-u C-u C-u TAB - #+findex: outline-show-all + #+findex: org-show-all Show all, including drawers. - {{{kbd(C-c C-r)}}} (~org-reveal~) :: @@ -529,18 +529,18 @@ Org uses just two commands, bound to {{{kbd(TAB)}}} and headings. With a double prefix argument, also show the entire subtree of the parent. -- {{{kbd(C-c C-k)}}} (~outline-show-branches~) :: +- {{{kbd(C-c C-k)}}} (~org-show-branches~) :: #+cindex: show branches, command #+kindex: C-c C-k - #+findex: outline-show-branches + #+findex: org-show-branches Expose all the headings of the subtree, but not their bodies. -- {{{kbd(C-c TAB)}}} (~outline-show-children~) :: +- {{{kbd(C-c TAB)}}} (~org-show-children~) :: #+cindex: show children, command #+kindex: C-c TAB - #+findex: outline-show-children + #+findex: org-show-children Expose all direct children of the subtree. With a numeric prefix argument {{{var(N)}}}, expose all children down to level {{{var(N)}}}. @@ -7294,7 +7294,7 @@ its location in the outline tree, but behaves in the following way: command (see [[*Visibility Cycling]]). You can force cycling archived subtrees with {{{kbd(C-TAB)}}}, or by setting the option ~org-cycle-open-archived-trees~. Also normal outline commands, like - ~outline-show-all~, open archived subtrees. + ~org-show-all~, open archived subtrees. - #+vindex: org-sparse-tree-open-archived-trees diff --git a/lisp/org-agenda.el b/lisp/org-agenda.el index 9fbeb2a1e..2f121f743 100644 --- a/lisp/org-agenda.el +++ b/lisp/org-agenda.el @@ -6824,7 +6824,7 @@ and stored in the variable `org-prefix-format-compiled'." (t " %-12:c%?-12t% s"))) (start 0) varform vars var e c f opt) - (while (string-match "%\\(\\?\\)?\\([-+]?[0-9.]*\\)\\([ .;,:!?=|/<>]?\\)\\([cltseib]\\|(.+)\\)" + (while (string-match "%\\(\\?\\)?\\([-+]?[0-9.]*\\)\\([ .;,:!?=|/<>]?\\)\\([cltseib]\\|(.+?)\\)" s start) (setq var (or (cdr (assoc (match-string 4 s) '(("c" . category) ("t" . time) ("l" . level) ("s" . extra) @@ -9136,20 +9136,20 @@ if it was hidden in the outline." ((and (called-interactively-p 'any) (= more 1)) (message "Remote: show with default settings")) ((= more 2) - (outline-show-entry) + (org-show-entry) (org-show-children) (save-excursion (org-back-to-heading) (run-hook-with-args 'org-cycle-hook 'children)) (message "Remote: CHILDREN")) ((= more 3) - (outline-show-subtree) + (org-show-subtree) (save-excursion (org-back-to-heading) (run-hook-with-args 'org-cycle-hook 'subtree)) (message "Remote: SUBTREE")) ((> more 3) - (outline-show-subtree) + (org-show-subtree) (message "Remote: SUBTREE AND ALL DRAWERS"))) (select-window win))) diff --git a/lisp/org-archive.el b/lisp/org-archive.el index d3e12d17b..d864dad8a 100644 --- a/lisp/org-archive.el +++ b/lisp/org-archive.el @@ -330,7 +330,7 @@ direct children of this heading." (insert (if datetree-date "" "\n") heading "\n") (end-of-line 0)) ;; Make the subtree visible - (outline-show-subtree) + (org-show-subtree) (if org-archive-reversed-order (progn (org-back-to-heading t) diff --git a/lisp/org-colview.el b/lisp/org-colview.el index e50a4d7c8..e656df555 100644 --- a/lisp/org-colview.el +++ b/lisp/org-colview.el @@ -699,7 +699,7 @@ FUN is a function called with no argument." (move-beginning-of-line 2) (org-at-heading-p t))))) (unwind-protect (funcall fun) - (when hide-body (outline-hide-entry))))) + (when hide-body (org-hide-entry))))) (defun org-columns-previous-allowed-value () "Switch to the previous allowed value for this column." diff --git a/lisp/org-compat.el b/lisp/org-compat.el index 5953f89d2..09a09472a 100644 --- a/lisp/org-compat.el +++ b/lisp/org-compat.el @@ -138,12 +138,8 @@ This is a floating point number if the size is too large for an integer." ;;; Emacs < 25.1 compatibility (when (< emacs-major-version 25) - (defalias 'outline-hide-entry 'hide-entry) - (defalias 'outline-hide-sublevels 'hide-sublevels) - (defalias 'outline-hide-subtree 'hide-subtree) (defalias 'outline-show-branches 'show-branches) (defalias 'outline-show-children 'show-children) - (defalias 'outline-show-entry 'show-entry) (defalias 'outline-show-subtree 'show-subtree) (defalias 'xref-find-definitions 'find-tag) (defalias 'format-message 'format) @@ -644,7 +640,7 @@ When optional argument ELEMENT is a parsed drawer, as returned by When buffer positions BEG and END are provided, hide or show that region as a drawer without further ado." (declare (obsolete "use `org-hide-drawer-toggle' instead." "Org 9.4")) - (if (and beg end) (org-flag-region beg end flag 'outline) + (if (and beg end) (org-flag-region beg end flag 'org-hide-drawer) (let ((drawer (or element (and (save-excursion @@ -658,7 +654,7 @@ region as a drawer without further ado." (save-excursion (goto-char (org-element-property :end drawer)) (skip-chars-backward " \t\n") (line-end-position)) - flag 'outline) + flag 'org-hide-drawer) ;; When the drawer is hidden away, make sure point lies in ;; a visible part of the buffer. (when (invisible-p (max (1- (point)) (point-min))) diff --git a/lisp/org-element.el b/lisp/org-element.el index ac41b7650..2d5c8d771 100644 --- a/lisp/org-element.el +++ b/lisp/org-element.el @@ -4320,7 +4320,7 @@ element or object. Meaningful values are `first-section', TYPE is the type of the current element or object. If PARENT? is non-nil, assume the next element or object will be -located inside the current one. " +located inside the current one." (if parent? (pcase type (`headline 'section) diff --git a/lisp/org-keys.el b/lisp/org-keys.el index 37df29983..a714dec0f 100644 --- a/lisp/org-keys.el +++ b/lisp/org-keys.el @@ -437,7 +437,7 @@ COMMANDS is a list of alternating OLDDEF NEWDEF command names." #'org-next-visible-heading) (define-key org-mode-map [remap outline-previous-visible-heading] #'org-previous-visible-heading) -(define-key org-mode-map [remap show-children] #'org-show-children) +(define-key org-mode-map [remap outline-show-children] #'org-show-children) ;;;; Make `C-c C-x' a prefix key (org-defkey org-mode-map (kbd "C-c C-x") (make-sparse-keymap)) diff --git a/lisp/org-macs.el b/lisp/org-macs.el index a02f713ca..b17c0cb4d 100644 --- a/lisp/org-macs.el +++ b/lisp/org-macs.el @@ -682,7 +682,7 @@ When NEXT is non-nil, check the next line instead." -;;; Overlays +;;; Overlays and text properties (defun org-overlay-display (ovl text &optional face evap) "Make overlay OVL display TEXT with face FACE." @@ -705,26 +705,126 @@ If DELETE is non-nil, delete all those overlays." (delete (delete-overlay ov)) (t (push ov found)))))) +(defun org-remove-text-properties (start end properties &optional object) + "Remove text properties as in `remove-text-properties', but keep 'invisibility specs for folded regions. +Do not remove invisible text properties specified by 'outline, +'org-hide-block, and 'org-hide-drawer (but remove i.e. 'org-link) this +is needed to keep outlines, drawers, and blocks hidden unless they are +toggled by user. +Note: The below may be too specific and create troubles if more +invisibility specs are added to org in future" + (when (plist-member properties 'invisible) + (let ((pos start) + next spec) + (while (< pos end) + (setq next (next-single-property-change pos 'invisible nil end) + spec (get-text-property pos 'invisible)) + (unless (memq spec (list 'org-hide-block + 'org-hide-drawer + 'outline)) + (remove-text-properties pos next '(invisible nil) object)) + (setq pos next)))) + (when-let ((properties-stripped (org-plist-delete properties 'invisible))) + (remove-text-properties start end properties-stripped object))) + +(defun org--find-text-property-region (pos prop) + "Find a region containing PROP text property around point POS." + (let* ((beg (and (get-text-property pos prop) pos)) + (end beg)) + (when beg + ;; when beg is the first point in the region, `previous-single-property-change' + ;; will return nil. + (setq beg (or (previous-single-property-change pos prop) + beg)) + ;; when end is the last point in the region, `next-single-property-change' + ;; will return nil. + (setq end (or (next-single-property-change pos prop) + end)) + (unless (= beg end) ; this should not happen + (cons beg end))))) + +(defun org--add-to-list-text-property (from to prop element) + "Add element to text property PROP, whos value should be a list." + (add-text-properties from to `(,prop ,(list element))) ; create if none + ;; add to existing + (alter-text-property from to + prop + (lambda (val) + (if (member element val) + val + (cons element val))))) + +(defun org--remove-from-list-text-property (from to prop element) + "Remove ELEMENT from text propery PROP, whos value should be a list." + (let ((pos from)) + (while (< pos to) + (when-let ((val (get-text-property pos prop))) + (if (equal val (list element)) + (remove-text-properties pos (next-single-char-property-change pos prop nil to) (list prop nil)) + (put-text-property pos (next-single-char-property-change pos prop nil to) + prop (remove element (get-text-property pos prop))))) + (setq pos (next-single-char-property-change pos prop nil to))))) + +(defvar org--invisible-spec-priority-list '(outline org-hide-drawer org-hide-block) + "Priority of invisibility specs.") + +(defun org--get-buffer-local-invisible-property-symbol (spec &optional buffer return-only) + "Return unique symbol suitable to be used as buffer-local in BUFFER for 'invisible SPEC. +If the buffer already have buffer-local setup in `char-property-alias-alist' +and the setup appears to be created for different buffer, +copy the old invisibility state into new buffer-local text properties, +unless RETURN-ONLY is non-nil." + (if (not (member spec org--invisible-spec-priority-list)) + (user-error "%s should be a valid invisibility spec" spec) + (let* ((buf (or buffer (current-buffer)))) + (let ((local-prop (intern (format "org--invisible-%s-buffer-local-%S" + (symbol-name spec) + ;; (sxhash buf) appears to be not constant over time. + ;; Using buffer-name is safe, since the only place where + ;; buffer-local text property actually matters is an indirect + ;; buffer, where the name cannot be same anyway. + (sxhash (buffer-name buf)))))) + (prog1 + local-prop + (unless return-only + (with-current-buffer buf + (unless (member local-prop (alist-get 'invisible char-property-alias-alist)) + ;; copy old property + (dolist (old-prop (alist-get 'invisible char-property-alias-alist)) + (org-with-wide-buffer + (let* ((pos (point-min)) + (spec (seq-find (lambda (spec) + (string-match-p (symbol-name spec) + (symbol-name old-prop))) + org--invisible-spec-priority-list)) + (new-prop (org--get-buffer-local-invisible-property-symbol spec nil 'return-only))) + (while (< pos (point-max)) + (when-let (val (get-text-property pos old-prop)) + (put-text-property pos (next-single-char-property-change pos old-prop) new-prop val)) + (setq pos (next-single-char-property-change pos old-prop)))))) + (setq-local char-property-alias-alist + (cons (cons 'invisible + (mapcar (lambda (spec) + (org--get-buffer-local-invisible-property-symbol spec nil 'return-only)) + org--invisible-spec-priority-list)) + (remove (assq 'invisible char-property-alias-alist) + char-property-alias-alist))))))))))) + (defun org-flag-region (from to flag spec) "Hide or show lines from FROM to TO, according to FLAG. SPEC is the invisibility spec, as a symbol." - (remove-overlays from to 'invisible spec) - ;; Use `front-advance' since text right before to the beginning of - ;; the overlay belongs to the visible line than to the contents. - (when flag - (let ((o (make-overlay from to nil 'front-advance))) - (overlay-put o 'evaporate t) - (overlay-put o 'invisible spec) - (overlay-put o 'isearch-open-invisible #'delete-overlay)))) - + (with-silent-modifications + (remove-text-properties from to (list (org--get-buffer-local-invisible-property-symbol spec) nil)) + (when flag + (put-text-property from to (org--get-buffer-local-invisible-property-symbol spec) spec)))) ;;; Regexp matching (defsubst org-pos-in-match-range (pos n) - (and (match-beginning n) - (<= (match-beginning n) pos) - (>= (match-end n) pos))) +(and (match-beginning n) + (<= (match-beginning n) pos) + (>= (match-end n) pos))) (defun org-skip-whitespace () "Skip over space, tabs and newline characters." diff --git a/lisp/org-src.el b/lisp/org-src.el index 6f6c544dc..9e8a50044 100644 --- a/lisp/org-src.el +++ b/lisp/org-src.el @@ -529,8 +529,8 @@ Leave point in edit buffer." (org-src-switch-to-buffer buffer 'edit) ;; Insert contents. (insert contents) - (remove-text-properties (point-min) (point-max) - '(display nil invisible nil intangible nil)) + (org-remove-text-properties (point-min) (point-max) + '(display nil invisible nil intangible nil)) (unless preserve-ind (org-do-remove-indentation)) (set-buffer-modified-p nil) (setq buffer-file-name nil) diff --git a/lisp/org-table.el b/lisp/org-table.el index 6462b99c4..75801161b 100644 --- a/lisp/org-table.el +++ b/lisp/org-table.el @@ -2001,7 +2001,7 @@ toggle `org-table-follow-field-mode'." (arg (let ((b (save-excursion (skip-chars-backward "^|") (point))) (e (save-excursion (skip-chars-forward "^|\r\n") (point)))) - (remove-text-properties b e '(invisible t intangible t)) + (org-remove-text-properties b e '(invisible t intangible t)) (if (and (boundp 'font-lock-mode) font-lock-mode) (font-lock-fontify-block)))) (t @@ -2028,7 +2028,7 @@ toggle `org-table-follow-field-mode'." (setq word-wrap t) (goto-char (setq p (point-max))) (insert (org-trim field)) - (remove-text-properties p (point-max) '(invisible t intangible t)) + (org-remove-text-properties p (point-max) '(invisible t intangible t)) (goto-char p) (setq-local org-finish-function 'org-table-finish-edit-field) (setq-local org-window-configuration cw) diff --git a/lisp/org.el b/lisp/org.el index e5cea04c6..3d4a7b072 100644 --- a/lisp/org.el +++ b/lisp/org.el @@ -114,6 +114,7 @@ Stars are put in group 1 and the trimmed body in group 2.") (declare-function cdlatex-math-symbol "ext:cdlatex") (declare-function Info-goto-node "info" (nodename &optional fork strict-case)) (declare-function isearch-no-upper-case-p "isearch" (string regexp-flag)) +(declare-function isearch-filter-visible "isearch" (beg end)) (declare-function org-add-archive-files "org-archive" (files)) (declare-function org-agenda-entry-get-agenda-timestamp "org-agenda" (pom)) (declare-function org-agenda-list "org-agenda" (&optional arg start-day span with-hour)) @@ -192,6 +193,9 @@ Stars are put in group 1 and the trimmed body in group 2.") (defvar ffap-url-regexp) (defvar org-element-paragraph-separate) +(defvar org-element-all-objects) +(defvar org-element-all-elements) +(defvar org-element-greater-elements) (defvar org-indent-indentation-per-level) (defvar org-radio-target-regexp) (defvar org-target-link-regexp) @@ -4734,9 +4738,153 @@ This is for getting out of special buffers like capture.") ;;;; Define the Org mode +;;; Handling buffer modifications + (defun org-before-change-function (_beg _end) "Every change indicates that a table might need an update." (setq org-table-may-need-update t)) + +(defun org-after-change-function (from to len) + "Process changes in folded elements. +If a text was inserted into invisible region, hide the inserted text. +If the beginning/end line of a folded drawer/block was changed, unfold it. +If a valid end line was inserted in the middle of the folded drawer/block, unfold it." + + ;; re-hide text inserted in the middle of a folded region + (dolist (spec org--invisible-spec-priority-list) + (when-let ((spec-to (get-text-property to (org--get-buffer-local-invisible-property-symbol spec))) + (spec-from (get-text-property (max (point-min) (1- from)) (org--get-buffer-local-invisible-property-symbol spec)))) + (when (eq spec-to spec-from) + (org-flag-region from to 't spec-to)))) + + ;; Process all the folded text between `from' and `to' + (org-with-wide-buffer + + (if (< to from) + (let ((tmp from)) + (setq from to) + (setq to tmp))) + + ;; Include next/previous line into the changed region. + ;; This is needed to catch edits in beginning line of a folded + ;; element. + (setq to (save-excursion (goto-char to) (forward-line) (point))) + (setq from (save-excursion (goto-char from) (forward-line -1) (point))) + + ;; Expand the considered region to include partially present folded + ;; drawer/block. + (when (get-text-property from (org--get-buffer-local-invisible-property-symbol 'org-hide-drawer)) + (setq from (previous-single-char-property-change from (org--get-buffer-local-invisible-property-symbol 'org-hide-drawer)))) + (when (get-text-property from (org--get-buffer-local-invisible-property-symbol 'org-hide-block)) + (setq from (previous-single-char-property-change from (org--get-buffer-local-invisible-property-symbol 'org-hide-block)))) + + ;; check folded drawers + (let ((pos from)) + (unless (get-text-property pos (org--get-buffer-local-invisible-property-symbol 'org-hide-drawer)) + (setq pos (next-single-char-property-change pos + (org--get-buffer-local-invisible-property-symbol 'org-hide-drawer)))) + (while (< pos to) + (when-let ((drawer-begin (and (get-text-property pos (org--get-buffer-local-invisible-property-symbol 'org-hide-drawer)) + pos)) + (drawer-end (next-single-char-property-change pos (org--get-buffer-local-invisible-property-symbol 'org-hide-drawer)))) + + (let (unfold?) + ;; the line before folded text should be beginning of the drawer + (save-excursion + (goto-char drawer-begin) + (backward-char) + (beginning-of-line) + (unless (looking-at-p org-drawer-regexp) + (setq unfold? t))) + ;; the last line of the folded text should be :END: + (save-excursion + (goto-char drawer-end) + (beginning-of-line) + (unless (let ((case-fold-search t)) (looking-at-p org-property-end-re)) + (setq unfold? t))) + ;; there should be no :END: anywhere in the drawer body + (save-excursion + (goto-char drawer-begin) + (when (save-excursion + (let ((case-fold-search t)) + (re-search-forward org-property-end-re + (max (point) + (1- (save-excursion + (goto-char drawer-end) + (line-beginning-position)))) + 't))) + (setq unfold? t))) + ;; there should be no new entry anywhere in the drawer body + (save-excursion + (goto-char drawer-begin) + (when (save-excursion + (let ((case-fold-search t)) + (re-search-forward org-outline-regexp-bol + (max (point) + (1- (save-excursion + (goto-char drawer-end) + (line-beginning-position)))) + 't))) + (setq unfold? t))) + + (when unfold? (org-flag-region drawer-begin drawer-end nil 'org-hide-drawer)))) + + (setq pos (next-single-char-property-change pos (org--get-buffer-local-invisible-property-symbol 'org-hide-drawer))))) + + ;; check folded blocks + (let ((pos from)) + (unless (get-text-property pos (org--get-buffer-local-invisible-property-symbol 'org-hide-block)) + (setq pos (next-single-char-property-change pos + (org--get-buffer-local-invisible-property-symbol 'org-hide-block)))) + (while (< pos to) + (when-let ((block-begin (and (get-text-property pos (org--get-buffer-local-invisible-property-symbol 'org-hide-block)) + pos)) + (block-end (next-single-char-property-change pos (org--get-buffer-local-invisible-property-symbol 'org-hide-block)))) + + (let (unfold?) + ;; the line before folded text should be beginning of the block + (save-excursion + (goto-char block-begin) + (backward-char) + (beginning-of-line) + (unless (looking-at-p org-dblock-start-re) + (setq unfold? t))) + ;; the last line of the folded text should be end of the block + (save-excursion + (goto-char block-end) + (beginning-of-line) + (unless (looking-at-p org-dblock-end-re) + (setq unfold? t))) + ;; there should be no #+end anywhere in the block body + (save-excursion + (goto-char block-begin) + (when (save-excursion + (re-search-forward org-dblock-end-re + (max (point) + (1- (save-excursion + (goto-char block-end) + (line-beginning-position)))) + 't)) + (setq unfold? t))) + ;; there should be no new entry anywhere in the block body + (save-excursion + (goto-char block-begin) + (when (save-excursion + (let ((case-fold-search t)) + (re-search-forward org-outline-regexp-bol + (max (point) + (1- (save-excursion + (goto-char block-end) + (line-beginning-position)))) + 't))) + (setq unfold? t))) + + (when unfold? (org-flag-region block-begin block-end nil 'org-hide-block)))) + + (setq pos + (next-single-char-property-change pos + (org--get-buffer-local-invisible-property-symbol 'org-hide-block))))))) + (defvar org-mode-map) (defvar org-inhibit-startup-visibility-stuff nil) ; Dynamically-scoped param. (defvar org-agenda-keep-modes nil) ; Dynamically-scoped param. @@ -4789,6 +4937,7 @@ The following commands are available: (org-install-agenda-files-menu) (when org-link-descriptive (add-to-invisibility-spec '(org-link))) (add-to-invisibility-spec '(org-hide-block . t)) + (add-to-invisibility-spec '(org-hide-drawer . t)) (setq-local outline-regexp org-outline-regexp) (setq-local outline-level 'org-outline-level) (setq bidi-paragraph-direction 'left-to-right) @@ -4817,6 +4966,8 @@ The following commands are available: ;; Activate before-change-function (setq-local org-table-may-need-update t) (add-hook 'before-change-functions 'org-before-change-function nil 'local) + ;; Activate after-change-function + (add-hook 'after-change-functions 'org-after-change-function nil 'local) ;; Check for running clock before killing a buffer (add-hook 'kill-buffer-hook 'org-check-running-clock nil 'local) ;; Initialize macros templates. @@ -4868,6 +5019,10 @@ The following commands are available: (setq-local outline-isearch-open-invisible-function (lambda (&rest _) (org-show-context 'isearch))) + ;; Make isearch search in blocks hidden via text properties + (setq-local isearch-filter-predicate #'org--isearch-filter-predicate) + (add-hook 'isearch-mode-end-hook #'org--clear-isearch-overlays nil 'local) + ;; Setup the pcomplete hooks (setq-local pcomplete-command-completion-function #'org-pcomplete-initial) (setq-local pcomplete-command-name-function #'org-command-at-point) @@ -5049,8 +5204,8 @@ stacked delimiters is N. Escaping delimiters is not possible." (when verbatim? (org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0)) - (remove-text-properties (match-beginning 2) (match-end 2) - '(display t invisible t intangible t))) + (org-remove-text-properties (match-beginning 2) (match-end 2) + '(display t invisible t intangible t))) (add-text-properties (match-beginning 2) (match-end 2) '(font-lock-multiline t org-emphasis t)) (when (and org-hide-emphasis-markers @@ -5165,7 +5320,7 @@ This includes angle, plain, and bracket links." (if (not (eq 'bracket style)) (add-text-properties start end properties) ;; Handle invisible parts in bracket links. - (remove-text-properties start end '(invisible nil)) + (org-remove-text-properties start end '(invisible nil)) (let ((hidden (append `(invisible ,(or (org-link-get-parameter type :display) @@ -5185,8 +5340,8 @@ This includes angle, plain, and bracket links." (defun org-activate-code (limit) (when (re-search-forward "^[ \t]*\\(:\\(?: .*\\|$\\)\n?\\)" limit t) (org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0)) - (remove-text-properties (match-beginning 0) (match-end 0) - '(display t invisible t intangible t)) + (org-remove-text-properties (match-beginning 0) (match-end 0) + '(display t invisible t intangible t)) t)) (defcustom org-src-fontify-natively t @@ -5257,8 +5412,8 @@ by a #." (setq block-end (match-beginning 0)) ; includes the final newline. (when quoting (org-remove-flyspell-overlays-in bol-after-beginline nl-before-endline) - (remove-text-properties beg end-of-endline - '(display t invisible t intangible t))) + (org-remove-text-properties beg end-of-endline + '(display t invisible t intangible t))) (add-text-properties beg end-of-endline '(font-lock-fontified t font-lock-multiline t)) (org-remove-flyspell-overlays-in beg bol-after-beginline) @@ -5312,9 +5467,9 @@ by a #." '(font-lock-fontified t face org-document-info)))) ((string-prefix-p "+caption" dc1) (org-remove-flyspell-overlays-in (match-end 2) (match-end 0)) - (remove-text-properties (match-beginning 0) (match-end 0) - '(display t invisible t intangible t)) - ;; Handle short captions + (org-remove-text-properties (match-beginning 0) (match-end 0) + '(display t invisible t intangible t)) + ;; Handle short captions. (save-excursion (beginning-of-line) (looking-at (rx (group (zero-or-more blank) @@ -5335,8 +5490,8 @@ by a #." '(font-lock-fontified t face font-lock-comment-face))) (t ;; Just any other in-buffer setting, but not indented (org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0)) - (remove-text-properties (match-beginning 0) (match-end 0) - '(display t invisible t intangible t)) + (org-remove-text-properties (match-beginning 0) (match-end 0) + '(display t invisible t intangible t)) (add-text-properties beg (match-end 0) '(font-lock-fontified t face org-meta-line)) t)))))) @@ -5721,35 +5876,59 @@ needs to be inserted at a specific position in the font-lock sequence.") (decompose-region (point-min) (point-max)) (message "Entities are now displayed as plain text")))) -(defvar-local org-custom-properties-overlays nil - "List of overlays used for custom properties.") +(defvar-local org-custom-properties-hidden-p nil + "Non-nil when custom properties are hidden.") + +(defcustom org-custom-properties-hide-emptied-drawers nil + "Non-nil means that drawers containing only `org-custom-properties' will be hidden together with the properties." + :group 'org + :type '(choice + (const :tag "Don't hide emptied drawers" nil) + (const :tag "Hide emptied drawers" t))) (defun org-toggle-custom-properties-visibility () "Display or hide properties in `org-custom-properties'." (interactive) - (if org-custom-properties-overlays - (progn (mapc #'delete-overlay org-custom-properties-overlays) - (setq org-custom-properties-overlays nil)) + (require 'org-macs) + (add-to-invisibility-spec 'org-hide-custom-property) + (add-to-list 'org--invisible-spec-priority-list 'org-hide-custom-property) + (if org-custom-properties-hidden-p + (let (match) + (setq org-custom-properties-hidden-p nil) + (org-with-wide-buffer + (goto-char (point-min)) + (with-silent-modifications + (while (setq match (text-property-search-forward (org--get-buffer-local-invisible-property-symbol 'org-hide-custom-property) 'org-hide-custom-property t)) + (org-flag-region (prop-match-beginning match) + (prop-match-end match) + nil 'org-hide-custom-property))))) (when org-custom-properties + (setq org-custom-properties-hidden-p t) (org-with-wide-buffer - (goto-char (point-min)) - (let ((regexp (org-re-property (regexp-opt org-custom-properties) t t))) + (let* ((regexp (org-re-property (regexp-opt org-custom-properties) t t)) + (regexp-drawer (format "%s\n\\(?:%s\\)+\n%s" + (replace-regexp-in-string "\\$$" "" org-drawer-regexp) + (replace-regexp-in-string "\\(^\\^\\|\\$$\\)" "" regexp) + (replace-regexp-in-string "^\\^" "" org-property-end-re)))) + + (when org-custom-properties-hide-emptied-drawers + (goto-char (point-min)) + (while (re-search-forward regexp-drawer nil t) + (with-silent-modifications + (org-flag-region (1- (match-beginning 0)) (match-end 0) t 'org-hide-custom-property)))) + + (goto-char (point-min)) (while (re-search-forward regexp nil t) (let ((end (cdr (save-match-data (org-get-property-block))))) (when (and end (< (point) end)) ;; Hide first custom property in current drawer. - (let ((o (make-overlay (match-beginning 0) (1+ (match-end 0))))) - (overlay-put o 'invisible t) - (overlay-put o 'org-custom-property t) - (push o org-custom-properties-overlays)) - ;; Hide additional custom properties in the same drawer. - (while (re-search-forward regexp end t) - (let ((o (make-overlay (match-beginning 0) (1+ (match-end 0))))) - (overlay-put o 'invisible t) - (overlay-put o 'org-custom-property t) - (push o org-custom-properties-overlays))))) - ;; Each entry is limited to a single property drawer. - (outline-next-heading))))))) + (with-silent-modifications + (org-flag-region (match-beginning 0) (1+ (match-end 0)) t 'org-hide-custom-property) + ;; Hide additional custom properties in the same drawer. + (while (re-search-forward regexp end t) + (org-flag-region (match-beginning 0) (1+ (match-end 0)) t 'org-hide-custom-property)))))) + ;; Each entry is limited to a single property drawer. + (outline-next-heading)))))) (defun org-fontify-entities (limit) "Find an entity to fontify." @@ -5858,10 +6037,11 @@ If TAG is a number, get the corresponding match group." (inhibit-modification-hooks t) deactivate-mark buffer-file-name buffer-file-truename) (decompose-region beg end) - (remove-text-properties beg end - '(mouse-face t keymap t org-linked-text t - invisible t intangible t - org-emphasis t)) + (org-remove-text-properties beg end + '(mouse-face t keymap t org-linked-text t + invisible t + intangible t + org-emphasis t)) (org-remove-font-lock-display-properties beg end))) (defconst org-script-display '(((raise -0.3) (height 0.7)) @@ -5969,6 +6149,29 @@ open and agenda-wise Org files." ;;;; Headlines visibility +(defun org-hide-entry () + "Hide the body directly following this heading." + (interactive) + (save-excursion + (outline-back-to-heading) + (outline-end-of-heading) + (org-flag-region (point) (progn (outline-next-preface) (point)) t 'outline))) + +(defun org-hide-subtree () + "Hide everything after this heading at deeper levels." + (interactive) + (org-flag-subtree t)) + +(defun org-hide-sublevels (levels) + "Hide everything but the top LEVELS levels of headers, in whole buffer. +This also unhides the top heading-less body, if any. + +Interactively, the prefix argument supplies the value of LEVELS. +When invoked without a prefix argument, LEVELS defaults to the level +of the current heading, or to 1 if the current line is not a heading." + (cl-letf (((symbol-function 'outline-flag-region) #'org-flag-region)) + (org-hide-sublevels levels))) + (defun org-show-entry () "Show the body directly following this heading. Show the heading too, if it is currently invisible." @@ -5980,13 +6183,24 @@ Show the heading too, if it is currently invisible." (line-end-position 0) (save-excursion (if (re-search-forward - (concat "[\r\n]\\(" org-outline-regexp "\\)") nil t) + (concat "[\r\n]\\(" (org-get-limited-outline-regexp) "\\)") nil t) (match-beginning 1) (point-max))) nil 'outline) (org-cycle-hide-drawers 'children)))) +(defun org-show-heading () + "Show the current heading and move to its end." + (org-flag-region (- (point) + (if (bobp) 0 + (if (and outline-blank-line + (eq (char-before (1- (point))) ?\n)) + 2 1))) + (progn (outline-end-of-heading) (point)) + nil + 'outline)) + (defun org-show-children (&optional level) "Show all direct subheadings of this heading. Prefix arg LEVEL is how many levels below the current level @@ -6030,6 +6244,11 @@ heading to appear." (org-flag-region (point) (save-excursion (org-end-of-subtree t t)) nil 'outline)) +(defun org-show-branches () + "Show all subheadings of this heading, but not their bodies." + (interactive) + (org-show-children 1000)) + ;;;; Blocks and drawers visibility (defun org--hide-wrapper-toggle (element category force no-error) @@ -6062,7 +6281,9 @@ Return a non-nil value when toggling is successful." ;; at the block closing line. (unless (let ((eol (line-end-position))) (and (> eol start) (/= eol end))) - (let* ((spec (if (eq category 'block) 'org-hide-block 'outline)) + (let* ((spec (cond ((eq category 'block) 'org-hide-block) + ((eq category 'drawer) 'org-hide-drawer) + (t 'outline))) (flag (cond ((eq force 'off) nil) (force t) @@ -6115,24 +6336,24 @@ Return a non-nil value when toggling is successful." (defun org-hide-drawer-all () "Fold all drawers in the current buffer." + (org-show-all '(drawers)) (save-excursion (goto-char (point-min)) (while (re-search-forward org-drawer-regexp nil t) - (let* ((pair (get-char-property-and-overlay (line-beginning-position) - 'invisible)) - (o (cdr-safe pair))) - (if (overlayp o) (goto-char (overlay-end o)) ;invisible drawer - (pcase (get-char-property-and-overlay (point) 'invisible) - (`(outline . ,o) (goto-char (overlay-end o))) ;already folded - (_ - (let* ((drawer (org-element-at-point)) - (type (org-element-type drawer))) - (when (memq type '(drawer property-drawer)) - (org-hide-drawer-toggle t nil drawer) - ;; Make sure to skip drawer entirely or we might flag it - ;; another time when matching its ending line with - ;; `org-drawer-regexp'. - (goto-char (org-element-property :end drawer))))))))))) + (let* ((drawer (org-element-at-point)) + (type (org-element-type drawer))) + (when (memq type '(drawer property-drawer)) + ;; We are sure regular drawers are unfolded because of + ;; `org-show-all' call above. However, property drawers may + ;; be folded, or in a folded headline. In that case, do not + ;; re-hide it. + (unless (and (eq type 'property-drawer) + (eq 'org-hide-drawer (get-char-property (point) 'invisible))) + (org-hide-drawer-toggle t nil drawer)) + ;; Make sure to skip drawer entirely or we might flag it + ;; another time when matching its ending line with + ;; `org-drawer-regexp'. + (goto-char (org-element-property :end drawer))))))) (defun org-cycle-hide-drawers (state) "Re-hide all drawers after a visibility state change. @@ -6147,9 +6368,10 @@ STATE should be one of the symbols listed in the docstring of (t (save-excursion (org-end-of-subtree t t)))))) (org-with-point-at beg (while (re-search-forward org-drawer-regexp end t) - (pcase (get-char-property-and-overlay (point) 'invisible) + (pcase (get-char-property (point) 'invisible) ;; Do not fold already folded drawers. - (`(outline . ,o) (goto-char (overlay-end o))) + ('outline + (goto-char (min end (next-single-char-property-change (point) 'invisible)))) (_ (let ((drawer (org-element-at-point))) (when (memq (org-element-type drawer) '(drawer property-drawer)) @@ -6172,31 +6394,13 @@ By default, the function expands headings, blocks and drawers. When optional argument TYPE is a list of symbols among `blocks', `drawers' and `headings', to only expand one specific type." (interactive) - (let ((types (or types '(blocks drawers headings)))) - (when (memq 'blocks types) - (org-flag-region (point-min) (point-max) nil 'org-hide-block)) - (cond - ;; Fast path. Since headings and drawers share the same - ;; invisible spec, clear everything in one go. - ((and (memq 'headings types) - (memq 'drawers types)) - (org-flag-region (point-min) (point-max) nil 'outline)) - ((memq 'headings types) - (org-flag-region (point-min) (point-max) nil 'outline) - (org-cycle-hide-drawers 'all)) - ((memq 'drawers types) - (save-excursion - (goto-char (point-min)) - (while (re-search-forward org-drawer-regexp nil t) - (let* ((pair (get-char-property-and-overlay (line-beginning-position) - 'invisible)) - (o (cdr-safe pair))) - (if (overlayp o) (goto-char (overlay-end o)) - (pcase (get-char-property-and-overlay (point) 'invisible) - (`(outline . ,o) - (goto-char (overlay-end o)) - (delete-overlay o)) - (_ nil)))))))))) + (dolist (type (or types '(blocks drawers headings))) + (org-flag-region (point-min) (point-max) nil + (pcase type + (`blocks 'org-hide-block) + (`drawers 'org-hide-drawer) + (`headings 'outline) + (_ (error "Invalid type: %S" type)))))) ;;;###autoload (defun org-cycle (&optional arg) @@ -6552,7 +6756,7 @@ With a numeric prefix, show all headlines up to that level." (org-narrow-to-subtree) (org-content)))) ((or "all" "showall") - (outline-show-subtree)) + (org-show-subtree)) (_ nil))) (org-end-of-subtree))))))) @@ -6625,7 +6829,7 @@ This function is the default value of the hook `org-cycle-hook'." (while (re-search-forward re nil t) (when (and (not (org-invisible-p)) (org-invisible-p (line-end-position))) - (outline-hide-entry)))) + (org-hide-entry)))) (org-cycle-hide-drawers 'all) (org-cycle-show-empty-lines 'overview))))) @@ -6697,10 +6901,11 @@ information." (org-show-entry) ;; If point is hidden within a drawer or a block, make sure to ;; expose it. - (dolist (o (overlays-at (point))) - (when (memq (overlay-get o 'invisible) - '(org-hide-block org-hide-drawer outline)) - (delete-overlay o))) + (when (memq (get-text-property (point) 'invisible) + '(org-hide-block org-hide-drawer)) + (let ((spec (get-text-property (point) 'invisible)) + (region (org--find-text-property-region (point) 'invisible))) + (org-flag-region (car region) (cdr region) nil spec))) (unless (org-before-first-heading-p) (org-with-limited-levels (cl-case detail @@ -6916,9 +7121,10 @@ unconditionally." ;; When INVISIBLE-OK is non-nil, ensure newly created headline ;; is visible. (unless invisible-ok - (pcase (get-char-property-and-overlay (point) 'invisible) - (`(outline . ,o) - (move-overlay o (overlay-start o) (line-end-position 0))) + (pcase (get-char-property (point) 'invisible) + ('outline + (let ((region (org--find-text-property-region (point) 'invisible))) + (org-flag-region (line-end-position 0) (cdr region) nil 'outline))) (_ nil)))) ;; At a headline... ((org-at-heading-p) @@ -7515,7 +7721,6 @@ case." (setq txt (buffer-substring beg end)) (org-save-markers-in-region beg end) (delete-region beg end) - (org-remove-empty-overlays-at beg) (unless (= beg (point-min)) (org-flag-region (1- beg) beg nil 'outline)) (unless (bobp) (org-flag-region (1- (point)) (point) nil 'outline)) (and (not (bolp)) (looking-at "\n") (forward-char 1)) @@ -7677,7 +7882,7 @@ When REMOVE is non-nil, remove the subtree from the clipboard." (skip-chars-forward " \t\n\r") (setq beg (point)) (when (and (org-invisible-p) visp) - (save-excursion (outline-show-heading))) + (save-excursion (org-show-heading))) ;; Shift if necessary. (unless (= shift 0) (save-restriction @@ -8119,7 +8324,7 @@ function is being called interactively." (point)) what "children") (goto-char start) - (outline-show-subtree) + (org-show-subtree) (outline-next-heading)) (t ;; we will sort the top-level entries in this file @@ -10736,7 +10941,8 @@ narrowing." (let ((beg (point))) (insert ":" drawer ":\n:END:\n") (org-indent-region beg (point)) - (org-flag-region (line-end-position -1) (1- (point)) t 'outline)) + (org-flag-region + (line-end-position -1) (1- (point)) t 'org-hide-drawer)) (end-of-line -1))))) (t (org-end-of-meta-data org-log-state-notes-insert-after-drawers) @@ -13173,7 +13379,7 @@ drawer is immediately hidden." (inhibit-read-only t)) (unless (bobp) (insert "\n")) (insert ":PROPERTIES:\n:END:") - (org-flag-region (line-end-position 0) (point) t 'outline) + (org-flag-region (line-end-position 0) (point) t 'org-hide-drawer) (when (or (eobp) (= begin (point-min))) (insert "\n")) (org-indent-region begin (point)))))) @@ -16553,7 +16759,7 @@ The detailed reaction depends on the user option `org-catch-invisible-edits'." (when (or invisible-at-point invisible-before-point) (when (eq org-catch-invisible-edits 'error) (user-error "Editing in invisible areas is prohibited, make them visible first")) - (if (and org-custom-properties-overlays + (if (and org-custom-properties-hidden-p (y-or-n-p "Display invisible properties in this buffer? ")) (org-toggle-custom-properties-visibility) ;; Make the area visible @@ -17636,11 +17842,11 @@ Move point to the beginning of first heading or end of buffer." (defun org-show-branches-buffer () "Show all branches in the buffer." (org-flag-above-first-heading) - (outline-hide-sublevels 1) + (org-hide-sublevels 1) (unless (eobp) - (outline-show-branches) + (org-show-branches) (while (outline-get-next-sibling) - (outline-show-branches))) + (org-show-branches))) (goto-char (point-min))) (defun org-kill-note-or-show-branches () @@ -17654,8 +17860,8 @@ Move point to the beginning of first heading or end of buffer." (t (let ((beg (progn (org-back-to-heading) (point))) (end (save-excursion (org-end-of-subtree t t) (point)))) - (outline-hide-subtree) - (outline-show-branches) + (org-hide-subtree) + (org-show-branches) (org-hide-archived-subtrees beg end))))) (defun org-delete-indentation (&optional arg) @@ -17811,9 +18017,9 @@ Otherwise, call `org-show-children'. ARG is the level to hide." (if (org-before-first-heading-p) (progn (org-flag-above-first-heading) - (outline-hide-sublevels (or arg 1)) + (org-hide-sublevels (or arg 1)) (goto-char (point-min))) - (outline-hide-subtree) + (org-hide-subtree) (org-show-children arg)))) (defun org-ctrl-c-star () @@ -20489,20 +20695,20 @@ With ARG, repeats or can move backward if negative." (end-of-line)) (while (and (< arg 0) (re-search-backward regexp nil :move)) (unless (bobp) - (while (pcase (get-char-property-and-overlay (point) 'invisible) - (`(outline . ,o) - (goto-char (overlay-start o)) - (re-search-backward regexp nil :move)) - (_ nil)))) + (pcase (get-char-property (point) 'invisible) + ('outline + (goto-char (car (org--find-text-property-region (point) 'invisible))) + (beginning-of-line)) + (_ nil))) (cl-incf arg)) - (while (and (> arg 0) (re-search-forward regexp nil t)) - (while (pcase (get-char-property-and-overlay (point) 'invisible) - (`(outline . ,o) - (goto-char (overlay-end o)) - (re-search-forward regexp nil :move)) - (_ - (end-of-line) - nil))) ;leave the loop + (while (and (> arg 0) (re-search-forward regexp nil :move)) + (pcase (get-char-property (point) 'invisible) + ('outline + (goto-char (cdr (org--find-text-property-region (point) 'invisible))) + (skip-chars-forward " \t\n") + (end-of-line)) + (_ + (end-of-line))) (cl-decf arg)) (if (> arg 0) (goto-char (point-max)) (beginning-of-line)))) @@ -20957,6 +21163,80 @@ Started from `gnus-info-find-node'." (t default-org-info-node)))))) + +;;; Make isearch search in some text hidden via text propertoes + +(defvar org--isearch-overlays nil + "List of overlays temporarily created during isearch. +This is used to allow searching in regions hidden via text properties. +As for [2020-05-09 Sat], Isearch only has special handling of hidden overlays. +Any text hidden via text properties is not revealed even if `search-invisible' +is set to 't.") + +;; Not sure if it needs to be a user option +;; One might want to reveal hidden text in, for example, hidden parts of the links. +;; Currently, hidden text in links is never revealed by isearch. +(defvar org-isearch-specs '(org-hide-block + org-hide-drawer) + "List of text invisibility specs to be searched by isearch. +By default ([2020-05-09 Sat]), isearch does not search in hidden text, +which was made invisible using text properties. Isearch will be forced +to search in hidden text with any of the listed 'invisible property value.") + +(defun org--create-isearch-overlays (beg end) + "Replace text property invisibility spec by overlays between BEG and END. +All the regions with invisibility text property spec from +`org-isearch-specs' will be changed to use overlays instead +of text properties. The created overlays will be stored in +`org--isearch-overlays'." + (let ((pos beg)) + (while (< pos end) + (when-let* ((spec (get-text-property pos 'invisible)) + (spec (memq spec org-isearch-specs)) + (region (org--find-text-property-region pos 'invisible))) + (setq spec (get-text-property pos 'invisible)) + ;; Changing text properties is considered buffer modification. + ;; We do not want it here. + (with-silent-modifications + ;; The overlay is modelled after `org-flag-region' [2020-05-09 Sat] + ;; overlay for 'outline blocks. + (let ((o (make-overlay (car region) (cdr region) nil 'front-advance))) + (overlay-put o 'evaporate t) + (overlay-put o 'invisible spec) + ;; `delete-overlay' here means that spec information will be lost + ;; for the region. The region will remain visible. + (overlay-put o 'isearch-open-invisible #'delete-overlay) + (push o org--isearch-overlays)) + (org-flag-region (car region) (cdr region) nil spec))) + (setq pos (next-single-property-change pos 'invisible nil end))))) + +(defun org--isearch-filter-predicate (beg end) + "Return non-nil if text between BEG and END is deemed visible by Isearch. +This function is intended to be used as `isearch-filter-predicate'. +Unlike `isearch-filter-visible', make text with 'invisible text property +value listed in `org-isearch-specs' visible to Isearch." + (org--create-isearch-overlays beg end) ;; trick isearch by creating overlays in place of invisible text + (isearch-filter-visible beg end)) + +(defun org--clear-isearch-overlay (ov) + "Convert OV region back into using text properties." + (when-let ((spec (overlay-get ov 'invisible))) ;; ignore deleted overlays + ;; Changing text properties is considered buffer modification. + ;; We do not want it here. + (with-silent-modifications + (org-flag-region (overlay-start ov) (overlay-end ov) t spec))) + (when (member ov isearch-opened-overlays) + (setq isearch-opened-overlays (delete ov isearch-opened-overlays))) + (delete-overlay ov)) + +(defun org--clear-isearch-overlays () + "Convert overlays from `org--isearch-overlays' back into using text properties." + (when org--isearch-overlays + (mapc #'org--clear-isearch-overlay org--isearch-overlays) + (setq org--isearch-overlays nil))) + + + ;;; Finish up (add-hook 'org-mode-hook ;remove overlays when changing major mode --=-=-= Content-Type: text/plain Ihor Radchenko writes: > Hello, > > [The patch itself will be provided in the following email] > > I have four more updates from the previous version of the patch: > > 1. All the code handling modifications in folded drawers/blocks is moved > to after-change-function. It works as follows: > - if any text is inserted in the middle of hidden region, that text > is also hidden; > - if BEGIN/END line of a folded drawer do not match org-drawer-regexp > and org-property-end-re, unfold it; > - if org-property-end-re or new org-outline-regexp-bol is inserted in > the middle of the drawer, unfold it; > - the same logic for blocks. > > 2. The text property stack is rewritten using char-property-alias-alist. > This is faster in comparison with previous approach, which involved > modifying all the text properties every timer org-flag-region was > called. > > 3. org-toggle-custom-properties-visibility is rewritten using text > properties. I also took a freedom to implement a new feature here. > Now, setting new `org-custom-properties-hide-emptied-drawers' to > non-nil will result in hiding the whole property drawer if it > contains only org-custom-properties. > > 4. This patch should work against 1aa095ccf. However, the merge was not > trivial here. Recent commits actively used the fact that drawers and > outlines are hidden via 'outline invisibility spec, which is not the > case in this branch. I am not confident that I did not break anything > during the merge, especially 1aa095ccf. > > ----------------------------------------------------------------------- > ----------------------------------------------------------------------- > > More details on the new implementation for tracking changes: > >> I gave you a few ideas to quickly check if a change requires expansion, >> in an earlier mail. I suggest to start out from that. Let me know if you >> have questions about it. > > All the code lives in org-after-change-function. I tried to incorporate > the earlier Nicholas' suggestions, except the parts related to > intersecting blocks and drawers. I am not sure if I understand the > parsing priority of blocks vs. drawers. > > ----------------------------------------------------------------------- > ----------------------------------------------------------------------- > > More details on the text property stack: > > The earlier version of the code literally used stack to save > pre-existing 'invisibility specs in org-flag-region. This was done on > every invocation of org-flag-region, which made org-flag-region > significantly slower. I re-implemented the same feature using > char-property-alias-alist. Now, different invisibility specs live in > separate text properties and can be safely modified independently. The > specs are applied according to org--invisible-spec-priority-list. A side > effect of current implementation is that char-property-alias-alist is > fully controlled by org. All the pre-existing settings for 'invisible > text property will be overwritten by org. > >> `gensym' is just a shorter, and somewhat standard way, to create a new >> uninterned symbol with a given prefix. You seem to re-invent it. What >> you do with that new symbol is orthogonal to that suggestion, of course. > > I do not think that `gensym' is suitable here. We don't want a new > symbol every time org--get-buffer-local-invisible-property-symbol is > called. It should return the same symbol if it is called from the same > buffer multiple times. > > ----------------------------------------------------------------------- > ----------------------------------------------------------------------- > > More details on the org-toggle-custom-properties-visibility: > > The implementation showcases how to introduce new invisibility specs to > org. Apart from expected (add-to-invisibility-spec 'org-hide-custom-property) > one also needs to add the spec into org--invisible-spec-priority-list: > > (add-to-list 'org--invisible-spec-priority-list 'org-hide-custom-property) > > Searching for text with the given invisibility spec is done as > follows: > > (text-property-search-forward (org--get-buffer-local-invisible-property-symbol 'org-hide-custom-property) 'org-hide-custom-property t) > > This last piece of code is probably not the most elegant. I am thinking > if creating some higher-level interface would be more reasonable here. > What do you think? > > > The new customisation `org-custom-properties-hide-emptied-drawers' > sounds logical for me since empty property drawers left after invoking > org-toggle-custom-properties-visibility are rather useless according to > my experience. If one already wants to hide parts of property drawers, I > do not see a reason to show leftover > > :PROPERTIES: > :END: > > ----------------------------------------------------------------------- > ----------------------------------------------------------------------- > > More details on the merge with the latest master: > > I tried my best to not break anything. However, I am not sure if I > understand all the recent commits. Could someone take a look if there is > anything suspicious in org-next-visible-heading? > > Also, I have seen some optimisations making use of the fact that drawers > and headlines both use 'outline invisibility spec. This change in the > implementation details supposed to improve performance and should not be > necessary if this patch is going to be merged. Would it be possible to > refrain from abusing this particular implementation detail in the > nearest commits on master (unless really necessary)? > > ----------------------------------------------------------------------- > ----------------------------------------------------------------------- > > Further work: > > I would like to finalise the current patch and work on other code using > overlays separately. This patch is already quite complicated as is. I do > not want to introduce even more potential bugs by working on things not > directly affected by this version of the patch. > > Best, > Ihor > > > Nicolas Goaziou writes: > >> Ihor Radchenko writes: >> >>>> See also `gensym'. Do we really need to use it for something else than >>>> `invisible'? If not, the tool doesn't need to be generic. >>> >>> For now, I also use it for buffer-local 'invisible stack. The stack is >>> needed to preserve folding state of drawers/blocks inside folded >>> outline. Though I am thinking about replacing the stack with separate >>> text properties, like 'invisible-outline-buffer-local + >>> 'invisible-drawer-buffer-local + 'invisible-block-buffer-local. >>> Maintaining stack takes a noticeable percentage of CPU time in profiler. >>> >>> org--get-buffer-local-text-property-symbol must take care about >>> situation with indirect buffers. When an indirect buffer is created from >>> some org buffer, the old value of char-property-alias-alist is carried >>> over. We need to detect this case and create new buffer-local symbol, >>> which is unique to the newly created buffer (but not create it if the >>> buffer-local property is already there). Then, the new symbol must >>> replace the old alias in char-property-alias-alist + old folding state >>> must be preserved (via copying the old invisibility specs into the new >>> buffer-local text property). I do not see how gensym can benefit this >>> logic. >> >> `gensym' is just a shorter, and somewhat standard way, to create a new >> uninterned symbol with a given prefix. You seem to re-invent it. What >> you do with that new symbol is orthogonal to that suggestion, of course. >> >>>> OK, but this may not be sufficient if we want to do slightly better than >>>> overlays in that area. This is not mandatory, though. >>> >>> Could you elaborate on what can be "slightly better"? >> >> IIRC, I gave examples of finer control of folding state after a change. >> Consider this _folded_ drawer: >> >> :BEGIN: >> Foo >> :END: >> >> Inserting ":END" in it should not unfold it, as it is currently the case >> with overlays, >> >> :BEGIN >> Foo >> :END >> :END: >> >> but a soon as the last ":" is inserted, the initial drawer could be >> expanded. >> >> :BEGIN >> Foo >> :END: >> :END: >> >> The latter case is not currently handled by overlays. This is what >> I call "slightly better". >> >> Also, note that this change is not related to opening and closing lines >> of the initial drawer, so sticking text properties on them would not >> help here. >> >> Another case is modifying those borders, e.g., >> >> >> :BEGIN: :BEGIN: >> Foo ------> Foo >> :END: :ND: >> >> which should expand the drawer. Your implementation catches this, but >> I'm pointing out that current implementation with overlays does not. >> Even though that's not strictly required for compatibility with >> overlays, it is a welcome slight improvement. >> >>>> As discussed before, I don't think you need to use `modification-hooks' >>>> or `insert-behind-hooks' if you already use `after-change-functions'. >>>> >>>> `after-change-functions' are also triggered upon text properties >>>> changes. So, what is the use case for the other hooks? >>> >>> The problem is that `after-change-functions' cannot be a text property. >>> Only `modification-hooks' and `insert-in-front/behind-hooks' can be a >>> valid text property. If we use `after-change-functions', they will >>> always be triggered, regardless if the change was made inside or outside >>> folded region. >> >> As discussed, text properties are local to the change, but require extra >> care when moving text around. You also observed serious overhead when >> using them. >> >> OTOH, even if `a-c-f' is not local, you can quickly determine if the >> change altered a folded element, so the overhead is limited, i.e., >> mostly checking for a text property at a given buffer position. >> >> To be clear, I initially thought that text properties were a superior >> choice, but I changed my mind a while ago, and I thought you had, too. >> IOW, `after-change-functions' is the way to go, since you have no strong >> reason to stick to text properties for this kind of function. >> >>>>> :asd: >>>>> :drawer: >>>>> lksjdfksdfjl >>>>> sdfsdfsdf >>>>> :end: >>>>> >>>>> If :asd: was inserted in front of folded :drawer:, changes in :drawer: >>>>> line of the new folded :asd: drawer would reveal the text between >>>>> :drawer: and :end:. >>>>> >>>>> Let me know what you think on this. >>> >>>> I have first to understand the use case for `modification-hook'. But >>>> I think unfolding is the right thing to do in this situation, isn't it? >>> >>> That situation arises because the modification-hooks from ":drawer:" >>> (they are set via text properties) only have information about the >>> :drawer:...:end: drawer before the modifications (they were set when >>> :drawer: was folded last time). So, they will only unfold a part of the >>> new :asd: drawer. I do not see a simple way to unfold everything without >>> re-parsing the drawer around the changed text. >> >> Oh! I misread your message. I withdraw what I wrote. In this case, we >> don't want to unfold anything. The situation is not worse than what we >> have now, and trying to fix it would have repercussions down in the >> buffer, e.g., expanding drawers screen below. >> >> As a rule of thumb, I think we can pay attention to changes in the >> folded text, and its immediate surroundings (e.g., the opening line, >> which is not folded), but no further. >> >> As written above, slight changes are welcome, but let's not go overboard >> and parse a whole section just to know if we can expand a drawer. >> >>> Actually, I am quite unhappy with the performance of modification-hooks >>> set via text properties (I am using this patch on my Emacs during this >>> week). It appears that setting the text properties costs a significant >>> CPU time in practice, even though running the hooks is pretty fast. >>> I will think about a way to handle modifications using global >>> after-change-functions. >> >> That's better, IMO. >> >> I gave you a few ideas to quickly check if a change requires expansion, >> in an earlier mail. I suggest to start out from that. Let me know if you >> have questions about it. > > -- > Ihor Radchenko, > PhD, > Center for Advancing Materials Performance from the Nanoscale (CAMP-nano) > State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong University, Xi'an, China > Email: yantar92@gmail.com, ihor_radchenko@alumni.sutd.edu.sg -- Ihor Radchenko, PhD, Center for Advancing Materials Performance from the Nanoscale (CAMP-nano) State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong University, Xi'an, China Email: yantar92@gmail.com, ihor_radchenko@alumni.sutd.edu.sg --=-=-=--