Recent folding issues

Recent folding issues

[Jack Kamm]

I started noticing a couple issues with folding, after updating my
org-mode in recent months:

1. Inserting text below or above a folded headline will cause it to
unfold. I am not sure if this is an intentional change, but I find the
new behavior confusing -- usually I am trying to enter a new headline,
and the unfolding causes me to lose my context and forget what level I
wanted my new headline to be.

2. When folded, I frequently found multiple headlines to be displayed on
the same line, like so:

* Headline 1...
* Headline 2...* Headline 3...
* Headline 4

Hitting Shift-Tab a few times (org-global-cycle) usually fixed the
problem.

After reading ORG-NEWS, I found out about org-fold-core-style, and have
set it to 'overlays, which solved both of my problems.

However, this makes me wonder whether it should be the default
value. The help for it says:

> Can be either ‘text-properties’ or ‘overlays’.
> The former is faster on large files, while the latter is generally
> less error-prone.

Since the latter is less error-prone, shouldn't it be the default? And
then a user can switch the value if they have really large files and
need better performance. It's great that performance is being improved
for large files, but I'm not sure this is ready to be the default yet.

Re: Recent folding issues

[Ihor Radchenko]

Jack Kamm <jackkamm@gmail.com> writes:

> I started noticing a couple issues with folding, after updating my
> org-mode in recent months:

May you provide the output of M-x org-version?

> 1. Inserting text below or above a folded headline will cause it to
> unfold. I am not sure if this is an intentional change, but I find the
> new behavior confusing -- usually I am trying to enter a new headline,
> and the unfolding causes me to lose my context and forget what level I
> wanted my new headline to be.

I cannot reproduce. Please, update your Org to the latest version, try
to reproduce, and provide the detailed steps required to obtain the
confusing behaviour you are seeing.
See https://orgmode.org/manual/Feedback.html

> 2. When folded, I frequently found multiple headlines to be displayed on
> the same line, like so:
>
> * Headline 1...
> * Headline 2...* Headline 3...
> * Headline 4
>
> Hitting Shift-Tab a few times (org-global-cycle) usually fixed the
> problem.

Again, I cannot reproduce. If you can reliably obtain this erroneous
behaviour, please give us the information how to reproduce it. Then, we
will be able to fix it.

> After reading ORG-NEWS, I found out about org-fold-core-style, and have
> set it to 'overlays, which solved both of my problems.
>
> However, this makes me wonder whether it should be the default
> value. The help for it says:
>
>> Can be either ‘text-properties’ or ‘overlays’.
>> The former is faster on large files, while the latter is generally
>> less error-prone.
>
> Since the latter is less error-prone, shouldn't it be the default? And
> then a user can switch the value if they have really large files and
> need better performance. It's great that performance is being improved
> for large files, but I'm not sure this is ready to be the default yet.

I now clarified the docstring of org-fold-core-style.
"Error-prone" there refers to third-party packages that are relying on
the old implementation detail of folding.

Note that this feature is not a part of the stable Org release. It is
the development branch. We are trying our best to avoid bugs, but bugs
are pretty hard to avoid when introducing major changes like this.

Of course, we will fix the reported bugs given that enough information
is provided to reproduce them.

So, no, 'overlays should not be the default. You may also check out
https://blog.tecosaur.com/tmio/2022-05-31-folding.html

Best,
Ihor

Re: Recent folding issues

[William Denton]

On 11 July 2022, Jack Kamm wrote:

> 2. When folded, I frequently found multiple headlines to be displayed on
> the same line, like so:
>
> * Headline 1...
> * Headline 2...* Headline 3...
> * Headline 4
>
> Hitting Shift-Tab a few times (org-global-cycle) usually fixed the
> problem.

That's very much like a problem I've been getting (see the message I just sent 
to the list), which Ihor has also been helping with.  For me, just now when the 
problem happened again (I realize this is anecdotal, not reproducible) hitting 
TAB a lot on the headline might work, but hitting TAB on other headlines did 
nothing, and Shift-TAB would expand /some/ headlines but not the one where the 
pointer was!

So I'm in the same boat as you, and trying various things to figure out what's 
going on.


Cheers,

Bill

--
William Denton
https://www.miskatonic.org/
Librarian, artist and licensed private investigator.
Toronto, Canada

Re: Recent folding issues

[Jack Kamm]

[-- Attachment #1: Type: text/plain, Size: 2344 bytes --]

Hi Ihor,

> May you provide the output of M-x org-version?

Org mode version 9.5.4 (release_9.5.4-623-gc66bdb @ /home/jack/dev/org-mode/lisp/)

I also updated org-mode to the most recent version (last commit 5
hours ago).

> I cannot reproduce. Please, update your Org to the latest version, try
> to reproduce, and provide the detailed steps required to obtain the
> confusing behaviour you are seeing.

I attach a couple files for a minimal reproducible example of both
issues.

Steps to reproduce:

0. Download the attached init.el and test.org. Edit the absolute path
   of "test.org" in the capture template in "init.el" to the location
   you downloaded it.

   Then do "emacs -q --load init.el test.org"

   test.org should look like this now:

> * Headline 1
> * 2021
> * 2022
> * Headline 4

1. Next, do "M-x org-capture". Then type "c". Then create the following note:

> **** Test entry 1

Then press C-c C-c. Buffer should look like this now:

> * Headline 1
> * 2021
> * 2022
> 
> ** 2022-07 July
> 
> *** 2022-07-11 Monday
> 
> **** Test entry 1
> 
> * Headline 4

2. Shift-TAB until all headlines are collapsed. Buffer should look like this:

> * Headline 1
> * 2021
> * 2022...
> * Headline 4

3. Capture a second note: do M-x org-capture, type "c", and create the entry:

> **** Test entry 2

And press C-c C-c.

Now, an error appears: I observe two headlines running into each other
on the last line, like so:

> * Headline 1
> * 2021
> * 2022...
> ** 2022-07 July...
> *** 2022-07-11 Monday...
> **** Test entry 2...* Headline 4

Now, to produce the other issue. Shift-tab until all headlines are
collapsed again. Doing this a couple times will also fix the above
error, and the buffer should look like this now:

> * Headline 1
> * 2021
> * 2022...
> * Headline 4

Put the pointer at the start of the line "* Headline 4". Then press Enter.

A newline is placed above "* Headline 4", but also, the preceding
headline "* 2022", has been expanded, so the buffer looks like this to me:

> * Headline 1
> * 2021
> * 2022
> 
> ** 2022-07 July
> 
> *** 2022-07-11 Monday
> 
> **** Test entry 1
> 
> **** Test entry 2
> 
> 
> * Headline 4

However, I would instead expect to see the following:

> * Headline 1
> * 2021
> * 2022...
> 
> * Headline 4

Which is what I observe when org-fold-core-style is set to overlays.

Best,
Jack


[-- Attachment #2: init.el --]
[-- Type: text/plain, Size: 272 bytes --]

(add-to-list 'load-path "~/dev/org-mode/lisp")

(customize-set-variable
 'org-capture-templates
 '(("c" "capture" entry (file+olp+datetree
                         "~/dev/orgmode-devel/220711-folding-issues/test.org")
    "* %?"
    :empty-lines 1 :jump-to-captured t)))


[-- Attachment #3: test.org --]
[-- Type: text/plain, Size: 41 bytes --]


* Headline 1
* 2021
* 2022
* Headline 4

Re: Recent folding issues

[Ihor Radchenko]

[-- Attachment #1: Type: text/plain, Size: 387 bytes --]

Jack Kamm <jackkamm@gmail.com> writes:

>> I cannot reproduce. Please, update your Org to the latest version, try
>> to reproduce, and provide the detailed steps required to obtain the
>> confusing behaviour you are seeing.
>
> I attach a couple files for a minimal reproducible example of both
> issues.
>
> Steps to reproduce:

Thanks!
Can you try the attached patch set?

Best,
Ihor


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Revert-org.el-Improve-performance-of-deletion-comman.patch --]
[-- Type: text/x-patch, Size: 3995 bytes --]

From a79a742cbff01b7815cffba806c28ea45a4da63c Mon Sep 17 00:00:00 2001
Message-Id: <a79a742cbff01b7815cffba806c28ea45a4da63c.1657634233.git.yantar92@gmail.com>
From: Ihor Radchenko <yantar92@gmail.com>
Date: Tue, 12 Jul 2022 15:05:45 +0800
Subject: [PATCH 1/3] Revert "org.el: Improve performance of deletion commands"

This reverts commit 46df681336c83c826b367d2803f59560165bdeba.

The optimization broke folding fragility checks.
---
 lisp/org.el | 74 ++++++++++++++++++++++++++---------------------------
 1 file changed, 36 insertions(+), 38 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 3d4de5b4f..d85b5818e 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -16156,19 +16156,18 @@ (defun org-delete-backward-char (N)
 still be marked for re-alignment if the field did fill the entire column,
 because, in this case the deletion might narrow the column."
   (interactive "p")
-  (org-fold-core-ignore-modifications
-    (save-match-data
-      (org-fold-check-before-invisible-edit 'delete-backward)
-      (if (and (= N 1)
-	       (not overwrite-mode)
-	       (not (org-region-active-p))
-	       (not (eq (char-before) ?|))
-	       (save-excursion (skip-chars-backward " \t") (not (bolp)))
-	       (looking-at-p ".*?|")
-	       (org-at-table-p))
-	  (progn (forward-char -1) (org-delete-char 1))
-        (backward-delete-char N)
-        (org-fix-tags-on-the-fly)))))
+  (save-match-data
+    (org-fold-check-before-invisible-edit 'delete-backward)
+    (if (and (= N 1)
+	     (not overwrite-mode)
+	     (not (org-region-active-p))
+	     (not (eq (char-before) ?|))
+	     (save-excursion (skip-chars-backward " \t") (not (bolp)))
+	     (looking-at-p ".*?|")
+	     (org-at-table-p))
+	(progn (forward-char -1) (org-delete-char 1))
+      (backward-delete-char N)
+      (org-fix-tags-on-the-fly))))
 
 (defun org-delete-char (N)
   "Like `delete-char', but insert whitespace at field end in tables.
@@ -16177,31 +16176,30 @@ (defun org-delete-char (N)
 still be marked for re-alignment if the field did fill the entire column,
 because, in this case the deletion might narrow the column."
   (interactive "p")
-  (org-fold-core-ignore-modifications
-    (save-match-data
-      (org-fold-check-before-invisible-edit 'delete)
-      (cond
-       ((or (/= N 1)
-	    (eq (char-after) ?|)
-	    (save-excursion (skip-chars-backward " \t") (bolp))
-	    (not (org-at-table-p)))
-        (delete-char N)
-        (org-fix-tags-on-the-fly))
-       ((looking-at ".\\(.*?\\)|")
-        (let* ((update? org-table-may-need-update)
-	       (noalign (looking-at-p ".*?  |")))
-	  (delete-char 1)
-	  (org-table-with-shrunk-field
-	   (save-excursion
-	     ;; Last space is `org-table-separator-space', so insert
-	     ;; a regular one before it instead.
-	     (goto-char (- (match-end 0) 2))
-	     (insert " ")))
-	  ;; If there were two spaces at the end, this field does not
-	  ;; determine the width of the column.
-	  (when noalign (setq org-table-may-need-update update?))))
-       (t
-        (delete-char N))))))
+  (save-match-data
+    (org-fold-check-before-invisible-edit 'delete)
+    (cond
+     ((or (/= N 1)
+	  (eq (char-after) ?|)
+	  (save-excursion (skip-chars-backward " \t") (bolp))
+	  (not (org-at-table-p)))
+      (delete-char N)
+      (org-fix-tags-on-the-fly))
+     ((looking-at ".\\(.*?\\)|")
+      (let* ((update? org-table-may-need-update)
+	     (noalign (looking-at-p ".*?  |")))
+	(delete-char 1)
+	(org-table-with-shrunk-field
+	 (save-excursion
+	   ;; Last space is `org-table-separator-space', so insert
+	   ;; a regular one before it instead.
+	   (goto-char (- (match-end 0) 2))
+	   (insert " ")))
+	;; If there were two spaces at the end, this field does not
+	;; determine the width of the column.
+	(when noalign (setq org-table-may-need-update update?))))
+     (t
+      (delete-char N)))))
 
 ;; Make `delete-selection-mode' work with Org mode and Orgtbl mode
 (put 'org-self-insert-command 'delete-selection
-- 
2.35.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-test-org-fold-Cover-the-issue-fixed-by-previous-comm.patch --]
[-- Type: text/x-patch, Size: 1284 bytes --]

From f29ccf3161a51f700bb3375a16acab563baceb49 Mon Sep 17 00:00:00 2001
Message-Id: <f29ccf3161a51f700bb3375a16acab563baceb49.1657634233.git.yantar92@gmail.com>
In-Reply-To: <a79a742cbff01b7815cffba806c28ea45a4da63c.1657634233.git.yantar92@gmail.com>
References: <a79a742cbff01b7815cffba806c28ea45a4da63c.1657634233.git.yantar92@gmail.com>
From: Ihor Radchenko <yantar92@gmail.com>
Date: Tue, 12 Jul 2022 21:30:58 +0800
Subject: [PATCH 2/3] test-org-fold: Cover the issue fixed by previous commit

*
testing/lisp/test-org-fold.el (test-org-fold/org-fold-reveal-broken-structure):
Test `org-delete-char' instead of `delete-char'.  The former is the
function used interactively.
---
 testing/lisp/test-org-fold.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/testing/lisp/test-org-fold.el b/testing/lisp/test-org-fold.el
index 40afe55ae..a26346175 100644
--- a/testing/lisp/test-org-fold.el
+++ b/testing/lisp/test-org-fold.el
@@ -410,7 +410,7 @@ (ert-deftest test-org-fold/org-fold-reveal-broken-structure ()
       (re-search-forward "Text")
       (should (org-invisible-p))
       (goto-char 1)
-      (delete-char 1)
+      (org-delete-char 1)
       (re-search-forward "Text")
       (should-not (org-invisible-p)))
     (org-test-with-temp-text
-- 
2.35.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: 0003-org-fold-Make-sure-that-changes-do-not-leave-partial.patch --]
[-- Type: text/x-patch, Size: 10403 bytes --]

From 8e2fed82ed9335680f6b5a7d02028786d8479ca3 Mon Sep 17 00:00:00 2001
Message-Id: <8e2fed82ed9335680f6b5a7d02028786d8479ca3.1657634233.git.yantar92@gmail.com>
In-Reply-To: <a79a742cbff01b7815cffba806c28ea45a4da63c.1657634233.git.yantar92@gmail.com>
References: <a79a742cbff01b7815cffba806c28ea45a4da63c.1657634233.git.yantar92@gmail.com>
From: Ihor Radchenko <yantar92@gmail.com>
Date: Tue, 12 Jul 2022 21:32:08 +0800
Subject: [PATCH 3/3] org-fold: Make sure that changes do not leave partially
 folded headings

* lisp/org-fold-core.el (org-fold-core--fix-folded-region): Check
fragility in all the indirect buffers.
* lisp/org-fold.el (org-fold--reveal-headline-at-point): New function
revealing heading line or full heading when the heading only contains
blank lines.
(org-fold--reveal-outline-maybe): Check headline at the end of folded
region as well.  Use `org-fold-core--fix-folded-region'.
* lisp/org.el (org-insert-heading): Make sure that the newline in
front of heading is revealed.
---
 lisp/org-fold-core.el | 71 ++++++++++++++++++++++---------------------
 lisp/org-fold.el      | 52 ++++++++++++++++++++++---------
 lisp/org.el           |  5 ++-
 3 files changed, 77 insertions(+), 51 deletions(-)

diff --git a/lisp/org-fold-core.el b/lisp/org-fold-core.el
index 23507967f..3fcacb975 100644
--- a/lisp/org-fold-core.el
+++ b/lisp/org-fold-core.el
@@ -1269,41 +1269,42 @@ (defun org-fold-core--fix-folded-region (from to _)
           (let ((new-region (funcall func from to)))
             (setq from (car new-region))
             (setq to (cdr new-region))))
-        (dolist (spec (org-fold-core-folding-spec-list))
-          ;; No action is needed when :fragile is nil for the spec.
-          (when (org-fold-core-get-folding-spec-property spec :fragile)
-            (org-with-wide-buffer
-             ;; Expand the considered region to include partially present fold.
-             ;; Note: It is important to do this inside loop over all
-             ;; specs.  Otherwise, the region may be expanded to huge
-             ;; outline fold, potentially involving majority of the
-             ;; buffer.  That would cause the below code to loop over
-             ;; almost all the folds in buffer, which would be too slow.
-             (let ((local-from from)
-                   (local-to to)
-                   (region-from (org-fold-core-get-region-at-point spec (max (point-min) (1- from))))
-                   (region-to (org-fold-core-get-region-at-point spec (min to (1- (point-max))))))
-               (when region-from (setq local-from (car region-from)))
-               (when region-to (setq local-to (cdr region-to)))
-               (let ((pos local-from))
-                 ;; Move to the first hidden region.
-                 (unless (org-fold-core-get-folding-spec spec pos)
-                   (setq pos (org-fold-core-next-folding-state-change spec pos local-to)))
-                 ;; Cycle over all the folds.
-                 (while (< pos local-to)
-                   (save-match-data ; we should not clobber match-data in after-change-functions
-                     (let ((fold-begin (and (org-fold-core-get-folding-spec spec pos)
-                                            pos))
-                           (fold-end (org-fold-core-next-folding-state-change spec pos local-to)))
-                       (when (and fold-begin fold-end)
-                         (when (save-excursion
-                                 (funcall (org-fold-core-get-folding-spec-property spec :fragile)
-                                          (cons fold-begin fold-end)
-                                          spec))
-                           ;; Reveal completely, not just from the SPEC.
-                           (org-fold-core-region fold-begin fold-end nil)))))
-                   ;; Move to next fold.
-                   (setq pos (org-fold-core-next-folding-state-change spec pos local-to))))))))))))
+        (org-fold-core-cycle-over-indirect-buffers
+          (dolist (spec (org-fold-core-folding-spec-list))
+            ;; No action is needed when :fragile is nil for the spec.
+            (when (org-fold-core-get-folding-spec-property spec :fragile)
+              (org-with-wide-buffer
+               ;; Expand the considered region to include partially present fold.
+               ;; Note: It is important to do this inside loop over all
+               ;; specs.  Otherwise, the region may be expanded to huge
+               ;; outline fold, potentially involving majority of the
+               ;; buffer.  That would cause the below code to loop over
+               ;; almost all the folds in buffer, which would be too slow.
+               (let ((local-from from)
+                     (local-to to)
+                     (region-from (org-fold-core-get-region-at-point spec (max (point-min) (1- from))))
+                     (region-to (org-fold-core-get-region-at-point spec (min to (1- (point-max))))))
+                 (when region-from (setq local-from (car region-from)))
+                 (when region-to (setq local-to (cdr region-to)))
+                 (let ((pos local-from))
+                   ;; Move to the first hidden region.
+                   (unless (org-fold-core-get-folding-spec spec pos)
+                     (setq pos (org-fold-core-next-folding-state-change spec pos local-to)))
+                   ;; Cycle over all the folds.
+                   (while (< pos local-to)
+                     (save-match-data ; we should not clobber match-data in after-change-functions
+                       (let ((fold-begin (and (org-fold-core-get-folding-spec spec pos)
+                                              pos))
+                             (fold-end (org-fold-core-next-folding-state-change spec pos local-to)))
+                         (when (and fold-begin fold-end)
+                           (when (save-excursion
+                                   (funcall (org-fold-core-get-folding-spec-property spec :fragile)
+                                            (cons fold-begin fold-end)
+                                            spec))
+                             ;; Reveal completely, not just from the SPEC.
+                             (org-fold-core-region fold-begin fold-end nil)))))
+                     ;; Move to next fold.
+                     (setq pos (org-fold-core-next-folding-state-change spec pos local-to)))))))))))))
 
 ;;; Handling killing/yanking of folded text
 
diff --git a/lisp/org-fold.el b/lisp/org-fold.el
index ce8afd9b4..6ff21dfc7 100644
--- a/lisp/org-fold.el
+++ b/lisp/org-fold.el
@@ -55,6 +55,7 @@ (defvar org-link-descriptive)
 (defvar org-outline-regexp-bol)
 (defvar org-archive-tag)
 (defvar org-custom-properties-overlays)
+(defvar org-element-headline-re)
 
 (declare-function isearch-filter-visible "isearch" (beg end))
 (declare-function org-element-type "org-element" (element))
@@ -930,6 +931,30 @@ (defun org-fold--extend-changed-region (from to)
   (setq from (save-excursion (goto-char from) (line-beginning-position 0)))
   (cons from to))
 
+(defun org-fold--reveal-headline-at-point ()
+  "Reveal header line and empty contents inside.
+Reveal the header line and, if present, also reveal its contents, when
+the contents consists of blank lines.
+
+Assume that point is located at the header line."
+  (org-with-wide-buffer
+   (beginning-of-line)
+   (org-fold-region
+    (max (point-min) (1- (point)))
+    (let ((endl (line-end-position)))
+      (save-excursion
+        (goto-char endl)
+        (skip-chars-forward "\n\t\r ")
+        ;; Unfold blank lines after newly inserted headline.
+        (if (equal (point)
+                   (save-excursion
+                     (goto-char endl)
+                     (org-end-of-subtree)
+                     (skip-chars-forward "\n\t\r ")))
+            (point)
+          endl)))
+    nil 'headline)))
+
 (defun org-fold--reveal-outline-maybe (region _)
   "Reveal folded outline in REGION when needed.
 
@@ -942,28 +967,25 @@ (defun org-fold--reveal-outline-maybe (region _)
      ;; headline or a list item.
      (backward-char)
      (beginning-of-line)
-     ;; Make sure that headline is not partially hidden
+     ;; Make sure that headline is not partially hidden.
      (unless (org-fold-folded-p nil 'headline)
-       (org-fold-region
-        (max (point-min) (1- (point)))
-        (let ((endl (line-end-position)))
-          (save-excursion
-            (goto-char endl)
-            (skip-chars-forward "\n\t\r ")
-            ;; Unfold blank lines.
-            (if (or (and (looking-at-p "\\*")
-                         (> (point) (1+ endl)))
-                    (eq (point) (point-max)))
-                (point)
-              endl)))
-        nil 'headline))
+       (org-fold--reveal-headline-at-point))
      ;; Never hide level 1 headlines
      (save-excursion
        (goto-char (line-end-position))
        (unless (>= (point) (cdr region))
          (when (re-search-forward (rx bol "* ") (cdr region) t)
-           (org-fold-region (match-beginning 0) (line-end-position) nil 'headline))))
+           (org-fold--reveal-headline-at-point))))
+     ;; Make sure that headline after is not partially hidden.
+     (goto-char (cdr region))
+     (beginning-of-line)
+     (unless (org-fold-folded-p nil 'headline)
+       (when (looking-at-p org-element-headline-re)
+         (org-fold--reveal-headline-at-point)))
      ;; Check the validity of headline
+     (goto-char (car region))
+     (backward-char)
+     (beginning-of-line)
      (unless (let ((case-fold-search t))
 	       (looking-at (rx-to-string
                             `(or (regex ,(org-item-re))
diff --git a/lisp/org.el b/lisp/org.el
index d85b5818e..1c9eaf09a 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -6141,7 +6141,10 @@ (defun org-insert-heading (&optional arg invisible-ok top)
       (unless invisible-ok
         (if (eq org-fold-core-style 'text-properties)
 	    (cond
-	     ((org-fold-folded-p (line-beginning-position) 'headline)
+	     ((org-fold-folded-p
+               (max (point-min)
+                    (1- (line-beginning-position)))
+               'headline)
 	      (org-fold-region (line-end-position 0) (line-end-position) nil 'headline))
 	     (t nil))
           (pcase (get-char-property-and-overlay (point) 'invisible)
-- 
2.35.1

Re: Recent folding issues

[Jack Kamm]

Hi Ihor,

> Can you try the attached patch set?

Thanks for the very prompt patchset!

Testing on the minimal example, the patchset solves the problem I
reported.

After some brief testing with my usual config, it solves the problem of
headlines running together after capture, and also solves the problem of
unfolding the above headline when calling evil-open-above (vim's "O"
key).

I'm still having some issues with evil-open-below (vim's "o" key). I
don't have a minimal config for this yet -- it requires the 3rd party
package evil, and I'm not sure if emacs has a built-in equivalent to
evil-open-below. But here's a description of what I observe.

Suppose I have an org file like this:

> * Headline 1
> ** Headline 1a
> ** Headline 1b
> *** Headline 1b1
> 
> Stuff
> 
> * Headline 2

Then I collapse the buffer to look like this:

> * Headline 1...
> * Headline 2

If I put the cursor on Headline 1 and call evil-open-below, I don't
immediately see a new line, and the buffer looks as above, but with the
pointer "inside" the elipses. As soon as I start typing, it will unfold
the last subheading of Headline 1, and the buffer looks like this after
I finished typing:

> * Headline 1
> *** Headline 1b1
>
> Stuff
>
> * New typing here
> * Headline 2

And also I got the message:

> org-fold-check-before-invisible-edit--text-properties: Edit in invisible region aborted, repeat to confirm with text visible

Whereas with org-fold-core-style set to "overlays", I get the old
behavior -- after typing "o" on Headline 1, I see this:

> * Headline 1...
>  
> * Headline 2

With the pointer on the new line, and after typing I see:

> * Headline 1...
> * New typing here 
> * Headline 2

Re: Recent folding issues

[Ihor Radchenko]

Jack Kamm <jackkamm@gmail.com> writes:

>> Can you try the attached patch set?
>
> Thanks for the very prompt patchset!
>
> Testing on the minimal example, the patchset solves the problem I
> reported.
>
> After some brief testing with my usual config, it solves the problem of
> headlines running together after capture, and also solves the problem of
> unfolding the above headline when calling evil-open-above (vim's "O"
> key).

Thanks for testing!
Applied onto main via a79a742cb, f29ccf316, and 8e2fed82e.

I haven't looked into the other problem yet. Too tired today.

Best,
Ihor

Re: Recent folding issues

[Ihor Radchenko]

Jack Kamm <jackkamm@gmail.com> writes:

> If I put the cursor on Headline 1 and call evil-open-below, I don't
> immediately see a new line, and the buffer looks as above, but with the
> pointer "inside" the elipses. As soon as I start typing, it will unfold
> the last subheading of Headline 1, and the buffer looks like this after
> I finished typing:
>
>> * Headline 1
>> *** Headline 1b1
>>
>> Stuff
>>
>> * New typing here
>> * Headline 2
>
> And also I got the message:
>
>> org-fold-check-before-invisible-edit--text-properties: Edit in invisible region aborted, repeat to confirm with text visible
>
> Whereas with org-fold-core-style set to "overlays", I get the old
> behavior -- after typing "o" on Headline 1, I see this:
>
>> * Headline 1...
>>  
>> * Headline 2
>
> With the pointer on the new line, and after typing I see:
>
>> * Headline 1...
>> * New typing here 
>> * Headline 2

Thanks for another detailed report!
Fixed on main via 39005dc09.

Let me know if you notice any other inconsistencies.
For clarity, this particular change had been tested for over a year
before it was merged.

Best,
Ihor

Re: Recent folding issues

[Jack Kamm]

Ihor Radchenko <yantar92@gmail.com> writes:

> Thanks for another detailed report!
> Fixed on main via 39005dc09.
>
> Let me know if you notice any other inconsistencies.

Thanks very much Ihor! Much appreciated :)

I'll let you know if I notice any other issues -- everything seems to be working well so far.

Best,
Jack