[BUG] org-insert-heading changed behavior with Emacs 29.2 [9.6.15 (release_9.6.15 @ /usr/local/share/emacs/29.2/lisp/org/)]

[BUG] org-insert-heading changed behavior with Emacs 29.2 [9.6.15 (release_9.6.15 @ /usr/local/share/emacs/29.2/lisp/org/)]

[gusbrs]

Hi All,

I recently upgraded to Emacs 29.2 (from 29.1) and observed that
`org-insert-heading-respect-content' has changed behavior with regard
to how it handles blank lines at the end of the entry at which the
command was called.

Consider the following document (with "|" representing point):

#+begin_src org
,* Sec1

,** |SubSec1

text

,** SubSec2

text

#+end_src

Calling `org-insert-heading-respect-content' ("C-RET") with Org
version 9.6.6 (built-in Emacs 29.1) would result in:

#+begin_src org
,* Sec1

,** SubSec1

text

,** |

,** SubSec2

text

#+end_src

Now, with Org version 9.6.15 (built-in Emacs 29.2), it results in:

#+begin_src org
,* Sec1

,** SubSec1

text

,** |
,** SubSec2

text

#+end_src

Note the missing blank line between the inserted heading and SubSec2.

That is an unfortunate change of behavior since those who like to keep
some breathing space at the end of entries now have to deal with it
manually after the heading is inserted.  So the handy "C-RET" becomes
something like "C-RET RET C-b SPC".  Plus the cost of having to think
about it, and that of occasionally forgetting it, consistency is just
harder to maintain.

As far as I can tell, the commit which introduced this change of
behavior was https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=52bc95676.
It was introduced to deal with a real issue which had been reported:
to avoid the possibility of those blank lines being shown folded after
insertion in some cases (link on the commit).

However, as far as I understand, the issue the commit tried to fix was
one of visibility of the buffer.  The actual contents were expected,
correct.  But to fix the visibility issue, the behavior of
`org-insert-heading' with regard to the actual contents was changed:
to avoid the blank lines being shown folded the blank lines were
actually removed.

So I'd like to suggest / propose that previous behavior of
`org-insert-heading' with respect to blank lines be somehow restored
and that the problem of the blank lines being shown folded be treated
as one of visibility / folding, not one of buffer contents.

Best regards,
gusbrs.





Emacs  : GNU Emacs 29.2 (build 2, x86_64-pc-linux-gnu, GTK+ Version
3.24.33, cairo version 1.16.0)
 of 2024-01-18
Package: Org mode version 9.6.15 (release_9.6.15 @
/usr/local/share/emacs/29.2/lisp/org/)

current state:
==============
(setq
 org-link-elisp-confirm-function 'yes-or-no-p
 org-bibtex-headline-format-function #[257 "\300 \236A\207" [:title] 3
"\n\n(fn ENTRY)"]
 org-persist-after-read-hook '(org-element--cache-persist-after-read)
 org-export-before-parsing-hook '(org-attach-expand-links)
 org-cycle-tab-first-hook '(org-babel-hide-result-toggle-maybe
                org-babel-header-arg-expand)
 org-archive-hook '(org-attach-archive-delete-maybe)
 org-cycle-hook '(org-cycle-hide-archived-subtrees org-cycle-show-empty-lines
          org-cycle-optimize-window-after-visibility-change
          org-cycle-display-inline-images)
 org-persist-before-read-hook '(org-element--cache-persist-before-read)
 org-mode-hook '(#[0 "\300\301\302\303\304$\207"
           [add-hook change-major-mode-hook org-fold-show-all append
            local]
           5]
         #[0 "\300\301\302\303\304$\207"
           [add-hook change-major-mode-hook org-babel-show-result-all
            append local]
           5]
         org-babel-result-hide-spec org-babel-hide-all-hashes)
 org-confirm-shell-link-function 'yes-or-no-p
 outline-isearch-open-invisible-function 'outline-isearch-open-invisible
 org-agenda-before-write-hook '(org-agenda-add-entry-text)
 org-src-mode-hook '(org-src-babel-configure-edit-buffer
             org-src-mode-configure-edit-buffer)
 org-confirm-elisp-link-function 'yes-or-no-p
 org-speed-command-hook '(org-speed-command-activate
              org-babel-speed-command-activate)
 org-persist-directory "/tmp/org-persist-St7aom"
 org-fold-core-isearch-open-function 'org-fold--isearch-reveal
 org-persist-before-write-hook '(org-element--cache-persist-before-write)
 org-tab-first-hook '(org-babel-hide-result-toggle-maybe
              org-babel-header-arg-expand)
 org-link-shell-confirm-function 'yes-or-no-p
 org-babel-pre-tangle-hook '(save-buffer)
 org-agenda-loop-over-headlines-in-active-region nil
 org-occur-hook '(org-first-headline-recenter)
 org-metadown-hook '(org-babel-pop-to-session-maybe)
 org-link-parameters '(("attachment" :follow org-attach-follow :complete
            org-attach-complete-link)
               ("id" :follow org-id-open)
               ("eww" :follow org-eww-open :store org-eww-store-link)
               ("rmail" :follow org-rmail-open :store
            org-rmail-store-link)
               ("mhe" :follow org-mhe-open :store org-mhe-store-link)
               ("irc" :follow org-irc-visit :store org-irc-store-link
            :export org-irc-export)
               ("info" :follow org-info-open :export org-info-export
            :store org-info-store-link :insert-description
            org-info-description-as-command)
               ("gnus" :follow org-gnus-open :store
            org-gnus-store-link)
               ("docview" :follow org-docview-open :export
            org-docview-export :store org-docview-store-link)
               ("bibtex" :follow org-bibtex-open :store
            org-bibtex-store-link)
               ("bbdb" :follow org-bbdb-open :export org-bbdb-export
            :complete org-bbdb-complete-link :store
            org-bbdb-store-link)
               ("w3m" :store org-w3m-store-link)
               ("doi" :follow org-link-doi-open :export
            org-link-doi-export)
               ("file+sys") ("file+emacs")
               ("shell" :follow org-link--open-shell)
               ("news" :follow
            #[514 "\301\300\302 Q \"\207"
              ["news" browse-url ":"] 6 "\n\n(fn URL ARG)"]
            )
               ("mailto" :follow
            #[514 "\301\300\302 Q \"\207"
              ["mailto" browse-url ":"] 6 "\n\n(fn URL ARG)"]
            )
               ("https" :follow
            #[514 "\301\300\302 Q \"\207"
              ["https" browse-url ":"] 6 "\n\n(fn URL ARG)"]
            )
               ("http" :follow
            #[514 "\301\300\302 Q \"\207"
              ["http" browse-url ":"] 6 "\n\n(fn URL ARG)"]
            )
               ("ftp" :follow
            #[514 "\301\300\302 Q \"\207" ["ftp" browse-url ":"]
              6 "\n\n(fn URL ARG)"]
            )
               ("help" :follow org-link--open-help :store
            org-link--store-help)
               ("file" :complete org-link-complete-file)
               ("elisp" :follow org-link--open-elisp))
 org-metaup-hook '(org-babel-load-in-session-maybe)
 )

Re: [BUG] org-insert-heading changed behavior with Emacs 29.2 [9.6.15 (release_9.6.15 @ /usr/local/share/emacs/29.2/lisp/org/)]

[Max Nikulin]

On 02/02/2024 01:46, gusbrs wrote:
> That is an unfortunate change of behavior since those who like to keep
> some breathing space at the end of entries now have to deal with it
> manually after the heading is inserted.  So the handy "C-RET" becomes
> something like "C-RET RET C-b SPC".

C-o `org-open-line` (or just `open-line') adds newline after cursor.

C-j allows to keep space after stars.

Re: [BUG] org-insert-heading changed behavior with Emacs 29.2 [9.6.15 (release_9.6.15 @ /usr/local/share/emacs/29.2/lisp/org/)]

[Ihor Radchenko]

gusbrs <gusbrs.2016@gmail.com> writes:

> I recently upgraded to Emacs 29.2 (from 29.1) and observed that
> `org-insert-heading-respect-content' has changed behavior with regard
> to how it handles blank lines at the end of the entry at which the
> command was called.
>
> Consider the following document (with "|" representing point):
>
> ** |SubSec1
>
> text
>
> ** SubSec2

> Calling `org-insert-heading-respect-content' ("C-RET") with Org
> version 9.6.6 (built-in Emacs 29.1) would result in:
> ...
> ** |
>
> ** SubSec2
>
> Now, with Org version 9.6.15 (built-in Emacs 29.2), it results in:
> ...
> ,** |
> ,** SubSec2
>
> That is an unfortunate change of behavior since those who like to keep
> some breathing space at the end of entries now have to deal with it
> manually after the heading is inserted.  So the handy "C-RET" becomes
> something like "C-RET RET C-b SPC".  Plus the cost of having to think
> about it, and that of occasionally forgetting it, consistency is just
> harder to maintain.

The number of blank lines after newly inserted is not defined by Org
mode, unlike the number of blank lines before heading that is controlled
by `org-blank-before-new-entry'.

If you use M-<RET> rather then C-<RET> and set
(setq org-blank-before-new-entry '((heading) (plain-list-item)))
, you will get no newlines after the new heading inserted before current:

* Sec1

** <point>SubSec1

M-<RET> will yield

* Sec1
** <point>
** SubSec1

Would it make sense to add a new `org-blank-after-new-entry'
customization that will provide explicit user control over what Org does
when inserting a new heading?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] org-insert-heading changed behavior with Emacs 29.2 [9.6.15 (release_9.6.15 @ /usr/local/share/emacs/29.2/lisp/org/)]

[gusbrs]

Hi Ihor,

Thanks for looking into this.

On Sat, 3 Feb 2024 at 09:32, Ihor Radchenko <yantar92@posteo.net> wrote:

> The number of blank lines after newly inserted is not defined by Org
> mode, unlike the number of blank lines before heading that is controlled
> by `org-blank-before-new-entry'.

Well, "not defined" is strong, I'd say. There is no exposed option.
But the behavior used to be regular, and "do the right thing" (OK, you
may wish to debate that). And now the behavior has changed.

The commit basically only changes from:

    (org-end-of-subtree)

to:

     (org-end-of-subtree invisible-ok 'to-heading)

Which essentially changes the point position before the task of
actually inserting the heading is carried out.

It goes from here:

#+begin_src org
,* Sec1

,** SubSec1

text<point>

,** SubSec2

text

#+end_src

To here:

#+begin_src org
,* Sec1

,** SubSec1

text

,<point>** SubSec2

text

#+end_src

In practice, the previous behavior would "honor" whatever blank lines
existed in the current heading. That's why
`org-blank-before-new-entry' was needed, but something like
`org-blank-after-new-entry' not so much. Consistency was naturally
kept, because of this regularity.

> If you use M-<RET> rather then C-<RET> and set
> (setq org-blank-before-new-entry '((heading) (plain-list-item)))
> , you will get no newlines after the new heading inserted before current:
>
> * Sec1
>
> ** <point>SubSec1
>
> M-<RET> will yield
>
> * Sec1
> ** <point>
> ** SubSec1

Not the same thing, C-RET calls `org-insert-heading-respect-content'
and `org-meta-return' calls `org-insert-heading'. Besides, what we get
seems to align with what you asked in `org-blank-before-new-entry'. I
tend to use mostly C-RET, because it is more regular, and I use lists
a lot, so that's the issue I'm raising.

> Would it make sense to add a new `org-blank-after-new-entry'
> customization that will provide explicit user control over what Org does
> when inserting a new heading?

Looking from the perspective of C-RET alone, I'd be inclined to
restate my suggestion of treating the issue that motivated that commit
as a "visibility / folding" problem. However, including M-RET into the
mix, I think you have a good point, and something like
`org-blank-after-new-entry' would possibly help improve blank line
consistency in general.

Best regards,
gusbrs

Re: [BUG] org-insert-heading changed behavior with Emacs 29.2 [9.6.15 (release_9.6.15 @ /usr/local/share/emacs/29.2/lisp/org/)]

[gusbrs]

Hi,

On Sat, 3 Feb 2024 at 10:27, gusbrs <gusbrs.2016@gmail.com> wrote:
> On Sat, 3 Feb 2024 at 09:32, Ihor Radchenko <yantar92@posteo.net> wrote:

> > Would it make sense to add a new `org-blank-after-new-entry'
> > customization that will provide explicit user control over what Org does
> > when inserting a new heading?
>
> Looking from the perspective of C-RET alone, I'd be inclined to
> restate my suggestion of treating the issue that motivated that commit
> as a "visibility / folding" problem. However, including M-RET into the
> mix, I think you have a good point, and something like
> `org-blank-after-new-entry' would possibly help improve blank line
> consistency in general.

A couple of further comments about this. First, I think exposing a
`org-blank-after-new-entry' is a good idea also on the grounds that it
becomes a commitment to a desired/desirable behavior. And would offer
more control.

Also, something like `(setq org-blank-after-new-entry '((heading .
auto)))' would arguably have to keep the number of lines existing at
the end of the current entry. In other words, implement (by other
means) the behavior which used to prevail before the commit in
discussion.

Best,
gusbrs

Re: [BUG] org-insert-heading changed behavior with Emacs 29.2 [9.6.15 (release_9.6.15 @ /usr/local/share/emacs/29.2/lisp/org/)]

[Ihor Radchenko]

Ihor Radchenko <yantar92@posteo.net> writes:

> The number of blank lines after newly inserted is not defined by Org
> mode, unlike the number of blank lines before heading that is controlled
> by `org-blank-before-new-entry'.
> ...
> Would it make sense to add a new `org-blank-after-new-entry'
> customization that will provide explicit user control over what Org does
> when inserting a new heading?

I studied the code closer, and it looks like some parts of Org mode
already use `org-blank-before-new-entry' when deciding the blank lines
after newly inserted list items.  So, instead of introducing a new
customization, I went with making sure that blank is inserted after new
heading when that heading has blank lines before and there is no blank
before the next heading.

Fixed, on main.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=f64c8a5a5

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] org-insert-heading changed behavior with Emacs 29.2 [9.6.15 (release_9.6.15 @ /usr/local/share/emacs/29.2/lisp/org/)]

[gusbrs]

Hi Ihor,

On Fri, 10 May 2024 at 06:17, Ihor Radchenko <yantar92@posteo.net> wrote:
>
> I studied the code closer, and it looks like some parts of Org mode
> already use `org-blank-before-new-entry' when deciding the blank lines
> after newly inserted list items.  So, instead of introducing a new
> customization, I went with making sure that blank is inserted after new
> heading when that heading has blank lines before and there is no blank
> before the next heading.
>
> Fixed, on main.
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=f64c8a5a5

Thank you very much!