Bug: refiling gobbles a newline and absorbs the next heading [9.1.9 (release_9.1.9-65-g5e4542 @ /Applications/Emacs.app/Contents/Resources/lisp/org/)]

[Miguel Morin <blind.pew@cantab.net>]

Hello,

TL;DR: starting a capture item and deleting the new lines at the end of the capture buffer gobbles the next heading, and refiling moves the gobbled heading along with the new capture item.

I asked on emacs.stackexchange and found no solution:

https://emacs.stackexchange.com/questions/53526/org-mode-refiling-gobbles-a-newline-and-absorbs-the-next-heading

In the course of fixing this, I found another bug in using the hook `org-capture-before-finalize-hook`, which adds text at the beginning and end of the org buffer and not in the beggining and end of the capture item.

Full details, copied from the StackExchange thread:

I often have this problem in Org mode: I write an item with org-capture, refile it, and days later I see a result like this:

<pre>
*** TODO some task
some notes** StackExchange
</pre>

I believe that the problem is that org capture inserts the text into the buffer, and if the item does not end with a newline, it gobbles the delimiter to the next heading and both get refiled together.

I already added two newlines to my capture template to minimize this problem. My capture template is:

``` lisp
(setq org-capture-templates
      '(("t" "Todo" entry (file+headline "~/org/gtd.org" "Reminders")
	 "* TODO %?\n\n"))
```

If I try a new item with `C-c c t` and type something, I see it appear in the main buffer. If I delete the newline at the end, the newline is deleted in the buffer too and gobbles the next line. Deleting further characters in the capture buffer does not delete them from the main buffer.

I think I could add a hook such as `(add-hook 'org-capture-before-finalize-hook ...)` but I'm not sure how to include a newline and how to do it before refiling, to avoid the next item being refiled with the captured item.

How can I fix this problem?

**Update**: Following @Aquaactress's suggestions, I tried two methods. Both fail and the versions may be important:

<pre>
Org mode version 9.1.9 (release_9.1.9-65-g5e4542 @ /Applications/Emacs.app/Contents/Resources/lisp/org/)
GNU Emacs 26.3 (build 1, x86_64-apple-darwin18.2.0, NS appkit-1671.20 Version 10.14.3 (Build 18D109)) of 2019-09-02
</pre>

In the first method, I added `:empty-lines` in the org-capture template:

```lisp
(setq org-capture-before-finalize-hook nil)

(setq org-capture-templates
      '(("t" "Todo" entry (file+headline "~/test.org" "heading 1")
	 "* TODO %?\n" :empty-lines 1)))
```

The result in screenshots is below: I add a capture item, delete newlines from the end of the org-capture buffer, and refile. The heading next to the refiled item is gobbled:

[![Empty lines method, step 1 - create an org-capture item][1]][1]

[![Empty lines method, step 2 - delete lines from end of org-capture buffer][2]][2]

[![Empty lines method, step 3 - refile][3]][3]

In the second method, I added a function that inserts new lines at the beginning and end of the org capture buffer (for exposition, I also add `hello world!` to show that these insertions go at the beginning and end of the whole org buffer, not the org-capture buffer):

```lisp
(setq org-capture-before-finalize-hook nil)

(defun +org|insert-newlines ()
  (beginning-of-buffer) (insert "\nhello world!\n")
  (end-of-buffer) (insert "\nhello world!\n"))

(add-hook 'org-capture-before-finalize-hook #'+org|insert-newlines)

(setq org-capture-templates
      '(("t" "Todo" entry (file+headline "~/test.org" "heading 1")
	 "* TODO %?\n" :empty-lines 1)))
```

The result in screenshots is below: I add a capture item, delete newlines from the end of the org-capture buffer, and refile. The heading next to the refiled item is gobbled and the lines inserted go at the beginning and end of the buffer, instead of the org-capture buffer.

[![Insert lines method, step 1 - add org-capture item][4]][4]

[![Insert lines method, step 2 - delete newlines from end of org-capture buffer][5]][5]

[![Insert lines method, step 3 - refile][6]][6]

  [1]: https://i.stack.imgur.com/6cUlo.png
  [2]: https://i.stack.imgur.com/GPwg8.png
  [3]: https://i.stack.imgur.com/tzAJI.png
  [4]: https://i.stack.imgur.com/xcKva.png
  [5]: https://i.stack.imgur.com/g5sIT.png
  [6]: https://i.stack.imgur.com/D1H8l.png
  [7]: https://i.stack.imgur.com/l6a55.png

------------------------------------------------------------------------



Emacs  : GNU Emacs 26.3 (build 1, x86_64-apple-darwin18.2.0, NS appkit-1671.20 Version 10.14.3 (Build 18D109))
 of 2019-09-02
Package: Org mode version 9.1.9 (release_9.1.9-65-g5e4542 @ /Applications/Emacs.app/Contents/Resources/lisp/org/)

current state:
==============
(setq
 org-src-mode-hook '(org-src-babel-configure-edit-buffer
		     org-src-mode-configure-edit-buffer)
 org-after-todo-state-change-hook '(org-clock-out-if-current)
 org-metadown-hook '(org-babel-pop-to-session-maybe)
 org-clock-out-hook '(org-clock-remove-empty-clock-drawer)
 org-mode-hook '(#[0 "\300\301\302\303\304$\207"
		   [add-hook change-major-mode-hook org-show-block-all append
		    local]
		   5]
		 #[0 "\300\301\302\303\304$\207"
		   [add-hook change-major-mode-hook org-babel-show-result-all
		    append local]
		   5]
		 org-babel-result-hide-spec org-babel-hide-all-hashes)
 org-archive-hook '(org-attach-archive-delete-maybe)
 org-confirm-elisp-link-function 'yes-or-no-p
 org-agenda-before-write-hook '(org-agenda-add-entry-text)
 org-metaup-hook '(org-babel-load-in-session-maybe)
 org-bibtex-headline-format-function #[257 "\300\x01\236A\207" [:title] 3 "\n\n(fn ENTRY)"]
 org-babel-pre-tangle-hook '(save-buffer)
 org-tab-first-hook '(org-babel-hide-result-toggle-maybe
		      org-babel-header-arg-expand)
 org-occur-hook '(org-first-headline-recenter)
 org-cycle-hook '(org-cycle-hide-archived-subtrees org-cycle-hide-drawers
		  org-cycle-show-empty-lines
		  org-optimize-window-after-visibility-change)
 org-speed-command-hook '(org-speed-command-activate
			  org-babel-speed-command-activate)
 org-confirm-shell-link-function 'yes-or-no-p
 org-link-parameters '(("id" :follow org-id-open)
		       ("rmail" :follow org-rmail-open :store
			org-rmail-store-link)
		       ("mhe" :follow org-mhe-open :store org-mhe-store-link)
		       ("irc" :follow org-irc-visit :store org-irc-store-link)
		       ("info" :follow org-info-open :export org-info-export
			:store org-info-store-link)
		       ("gnus" :follow org-gnus-open :store
			org-gnus-store-link)
		       ("docview" :follow org-docview-open :export
			org-docview-export :store org-docview-store-link)
		       ("bibtex" :follow org-bibtex-open :store
			org-bibtex-store-link)
		       ("bbdb" :follow org-bbdb-open :export org-bbdb-export
			:complete org-bbdb-complete-link :store
			org-bbdb-store-link)
		       ("w3m" :store org-w3m-store-link) ("file+sys")
		       ("file+emacs") ("doi" :follow org--open-doi-link)
		       ("elisp" :follow org--open-elisp-link)
		       ("file" :complete org-file-complete-link)
		       ("ftp" :follow
			(lambda (path) (browse-url (concat "ftp:" path))))
		       ("help" :follow org--open-help-link)
		       ("http" :follow
			(lambda (path) (browse-url (concat "http:" path))))
		       ("https" :follow
			(lambda (path) (browse-url (concat "https:" path))))
		       ("mailto" :follow
			(lambda (path) (browse-url (concat "mailto:" path))))
		       ("news" :follow
			(lambda (path) (browse-url (concat "news:" path))))
		       ("shell" :follow org--open-shell-link))
 )