org-refile.el: Fix the case of emtpy buffer name

org-refile.el: Fix the case of emtpy buffer name

[satotake]

Dear Org maintainers,

I often use org-mode for note-taking and so on.
Sometimes, I would like to refile the headings in the scratch to other
org files while I am writing.
When I call `org-refile' in the buffer, Emacs throws errors on 
`org-refile-get-targets' because, typically,
these buffers do not have `buffer-name' and I set `org-refile-use-outline-path' `file'.
To fix it, I have created my first patch.

Regards
Takeshi SATO

[PATCH] org-refile.el: Fix the case of emtpy buffer name

[satotake]

* lisp/org-refile.el (org-refile-get-targets): Ensure
arg of `file-name-non' and `file-truename' is non-nil.

If you set `org-refile-use-outline-path' `file' or `full-file-path',
and call `org-refile' in the buffer before visiting file,
errors are raised at these point. To fix them,
check if they are nil or not.

TINYCHANGE
---
 lisp/org-refile.el | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/lisp/org-refile.el b/lisp/org-refile.el
index 24a1bde51..6f5b8acee 100644
--- a/lisp/org-refile.el
+++ b/lisp/org-refile.el
@@ -310,11 +310,14 @@ converted to a headline before refiling."
 		 (setq f (buffer-file-name (buffer-base-buffer f))))
 	       (setq f (and f (expand-file-name f)))
 	       (when (eq org-refile-use-outline-path 'file)
-		 (push (list (file-name-nondirectory f) f nil nil) tgs))
+		 (push (list (if f (file-name-nondirectory f) nil) f nil nil) tgs))
 	       (when (eq org-refile-use-outline-path 'buffer-name)
 		 (push (list (buffer-name (buffer-base-buffer)) f nil nil) tgs))
 	       (when (eq org-refile-use-outline-path 'full-file-path)
-		 (push (list (file-truename (buffer-file-name (buffer-base-buffer))) f nil nil) tgs))
+		 (push (list (if (buffer-base-buffer)
+                                 (file-truename (buffer-file-name (buffer-base-buffer)))
+                               nil)
+                             f nil nil) tgs))
 	       (org-with-wide-buffer
 		(goto-char (point-min))
 		(setq org-outline-path-cache nil)
@@ -337,9 +340,11 @@ converted to a headline before refiling."
 				#'identity
 				(append
 				 (pcase org-refile-use-outline-path
-				   (`file (list (file-name-nondirectory
-						 (buffer-file-name
-						  (buffer-base-buffer)))))
+				   (`file (list
+                                           (if (buffer-base-buffer)
+                                               (file-name-nondirectory
+                                                (buffer-file-name (buffer-base-buffer)))
+                                                  nil)))
 				   (`full-file-path
 				    (list (buffer-file-name
 					   (buffer-base-buffer))))
-- 
2.30.0

Re: org-refile.el: Fix the case of emtpy buffer name

[satotake]

I am very sorry but related tests failed with my previous patch.
I fixed it with this additinal patch.

Bests,
Takehsi SATO

[PATCH 2/2] org-refile.el: Fix test case

[satotake]

* lisp/org-refile.el (org-refile-get-targets): Check
`buffer-file-name' return value instead of `buffer-base-buffer'.

To pass the related tests, we need to check `buffer-file-name'.

TINYCHANGE
---
 lisp/org-refile.el | 14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)

diff --git a/lisp/org-refile.el b/lisp/org-refile.el
index 6f5b8acee..2900be27e 100644
--- a/lisp/org-refile.el
+++ b/lisp/org-refile.el
@@ -310,13 +310,12 @@ converted to a headline before refiling."
 		 (setq f (buffer-file-name (buffer-base-buffer f))))
 	       (setq f (and f (expand-file-name f)))
 	       (when (eq org-refile-use-outline-path 'file)
-		 (push (list (if f (file-name-nondirectory f) nil) f nil nil) tgs))
+		 (push (list (and f (file-name-nondirectory f)) f nil nil) tgs))
 	       (when (eq org-refile-use-outline-path 'buffer-name)
 		 (push (list (buffer-name (buffer-base-buffer)) f nil nil) tgs))
 	       (when (eq org-refile-use-outline-path 'full-file-path)
-		 (push (list (if (buffer-base-buffer)
-                                 (file-truename (buffer-file-name (buffer-base-buffer)))
-                               nil)
+		 (push (list (and (buffer-file-name (buffer-base-buffer))
+                                  (file-truename (buffer-file-name (buffer-base-buffer))))
                              f nil nil) tgs))
 	       (org-with-wide-buffer
 		(goto-char (point-min))
@@ -341,10 +340,9 @@ converted to a headline before refiling."
 				(append
 				 (pcase org-refile-use-outline-path
 				   (`file (list
-                                           (if (buffer-base-buffer)
-                                               (file-name-nondirectory
-                                                (buffer-file-name (buffer-base-buffer)))
-                                                  nil)))
+                                           (and (buffer-file-name (buffer-base-buffer))
+                                                (file-name-nondirectory
+                                                 (buffer-file-name (buffer-base-buffer))))))
 				   (`full-file-path
 				    (list (buffer-file-name
 					   (buffer-base-buffer))))
-- 
2.30.0

[PATCH 2/2] org-refile.el: Fix test case

[satotake]

* lisp/org-refile.el (org-refile-get-targets): Check
`buffer-file-name' return value instead of `buffer-base-buffer'.

To pass the related tests, we need to check `buffer-file-name'.

TINYCHANGE
---
 lisp/org-refile.el | 14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)

diff --git a/lisp/org-refile.el b/lisp/org-refile.el
index 6f5b8acee..2900be27e 100644
--- a/lisp/org-refile.el
+++ b/lisp/org-refile.el
@@ -310,13 +310,12 @@ converted to a headline before refiling."
 		 (setq f (buffer-file-name (buffer-base-buffer f))))
 	       (setq f (and f (expand-file-name f)))
 	       (when (eq org-refile-use-outline-path 'file)
-		 (push (list (if f (file-name-nondirectory f) nil) f nil nil) tgs))
+		 (push (list (and f (file-name-nondirectory f)) f nil nil) tgs))
 	       (when (eq org-refile-use-outline-path 'buffer-name)
 		 (push (list (buffer-name (buffer-base-buffer)) f nil nil) tgs))
 	       (when (eq org-refile-use-outline-path 'full-file-path)
-		 (push (list (if (buffer-base-buffer)
-                                 (file-truename (buffer-file-name (buffer-base-buffer)))
-                               nil)
+		 (push (list (and (buffer-file-name (buffer-base-buffer))
+                                  (file-truename (buffer-file-name (buffer-base-buffer))))
                              f nil nil) tgs))
 	       (org-with-wide-buffer
 		(goto-char (point-min))
@@ -341,10 +340,9 @@ converted to a headline before refiling."
 				(append
 				 (pcase org-refile-use-outline-path
 				   (`file (list
-                                           (if (buffer-base-buffer)
-                                               (file-name-nondirectory
-                                                (buffer-file-name (buffer-base-buffer)))
-                                                  nil)))
+                                           (and (buffer-file-name (buffer-base-buffer))
+                                                (file-name-nondirectory
+                                                 (buffer-file-name (buffer-base-buffer))))))
 				   (`full-file-path
 				    (list (buffer-file-name
 					   (buffer-base-buffer))))
-- 
2.30.0

Re: [PATCH 2/2] org-refile.el: Fix test case

[Maxim Nikulin]

On 09/05/2021 23:05, satotake wrote:
> * lisp/org-refile.el (org-refile-get-targets): Check
> `buffer-file-name' return value instead of `buffer-base-buffer'.
> 
> To pass the related tests, we need to check `buffer-file-name'.

Thank you for the patches. Maybe I have missed something, but it seems 2 
patches are not logically independent, second one touches the same code 
to fix regression in tests. If so, could you, please, squash both 
patches into a single one (e.g. using git rebase --interactive)? I hope, 
it will help to review the change and to avoid confusion with several 
versions of second patch in 2 threads.

Code around has at least one more problem: questionable behavior in the 
case of indirect buffers (e.g. C-x 4 c). Refile targets cache, when 
enabled, is not reused for indirect buffer.

On 09/05/2021 21:47, satotake wrote:
> I often use org-mode for note-taking and so on.
> Sometimes, I would like to refile the headings in the scratch to other
> org files while I am writing.

Have you tried org-capture? https://orgmode.org/manual/Capture.html
https://sachachua.com/blog/2015/02/learn-take-notes-efficiently-org-mode/#unnumbered-3
I think, existence of this feature is the reason why this bug was 
remaining unnoticed.

Re: org-refile.el: Fix test case (Squashed)

[satotake]

Thank you for your response.

> Maybe I have missed something, but it seems 2 
> patches are not logically independent, second one touches the same code 
> to fix regression in tests. 
> If so, could you, please, squash both 
> patches into a single one (e.g. using git rebase --interactive)? 
Yes. They should be a single patch. I squashed them.

> Have you tried org-capture? 
Yes. Mostly I use org-capture, especially, for creating TODO tasks.
I sometimes start with fundamental-mode new buffer.
After writing some texts, I switch to org-mode and try to call refile.
I do not know why I do it by myself clearly but I tend to do it when I
do not have any clear goal for the file.

I see this issue not critical for now because I know why this happens 
and just save it anywhere temporarily to avoid it.
But when I was new to Emacs and org-mode, I did not have any idea.
Thus, at least, we should show friendly error message.


> Code around has at least one more problem: questionable behavior in the 
> case of indirect buffers (e.g. C-x 4 c). Refile targets cache, when 
> enabled, is not reused for indirect buffer.

In addition to your point, I found that we cannot refile internally even with

my patch. In other words, if we can cache and reuse it, error ("Please 
indicate a target file in the refile path") is raised
when we select it as refile target.
Probably, we need to some additional fixations.
It may be good to filter `files' which does not have `buffer-file-name'.

[PATCH] org-refile.el: Fix the case of emtpy buffer name

[satotake]

* lisp/org-refile.el (org-refile-get-targets): Ensure
arg of `file-name-non' and `file-truename' is non-nil.

If you set `org-refile-use-outline-path' `file' or `full-file-path',
and call `org-refile' in the buffer before visiting file,
errors are raised at these point. To fix them,
check if they are nil or not.

TINYCHANGE
---
 lisp/org-refile.el | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/lisp/org-refile.el b/lisp/org-refile.el
index 24a1bde51..2900be27e 100644
--- a/lisp/org-refile.el
+++ b/lisp/org-refile.el
@@ -310,11 +310,13 @@ converted to a headline before refiling."
 		 (setq f (buffer-file-name (buffer-base-buffer f))))
 	       (setq f (and f (expand-file-name f)))
 	       (when (eq org-refile-use-outline-path 'file)
-		 (push (list (file-name-nondirectory f) f nil nil) tgs))
+		 (push (list (and f (file-name-nondirectory f)) f nil nil) tgs))
 	       (when (eq org-refile-use-outline-path 'buffer-name)
 		 (push (list (buffer-name (buffer-base-buffer)) f nil nil) tgs))
 	       (when (eq org-refile-use-outline-path 'full-file-path)
-		 (push (list (file-truename (buffer-file-name (buffer-base-buffer))) f nil nil) tgs))
+		 (push (list (and (buffer-file-name (buffer-base-buffer))
+                                  (file-truename (buffer-file-name (buffer-base-buffer))))
+                             f nil nil) tgs))
 	       (org-with-wide-buffer
 		(goto-char (point-min))
 		(setq org-outline-path-cache nil)
@@ -337,9 +339,10 @@ converted to a headline before refiling."
 				#'identity
 				(append
 				 (pcase org-refile-use-outline-path
-				   (`file (list (file-name-nondirectory
-						 (buffer-file-name
-						  (buffer-base-buffer)))))
+				   (`file (list
+                                           (and (buffer-file-name (buffer-base-buffer))
+                                                (file-name-nondirectory
+                                                 (buffer-file-name (buffer-base-buffer))))))
 				   (`full-file-path
 				    (list (buffer-file-name
 					   (buffer-base-buffer))))
-- 
2.30.0

Re: [PATCH] org-refile.el: Fix the case of emtpy buffer name

[Maxim Nikulin]

In my opinion, patch is the minimal change that fixes particular 
workflow and can be committed in the current form. Commit message may be 
improved a bit.

I have a question (mainly to maintainer) if another approach could lead 
to undesired effects, see below.

I have noticed a couple of old issues that can be improved later since 
they may require more changes than allowed for "TINYCHANGE".

On 14/05/2021 01:20, satotake wrote:
> [PATCH] org-refile.el: Fix the case of emtpy buffer name

Buffer not associated with a file may be more precise since "*scratch*" 
has name.

> * lisp/org-refile.el (org-refile-get-targets): Ensure
> arg of `file-name-non' and `file-truename' is non-nil.

truncated function name

Comments below a loosely related to the suggested patch.

> --- a/lisp/org-refile.el
> +++ b/lisp/org-refile.el
> @@ -310,11 +310,13 @@ converted to a headline before refiling."
>   		 (setq f (buffer-file-name (buffer-base-buffer f))))
>   	       (setq f (and f (expand-file-name f)))
>   	       (when (eq org-refile-use-outline-path 'file)
> -		 (push (list (file-name-nondirectory f) f nil nil) tgs))
> +		 (push (list (and f (file-name-nondirectory f)) f nil nil) tgs))
>   	       (when (eq org-refile-use-outline-path 'buffer-name)
>   		 (push (list (buffer-name (buffer-base-buffer)) f nil nil) tgs))
>   	       (when (eq org-refile-use-outline-path 'full-file-path)
> -		 (push (list (file-truename (buffer-file-name (buffer-base-buffer))) f nil nil) tgs))

Notice `file-truename'

> +		 (push (list (and (buffer-file-name (buffer-base-buffer))
> +                                  (file-truename (buffer-file-name (buffer-base-buffer))))
> +                             f nil nil) tgs))
>   	       (org-with-wide-buffer
>   		(goto-char (point-min))
>   		(setq org-outline-path-cache nil)
> @@ -337,9 +339,10 @@ converted to a headline before refiling."
>   				#'identity
>   				(append
>   				 (pcase org-refile-use-outline-path
> -				   (`file (list (file-name-nondirectory
> -						 (buffer-file-name
> -						  (buffer-base-buffer)))))
> +				   (`file (list
> +                                           (and (buffer-file-name (buffer-base-buffer))

I hope, additional operation in the inner loop has minimal performance 
penalty. Additional variable may be introduced and initialized before 
the loop over headings. It may help to avoid discrepancy similar to 
`file-truename'.

> +                                                (file-name-nondirectory
> +                                                 (buffer-file-name (buffer-base-buffer))))))
>   				   (`full-file-path
>   				    (list (buffer-file-name
>   					   (buffer-base-buffer))))

Earlier patch, that added `file-truename' above, missed this point. At 
first I was surprised why this clause does not require modification.

To maintainers:
What are negative consequences of completely skipping of buffers that 
have no associated files? I mean not to add them to the list for 
iteration. I can anticipate some tests should be fixed but I do not 
think it is a problem. Anyway `org-goto' and `org-refile' do not work in 
such buffers (from my point of view, reaction to such issue should be 
saving content of buffer to some file since critical size of notes is 
achieved):

On 14/05/2021 01:20, satotake wrote:
> In addition to your point, I found that we cannot refile internally even with
> 
> my patch. In other words, if we can cache and reuse it, error ("Please 
> indicate a target file in the refile path") is raised
> when we select it as refile target.
> Probably, we need to some additional fixations.
> It may be good to filter `files' which does not have `buffer-file-name'.

>> Have you tried org-capture? 
> Yes. Mostly I use org-capture, especially, for creating TODO tasks.
> I sometimes start with fundamental-mode new buffer.
> After writing some texts, I switch to org-mode and try to call refile.
> I do not know why I do it by myself clearly but I tend to do it when I
> do not have any clear goal for the file.

Thank you for details. I think, it is a kind of valid workflow. 
Personally I would consider it unreliable however. I suppose, 
`org-default-notes-file' (default capture target) better suited for 
temporary notes since it minimizes risk of lost text in the case of 
failure. It requires to change habits a bit to call org-capture with 
simple template in advance.

Re: [PATCH] org-refile.el: Fix the case of emtpy buffer name

[Bastien]

Hi,

satotake <doublequotation@gmail.com> writes:

> * lisp/org-refile.el (org-refile-get-targets): Ensure
> arg of `file-name-non' and `file-truename' is non-nil.
>
> If you set `org-refile-use-outline-path' `file' or `full-file-path',
> and call `org-refile' in the buffer before visiting file,
> errors are raised at these point. To fix them,
> check if they are nil or not.
>
> TINYCHANGE

Applied in maint as d285831af with a slightly updated commit message,
thank you.

-- 
 Bastien

Re: [PATCH] org-refile.el: Fix the case of emtpy buffer name

[satotake]

I was just refactoring my earlier patch based on Maxim's suggestions.
I attached it but it might be too late.
If the revision make you confused, please ignore it.

Anyway, 
thank you for your reviewing, Maxim and Bastien.

[PATCH] org-refile.el: Fix the case of *scratch* buffer

[satotake]

* lisp/org-refile.el (org-refile-get-targets): Ensure
arg of `file-name-nondirectory' and `file-truename' is non-nil.

If you set `org-refile-use-outline-path' `file' or `full-file-path',
and call `org-refile' in the buffer before visiting file,
errors are raised at these point. To fix them,
check if they are nil or not.

TINYCHANGE
---
 lisp/org-refile.el | 34 ++++++++++++++++------------------
 1 file changed, 16 insertions(+), 18 deletions(-)

diff --git a/lisp/org-refile.el b/lisp/org-refile.el
index 24a1bde51..01a0a6f7f 100644
--- a/lisp/org-refile.el
+++ b/lisp/org-refile.el
@@ -267,7 +267,7 @@ converted to a headline before refiling."
   (let ((case-fold-search nil)
 	;; otherwise org confuses "TODO" as a kw and "Todo" as a word
 	(entries (or org-refile-targets '((nil . (:level . 1)))))
-	targets tgs files desc descre)
+	targets tgs files desc descre tg-base)
     (message "Getting targets...")
     (with-current-buffer (or default-buffer (current-buffer))
       (dolist (entry entries)
@@ -309,12 +309,17 @@ converted to a headline before refiling."
 	       (when (bufferp f)
 		 (setq f (buffer-file-name (buffer-base-buffer f))))
 	       (setq f (and f (expand-file-name f)))
-	       (when (eq org-refile-use-outline-path 'file)
-		 (push (list (file-name-nondirectory f) f nil nil) tgs))
-	       (when (eq org-refile-use-outline-path 'buffer-name)
-		 (push (list (buffer-name (buffer-base-buffer)) f nil nil) tgs))
-	       (when (eq org-refile-use-outline-path 'full-file-path)
-		 (push (list (file-truename (buffer-file-name (buffer-base-buffer))) f nil nil) tgs))
+               (setq tg-base
+                     (pcase org-refile-use-outline-path
+                       (`file (and f (file-name-nondirectory f)))
+                       (`full-file-path
+                        (and (buffer-file-name (buffer-base-buffer))
+                             (file-truename (buffer-file-name (buffer-base-buffer)))))
+                       (`buffer-name (buffer-name (buffer-base-buffer)))
+                       (_ nil)))
+               (when (member org-refile-use-outline-path
+                             '(file full-file-path buffer-name))
+                 (push (list tg-base f nil nil) tgs))
 	       (org-with-wide-buffer
 		(goto-char (point-min))
 		(setq org-outline-path-cache nil)
@@ -336,17 +341,10 @@ converted to a headline before refiling."
 			       (mapconcat
 				#'identity
 				(append
-				 (pcase org-refile-use-outline-path
-				   (`file (list (file-name-nondirectory
-						 (buffer-file-name
-						  (buffer-base-buffer)))))
-				   (`full-file-path
-				    (list (buffer-file-name
-					   (buffer-base-buffer))))
-				   (`buffer-name
-				    (list (buffer-name
-					   (buffer-base-buffer))))
-				   (_ nil))
+                                 (if (member org-refile-use-outline-path
+                                             '(file full-file-path buffer-name))
+                                     (list tg-base)
+                                   nil)
 				 (mapcar (lambda (s) (replace-regexp-in-string
 						      "/" "\\/" s nil t))
 					 (org-get-outline-path t t)))
-- 
2.30.0

[PATCH 2/2] org-refile.el: Fix test case

[satotake]

* lisp/org-refile.el (org-refile-get-targets): Check
`buffer-file-name' return value instead of `buffer-base-buffer'.

To pass the related tests, we need to check `buffer-file-name'.

TINYCHANGE
---
 lisp/org-refile.el | 14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)

diff --git a/lisp/org-refile.el b/lisp/org-refile.el
index 6f5b8acee..2900be27e 100644
--- a/lisp/org-refile.el
+++ b/lisp/org-refile.el
@@ -310,13 +310,12 @@ converted to a headline before refiling."
 		 (setq f (buffer-file-name (buffer-base-buffer f))))
 	       (setq f (and f (expand-file-name f)))
 	       (when (eq org-refile-use-outline-path 'file)
-		 (push (list (if f (file-name-nondirectory f) nil) f nil nil) tgs))
+		 (push (list (and f (file-name-nondirectory f)) f nil nil) tgs))
 	       (when (eq org-refile-use-outline-path 'buffer-name)
 		 (push (list (buffer-name (buffer-base-buffer)) f nil nil) tgs))
 	       (when (eq org-refile-use-outline-path 'full-file-path)
-		 (push (list (if (buffer-base-buffer)
-                                 (file-truename (buffer-file-name (buffer-base-buffer)))
-                               nil)
+		 (push (list (and (buffer-file-name (buffer-base-buffer))
+                                  (file-truename (buffer-file-name (buffer-base-buffer))))
                              f nil nil) tgs))
 	       (org-with-wide-buffer
 		(goto-char (point-min))
@@ -341,10 +340,9 @@ converted to a headline before refiling."
 				(append
 				 (pcase org-refile-use-outline-path
 				   (`file (list
-                                           (if (buffer-base-buffer)
-                                               (file-name-nondirectory
-                                                (buffer-file-name (buffer-base-buffer)))
-                                                  nil)))
+                                           (and (buffer-file-name (buffer-base-buffer))
+                                                (file-name-nondirectory
+                                                 (buffer-file-name (buffer-base-buffer))))))
 				   (`full-file-path
 				    (list (buffer-file-name
 					   (buffer-base-buffer))))
-- 
2.30.0