org-refile-use-cache and jumps using org-refile or org-goto

org-refile-use-cache and jumps using org-refile or org-goto

[Maxim Nikulin]

It seems, something goes a bit wrong. However it is rather confusion 
than anything really broken.

Maybe everybody uses helm, ivy, etc., so nobody is affected.

Usually I jump from one note to another heading with C-u C-c C-j 
(org-goto interface with target completion). I have not setup helm or 
other similar package. Completion works, maybe it is not really perfect 
(e.g. completion of top-level headings and deeper ones differ a bit). I 
have heard that C-u C-c C-w (org-refile) could work in a similar way. I 
did not like the latter variant due to mandatory file name in the 
beginning of a target.

I have tried to enable org-refile-use-cache. Now completion options 
depends on what method was called earlier: org-refile or org-goto. If 
after cache clean up (C-u C-u C-u C-c C-w) first command was C-u C-c C-j 
(org-goto) than there is no file name as a prefix for both commands. 
However file name is prepended for both C-u C-c C-j and C-u C-c C-w if 
at first org-refile was called.

There is one issue however. Default option option does not work if after 
cache clean other command is called, e.g.
- jump using C-u C-c C-j
- clean cache C-u C-u C-u C-c C-w
- try to jump or to refile [C-u] C-c C-w to default offered option
- "user-error: Invalid target location"

I have just one file in the org-agenda-files list. It is becoming 
larger, that is why I decided to try org-refile-use-cache. My 
customization is minimal (besides org-default-dir and org-agenda-files):

(org-refile-targets (quote ((org-agenda-files :maxlevel . 5))))
(org-outline-path-complete-in-steps nil)

I have tried to add (nil :maxlevel . 5) to org-refile-targets to check 
if options without file name will appear in addition to ones with file 
name for org-refile, but it has not happened. It was not clear at first 
that it is possible to start query with "/" and file name is added when 
TAB is pressed.

Is it expected that with cache enabled, command that causes cache update 
affects appearance of completion options to its sibling (org-refile and 
org-goto)?

Re: org-refile-use-cache and jumps using org-refile or org-goto

[Samuel Wales]

idk if thi will help buit a fwe not3es.  just in case they are
relevant to your case.

fwiw i have not heard of interactions between org-goto and the refile
mechanism and the cache.  i hafve not heard that the latter used a
cache but i do not use it.  refile with and withoyut the goto prefix
should normatively be congruent and both use cache.

i used to use the refile cache, then discovered that ido-hacks sped
ido so much that i didn't need it.  it would cache one fileset and
then be potentially not usable for another.  sort of like one cache
and multiple callers.  idk if htat got fixed.  probably not used much.

until recently in maint, ido and ido hacks with both refile and refile
goto [note: org-refile with a goto arg, not org-goto] has worked
perfectly.  with no cache.  now, there is an issue, where with no
cache that i know of, the first use, or the first use in a long time,
will actually present a huge file list that includes crazy elements
and is not constrained by even the verify function.  thus if i type
faster than it goes, the wrong item gets selected.  even if i use the
same selectors as normal.  so there's some bug or race condition.  it
is slow at those times.  this might not be relevant to your case but
seems worth mentioning.  i think i have not filed a bug report as i
don't know how to repro it at this time.


On 3/2/21, Maxim Nikulin <manikulin@gmail.com> wrote:
> It seems, something goes a bit wrong. However it is rather confusion
> than anything really broken.
>
> Maybe everybody uses helm, ivy, etc., so nobody is affected.
>
> Usually I jump from one note to another heading with C-u C-c C-j
> (org-goto interface with target completion). I have not setup helm or
> other similar package. Completion works, maybe it is not really perfect
> (e.g. completion of top-level headings and deeper ones differ a bit). I
> have heard that C-u C-c C-w (org-refile) could work in a similar way. I
> did not like the latter variant due to mandatory file name in the
> beginning of a target.
>
> I have tried to enable org-refile-use-cache. Now completion options
> depends on what method was called earlier: org-refile or org-goto. If
> after cache clean up (C-u C-u C-u C-c C-w) first command was C-u C-c C-j
> (org-goto) than there is no file name as a prefix for both commands.
> However file name is prepended for both C-u C-c C-j and C-u C-c C-w if
> at first org-refile was called.
>
> There is one issue however. Default option option does not work if after
> cache clean other command is called, e.g.
> - jump using C-u C-c C-j
> - clean cache C-u C-u C-u C-c C-w
> - try to jump or to refile [C-u] C-c C-w to default offered option
> - "user-error: Invalid target location"
>
> I have just one file in the org-agenda-files list. It is becoming
> larger, that is why I decided to try org-refile-use-cache. My
> customization is minimal (besides org-default-dir and org-agenda-files):
>
> (org-refile-targets (quote ((org-agenda-files :maxlevel . 5))))
> (org-outline-path-complete-in-steps nil)
>
> I have tried to add (nil :maxlevel . 5) to org-refile-targets to check
> if options without file name will appear in addition to ones with file
> name for org-refile, but it has not happened. It was not clear at first
> that it is possible to start query with "/" and file name is added when
> TAB is pressed.
>
> Is it expected that with cache enabled, command that causes cache update
> affects appearance of completion options to its sibling (org-refile and
> org-goto)?
>
>
>


-- 
The Kafka Pandemic

Please learn what misopathy is.
https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html

Re: org-refile-use-cache and jumps using org-refile or org-goto

[Maxim Nikulin]

On 03/03/2021 09:34, Samuel Wales wrote:
> 
> until recently in maint, ido and ido hacks with both refile and refile
> goto [note: org-refile with a goto arg, not org-goto] has worked
> perfectly.  with no cache.  now, there is an issue, where with no
> cache that i know of, the first use, or the first use in a long time,
> will actually present a huge file list that includes crazy elements
> and is not constrained by even the verify function.

Concerning performance. After reading your message I have realized that 
it should be really fast to extract several thousand headings from a 
buffer using regexp. However actually it is not so:

(benchmark-run 10 (and (org-refile-get-targets) nil))
| 9.868742533999999 | 40 | 2.999932755999996 |

So preparing the list of refile targets currently takes almost precisely 
1 second. It is slow. Results are same for org-9.1.6 and 
release_9.4.4-231-gf46925. I am surprised however that 9.1.6 and 9.3.1 
are installed as system packages and have compiled files. I have not 
compiled git version but it works with the same speed.

(length (org-refile-get-targets))
: 3220

Preferences:

   (require 'org-refile)
   (setq org-agenda-files '("~/notes/notes.org"))
   (setq org-refile-use-cache nil)
   (setq org-refile-use-outline-path t)
   (setq org-outline-path-complete-in-steps nil)
   (setq org-refile-targets '((org-agenda-files :maxlevel . 5)))

Some lines from profile:

- org-refile-get-targets                   8414  74%
...   - org-get-outline-path               7403  65%
...     - org--get-outline-path-1          7286  64%
...       - org-up-heading-safe            6328  56%

There is a room for improvement. Outline paths could be obtained in a 
single pass without backward search. It should speed up building the 
list of targets by 2 or 3 times.

Though it is unrelated to issues with default option and cache entries 
when both ways to call org-refile are used: directly and through org-goto.

org-refile failed due to default option stored by org-goto

[Maxim Nikulin]

On 03/03/2021 00:15, Maxim Nikulin wrote:
> 
> There is one issue however. Default option option does not work if after 
> cache clean other command is called, e.g.
> - jump using C-u C-c C-j
> - clean cache C-u C-u C-u C-c C-w
> - try to jump or to refile [C-u] C-c C-w to default offered option
> - "user-error: Invalid target location"

I have realized that this issue is rather loosely related to 
org-refile-use-cache and I have seen it in a worse variant with disabled 
cache.

file init-refile.el:

(package-initialize)

(custom-set-variables
  '(org-agenda-files (quote ("~/examples/org/test-notes.org")))
  '(org-capture-templates
    (quote
     (("t" "Test" entry
       (file "")
       "* %?\n\n%U\n" :empty-lines 1))))
  '(org-default-notes-file "~/examples/org/capture.org")
  '(org-directory "~/examples/org/")
  '(org-modules (quote (org-refile)))
  '(org-outline-path-complete-in-steps nil)
  '(org-refile-targets (quote ((nil :maxlevel . 5) (org-agenda-files 
:maxlevel . 5))))
  '(org-refile-use-outline-path t)
)

emacs -Q -L ~/src/emacs/org-mode/lisp/ -L 
~/src/emacs/org-mode/contrib/lisp/ -l ~/examples/org/init-refile.el 
~/examples/org/test-notes.org

- C-u C-c C-j (org-goto) to some heading in the test-notes.org, e.g. 
"Two" / "Third" / "Theme"
- M-x org-capture RET t to capture some note
- Let's assume that target of previously executed org-goto suits for 
this note
- C-c C-w RET to refile the note from capture.org file to the heading in 
test-notes.org offered as the default option.

Actual result:
- "Invalid target location"
- Captured note is still in the capture.org file but it is not apparent 
since capture frame is closed.

Expected result:
- Default option works for refile even if it is remained from org-goto 
command

Side note. In some cases it not so easy to close capture frame. If 
org-default-note-file is configured to a file in a non-existing 
directory then C-c C-k asks to create that directory but does not 
discards the note.

Re: org-refile-use-cache and jumps using org-refile or org-goto

[Samuel Wales]

interesting.  that would be great to speed it up.  [i just meant that
the file list used to be correct.]

but it is slow these days.  for me, (benchmark-run 1 (length
(org-refile-get-targets))) is 8s for 5886 targets.

i presume you mean something like that up heading is unnecessary
because you can keep a running olpath as you search.


On 3/4/21, Maxim Nikulin <manikulin@gmail.com> wrote:
> On 03/03/2021 09:34, Samuel Wales wrote:
>>
>> until recently in maint, ido and ido hacks with both refile and refile
>> goto [note: org-refile with a goto arg, not org-goto] has worked
>> perfectly.  with no cache.  now, there is an issue, where with no
>> cache that i know of, the first use, or the first use in a long time,
>> will actually present a huge file list that includes crazy elements
>> and is not constrained by even the verify function.
>
> Concerning performance. After reading your message I have realized that
> it should be really fast to extract several thousand headings from a
> buffer using regexp. However actually it is not so:
>
> (benchmark-run 10 (and (org-refile-get-targets) nil))
> | 9.868742533999999 | 40 | 2.999932755999996 |
>
> So preparing the list of refile targets currently takes almost precisely
> 1 second. It is slow. Results are same for org-9.1.6 and
> release_9.4.4-231-gf46925. I am surprised however that 9.1.6 and 9.3.1
> are installed as system packages and have compiled files. I have not
> compiled git version but it works with the same speed.
>
> (length (org-refile-get-targets))
> : 3220
>
> Preferences:
>
>    (require 'org-refile)
>    (setq org-agenda-files '("~/notes/notes.org"))
>    (setq org-refile-use-cache nil)
>    (setq org-refile-use-outline-path t)
>    (setq org-outline-path-complete-in-steps nil)
>    (setq org-refile-targets '((org-agenda-files :maxlevel . 5)))
>
> Some lines from profile:
>
> - org-refile-get-targets                   8414  74%
> ...   - org-get-outline-path               7403  65%
> ...     - org--get-outline-path-1          7286  64%
> ...       - org-up-heading-safe            6328  56%
>
> There is a room for improvement. Outline paths could be obtained in a
> single pass without backward search. It should speed up building the
> list of targets by 2 or 3 times.
>
> Though it is unrelated to issues with default option and cache entries
> when both ways to call org-refile are used: directly and through org-goto.
>
>
>


-- 
The Kafka Pandemic

Please learn what misopathy is.
https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html

Re: org-refile failed due to default option stored by org-goto

[Samuel Wales]

if you are willing to try something as a complete stab in the dark,
here is something i have had from many years ago that fixed issues
possibly including a defaulting issue.  the issues include other stuff
too, but i do not understand the code now for health reasons to rule
this out as a solution to your problem in addition to the problem i
put in the comment and name.  i havce carried along this patch for
eyars.

commit deaa14b6ed264c259a1f3b805b67b2db3951ba20
Author: Your Name <you@example.com>
Date:   2020-09-20 13:30:55 -0700

    === alpha remove the parens from ido completion of olpaths

	Modified lisp/org-refile.el
diff --git a/lisp/org-refile.el b/lisp/org-refile.el
index 1e6872b46..8f98e9cf9 100644
--- a/lisp/org-refile.el
+++ b/lisp/org-refile.el
@@ -617,7 +617,9 @@ this function appends the default value from
 	 (tbl (mapcar
 	       (lambda (x)
 		 (if (and (not (member org-refile-use-outline-path
-				       '(file full-file-path)))
+                                       ;; === alpha remove the parens
from ido completion of olpaths
+				       '(nil full-file-path)))
+				       ;; '(file full-file-path)))
 			  (not (equal filename (nth 1 x))))
 		     (cons (concat (car x) extra " ("
 				   (file-name-nondirectory (nth 1 x)) ")")



On 3/4/21, Maxim Nikulin <manikulin@gmail.com> wrote:
> On 03/03/2021 00:15, Maxim Nikulin wrote:
>>
>> There is one issue however. Default option option does not work if after
>> cache clean other command is called, e.g.
>> - jump using C-u C-c C-j
>> - clean cache C-u C-u C-u C-c C-w
>> - try to jump or to refile [C-u] C-c C-w to default offered option
>> - "user-error: Invalid target location"
>
> I have realized that this issue is rather loosely related to
> org-refile-use-cache and I have seen it in a worse variant with disabled
> cache.
>
> file init-refile.el:
>
> (package-initialize)
>
> (custom-set-variables
>   '(org-agenda-files (quote ("~/examples/org/test-notes.org")))
>   '(org-capture-templates
>     (quote
>      (("t" "Test" entry
>        (file "")
>        "* %?\n\n%U\n" :empty-lines 1))))
>   '(org-default-notes-file "~/examples/org/capture.org")
>   '(org-directory "~/examples/org/")
>   '(org-modules (quote (org-refile)))
>   '(org-outline-path-complete-in-steps nil)
>   '(org-refile-targets (quote ((nil :maxlevel . 5) (org-agenda-files
> :maxlevel . 5))))
>   '(org-refile-use-outline-path t)
> )
>
> emacs -Q -L ~/src/emacs/org-mode/lisp/ -L
> ~/src/emacs/org-mode/contrib/lisp/ -l ~/examples/org/init-refile.el
> ~/examples/org/test-notes.org
>
> - C-u C-c C-j (org-goto) to some heading in the test-notes.org, e.g.
> "Two" / "Third" / "Theme"
> - M-x org-capture RET t to capture some note
> - Let's assume that target of previously executed org-goto suits for
> this note
> - C-c C-w RET to refile the note from capture.org file to the heading in
> test-notes.org offered as the default option.
>
> Actual result:
> - "Invalid target location"
> - Captured note is still in the capture.org file but it is not apparent
> since capture frame is closed.
>
> Expected result:
> - Default option works for refile even if it is remained from org-goto
> command
>
> Side note. In some cases it not so easy to close capture frame. If
> org-default-note-file is configured to a file in a non-existing
> directory then C-c C-k asks to create that directory but does not
> discards the note.
>
>
>


-- 
The Kafka Pandemic

Please learn what misopathy is.
https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html

[PATCH] optimize org-refile-get-targets

[Maxim Nikulin]

[-- Attachment #1: Type: text/plain, Size: 705 bytes --]

On 05/03/2021 04:03, Samuel Wales wrote:
> interesting.  that would be great to speed it up.  [i just meant that
> the file list used to be correct.]

I am a bit disappointed. I have managed to get x2 performance boost. At 
first, the result was x2 better but I realized that I did not added 
heading cleanup for the new strategy. It adds several regexp to the 
inner loop and severely hits performance. I do not like the idea to 
manually inline (with minor variations) existing functions and regexps, 
but the function is still slow.

For a while, improvement is significant, so I am attaching the patch 
that should make jumps using org-goto or org-refile faster. I hope, I 
have not broken anything.


[-- Attachment #2: refile-get-targets-outline-path-tests.patch --]
[-- Type: text/x-patch, Size: 2012 bytes --]

commit 45cfa5b15e9009fee4f6a688caa210ff543b1ac1
Author: Max Nikulin <manikulin@gmail.com>
Date:   Sat Mar 6 22:14:39 2021 +0700

    testing/lisp/test-org.el: More tests for `org-refile-get-targets'
    
    testing/lisp/test-org.el (test-org/refile-get-targets): Add a few more
    cases for the `org-refile-get-targets' function.
    
    - Fraction of completed subtasks is removed from heading.
    - `:level' filter ignores headings having over levels.
    - Outline path works with `:tag' filter.
    
    The aim is to increase coverage for experiments with optimizing
    of the function.

diff --git a/testing/lisp/test-org.el b/testing/lisp/test-org.el
index 2d727ba7a..da313b45b 100644
--- a/testing/lisp/test-org.el
+++ b/testing/lisp/test-org.el
@@ -6369,6 +6369,27 @@ Paragraph<point>"
 	    (let ((org-refile-use-outline-path t)
 		  (org-refile-targets `((nil :maxlevel . 1))))
 	      (mapcar #'car (org-refile-get-targets))))))
+  ;; When providing targets as paths, clean fraction cookie.
+  (should
+   (equal '("H1")
+	  (org-test-with-temp-text "* H1 [1/1]"
+	    (let ((org-refile-use-outline-path t)
+		  (org-refile-targets '((nil :maxlevel . 1))))
+	      (mapcar #'car (org-refile-get-targets))))))
+  ;; When providing targets as paths, intermediate paths are cached correctly.
+  (should
+   (equal '("H1/H2")
+	  (org-test-with-temp-text "* Skip 1\n* H1\n*** Skip 2\n** H2\n*** Skip 3"
+	    (let ((org-refile-use-outline-path t)
+		  (org-refile-targets '((nil :level . 2))))
+	      (mapcar #'car (org-refile-get-targets))))))
+  ;; When providing targets as paths, they are obtained correctly.
+  (should
+   (equal '("H1/H2" "H3")
+	  (org-test-with-temp-text "* Skip 1\n* H1\n** Skip 2\n** H2 :take:\n* H3 :take:"
+	    (let ((org-refile-use-outline-path t)
+		  (org-refile-targets '((nil :tag . "take"))))
+	      (mapcar #'car (org-refile-get-targets))))))
   ;; When `org-refile-use-outline-path' is `file', include file name
   ;; without directory in targets.
   (should

[-- Attachment #3: refile-get-targets-outline-path-optimize.patch --]
[-- Type: text/x-patch, Size: 4030 bytes --]

commit 58b477e999f3bb5b48c39fe0a4e5ad0d37e2bb9d
Author: Max Nikulin <manikulin@gmail.com>
Date:   Sat Mar 6 22:44:27 2021 +0700

    lisp/org-refile.el: Speed up `org-refile-get-targets'
    
    lisp/org-refile.el (org-refile-get-targets): Optimize performance
    by eliminating backward lookup of already seen headers.
    
    If configuration allows it, incrementally update current outline path.
    For dense target trees (`:maxlevel' and `:level') it allows to avoid
    "one step forward, two steps back" strategy that requires multiple
    backward searches for deeply nested headings.

diff --git a/lisp/org-refile.el b/lisp/org-refile.el
index 4e9f26eff..8e760f1c3 100644
--- a/lisp/org-refile.el
+++ b/lisp/org-refile.el
@@ -267,7 +267,8 @@ converted to a headline before refiling."
   (let ((case-fold-search nil)
 	;; otherwise org confuses "TODO" as a kw and "Todo" as a word
 	(entries (or org-refile-targets '((nil . (:level . 1)))))
-	targets tgs files desc descre)
+	targets tgs files desc descre
+	outline-path cache-outline-path target-outline-level)
     (message "Getting targets...")
     (with-current-buffer (or default-buffer (current-buffer))
       (dolist (entry entries)
@@ -281,6 +282,11 @@ converted to a headline before refiling."
 	 ((and (symbolp files) (boundp files))
 	  (setq files (symbol-value files))))
 	(when (stringp files) (setq files (list files)))
+	(setq cache-outline-path (and org-refile-use-outline-path
+				      (memq (car desc) '(:level :maxlevel))))
+	(setq target-outline-level
+	      (if (and cache-outline-path (eq (car desc) :level))
+		  (if org-odd-levels-only (1- (* 2 (cdr desc))) (cdr desc))))
 	(cond
 	 ((eq (car desc) :tag)
 	  (setq descre (concat "^\\*+[ \t]+.*?:" (regexp-quote (cdr desc)) ":")))
@@ -288,13 +294,13 @@ converted to a headline before refiling."
 	  (setq descre (concat "^\\*+[ \t]+" (regexp-quote (cdr desc)) "[ \t]")))
 	 ((eq (car desc) :regexp)
 	  (setq descre (cdr desc)))
-	 ((eq (car desc) :level)
+	 ((and (not target-outline-level) (eq (car desc) :level))
 	  (setq descre (concat "^\\*\\{" (number-to-string
 					  (if org-odd-levels-only
 					      (1- (* 2 (cdr desc)))
 					    (cdr desc)))
 			       "\\}[ \t]")))
-	 ((eq (car desc) :maxlevel)
+	 ((memq (car desc) '(:level :maxlevel))
 	  (setq descre (concat "^\\*\\{1," (number-to-string
 					    (if org-odd-levels-only
 						(1- (* 2 (cdr desc)))
@@ -318,13 +324,30 @@ converted to a headline before refiling."
 	       (org-with-wide-buffer
 		(goto-char (point-min))
 		(setq org-outline-path-cache nil)
+		(setq outline-path nil)
 		(while (re-search-forward descre nil t)
 		  (beginning-of-line)
 		  (let ((case-fold-search nil))
 		    (looking-at org-complex-heading-regexp))
 		  (let ((begin (point))
-			(heading (match-string-no-properties 4)))
-		    (unless (or (and
+			(heading (match-string-no-properties 4))
+			(heading-level (length (match-string-no-properties 1))))
+		    (when cache-outline-path
+		      (while (and outline-path (<= heading-level (caar outline-path)))
+			(pop outline-path))
+		      (push (cons heading-level
+				  ;; Taken from org--get-outline-path-1. It is really slow.
+				  (if (not heading)
+				      ""
+				    (org-trim
+				     (org-link-display-format
+				      (replace-regexp-in-string
+				       "\\[[0-9]+%\\]\\|\\[[0-9]+/[0-9]+\\]" ""
+				       heading)))))
+			    outline-path))
+		    (unless (or (and target-outline-level
+				    (not (eq heading-level target-outline-level)))
+				(and
 				 org-refile-target-verify-function
 				 (not
 				  (funcall org-refile-target-verify-function)))
@@ -349,7 +372,9 @@ converted to a headline before refiling."
 				   (_ nil))
 				 (mapcar (lambda (s) (replace-regexp-in-string
 						      "/" "\\/" s nil t))
-					 (org-get-outline-path t t)))
+					 (if outline-path
+					     (nreverse (mapcar #'cdr outline-path))
+					   (org-get-outline-path t t))))
 				"/"))))
 			(push (list target f re (org-refile-marker (point)))
 			      tgs)))

Re: org-refile failed due to default option stored by org-goto

[Maxim Nikulin]

On 05/03/2021 05:53, Samuel Wales wrote:
> diff --git a/lisp/org-refile.el b/lisp/org-refile.el
> index 1e6872b46..8f98e9cf9 100644
> --- a/lisp/org-refile.el
> +++ b/lisp/org-refile.el
> @@ -617,7 +617,9 @@ this function appends the default value from
>   	 (tbl (mapcar
>   	       (lambda (x)
>   		 (if (and (not (member org-refile-use-outline-path
> -				       '(file full-file-path)))
> +                                       ;; === alpha remove the parens
> from ido completion of olpaths
> +				       '(nil full-file-path)))
> +				       ;; '(file full-file-path)))
>   			  (not (equal filename (nth 1 x))))
>   		     (cons (concat (car x) extra " ("
>   				   (file-name-nondirectory (nth 1 x)) ")")

The patch changes appearance of "(file.org)" at the end of each 
completion option. I do not know value of your 
org-refile-use-outline-path setting so I am unsure if your are trying to 
avoid it or vice versa to force it.

I do not think that the patch is relevant to sharing of default option 
between org-goto and org-refile. I think, buffer (or file) name should 
be stored separately from outline path within the buffer in org-refile 
history and merged accordingly to current settings just before 
completing-read. org-refile-cache likely requires a similar approach.

Personally I was confused by behavior of (org-refile-use-outline-path 
t). Completion options appears as "A (file.org)", "A/B (file.org)", and 
I could not figure out how to refile directly to "A" headings. I suspect 
that org-refile-allow-creating-parent-nodes does not work with "t" at 
all. Recently I have realized that after typing of "A/B" TAB completes 
option to "A/B (file.org)" and I was trying RET after "A". It is 
necessary to add space "A " to refile to a non-leaf node.

That was the reason why I set (org-refile-use-outline-path 'file). 
Completion options appears as "file.org/A/B". I have to press TAB at 
first to get "file.org/" but then after "file.org/A" RET works fine.

Re: [PATCH] optimize org-refile-get-targets

[Bastien]

Hi Maxim,

Maxim Nikulin <manikulin@gmail.com> writes:

> For a while, improvement is significant, so I am attaching the patch
> that should make jumps using org-goto or org-refile faster. I hope, I
> have not broken anything.

Thanks a lot for the patch.

Are you still using it?  Did anyone tested it?

Re: [PATCH] optimize org-refile-get-targets

[Maxim Nikulin]

On 25/04/2021 19:25, Bastien wrote:
> 
> Are you still using it?  Did anyone tested it?

I realized that I break cache a bit, so the patch should not be applied 
in the current form. I have an idea how to fix it but I have found more 
problems around (independent of my patch). I did not worried since the 
patch did not appeared on updates.orgmode.org (I guess, due to "git 
diff" instead of "git format patch".)

Cache key is regexp. With my patch, level and max-level options share 
the same regexp. I think, cache for both options could be generated as 
for max-level and filtered in the case of "level". It may simplify the code.

I would be grateful for other comments.

I have not realized how to reduce number of regexps inside the loop 
without duplication of some regegexps. I have seen a statement that 
cache for compiled regular expressions in emacs is rather small so 
several patterns used on each cycle is enough to make it useless with 
significant performance penalty.

Other issues:
- Indirect buffer created with C-x 4 c does not reuse cache created for 
the main buffer, so first call in new buffer have to generate its own cache.
- C-u C-c C-j promises to use headings from the buffer, but in the case 
of subtree indirect buffer, headings are not filtered to the narrowed part.
- I have not solved the problem with default option for `org-refile' and 
`org-goto'. Maybe more detailed structure should be stored in history 
and formatted accordingly to current option at the moment of invocation.