Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Gustavo Barros]

Hi Org list,

`org-refile-get-target', when using `org-refile-use-outline-path' 
appends an "extra" slash at the end of every path, but candidates are 
stored in `org-refile-history' without that extra slash. As the default 
candidate passed to `completing-read' is the car of `org-refile-history' 
(the last refile target), the default candidate is provided in duplicity 
(once with the trailing slash and once without it). This is the case 
independent of the completion framework in use, but the difference is 
less prominent with the default `completing-read-default' and more so 
with, e.g., `ivy-completing-read'.

Steps to reproduce:

- Start 'emacs -Q'

- Do an initial setup:
  #+begin_src emacs-lisp
  (package-initialize)
  (setq org-refile-targets '(("~/org/test.org" :maxlevel . 2)))
  (setq org-refile-use-outline-path 'file)
  (setq org-outline-path-complete-in-steps nil)
  (ivy-mode)
  ;; as mentioned, Ivy just makes things clearer, the issue is 
  independent of it
  #+end_src

- Open file "~/org/test.org", with contents:
  #+begin_src org
  ,* Top heading 1
  ,* Top heading 2
  ,** Entry 1
  ,** Entry 2
  #+end_src

- Go to heading "Entry 1", refile it to "Top heading 1"

- Go to heading "Entry 2", and call `org-refile'

- Observe the available candidates, and notice "test.org/Top heading 1" 
  is there twice, once as the default candidate, without a trailing 
  slash, and also on the paths list, with the slash.

Best regards,
Gustavo Barros.



Emacs  : GNU Emacs 26.3 (build 1, x86_64-pc-linux-gnu, GTK+ Version 
3.22.30)
 of 2019-09-30
Package: Org mode version 9.2.6 (9.2.6-4-ge30905-elpaplus @ 
/home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)

current state:
==============
(setq
 org-src-mode-hook '(org-src-babel-configure-edit-buffer 
 org-src-mode-configure-edit-buffer)
 org-metadown-hook '(org-babel-pop-to-session-maybe)
 org-clock-out-hook '(org-clock-remove-empty-clock-drawer)
 org-refile-targets '(("~/org/test.org" :maxlevel . 2))
 org-mode-hook '(#[0 "\300\301\302\303\304$\207" [add-hook 
 change-major-mode-hook org-show-all append local] 5]
		 #[0 "\300\301\302\303\304$\207"
		   [add-hook change-major-mode-hook 
		   org-babel-show-result-all append local] 5]
		 org-babel-result-hide-spec org-babel-hide-all-hashes 
		 org-eldoc-load)
 org-outline-path-complete-in-steps nil
 org-archive-hook '(org-attach-archive-delete-maybe)
 org-confirm-elisp-link-function 'yes-or-no-p
 org-agenda-before-write-hook '(org-agenda-add-entry-text)
 org-metaup-hook '(org-babel-load-in-session-maybe)
 org-bibtex-headline-format-function #[257 "\300\236A\207" [:title] 3 
 "\n\n(fn ENTRY)"]
 org-babel-pre-tangle-hook '(save-buffer)
 org-tab-first-hook '(org-babel-hide-result-toggle-maybe 
 org-babel-header-arg-expand)
 org-src-lang-modes '(("arduino" . arduino) ("redis" . redis) ("php" 
 . php) ("C" . c) ("C++" . c++)
		      ("asymptote" . asy) ("bash" . sh) ("beamer" 
		      . latex) ("calc" . fundamental) ("cpp" . c++)
		      ("ditaa" . artist) ("dot" . fundamental) ("elisp" 
		      . emacs-lisp) ("ocaml" . tuareg)
		      ("screen" . shell-script) ("shell" . sh) ("sqlite" 
		      . sql))
 org-occur-hook '(org-first-headline-recenter)
 org-cycle-hook '(org-cycle-hide-archived-subtrees 
 org-cycle-show-empty-lines
		  org-optimize-window-after-visibility-change)
 org-speed-command-hook '(org-speed-command-activate 
 org-babel-speed-command-activate)
 org-refile-use-outline-path 'file
 org-confirm-shell-link-function 'yes-or-no-p
 org-link-parameters '(("id" :follow org-id-open) ("eww" :follow eww 
 :store org-eww-store-link)
		       ("rmail" :follow org-rmail-open :store 
		       org-rmail-store-link)
		       ("mhe" :follow org-mhe-open :store 
		       org-mhe-store-link)
		       ("irc" :follow org-irc-visit :store 
		       org-irc-store-link :export org-irc-export)
		       ("info" :follow org-info-open :export 
		       org-info-export :store org-info-store-link)
		       ("gnus" :follow org-gnus-open :store 
		       org-gnus-store-link)
		       ("docview" :follow org-docview-open :export 
		       org-docview-export :store
			org-docview-store-link)
		       ("bibtex" :follow org-bibtex-open :store 
		       org-bibtex-store-link)
		       ("bbdb" :follow org-bbdb-open :export 
		       org-bbdb-export :complete org-bbdb-complete-link
			:store org-bbdb-store-link)
		       ("w3m" :store org-w3m-store-link) ("file+sys") 
		       ("file+emacs")
		       ("elfeed" :follow elfeed-link-open :store 
		       elfeed-link-store-link)
		       ("doi" :follow org--open-doi-link) ("elisp" 
		       :follow org--open-elisp-link)
		       ("file" :complete org-file-complete-link)
		       ("ftp" :follow (lambda (path) (browse-url (concat 
		       "ftp:" path))))
		       ("help" :follow org--open-help-link)
		       ("http" :follow (lambda (path) (browse-url 
		       (concat "http:" path))))
		       ("https" :follow (lambda (path) (browse-url 
		       (concat "https:" path))))
		       ("mailto" :follow (lambda (path) (browse-url 
		       (concat "mailto:" path))))
		       ("news" :follow (lambda (path) (browse-url 
		       (concat "news:" path))))
		       ("shell" :follow org--open-shell-link))
 )

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Bastien]

Hi Gustavo,

this should be fixed in Org master branch, thanks for the detailed
report.  If you can confirm the fix, even better.

Best,

-- 
 Bastien

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Gustavo Barros]

Hi Bastien,

thank you very much for looking into this.

On Wed, Feb 12 2020, Bastien wrote:

> this should be fixed in Org master branch, thanks for the detailed
> report.  If you can confirm the fix, even better.

I tested it and indeed the duplicate candidate is gone. However, the 
last refile target no longer seems to be offered as the default for a 
subsequent refile operation. Was that intentional?

To be more precise, in terms of the ECM of the initial bug report, when 
refiling "Entry 2" the default target is "test.org/", when I would 
expect it to be "test.org/Top heading 1".

Best,
Gustavo.

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Bastien]

Hi Gustavo,

Gustavo Barros <gusbrs.2016@gmail.com> writes:

> I tested it and indeed the duplicate candidate is gone. However, the
> last refile target no longer seems to be offered as the default for a 
> subsequent refile operation. Was that intentional?

Nope, an oversight -- fixed in master.

Thanks!

-- 
 Bastien

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Samuel Wales]

dunno if this is useful (probably not), but i found the following was
necessary to fix ido.  i didn't really understand it, but it fixed it.

Date:   2014-05-19 19:57:44 -0700

    === remove the parens from ido completion of olpaths

	Modified lisp/org.el
diff --git a/lisp/org.el b/lisp/org.el
index 69a288add..b2d8ec755 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -11942,7 +11942,7 @@ this function appends the default value from
 	 (tbl (mapcar
 	       (lambda (x)
 		 (if (and (not (member org-refile-use-outline-path
-				       '(file full-file-path)))
+				       '(nil file full-file-path)))
 			  (not (equal filename (nth 1 x))))
 		     (cons (concat (car x) extra " ("
 				   (file-name-nondirectory (nth 1 x)) ")")



-- 
The Kafka Pandemic

What is misopathy?
https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Bastien]

Hi Samuel,

Samuel Wales <samologist@gmail.com> writes:

> dunno if this is useful (probably not), but i found the following was
> necessary to fix ido.

What was the problem when refiling with ido?

> i didn't really understand it, but it fixed it.

I have tested the patch, but I'd like to understand and reproduce the
bug before accepting the fix.

Thanks,

-- 
 Bastien

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Samuel Wales]

On 2/14/20, Bastien <bzg@gnu.org> wrote:
> What was the problem when refiling with ido?

idk if the following will be useful but here goes.

bear in mind this was a while back.  this is partly from memory.

i don't know if the fix fixed all of them, but it definitely seemed
to, at least with my ido settings.

i tried ido settings.  but i recall thinking none of them worked.  it
took years to fix it.  my computer use is limited.

- the default was screwed up (more below) causing misfiling
- there was distracting, inconsistent (sometimes appearing and
sometimes not), and possibly redundant (i.e. extra olpaths in list)
information in parentheses
- similar, except with trailing slashes (this one might have had
redundancy and inconsistency properties, the latter perhaps caused by
the former, as in: you don't know which version of the olpath you will
get in the default)

the mechanism seemed to be designed for fs paths, not olpaths.  fs
paths have a directory/nondirectory distinction, and this might have
been related.  that distinction is bad for olpaths.

i do not deal with files when refiling.  even if i did, there would
still be no need for olpath directories for ido/ivy refiling.

my olpaths when refiling are bare header lines.

i think some but not all, olpaths got trailing slashes appended.

i realize that's all vague.

however, the fix made refiling change from dangerous and iffy and
annoying to safer, more reliable, and fast (ido-clever-paths and
ido-hacks also helped).

i will not be able to retrace the problem less vaguely.


as for the issues with the default, i reported at least one to the
mailing list in some detail.  it provides repro instructions.

i suspect trailing slash also caused a default problem.

please note that i am using an old version of maint because i can't
fix a capture bug that was introduced a while back.  it has perhaps
been fixed since then, dunno.


here is the report, for what it is worth:

vvv
ido and org-refile are a superb combination that enhances
Org significantly.  It is a killer feature IMO.

However, in my usage there is a substantial risk of
misfiling:

  1) If I start from a fresh Emacs, "myorg" is sufficient to
     select "computer/emacs/org/myorg/".  Good.

  2) If I refile something to
     "computer/emacs/org/myorg/strategy and examples/various
     todo kw and maybe some tags/", "various" is enough to
     select it.  Note that this is below "myorg".  Good.

  3) (Just as an aside, if I then refile something else to
     the same entry, I don't even need to enter "various".
     It is the default.  Good.)

  4) Now suppose I want to refile something to "myorg".  I
     enter "myorg" but I get "various".

Detail on this subtle issue is below:


My expectation is that if "myorg" selects "myorg" once, it
should always do so no matter what my refile history is.
This expectation is violated.

It violates a sort of referential transparency.  The same
narrowing inputs should produce the same narrowed outputs
(in my expectation at least).


I suspect the reason for the behavior is the defaulting in
3, which is useful.  However, the false defaulting in 4 is
not useful in any obvious way.


Proposed solution:

I think that as soon as the user starts selecting something,
the default should be discarded.


With my expectation, it is never necessary to check the
offered olpaths except as a confirmation.

With the current behavior, checking is always necessary
because in edge cases, there will be a guaranteed misfile.

Here is why: the only reason that default showed up at all
is that the narrowing input matched both
headlines.  Otherwise the default would have been discarded.

So it is an edge case that allowed the offer to misfile.  In
other cases, the correct default would be provided.  If I
wanted "various", RET would be sufficient.

User checking is significantly more error-prone because one
olpath is a substring of the other.

Likewise, the requirement to navigate in the list is
burdensome as 4 is never useful as far as I can tell.


I tried looking at ido
variables and got lost.

If you can't reproduce witn your ido settings, I'll try to
provide an MCE at some point.

Thanks.

Samuel
^^^

-- 
The Kafka Pandemic

What is misopathy?
https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Bastien]

Hi Samuel,

Samuel Wales <samologist@gmail.com> writes:

> If you can't reproduce witn your ido settings, I'll try to
> provide an MCE at some point.

I don't use ido-mode, so I don't have any ido setting and It's
difficult for me to both understand and chase the bug.

I suggest you start by checking the bug is still here in latest
maint *and* latest master, then (only then) propose a MCE.

Thanks!

-- 
 Bastien

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Samuel Wales]

your quote is from a many years old report.  i was providing it in
case it was useful historically.

-- 
The Kafka Pandemic

What is misopathy?
https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Samuel Wales]

i found a note that says that there isa nother disticnitont hat was
causing the bugs: a distinction between the current file and
non-current file.

it wreaks havoc to make that distinction.

************* REF this works around the bugs
=this is in my personal patches

vvv
	Modified   lisp/org.el
diff --git a/lisp/org.el b/lisp/org.el
index ec74314..695305c 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -11737,7 +11737,7 @@ this is used for the GOTO interface."
 	 (tbl (mapcar
 	       (lambda (x)
 		 (if (and (not (member org-refile-use-outline-path
-				       '(file full-file-path)))
+				       '(nil file full-file-path)))
 			  (not (equal filename (nth 1 x))))
 		     (cons (concat (car x) extra " ("
 				   (file-name-nondirectory (nth 1 x)) ")")

also, the line after the + line is clearly related to the
duplicate olpath and pointless current file vs. other file
distinction.

the workaround works because there is no olpath and no
filename now, so flex matching cannot screw with assuming
the default as easily.

ideally, however, we would show the filename but not match
on it in ido.
^^^

-- 
The Kafka Pandemic

What is misopathy?
https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Bastien]

Hi Samuel,

sorry to insist, but what I really need is a *fiable and easy* way to
reproduce a bug.  Other comments (like code analysis) are only useful
when I do have a reproducible bug.

I'm willing to spend as much time as needed on a MCE, but I'm waiting
for this MCE before interpreting comments on code and possible bugs.

Thanks for your understanding,

-- 
 Bastien

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Samuel Wales]

On 2/16/20, Bastien <bzg@gnu.org> wrote:
> I'm willing to spend as much time as needed on a MCE, but I'm waiting
> for this MCE before interpreting comments on code and possible bugs.

i see.

fyi i was not specifically trying to report a bug or supply an mce.

instead, i was hoping only to contribute to understanding of the code
that is currently mystifying in this thread with a concrete 1-line fix
that wfm that i hoped would jog somebody's ideas to fix any issues
that are under discussion, such as the trailing slashes.

then you asked for more details so i tried to supply what i could.

if that is not useful, apologies for the noise.  i was not in a
position to get into the details, and am not, but i wanted to mention
something that might help.

as it has been many years, and for various reasons, i will not be able
to create an mce for you at this time.  in addition, even if such did
not apply, i have since changed my org refile settings and ido
settings, and cannot use master or recent maint.

i hope that i have not disrupted the thread.  my fix fixed trailing
slashes and duplicates, is all.

-- 
The Kafka Pandemic

What is misopathy?
https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Bastien]

Hi Samuel,

Samuel Wales <samologist@gmail.com> writes:

> then you asked for more details so i tried to supply what i could.
>
> if that is not useful, apologies for the noise.

no need to apologize!  Information is always useful.  We may have been
miscommunicating: in general, when I ask for details it means that I'd
like to have enough information to reproduce the bug myself. Sometimes
a formal MCE does the job, sometimes informal recollection does too.

I'll keep your patch in mind for further investigation when I have
some time.

Thanks,

-- 
 Bastien

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Samuel Wales]

non-leaf olpaths are not usefully considered to be directories in my
opinion.  all olpaths should be equal in status.


On 2/16/20, Samuel Wales <samologist@gmail.com> wrote:
> On 2/14/20, Bastien <bzg@gnu.org> wrote:
>> What was the problem when refiling with ido?
>
> idk if the following will be useful but here goes.
>
> bear in mind this was a while back.  this is partly from memory.
>
> i don't know if the fix fixed all of them, but it definitely seemed
> to, at least with my ido settings.
>
> i tried ido settings.  but i recall thinking none of them worked.  it
> took years to fix it.  my computer use is limited.
>
> - the default was screwed up (more below) causing misfiling
> - there was distracting, inconsistent (sometimes appearing and
> sometimes not), and possibly redundant (i.e. extra olpaths in list)
> information in parentheses
> - similar, except with trailing slashes (this one might have had
> redundancy and inconsistency properties, the latter perhaps caused by
> the former, as in: you don't know which version of the olpath you will
> get in the default)
>
> the mechanism seemed to be designed for fs paths, not olpaths.  fs
> paths have a directory/nondirectory distinction, and this might have
> been related.  that distinction is bad for olpaths.
>
> i do not deal with files when refiling.  even if i did, there would
> still be no need for olpath directories for ido/ivy refiling.
>
> my olpaths when refiling are bare header lines.
>
> i think some but not all, olpaths got trailing slashes appended.
>
> i realize that's all vague.
>
> however, the fix made refiling change from dangerous and iffy and
> annoying to safer, more reliable, and fast (ido-clever-paths and
> ido-hacks also helped).
>
> i will not be able to retrace the problem less vaguely.
>
>
> as for the issues with the default, i reported at least one to the
> mailing list in some detail.  it provides repro instructions.
>
> i suspect trailing slash also caused a default problem.
>
> please note that i am using an old version of maint because i can't
> fix a capture bug that was introduced a while back.  it has perhaps
> been fixed since then, dunno.
>
>
> here is the report, for what it is worth:
>
> vvv
> ido and org-refile are a superb combination that enhances
> Org significantly.  It is a killer feature IMO.
>
> However, in my usage there is a substantial risk of
> misfiling:
>
>   1) If I start from a fresh Emacs, "myorg" is sufficient to
>      select "computer/emacs/org/myorg/".  Good.
>
>   2) If I refile something to
>      "computer/emacs/org/myorg/strategy and examples/various
>      todo kw and maybe some tags/", "various" is enough to
>      select it.  Note that this is below "myorg".  Good.
>
>   3) (Just as an aside, if I then refile something else to
>      the same entry, I don't even need to enter "various".
>      It is the default.  Good.)
>
>   4) Now suppose I want to refile something to "myorg".  I
>      enter "myorg" but I get "various".
>
> Detail on this subtle issue is below:
>
>
> My expectation is that if "myorg" selects "myorg" once, it
> should always do so no matter what my refile history is.
> This expectation is violated.
>
> It violates a sort of referential transparency.  The same
> narrowing inputs should produce the same narrowed outputs
> (in my expectation at least).
>
>
> I suspect the reason for the behavior is the defaulting in
> 3, which is useful.  However, the false defaulting in 4 is
> not useful in any obvious way.
>
>
> Proposed solution:
>
> I think that as soon as the user starts selecting something,
> the default should be discarded.
>
>
> With my expectation, it is never necessary to check the
> offered olpaths except as a confirmation.
>
> With the current behavior, checking is always necessary
> because in edge cases, there will be a guaranteed misfile.
>
> Here is why: the only reason that default showed up at all
> is that the narrowing input matched both
> headlines.  Otherwise the default would have been discarded.
>
> So it is an edge case that allowed the offer to misfile.  In
> other cases, the correct default would be provided.  If I
> wanted "various", RET would be sufficient.
>
> User checking is significantly more error-prone because one
> olpath is a substring of the other.
>
> Likewise, the requirement to navigate in the list is
> burdensome as 4 is never useful as far as I can tell.
>
>
> I tried looking at ido
> variables and got lost.
>
> If you can't reproduce witn your ido settings, I'll try to
> provide an MCE at some point.
>
> Thanks.
>
> Samuel
> ^^^
>
> --
> The Kafka Pandemic
>
> What is misopathy?
> https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html
>


-- 
The Kafka Pandemic

What is misopathy?
https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Gustavo Barros]

Hi Bastien,

On Thu, Feb 13 2020, Bastien wrote:

>> I tested it and indeed the duplicate candidate is gone. However, the
>> last refile target no longer seems to be offered as the default for a 
>> subsequent refile operation. Was that intentional?
>
> Nope, an oversight -- fixed in master.

Thank you very much.

I've tested it again, and I believe it is working as intended.

I observe, however, a difference of behavior between 
"completing-read-default" and "ivy-completing-read" in the workings of 
"org-refile-get-location". Namely, with "completing-read-default" the 
chosen target is stored in "org-refile-history" with a trailing slash 
(the "extra" part), while with "ivy-completing-read" it is stored 
without the trailing slash.

I have no idea why this is so and also don't know if this stems from 
Org's end. As far as I can tell, functionality of the feature with 
respect to this bug report is working as intended: no duplicate 
candidates, and history is honored. But the difference surprised me and 
if you think it might be important, I can provide an ECM for it.
Otherwise, I think this can well closed as fixed.

Once again, thanks a lot for the fix.

Best,
Gustavo.

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Samuel Wales]

again not sure if relevant and probably not, but it seemed to me that
the problem was related to something thinking that unix paths were
being completed, and directories were treated differently from
non-directories.  whatever i did fixed it.  took years to find and
fix.

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Bastien]

Hi Gustavo,

Gustavo Barros <gusbrs.2016@gmail.com> writes:

> I've tested it again, and I believe it is working as intended.

Thanks for confirming.

> I observe, however, a difference of behavior between
> "completing-read-default" and "ivy-completing-read" in the workings of 
> "org-refile-get-location". Namely, with "completing-read-default" the
> chosen target is stored in "org-refile-history" with a trailing slash 
> (the "extra" part), while with "ivy-completing-read" it is stored
> without the trailing slash.
>
> I have no idea why this is so and also don't know if this stems from
> Org's end. As far as I can tell, functionality of the feature with 
> respect to this bug report is working as intended: no duplicate
> candidates, and history is honored. But the difference surprised me
> and if you think it might be important, I can provide an ECM for it.
> Otherwise, I think this can well closed as fixed.

I've revisited org-refile-get-location given your new information on
ivy-completing-read but I'm not able to see something wrong in the
current implementation of org-refile-get-location.  At the same time,
I don't see why ivy-completing-read would handle arguments differently
than completing-read-default, so _perhaps_ org-refile-get-location
still does something wrong.

If you - or other ivy users - have time to investigate and report,
please do so.

Thanks,

-- 
 Bastien

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Gustavo Barros]

Hi Bastien,

On Fri, Feb 14 2020, Bastien wrote:

> I've revisited org-refile-get-location given your new information on
> ivy-completing-read but I'm not able to see something wrong in the
> current implementation of org-refile-get-location.  At the same time,
> I don't see why ivy-completing-read would handle arguments differently
> than completing-read-default, so _perhaps_ org-refile-get-location
> still does something wrong.

That was pretty much my thought too.

> If you - or other ivy users - have time to investigate and report,
> please do so.

I've tried, when I originally reported, and now again, to pin this 
further down.  Alas, there is much in 'org-refile-get-location' I don't 
understand (in the sense of not understanding why certain things are 
being done). I also have much to learn with respect to 
'completing-read'. But I can provide an ECM to reproduce it.

- Start 'emacs -Q'

- Do an initial setup:
  #+begin_src emacs-lisp
  (add-to-list 'load-path "~/src/org-mode/lisp"); current master
  (setq org-refile-targets '(("~/org/test.org" :maxlevel . 2)))
  (setq org-refile-use-outline-path 'file)
  (setq org-outline-path-complete-in-steps nil)
  #+end_src

- Open file "~/org/test.org", with contents:
  #+begin_src org
  ,* Top heading 1
  ,* Top heading 2
  ,** Entry 1
  ,** Entry 2
  #+end_src

- Now, do some refile operations here. Inspecting 'org-refile-history' 
  will give us:
  #+begin_src emacs-lisp
  ("test.org/Top heading 2/" "test.org/Top heading 2/" "test.org/Top 
  heading 1/" "test.org/Top heading 1/")
  #+end_src

- Now, we reset 'org-refile-history' and start 'ivy-mode':
  #+begin_src emacs-lisp
  (setq org-refile-history nil)
  (add-to-list 'load-path ".emacs.d/elpa/ivy-20200211.1338"); current 
  Melpa
  (require 'ivy)
  (ivy-mode)
  #+end_src

- After some refile operations, the value of 'org-refile-history' is:
  #+begin_src emacs-lisp
  ("test.org/Top heading 2" "test.org/Top heading 2" "test.org/Top 
  heading 1" "test.org/Top heading 1")
  #+end_src

The difference is, as previously reported, the presence/absence of the 
trailing slash (the "extra" in terms of 'org-refile-get-location'). As 
far as I can tell, it is inconsequential to the working of the refile 
operation, but it did intrigue me.

Environment: Org mode version 9.3.6 (release_9.3.6-298-g0fa69d @ 
/home/gustavo/src/org-mode/lisp/); GNU Emacs 26.3 (build 1, 
x86_64-pc-linux-gnu, GTK+ Version 3.22.30) of 2019-11-11; 
ivy-20200211.1338

HTH,
Gustavo.

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Bastien]

Hi Gustavo,

thanks for investigating further.  From what I've seen, ivy-mode does
not set the org-refile-history variable correctly, always setting the
car of this variable to "^" after ivy-completing-read is called.

From what I understand, this is not a bug in Org.  I hope you can fix
this somehow, maybe upstream.

Thanks,

-- 
 Bastien

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Gustavo Barros]

Hi Bastien,

On Fri, Feb 14 2020, Bastien wrote:

> I hope you can fix
> this somehow, maybe upstream.

Unfortunately, I'm afraid I don't understand this enough, and what 
'ivy-completing-read' was supposed to do, to be able to provide a 
pertinent report there.

I personally don't have a problem locally. I had a workaround thus far, 
and with your fix, I won't even need that. As previously reported, the 
issue initially raised is indeed fixed, and things work as expected.

I don't claim a problem persists in Org, and I was just trying to be 
thorough in the testing you requested. And did so with pleasure. So...

> From what I understand, this is not a bug in Org.

... if you are satisfied, I'm happy with that too.

Thank you very very much!

Best,
Gustavo.

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Gustavo Barros]

Hi Bastien,

On Fri, Feb 14 2020, Bastien wrote:

> I've revisited org-refile-get-location given your new information on
> ivy-completing-read but I'm not able to see something wrong in the
> current implementation of org-refile-get-location.  At the same time,
> I don't see why ivy-completing-read would handle arguments differently
> than completing-read-default, so _perhaps_ org-refile-get-location
> still does something wrong.
>
> If you - or other ivy users - have time to investigate and report,
> please do so.

As I said before, there is much that eludes me in 
'org-refile-get-location', but I'm trying to pin this down further by 
getting some inspection points and trying at least to grasp where it 
happens.

In particular, I set one inspection point exactly after the 
completing-read function is called to store the value of local variable 
"answ" which is the return value of the completing-read function. That 
is, right after:

#+begin_src emacs-lisp
(setq answ (funcall cfunc prompt tbl nil (not new-nodes)
			nil 'org-refile-history
			(or cdef (concat (car org-refile-history) 
			extra))))
#+end_src

The value of "answ" right after this step is then:
- with 'ivy-mode' off, that is with 'completing-read-default' as 
  'completing-read-function': "test.org/Top heading 1//" (that is with a 
  *double trailing slash*).
- with 'ivy-mode' on, that is with 'ivy-completing-read' as 
  'completing-read-function': "test.org/Top heading 2/"

In both cases the last trailing slash seems (as far as I understand it) 
to be then trimmed off by 'org-refile--get-location' with:

#+begin_src emacs-lisp
(replace-regexp-in-string "/$" "" refloc)
#+end_src

Why 'ivy-completing-read' does not return the end double slash while 
'completing-read-default' does, I have no idea. But the fact that 
'completing-read-default' returns the refile location with a double 
trailing slash makes me think there is still something to be fixed in 
'org-refile-get-location'.

Best,
Gustavo.

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Bastien]

Hi Gustavo,

Gustavo Barros <gusbrs.2016@gmail.com> writes:

> But the fact that 
> 'completing-read-default' returns the refile location with a double 
> trailing slash makes me think there is still something to be fixed in 
> 'org-refile-get-location'.

if you're not tired of the hunt, please tell us when you find what
needs to be fixed--since there is no bug attached to this, I'll set
this aside for now.

Thanks,

-- 
 Bastien

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Gustavo Barros]

Hi Bastien,

On Sun, Feb 16 2020, Bastien wrote:

> Hi Gustavo,
>
> Gustavo Barros <gusbrs.2016@gmail.com> writes:
>
>> But the fact that
>> 'completing-read-default' returns the refile location with a double
>> trailing slash makes me think there is still something to be fixed in
>> 'org-refile-get-location'.
>
> if you're not tired of the hunt, please tell us when you find what
> needs to be fixed--since there is no bug attached to this, I'll set
> this aside for now.

I'm afraid I have broken proper sequence of this thread unintentionally.
This strip you quote comes from a message I eventually sent 14/Feb 
12:33, right after a message of yours at 12:29 in which you concluded:

> From what I understand, this is not a bug in Org.  I hope you can fix
> this somehow, maybe upstream.

The fact is that I was writing mine when yours came, and I didn't see 
yours until later. And when I did, the reply was:

> I don't claim a problem persists in Org, and I was just trying to be 
> thorough
> in the testing you requested. And did so with pleasure. So...
> 
>> From what I understand, this is not a bug in Org.
> 
> ... if you are satisfied, I'm happy with that too.
> 
> Thank you very very much!

So, with the messages in proper order, I was taken the issue as settled.

If, however, you do think in my message of 12:29 I hit something which 
might be relevant, I'd be happy to pursue this further. I'm "not tired 
of the hunt", but I fear the issue might well be out of my league, so 
there is a real risk of me wasting your time and increasing the noise on 
the list. Besides, as reported before, it appears not to impact 
functionality. But, if knowing that, you would like me to do so, I'll be 
glad to try. In this case, let me know. Otherwise just:

> set this aside for now.

And thanks again!

Best,
Gustavo.

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Bastien]

Hi Gustavo,

Gustavo Barros <gusbrs.2016@gmail.com> writes:

> Otherwise just:
>
>> set this aside for now.

I'll set this aside for now, but I've noted to revisit the issue later
on to see if I can find a related bug---all information in this thread
will be useful to revisit this, thanks again!

-- 
 Bastien

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Gustavo Barros]

Hi Bastien,

On Wed, Feb 12 2020, Bastien wrote:

> this should be fixed in Org master branch, thanks for the detailed
> report.  If you can confirm the fix, even better.

By the way, I almost forgot, a small "side-report" on this.
In going to test this from master, I followed the instructions in the 
manual (web version https://orgmode.org/manual/Installation.html) and 
those instruct users to:

> $ git clone git@code.orgmode.org:bzg/org-mode.git

I tried it, but lacked the password...

Am I missing something, or wouldn't it be more appropriate 
`https://code.orgmode.org/bzg/org-mode.git' in the manual?

Best,
Gustavo.

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Bastien]

Hi Gustavo,

Gustavo Barros <gusbrs.2016@gmail.com> writes:

> Am I missing something, or wouldn't it be more appropriate
> `https://code.orgmode.org/bzg/org-mode.git' in the manual?

Indeed, applied, thanks!

-- 
 Bastien

Re: Bug: org-refile-get-target offers default candidate in duplicity [9.2.6 (9.2.6-4-ge30905-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20191007/)]

[Gustavo Barros]

Hi Bastien,

On Thu, Feb 13 2020, Bastien wrote:

>> Am I missing something, or wouldn't it be more appropriate
>> `https://code.orgmode.org/bzg/org-mode.git' in the manual?
>
> Indeed, applied, thanks!

Thanks!