emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
* [BUG] org-link-search fails if search string contains new lines
@ 2017-03-26 17:35 Matt Lundin
  2017-03-26 20:50 ` Matt Lundin
  0 siblings, 1 reply; 10+ messages in thread
From: Matt Lundin @ 2017-03-26 17:35 UTC (permalink / raw)
  To: Org Mode

Following links fails to locate the appropriate location in text files
if the search string in the link contains new lines.

Steps to reproduce:

/usr/bin/emacs -Q ~/config/minimal.el

where minimal.el contains...

--8<---------------cut here---------------start------------->8---
(add-to-list 'load-path "~/org-mode/lisp/")
(add-to-list 'load-path "~/org-mode/contrib/lisp/")

(setq org-agenda-files '("~/config/test.org"))

(setq org-capture-templates
      '(("n" "Note" entry
	           (file "~/config/test.org")
                   "* Test\n %a\n")))
--8<---------------cut here---------------end--------------->8---

Open a file (~/text.txt) containing....

--8<---------------cut here---------------start------------->8---
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do
eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad
minim veniam, quis nostrud exercitation ullamco laboris nisi ut
aliquip ex ea commodo consequat. Duis aute irure dolor in
reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla
pariatur. Excepteur sint occaecat cupidatat non proident, sunt in
culpa qui officia deserunt mollit anim id est laborum.
--8<---------------cut here---------------end--------------->8---

Create an active region covering the third sentence and select the
org-capture "n" template, which creates the following link:

--8<---------------cut here---------------start------------->8---
* Test
 [[file:~/test.txt::Duis%20aute%20irure%20dolor%20in%0Areprehenderit%20in%20voluptate%20velit%20esse%20cillum%20dolore%20eu%20fugiat%20nulla%0Apariatur.]]
--8<---------------cut here---------------end--------------->8---

Try to follow the link. It will open test.txt, but it will also give the
message and fail to locate the correct position in the file:

        "No match for fuzzy expression: Duis aute irure dolor in
        reprehenderit in voluptate velit esse cillum dolore eu fugiat
        nulla pariatur."

For comparison, evaluate (setq org-context-in-file-links 1) and run
capture again. The captured snippet looks like this:

--8<---------------cut here---------------start------------->8---
* Test
 [[file:~/test.txt::Duis%20aute%20irure%20dolor%20in]]
--8<---------------cut here---------------end--------------->8---

Follow the link. The fuzzy search succeeds and finds the correct
position in the text.

Matt

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [BUG] org-link-search fails if search string contains new lines
  2017-03-26 17:35 [BUG] org-link-search fails if search string contains new lines Matt Lundin
@ 2017-03-26 20:50 ` Matt Lundin
  2017-03-27 11:37   ` Nicolas Goaziou
  0 siblings, 1 reply; 10+ messages in thread
From: Matt Lundin @ 2017-03-26 20:50 UTC (permalink / raw)
  To: Org Mode

Matt Lundin <mdl@imapmail.org> writes:

>
> Create an active region covering the third sentence and select the
> org-capture "n" template, which creates the following link:
>
> * Test
>  [[file:~/test.txt::Duis%20aute%20irure%20dolor%20in%0Areprehenderit%20in%20voluptate%20velit%20esse%20cillum%20dolore%20eu%20fugiat%20nulla%0Apariatur.]]
>
> Try to follow the link. It will open test.txt, but it will also give the
> message and fail to locate the correct position in the file:
>
>         "No match for fuzzy expression: Duis aute irure dolor in
>         reprehenderit in voluptate velit esse cillum dolore eu fugiat
>         nulla pariatur."
>

The problem, I think, is the regexp construction in org-link-search.
This was introduced back in August of 2015 with commit
cfe5bc97f8b18ccbf49d0764746c7563ce8d29da.

The problematic line in org.el is 10951:

(s-multi-re (mapconcat #'regexp-quote words "[ \t]+\\(?:\n[ \t]*\\)?"))

The constructed regexp fails because it assumes a newline will be
preceded by whitespace. But often newlines are not preceded by
whitespace.

Is there a reason the following won't work? 

(s-multi-re (mapconcat #'regexp-quote words "[ \t\r\n]+"))

This was the method org-link-search used prior to the commit above. Are
we trying to avoid matching across blank lines? 

Best,
Matt

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [BUG] org-link-search fails if search string contains new lines
  2017-03-26 20:50 ` Matt Lundin
@ 2017-03-27 11:37   ` Nicolas Goaziou
  2017-03-27 15:15     ` Matt Lundin
  2017-03-28 21:34     ` Skip Collins
  0 siblings, 2 replies; 10+ messages in thread
From: Nicolas Goaziou @ 2017-03-27 11:37 UTC (permalink / raw)
  To: Org Mode

Hello,

Matt Lundin <mdl@imapmail.org> writes:

> Matt Lundin <mdl@imapmail.org> writes:
>
>>
>> Create an active region covering the third sentence and select the
>> org-capture "n" template, which creates the following link:
>>
>> * Test
>>  [[file:~/test.txt::Duis%20aute%20irure%20dolor%20in%0Areprehenderit%20in%20voluptate%20velit%20esse%20cillum%20dolore%20eu%20fugiat%20nulla%0Apariatur.]]
>>
>> Try to follow the link. It will open test.txt, but it will also give the
>> message and fail to locate the correct position in the file:
>>
>>         "No match for fuzzy expression: Duis aute irure dolor in
>>         reprehenderit in voluptate velit esse cillum dolore eu fugiat
>>         nulla pariatur."
>>
>
> The problem, I think, is the regexp construction in org-link-search.
> This was introduced back in August of 2015 with commit
> cfe5bc97f8b18ccbf49d0764746c7563ce8d29da.
>
> The problematic line in org.el is 10951:
>
> (s-multi-re (mapconcat #'regexp-quote words "[ \t]+\\(?:\n[ \t]*\\)?"))
>
> The constructed regexp fails because it assumes a newline will be
> preceded by whitespace. But often newlines are not preceded by
> whitespace.
>
> Is there a reason the following won't work? 
>
> (s-multi-re (mapconcat #'regexp-quote words "[ \t\r\n]+"))
>
> This was the method org-link-search used prior to the commit above. Are
> we trying to avoid matching across blank lines?

Yes, we are.

Fixed, hopefully.

Regards,

-- 
Nicolas Goaziou

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [BUG] org-link-search fails if search string contains new lines
  2017-03-27 11:37   ` Nicolas Goaziou
@ 2017-03-27 15:15     ` Matt Lundin
  2017-03-29 12:19       ` Nicolas Goaziou
  2017-03-28 21:34     ` Skip Collins
  1 sibling, 1 reply; 10+ messages in thread
From: Matt Lundin @ 2017-03-27 15:15 UTC (permalink / raw)
  To: Nicolas Goaziou; +Cc: Org Mode

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

>> The problem, I think, is the regexp construction in org-link-search.
>> This was introduced back in August of 2015 with commit
>> cfe5bc97f8b18ccbf49d0764746c7563ce8d29da.
>>
>> The problematic line in org.el is 10951:
>>
>> (s-multi-re (mapconcat #'regexp-quote words "[ \t]+\\(?:\n[ \t]*\\)?"))
>>
>> The constructed regexp fails because it assumes a newline will be
>> preceded by whitespace. But often newlines are not preceded by
>> whitespace.
>>
>> Is there a reason the following won't work? 
>>
>> (s-multi-re (mapconcat #'regexp-quote words "[ \t\r\n]+"))
>>
>> This was the method org-link-search used prior to the commit above. Are
>> we trying to avoid matching across blank lines?
>
> Yes, we are.
>
> Fixed, hopefully.

Thanks Nicolas. Yes, it now works with multi-line searches that do not
contain blank lines.

I think there are some instances in which one might want to search
across blank lines, such as when one captures a region that contains one
or more blank lines. E.g., let's say org-context-in-file-links is t (the
default). If one captures a section in org.el containing a blank line,
org-link-search will fail to locate the section.

Here's a sample link to try it out. (It assumes an org-mode repository
at ~/org-mode).

 [[file:~/org-mode/lisp/org.el::(defsubst%20org-uniquify%20(list)%0A%20"Non-destructively%20remove%20duplicate%20elements%20from%20LIST."%0A%20(let%20((res%20(copy-sequence%20list)))%20(delete-dups%20res)))%0A%0A(defsubst%20org-get-at-bol%20(property)%0A%20"Get%20text%20property%20PROPERTY%20at%20the%20beginning%20of%20line."%0A%20(get-text-property%20(point-at-bol)%20property))%0A]]

Thanks,
Matt

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [BUG] org-link-search fails if search string contains new lines
  2017-03-27 11:37   ` Nicolas Goaziou
  2017-03-27 15:15     ` Matt Lundin
@ 2017-03-28 21:34     ` Skip Collins
  2017-03-29 12:19       ` Nicolas Goaziou
  1 sibling, 1 reply; 10+ messages in thread
From: Skip Collins @ 2017-03-28 21:34 UTC (permalink / raw)
  To: Nicolas Goaziou; +Cc: Org Mode

On Mon, Mar 27, 2017 at 7:37 AM, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:
> Fixed, hopefully.

Ran 718 tests, 717 results as expected, 1 unexpected (2017-03-28 17:27:55-0400)

9 expected failures


1 unexpected results:

   FAILED  test-org/fuzzy-links

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [BUG] org-link-search fails if search string contains new lines
  2017-03-28 21:34     ` Skip Collins
@ 2017-03-29 12:19       ` Nicolas Goaziou
  2017-03-29 15:17         ` Skip Collins
  0 siblings, 1 reply; 10+ messages in thread
From: Nicolas Goaziou @ 2017-03-29 12:19 UTC (permalink / raw)
  To: Skip Collins; +Cc: Org Mode

Hello,

Skip Collins <skip.collins@gmail.com> writes:

> 1 unexpected results:
>
>    FAILED  test-org/fuzzy-links

This is not the case at http://www.randomsample.de:4457/waterfall

The error report would help.

Regards,

-- 
Nicolas Goaziou                                                0x80A93738

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [BUG] org-link-search fails if search string contains new lines
  2017-03-27 15:15     ` Matt Lundin
@ 2017-03-29 12:19       ` Nicolas Goaziou
  0 siblings, 0 replies; 10+ messages in thread
From: Nicolas Goaziou @ 2017-03-29 12:19 UTC (permalink / raw)
  To: Org Mode

Hello,

Matt Lundin <mdl@imapmail.org> writes:

> Thanks Nicolas. Yes, it now works with multi-line searches that do not
> contain blank lines.
>
> I think there are some instances in which one might want to search
> across blank lines, such as when one captures a region that contains one
> or more blank lines. E.g., let's say org-context-in-file-links is t (the
> default). If one captures a section in org.el containing a blank line,
> org-link-search will fail to locate the section.
>
> Here's a sample link to try it out. (It assumes an org-mode repository
> at ~/org-mode).
>
>  [[file:~/org-mode/lisp/org.el::(defsubst%20org-uniquify%20(list)%0A%20"Non-destructively%20remove%20duplicate%20elements%20from%20LIST."%0A%20(let%20((res%20(copy-sequence%20list)))%20(delete-dups%20res)))%0A%0A(defsubst%20org-get-at-bol%20(property)%0A%20"Get%20text%20property%20PROPERTY%20at%20the%20beginning%20of%20line."%0A%20(get-text-property%20(point-at-bol)%20property))%0A]]

Fixed. Thank you.

Regards,

-- 
Nicolas Goaziou                                                0x80A93738

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [BUG] org-link-search fails if search string contains new lines
  2017-03-29 12:19       ` Nicolas Goaziou
@ 2017-03-29 15:17         ` Skip Collins
  2017-03-30 10:50           ` Nicolas Goaziou
  0 siblings, 1 reply; 10+ messages in thread
From: Skip Collins @ 2017-03-29 15:17 UTC (permalink / raw)
  To: Nicolas Goaziou; +Cc: Org Mode

Test test-org/fuzzy-links backtrace:
  (if (unwind-protect (setq value-10839 (let ((file (make-temp-file "o
  (let (form-description-10840) (if (unwind-protect (setq value-10839
  (let ((value-10839 (cl-gensym "ert-form-evaluation-aborted-"))) (let
  (lambda nil (let ((value-10802 (cl-gensym "ert-form-evaluation-abort
  ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc
  ert-run-test([cl-struct-ert-test test-org/fuzzy-links "Test fuzzy li
  ert-run-or-rerun-test([cl-struct-ert--stats "\\(org\\|ob\\)" [[cl-st
  ert-run-tests("\\(org\\|ob\\)" #[385 "\306\307\"\203G\211\211G\310
  ert-run-tests-batch("\\(org\\|ob\\)")
  ert-run-tests-batch-and-exit("\\(org\\|ob\\)")
  (let ((org-id-track-globally t) (org-test-selector (if org-test-sele
  org-test-run-batch-tests("\\(org\\|ob\\)")
  eval((org-test-run-batch-tests org-test-select-re))
  command-line-1(("--eval" "(setq vc-handled-backends nil org-startup-
  command-line()
  normal-top-level()
Test test-org/fuzzy-links condition:
    (ert-test-failed
     ((should
       (org-test-with-temp-text-in-file "Paragraph

line1
line2

"
(let ... ... ... ... ... ...)))
      :form
      (let
 ((file ...)
  (kill-buffer-query-functions nil)
  (inside-text ...)
  G10838)
(with-temp-file file
 (insert inside-text))
(find-file file)
(org-mode)
(setq G10838
     (progn ...))
(save-buffer)
(kill-buffer
(current-buffer))
(delete-file file)
G10838)
      :value nil))
   FAILED  656/718  test-org/fuzzy-links

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [BUG] org-link-search fails if search string contains new lines
  2017-03-29 15:17         ` Skip Collins
@ 2017-03-30 10:50           ` Nicolas Goaziou
  2017-04-07 15:47             ` Skip Collins
  0 siblings, 1 reply; 10+ messages in thread
From: Nicolas Goaziou @ 2017-03-30 10:50 UTC (permalink / raw)
  To: Skip Collins; +Cc: Org Mode

Hello,

Skip Collins <skip.collins@gmail.com> writes:

> Test test-org/fuzzy-links backtrace:
>   (if (unwind-protect (setq value-10839 (let ((file (make-temp-file "o
>   (let (form-description-10840) (if (unwind-protect (setq value-10839
>   (let ((value-10839 (cl-gensym "ert-form-evaluation-aborted-"))) (let
>   (lambda nil (let ((value-10802 (cl-gensym "ert-form-evaluation-abort
>   ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc
>   ert-run-test([cl-struct-ert-test test-org/fuzzy-links "Test fuzzy li
>   ert-run-or-rerun-test([cl-struct-ert--stats "\\(org\\|ob\\)" [[cl-st
>   ert-run-tests("\\(org\\|ob\\)" #[385 "\306\307\"\203G\211\211G\310
>   ert-run-tests-batch("\\(org\\|ob\\)")
>   ert-run-tests-batch-and-exit("\\(org\\|ob\\)")
>   (let ((org-id-track-globally t) (org-test-selector (if org-test-sele
>   org-test-run-batch-tests("\\(org\\|ob\\)")
>   eval((org-test-run-batch-tests org-test-select-re))
>   command-line-1(("--eval" "(setq vc-handled-backends nil org-startup-
>   command-line()
>   normal-top-level()
> Test test-org/fuzzy-links condition:
>     (ert-test-failed
>      ((should
>        (org-test-with-temp-text-in-file "Paragraph
>
> line1
> line2
>
> "
> (let ... ... ... ... ... ...)))
>       :form
>       (let
>  ((file ...)
>   (kill-buffer-query-functions nil)
>   (inside-text ...)
>   G10838)
> (with-temp-file file
>  (insert inside-text))
> (find-file file)
> (org-mode)
> (setq G10838
>      (progn ...))
> (save-buffer)
> (kill-buffer
> (current-buffer))
> (delete-file file)
> G10838)
>       :value nil))
>    FAILED  656/718  test-org/fuzzy-links

Thank you. Unfortunately, there is not much to do with this. Could you
do some debugging when running the faulty test? It may come from
something in your configuration.

Regards,

-- 
Nicolas Goaziou

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [BUG] org-link-search fails if search string contains new lines
  2017-03-30 10:50           ` Nicolas Goaziou
@ 2017-04-07 15:47             ` Skip Collins
  0 siblings, 0 replies; 10+ messages in thread
From: Skip Collins @ 2017-04-07 15:47 UTC (permalink / raw)
  To: Nicolas Goaziou; +Cc: Org Mode

[-- Attachment #1: Type: text/plain, Size: 685 bytes --]

On Thu, Mar 30, 2017 at 6:50 AM, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:
> Thank you. Unfortunately, there is not much to do with this. Could you
> do some debugging when running the faulty test? It may come from
> something in your configuration.

I think I figured it out. During the fuzzy-links test,
org-open-at-point is invoked to open a file that was created with
org-test-with-temp-text-in-file. The temp file does not have a .org
extension. As a result, my mac attempts to open the file outside of
emacs, which produces an unexpected result. The solution is to pass an
argument to org-open-at-point, which ensures that the file is opened
in emacs. A patch is attached.

[-- Attachment #2: 0001-Ensure-test-files-are-opened-in-Emacs-in-test-org-fu.patch --]
[-- Type: application/octet-stream, Size: 1125 bytes --]

From 6c3ad66280edc10a683bdfd4e730033d5f541721 Mon Sep 17 00:00:00 2001
From: Skip Collins <skip.collins@gmail.com>
Date: Fri, 7 Apr 2017 11:41:47 -0400
Subject: [PATCH] Ensure test files are opened in Emacs in test-org/fuzzy-links

---
 testing/lisp/test-org.el | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/testing/lisp/test-org.el b/testing/lisp/test-org.el
index b1d4449ef..3361a69a5 100644
--- a/testing/lisp/test-org.el
+++ b/testing/lisp/test-org.el
@@ -2292,7 +2292,7 @@ Foo Bar
        (insert (format "[[file:%s::line1 line2]]" file))
        (beginning-of-line)
        (let ((org-link-search-must-match-exact-headline nil))
-	 (org-open-at-point))
+	 (org-open-at-point 0))
        (looking-at-p "line1"))))
   (should
    (org-test-with-temp-text-in-file "Paragraph\n\nline1\n\nline2\n\n"
@@ -2301,7 +2301,7 @@ Foo Bar
        (insert (format "[[file:%s::line1 line2]]" file))
        (beginning-of-line)
        (let ((org-link-search-must-match-exact-headline nil))
-	 (org-open-at-point))
+	 (org-open-at-point 0))
        (looking-at-p "line1")))))
 
 ;;;; Link Escaping
-- 
2.12.2


^ permalink raw reply related	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2017-04-07 15:47 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-03-26 17:35 [BUG] org-link-search fails if search string contains new lines Matt Lundin
2017-03-26 20:50 ` Matt Lundin
2017-03-27 11:37   ` Nicolas Goaziou
2017-03-27 15:15     ` Matt Lundin
2017-03-29 12:19       ` Nicolas Goaziou
2017-03-28 21:34     ` Skip Collins
2017-03-29 12:19       ` Nicolas Goaziou
2017-03-29 15:17         ` Skip Collins
2017-03-30 10:50           ` Nicolas Goaziou
2017-04-07 15:47             ` Skip Collins

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).