Bug: org-babel only tangles first noweb reference on a line [9.4 (9.4-elpaplus @ /home/tom/.emacs.d/elpa/org-plus-contrib-20200914/)]

Bug: org-babel only tangles first noweb reference on a line [9.4 (9.4-elpaplus @ /home/tom/.emacs.d/elpa/org-plus-contrib-20200914/)]

[Tom Gillespie]

Hi,
   The 9.4 release has a bug where it will only tangle the first noweb
reference on a line.
This is also present at 9c31cba002a1ba93053aebea1f778be87f61ba06. It happens in
emacs-27 and emacs-28. The reproduction is below. Best!
Tom

The expected content of oops-3.el should be 1 2 1, but is instead 1
<<block2>> <<block1>>.
C-c C-v C-t or C-c C-v C-v will show the issue.

#+name: block1
#+begin_src elisp
1
#+end_src

#+name: block2
#+begin_src elisp
2
#+end_src

#+name: block3
#+begin_src elisp :noweb yes :tangle ./oops-3.el
<<block1>> <<block2>> <<block1>>
#+end_src

[PATCH] ob-core: Fix handling of multiple noweb refs in same line

[Kyle Meyer]

Tom Gillespie writes:

> Hi,
>    The 9.4 release has a bug where it will only tangle the first noweb
> reference on a line.
> This is also present at 9c31cba002a1ba93053aebea1f778be87f61ba06. It happens in
> emacs-27 and emacs-28. The reproduction is below. Best!

Thanks for the report and the reproducer.

This bisects to c1aed9f80 (ob-core: Refactor
`org-babel-expand-noweb-references', 2020-01-12).  The patch below
addresses the case you provided (added as a test) and doesn't cause any
of the other tests to fail.  However, I'm no tangler, so I'd appreciate
if you could give it some testing and report back.

(I should also look more closely at org-babel-expand-noweb-references
before applying and see if I spot anything that suggests this change is
problematic.)

-- >8 --
Subject: [PATCH] ob-core: Fix handling of multiple noweb refs in same line

* lisp/ob-core.el (org-babel-expand-noweb-references): Don't anchor
noweb regexp at start of line to allow multiple matches per line.
* testing/lisp/test-ob-tangle.el (ob-tangle/multiple-noweb-in-line):
Add test.

This fixes a regression introduced by c1aed9f80 (ob-core: Refactor
`org-babel-expand-noweb-references', 2020-01-12), which was part of
the 9.4 release.

Reported-by: Tom Gillespie <tgbugs@gmail.com>
Ref: https://orgmode.org/list/CA+G3_PO2yO1jMMpdrkc39BGQQ2eU5X4FzTEJVotjDJo-50dsqQ@mail.gmail.com
---
 lisp/ob-core.el                |  2 +-
 testing/lisp/test-ob-tangle.el | 32 ++++++++++++++++++++++++++++++++
 2 files changed, 33 insertions(+), 1 deletion(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index a5e079d9a..7300f239e 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -2781,7 +2781,7 @@ (defun org-babel-expand-noweb-references (&optional info parent-buffer)
          (lang (nth 0 info))
          (body (nth 1 info))
 	 (comment (string= "noweb" (cdr (assq :comments (nth 2 info)))))
-	 (noweb-re (format "^\\(.*?\\)\\(%s\\)"
+	 (noweb-re (format "\\(.*?\\)\\(%s\\)"
 			   (with-current-buffer parent-buffer
 			     (org-babel-noweb-wrap))))
 	 (cache nil)
diff --git a/testing/lisp/test-ob-tangle.el b/testing/lisp/test-ob-tangle.el
index e0e2ea52c..cfdf16d40 100644
--- a/testing/lisp/test-ob-tangle.el
+++ b/testing/lisp/test-ob-tangle.el
@@ -420,6 +420,38 @@ (ert-deftest ob-tangle/commented-src-blocks ()
 		    (org-split-string (buffer-string))))
 	      (delete-file file))))))
 
+(ert-deftest ob-tangle/multiple-noweb-in-line ()
+  "Test handling of multiple noweb references in a single line."
+  (should
+   (equal '("1" "2" "1")
+	  (let ((file (make-temp-file "org-tangle-")))
+	    (unwind-protect
+		(progn
+		  (org-test-with-temp-text-in-file
+		      (format "
+#+name: block1
+#+begin_src elisp
+1
+#+end_src
+
+#+name: block2
+#+begin_src elisp
+2
+#+end_src
+
+#+name: block3
+#+begin_src elisp :noweb yes :tangle %s
+<<block1>> <<block2>> <<block1>>
+#+end_src"
+			      file)
+		    (let ((org-babel-noweb-error-all-langs nil)
+			  (org-babel-noweb-error-langs nil))
+		      (org-babel-tangle)))
+		  (with-temp-buffer
+		    (insert-file-contents file)
+		    (org-split-string (buffer-string))))
+	      (delete-file file))))))
+
 (ert-deftest ob-tangle/detangle-false-positive ()
   "Test handling of false positive link during detangle."
   (let (buffer)

base-commit: e6021bc9b18982b30dd61417d98276b2984892cd
-- 
2.28.0

Re: [PATCH] ob-core: Fix handling of multiple noweb refs in same line

[Tom Gillespie]

Hi Kyle,
   This fixes the issue on my system, and looking over the bisected
commit it looks like the ^ was just an oversight. This also resolves
the issue for <<references()>> as well, I don't think we need a
separate test case for that since any issue with references breaking
should show up elsewhere (though I guess you never know). Thank you
for tracking this down, and now I know how to add ert tests for things
like this in the future! Best,
Tom

On Tue, Sep 15, 2020 at 12:21 AM Kyle Meyer <kyle@kyleam.com> wrote:
>
> Tom Gillespie writes:
>
> > Hi,
> >    The 9.4 release has a bug where it will only tangle the first noweb
> > reference on a line.
> > This is also present at 9c31cba002a1ba93053aebea1f778be87f61ba06. It happens in
> > emacs-27 and emacs-28. The reproduction is below. Best!
>
> Thanks for the report and the reproducer.
>
> This bisects to c1aed9f80 (ob-core: Refactor
> `org-babel-expand-noweb-references', 2020-01-12).  The patch below
> addresses the case you provided (added as a test) and doesn't cause any
> of the other tests to fail.  However, I'm no tangler, so I'd appreciate
> if you could give it some testing and report back.
>
> (I should also look more closely at org-babel-expand-noweb-references
> before applying and see if I spot anything that suggests this change is
> problematic.)
>
> -- >8 --
> Subject: [PATCH] ob-core: Fix handling of multiple noweb refs in same line
>
> * lisp/ob-core.el (org-babel-expand-noweb-references): Don't anchor
> noweb regexp at start of line to allow multiple matches per line.
> * testing/lisp/test-ob-tangle.el (ob-tangle/multiple-noweb-in-line):
> Add test.
>
> This fixes a regression introduced by c1aed9f80 (ob-core: Refactor
> `org-babel-expand-noweb-references', 2020-01-12), which was part of
> the 9.4 release.
>
> Reported-by: Tom Gillespie <tgbugs@gmail.com>
> Ref: https://orgmode.org/list/CA+G3_PO2yO1jMMpdrkc39BGQQ2eU5X4FzTEJVotjDJo-50dsqQ@mail.gmail.com
> ---
>  lisp/ob-core.el                |  2 +-
>  testing/lisp/test-ob-tangle.el | 32 ++++++++++++++++++++++++++++++++
>  2 files changed, 33 insertions(+), 1 deletion(-)
>
> diff --git a/lisp/ob-core.el b/lisp/ob-core.el
> index a5e079d9a..7300f239e 100644
> --- a/lisp/ob-core.el
> +++ b/lisp/ob-core.el
> @@ -2781,7 +2781,7 @@ (defun org-babel-expand-noweb-references (&optional info parent-buffer)
>           (lang (nth 0 info))
>           (body (nth 1 info))
>          (comment (string= "noweb" (cdr (assq :comments (nth 2 info)))))
> -        (noweb-re (format "^\\(.*?\\)\\(%s\\)"
> +        (noweb-re (format "\\(.*?\\)\\(%s\\)"
>                            (with-current-buffer parent-buffer
>                              (org-babel-noweb-wrap))))
>          (cache nil)
> diff --git a/testing/lisp/test-ob-tangle.el b/testing/lisp/test-ob-tangle.el
> index e0e2ea52c..cfdf16d40 100644
> --- a/testing/lisp/test-ob-tangle.el
> +++ b/testing/lisp/test-ob-tangle.el
> @@ -420,6 +420,38 @@ (ert-deftest ob-tangle/commented-src-blocks ()
>                     (org-split-string (buffer-string))))
>               (delete-file file))))))
>
> +(ert-deftest ob-tangle/multiple-noweb-in-line ()
> +  "Test handling of multiple noweb references in a single line."
> +  (should
> +   (equal '("1" "2" "1")
> +         (let ((file (make-temp-file "org-tangle-")))
> +           (unwind-protect
> +               (progn
> +                 (org-test-with-temp-text-in-file
> +                     (format "
> +#+name: block1
> +#+begin_src elisp
> +1
> +#+end_src
> +
> +#+name: block2
> +#+begin_src elisp
> +2
> +#+end_src
> +
> +#+name: block3
> +#+begin_src elisp :noweb yes :tangle %s
> +<<block1>> <<block2>> <<block1>>
> +#+end_src"
> +                             file)
> +                   (let ((org-babel-noweb-error-all-langs nil)
> +                         (org-babel-noweb-error-langs nil))
> +                     (org-babel-tangle)))
> +                 (with-temp-buffer
> +                   (insert-file-contents file)
> +                   (org-split-string (buffer-string))))
> +             (delete-file file))))))
> +
>  (ert-deftest ob-tangle/detangle-false-positive ()
>    "Test handling of false positive link during detangle."
>    (let (buffer)
>
> base-commit: e6021bc9b18982b30dd61417d98276b2984892cd
> --
> 2.28.0
>

Re: [PATCH] ob-core: Fix handling of multiple noweb refs in same line

[Kyle Meyer]

Tom Gillespie writes:

>    This fixes the issue on my system, and looking over the bisected
> commit it looks like the ^ was just an oversight. This also resolves
> the issue for <<references()>> as well, I don't think we need a
> separate test case for that since any issue with references breaking
> should show up elsewhere (though I guess you never know). Thank you
> for tracking this down, and now I know how to add ert tests for things
> like this in the future! Best,
> Tom
>
> On Tue, Sep 15, 2020 at 12:21 AM Kyle Meyer <kyle@kyleam.com> wrote:
[...]
>> Subject: [PATCH] ob-core: Fix handling of multiple noweb refs in same line

Thanks for testing on your end.  Pushed (469ee6340).

Re: Bug: org-babel only tangles first noweb reference on a line [9.4 (9.4-elpaplus @ /home/tom/.emacs.d/elpa/org-plus-contrib-20200914/)]

[Bastien]

Hi Tom,

Tom Gillespie <tgbugs@gmail.com> writes:

>    The 9.4 release has a bug where it will only tangle the first noweb
> reference on a line.
> This is also present at 9c31cba002a1ba93053aebea1f778be87f61ba06. It happens in
> emacs-27 and emacs-28. The reproduction is below. Best!
> Tom
>
> The expected content of oops-3.el should be 1 2 1, but is instead 1
> <<block2>> <<block1>>.
> C-c C-v C-t or C-c C-v C-v will show the issue.

Do you still have this issue?  I cannot reproduce it here.

Thanks

Re: Bug: org-babel only tangles first noweb reference on a line [9.4 (9.4-elpaplus @ /home/tom/.emacs.d/elpa/org-plus-contrib-20200914/)]

[Kyle Meyer]

Bastien writes:

> Hi Tom,
>
> Tom Gillespie <tgbugs@gmail.com> writes:
>
>>    The 9.4 release has a bug where it will only tangle the first noweb
>> reference on a line.
>> This is also present at 9c31cba002a1ba93053aebea1f778be87f61ba06. It happens in
>> emacs-27 and emacs-28. The reproduction is below. Best!
>> Tom
>>
>> The expected content of oops-3.el should be 1 2 1, but is instead 1
>> <<block2>> <<block1>>.
>> C-c C-v C-t or C-c C-v C-v will show the issue.
>
> Do you still have this issue?  I cannot reproduce it here.

A patch upthread (<https://orgmode.org/list/87tuvzllg7.fsf@kyleam.com>)
was applied in 469ee6340 (ob-core: Fix handling of multiple noweb refs
in same line, 2020-09-15).

Re: Bug: org-babel only tangles first noweb reference on a line [9.4 (9.4-elpaplus @ /home/tom/.emacs.d/elpa/org-plus-contrib-20200914/)]

[Bastien]

Hi Kyle,

Kyle Meyer <kyle@kyleam.com> writes:

> A patch upthread (<https://orgmode.org/list/87tuvzllg7.fsf@kyleam.com>)
> was applied in 469ee6340 (ob-core: Fix handling of multiple noweb refs
> in same line, 2020-09-15).

I missed this one, thanks for letting me know and sorry for the noise.