"make test" failure

"make test" failure

[Nick Dokos]

Just pulled and ran "make test". I get one failure with the
appended backtrace:

Nick

Test test-org/backward-element backtrace:
  (if (unwind-protect (setq value-4576 (apply fn-4574 args-4575)) (set
  (let (form-description-4578) (if (unwind-protect (setq value-4576 (a
  (let ((value-4576 (quote ert-form-evaluation-aborted-4577))) (let (f
  (let ((fn-4574 (function looking-at)) (args-4575 (list (regexp-quote
  (progn (org-mode) (progn (insert "\n* Head 1\n** Head 1.1\n*** Head 
  (unwind-protect (progn (org-mode) (progn (insert "\n* Head 1\n** Hea
  (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn
  (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-b
  (lambda nil (let ((temp-buffer (generate-new-buffer " *temp*"))) (sa
  #[0 "\306\307!r\211q\210\310\311\312\313\314\315\x06\x06!\316\"\317\320%DC
  funcall(#[0 "\306\307!r\211q\210\310\311\312\313\314\315\x06\x06!\316\"\31
  ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc
  #[0 "r\304\305!q\210\306 d\307\223)\310\311\312\313\314\315\x06\x06!\316\"
  funcall(#[0 "r\304\305!q\210\306 d\307\223)\310\311\312\313\314\315\x06
  ert-run-test([cl-struct-ert-test test-org/backward-element "Test `or
  ert-run-or-rerun-test([cl-struct-ert--stats "\\(org\\|ob\\)" [[cl-st
  ert-run-tests("\\(org\\|ob\\)" #[385 "\306\307\"\203D

Re: "make test" failure

[Bastien]

[-- Attachment #1: Type: text/plain, Size: 249 bytes --]

Hi Nick,

Nick Dokos <nicholas.dokos@hp.com> writes:

> Just pulled and ran "make test". I get one failure with the
> appended backtrace:

I submitted this patch to Nicolas so that he can approve/apply
it.  I think the test is wrong, not the code.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: test-fix.patch --]
[-- Type: text/x-patch, Size: 709 bytes --]

diff --git a/testing/lisp/test-org.el b/testing/lisp/test-org.el
index dac5fd2..561ac98 100644
--- a/testing/lisp/test-org.el
+++ b/testing/lisp/test-org.el
@@ -677,10 +677,7 @@ Outside."
     ;;      headline at the same level.
     (goto-line 3)
     (org-backward-element)
-    (should (looking-at (regexp-quote "* Head 1")))
-    ;; 4.3. At the first top-level headline: should error.
-    (goto-line 2)
-    (should-error (org-backward-element)))
+    (should (looking-at (regexp-quote "** Head 1.1"))))
   ;; 5. At beginning of first element inside a greater element:
   ;;    expected to move to greater element's beginning.
   (org-test-with-temp-text "Before.\n#+BEGIN_CENTER\nInside.\n#+END_CENTER"

[-- Attachment #3: Type: text/plain, Size: 14 bytes --]


-- 
 Bastien

``make test'' failure

[Nick Dokos]

I get one failure in test-org-publish/resolve-external-link.
Bisecting fingers this commit:

,----
| $ git bisect good
| 007bbddbccfd06ab9c97d51bf833a068cb1b07a2 is the first bad commit
| commit 007bbddbccfd06ab9c97d51bf833a068cb1b07a2
| Author: Nicolas Goaziou <mail@nicolasgoaziou.fr>
| Date:   Sun Sep 10 00:16:12 2017 +0200
| 
|     ox-html: Prevent spurious target below headlines
|     
|     * lisp/ox-html.el (org-html-headline):
|     (org-html-link): Do not insert an additional target.
|     
|     * lisp/ox-publish.el (org-publish-resolve-external-link): Add an
|       optional argument.
|     
|     * lisp/ox.el (org-export-get-reference): Improve docstring.
|     
|     * testing/examples/pub/a.org:
|     * testing/examples/pub/b.org: New files.
|     * testing/lisp/test-ox-publish.el (test-org-publish/resolve-external-link):
|       New test.
| 
| :040000 040000 fda080e0db653c9049448200be5ad03369379cc3 ec0eb8a5cd9b3c3b204fd7134e515fb360b85c79 M	etc
| :040000 040000 39afe6514335e2ccd549fc5b7fafcac344cbb471 a0fdf6967cc3ab95b046f81dab08095fe0d302de M	lisp
| :040000 040000 120e098640643a963336d77d65929ef5863eeba5 679329e78a4bbc04f6f84bb176821a3084732ab3 M	testing
`----

which *introduced* this test, but of course the test may be at fault,
rather than the code.

The failure looks like this:

,----
|    passed  571/765  test-org-publish/get-project-from-filename
| Publishing file /home/nick/src/emacs/org/org-mode/testing/examples/pub/b.org using ‘(closure ((backend . #s(org-export-backend nil nil ((headline lambda (h c i) (concat (org-export-get-reference h i)   c)) (paragraph lambda (p c i) c) (section lambda (s c i) c) (link lambda (l c i) (let ((option (org-element-property :search-option l)) (path (org-element-property :path l))) (and option (org-publish-resolve-external-link option path))))) nil nil nil nil)) (ids) (fn-14501 . apply) t) (plist filename pub-dir) (org-publish-org-to backend filename .test plist pub-dir))’
| Publishing file /home/nick/src/emacs/org/org-mode/testing/examples/pub/a.org using ‘(closure ((backend . #s(org-export-backend nil nil ((headline lambda (h c i) (concat (org-export-get-reference h i)   c)) (paragraph lambda (p c i) c) (section lambda (s c i) c) (link lambda (l c i) (let ((option (org-element-property :search-option l)) (path (org-element-property :path l))) (and option (org-publish-resolve-external-link option path))))) nil nil nil nil)) (ids) (fn-14501 . apply) t) (plist filename pub-dir) (org-publish-org-to backend filename .test plist pub-dir))’
| Publishing file /home/nick/src/emacs/org/org-mode/testing/examples/pub/b.org using ‘(closure ((backend . #s(org-export-backend nil nil ((headline lambda (h c i) c) (paragraph lambda (p c i) c) (section lambda (s c i) c) (link lambda (l c i) (let ((option (org-element-property :search-option l)) (path (org-element-property :path l))) (when option (throw :exit (org-publish-resolve-external-link option path t)))))) nil nil nil nil)) (ids) (fn-14506 . equal) t) (plist filename pub-dir) (setq ids (cons (catch :exit (org-publish-org-to backend filename .test plist pub-dir)) ids)))’
| Publishing file /home/nick/src/emacs/org/org-mode/testing/examples/pub/a.org using ‘(closure ((backend . #s(org-export-backend nil nil ((headline lambda (h c i) c) (paragraph lambda (p c i) c) (section lambda (s c i) c) (link lambda (l c i) (let ((option (org-element-property :search-option l)) (path (org-element-property :path l))) (when option (throw :exit (org-publish-resolve-external-link option path t)))))) nil nil nil nil)) (ids a1) (fn-14506 . equal) t) (plist filename pub-dir) (setq ids (cons (catch :exit (org-publish-org-to backend filename .test plist pub-dir)) ids)))’
| Test test-org-publish/resolve-external-link backtrace:
|   signal(ert-test-failed (((should (equal "a1" (let* ((ids nil) (backe
|   ert-fail(((should (equal "a1" (let* ((ids nil) (backend (org-export-
|   (if (unwind-protect (setq value-14508 (apply fn-14506 args-14507)) (
|   (let (form-description-14510) (if (unwind-protect (setq value-14508 
|   (let ((value-14508 'ert-form-evaluation-aborted-14509)) (let (form-d
|   (let* ((fn-14506 (function equal)) (args-14507 (condition-case err (
|   (closure (t) nil (let* ((fn-14501 (function apply)) (args-14502 (con
|   ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
|   ert-run-test(#s(ert-test :name test-org-publish/resolve-external-lin
|   ert-run-or-rerun-test(#s(ert--stats :selector "\\(org\\|ob\\)" :test
|   ert-run-tests("\\(org\\|ob\\)" #f(compiled-function (event-type &res
|   ert-run-tests-batch("\\(org\\|ob\\)")
|   ert-run-tests-batch-and-exit("\\(org\\|ob\\)")
|   (let ((org-id-track-globally t) (org-test-selector (if org-test-sele
|   org-test-run-batch-tests("\\(org\\|ob\\)")
|   eval((org-test-run-batch-tests org-test-select-re))
|   command-line-1(("--eval" "(setq vc-handled-backends nil org-startup-
|   command-line()
|   normal-top-level()
| Test test-org-publish/resolve-external-link condition:
|     (ert-test-failed
|      ((should
|        (equal "a1"
| 	      (let* ... ... ...)))
|       :form
|       (equal "a1" "org44ee821")
|       :value nil :explanation
|       (arrays-of-different-length 2 10 "a1" "org44ee821" first-mismatch-at 0)))
|    FAILED  572/765  test-org-publish/resolve-external-link
`----

-- 
Nick

Re: ``make test'' failure

[Nicolas Goaziou]

[-- Attachment #1: Type: text/plain, Size: 1615 bytes --]

Hello,

Nick Dokos <ndokos@gmail.com> writes:

> I get one failure in test-org-publish/resolve-external-link.
> Bisecting fingers this commit:
>
> ,----
> | $ git bisect good
> | 007bbddbccfd06ab9c97d51bf833a068cb1b07a2 is the first bad commit
> | commit 007bbddbccfd06ab9c97d51bf833a068cb1b07a2
> | Author: Nicolas Goaziou <mail@nicolasgoaziou.fr>
> | Date:   Sun Sep 10 00:16:12 2017 +0200
> | 
> |     ox-html: Prevent spurious target below headlines
> |     
> |     * lisp/ox-html.el (org-html-headline):
> |     (org-html-link): Do not insert an additional target.
> |     
> |     * lisp/ox-publish.el (org-publish-resolve-external-link): Add an
> |       optional argument.
> |     
> |     * lisp/ox.el (org-export-get-reference): Improve docstring.
> |     
> |     * testing/examples/pub/a.org:
> |     * testing/examples/pub/b.org: New files.
> |     * testing/lisp/test-ox-publish.el (test-org-publish/resolve-external-link):
> |       New test.
> | 
> | :040000 040000 fda080e0db653c9049448200be5ad03369379cc3 ec0eb8a5cd9b3c3b204fd7134e515fb360b85c79 M	etc
> | :040000 040000 39afe6514335e2ccd549fc5b7fafcac344cbb471 a0fdf6967cc3ab95b046f81dab08095fe0d302de M	lisp
> | :040000 040000 120e098640643a963336d77d65929ef5863eeba5 679329e78a4bbc04f6f84bb176821a3084732ab3 M	testing
> `----
>
> which *introduced* this test, but of course the test may be at fault,
> rather than the code.

There is a non-deterministic part in that failure since neither our
Build nor my machine can reproduce it.

Could you try the following patch and tell me if the test succeed?

Thank you.

Regards,

-- 
Nicolas Goaziou

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Fix-failing-test.patch --]
[-- Type: text/x-diff, Size: 2631 bytes --]

From dc8d8d1b9bf92d9fe826cfe8a13c6ad89c1f9c8e Mon Sep 17 00:00:00 2001
From: Nicolas Goaziou <mail@nicolasgoaziou.fr>
Date: Wed, 1 Nov 2017 17:54:12 +0100
Subject: [PATCH] Fix failing test

* testing/lisp/test-ox-publish.el (test-org-publish/resolve-external-link):
  Rewrite test to prevent race condition which could affect results.
* testing/examples/pub/b.org: Add missing keyword.
---
 testing/examples/pub/b.org      |  3 +++
 testing/lisp/test-ox-publish.el | 34 ++++++++++++++++++----------------
 2 files changed, 21 insertions(+), 16 deletions(-)

diff --git a/testing/examples/pub/b.org b/testing/examples/pub/b.org
index 60387460e..261152e99 100644
--- a/testing/examples/pub/b.org
+++ b/testing/examples/pub/b.org
@@ -2,5 +2,8 @@
 #+date: <2012-03-29 Thu>
 
 * Headline1
+:PROPERTIES:
+:CUSTOM_ID: b1
+:END:
 
 [[file:a.org::#a1]]
diff --git a/testing/lisp/test-ox-publish.el b/testing/lisp/test-ox-publish.el
index a51190578..0c3aaedca 100644
--- a/testing/lisp/test-ox-publish.el
+++ b/testing/lisp/test-ox-publish.el
@@ -366,27 +366,29 @@ Unless set otherwise in PROPERTIES, `:base-directory' is set to
   ;; instead of internal reference, whenever possible.
   (should
    (equal
-    "a1"
+    '("a1" "b1")
     (let* ((ids nil)
+	   (link-transcoder
+	    (lambda (l c i)
+	      (let ((option (org-element-property :search-option l))
+		    (path (org-element-property :path l)))
+		(push (org-publish-resolve-external-link option path t)
+		      ids)
+		"")))
 	   (backend
 	    (org-export-create-backend
-	     :transcoders
-	     '((headline . (lambda (h c i) c))
-	       (paragraph . (lambda (p c i) c))
-	       (section . (lambda (s c i) c))
-	       (link . (lambda (l c i)
-			 (let ((option (org-element-property :search-option l))
-			       (path (org-element-property :path l)))
-			   (when option
-			     (throw :exit (org-publish-resolve-external-link
-					   option path t)))))))))
+	     :transcoders `((headline . (lambda (h c i) c))
+			    (paragraph . (lambda (p c i) c))
+			    (section . (lambda (s c i) c))
+			    (link . ,link-transcoder))))
 	   (publish
 	    (lambda (plist filename pub-dir)
-	      (push (catch :exit
-		      (org-publish-org-to backend filename ".test" plist pub-dir))
-		    ids))))
-      (org-test-publish (list :publishing-function (list publish)) #'ignore)
-      (car ids)))))
+	      (org-publish-org-to backend filename ".test" plist pub-dir))))
+      (org-test-publish (list :publishing-function (list publish)
+			      :exclude "."
+			      :include '("a.org" "b.org"))
+	#'ignore)
+      (sort ids #'string<)))))
 
 \f
 ;;; Tools
-- 
2.14.3

Re: ``make test'' failure

[Nick Dokos]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> There is a non-deterministic part in that failure since neither our
> Build nor my machine can reproduce it.
>
> Could you try the following patch and tell me if the test succeed?
>

Yes, indeed: the test passes here with the patch.

Thanks!
--
Nick

Re: ``make test'' failure

[Nicolas Goaziou]

Hello,

Nick Dokos <ndokos@gmail.com> writes:

> Yes, indeed: the test passes here with the patch.

I merged the updated test in master branch. Thank you.

Regards,

-- 
Nicolas Goaziou

Re: ``make test'' failure

[Nick Dokos]

[-- Attachment #1: Type: text/plain, Size: 270 bytes --]

Thank you!

On Nov 3, 2017 17:17, "Nicolas Goaziou" <mail@nicolasgoaziou.fr> wrote:

Hello,

Nick Dokos <ndokos@gmail.com> writes:

> Yes, indeed: the test passes here with the patch.

I merged the updated test in master branch. Thank you.

Regards,

--
Nicolas Goaziou

[-- Attachment #2: Type: text/html, Size: 736 bytes --]