emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
blob 9601ea7ce66862127827361a646be9f6060e93c5 3896 bytes (raw)
name: testing/README 	 # note: path name is non-authoritative(*)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
 
# -*- mode:org -*-
#+TITLE: Org-mode Testing
#+PROPERTY: results silent

* Dependencies

The only dependency is [[http://www.emacswiki.org/emacs/ErtTestLibrary][ERT]] the Emacs testing library which ships with
Emacs24.  If you are running an older version of Emacs and don't
already have ERT installed it can be installed from its old [[https://github.com/ohler/ert][git
repository]].

* Non-interactive batch testing from the command line

The simplest way to run the Org-mode test suite is from the command
line with the following invocation.  Note that the paths below are
relative to the base of the Org-mode directory.

Also note that many of the current tests uses babel evaluation...

#+BEGIN_SRC sh :dir (expand-file-name "..")
  # For Emacs earlier than 24, add -L /path/to/ert
  emacs -Q --batch \
        -L lisp/ -L testing/ -L testing/lisp -l lisp/org.el \
        -l lisp/org-id.el -l testing/org-test.el \
        --eval "(progn (org-reload) (setq org-confirm-babel-evaluate nil) \
        (org-babel-do-load-languages 'org-babel-load-languages \
        '((emacs-lisp . t) (sh . t) (org . t))))" \
        -f org-test-run-batch-tests
#+END_SRC

The options in the above command are explained below.

| -Q      | ignores any personal configuration ensuring a vanilla Emacs instance is used |
| --batch | runs Emacs in "batch" mode with no gui and termination after execution       |
| -l      | loads Org-mode and the org mode test suite defined in testing/org-test.el    |
| --eval  | reloads Org-mode and allows evaluation of code blocks by the tests           |
| -f      | actually runs the tests using the `org-test-run-batch-tests' function        |

* Interactive testing from within Emacs

To run the Org-mode test suite from a current Emacs instance simply
load and run the test suite with the following commands.

1) First load the test suite.
   #+BEGIN_SRC emacs-lisp :var here=(buffer-file-name)
     (add-to-list 'load-path (file-name-directory here))
     (require 'org-test)
   #+END_SRC

2) Disable babel evaluation confirmation
   #+BEGIN_SRC emacs-lisp
     (setq org-confirm-babel-evaluate)
   #+END_SRC

3) Then run the test suite,
   #+BEGIN_SRC emacs-lisp
     (org-test-run-all-tests)
   #+END_SRC

   or when a test fails run it interactively and investigate the
   problem in the ERT results buffer.

   How to run one test:
   Use this as a demo example of a failing test
   #+BEGIN_SRC emacs-lisp
     (ert-deftest test-org/org-link-escape-ascii-character-demo-of-fail ()
       (should (string= "%5B"  ;; expected is right
                        (org-link-escape "[")))
       (should (string= "%5C"  ;; expected is wrong, "%5D" would be right
                        (org-link-escape "]"))))
   #+END_SRC
   or evaluate the ert-deftest form of the test you want to run.  Then
   "M-x ert RET test-org/org-link-escape-ascii-character-demo-of-fail RET"
   When not visible yet switch to the ERT results buffer named
   "\*ert\*".  When a test failed the ERT results buffer shows the
   details of the first "should" that failed.  See
   (info "(ert)Running Tests Interactively") on how to re-run, start
   the debugger etc.

   How to run all tests of a single test file:
   "M-x ert-delete-all-tests RET", confirm.  Open the file
   ./lisp/test-*.el, "M-x eval-buffer RET", "M-x ert RET t RET"

   Consider to set pp-escape-newlines nil before running the test when
   looking at "should" in the ERT results buffer.  Especially when
   using "l" to look at passed test results and possibly missing an
   appropriate setting of pp-escape-newlines made only temporarily for
   the running time of the test as e. g. tests using
   org-test-table-target-expect-tblfm do.

* Troubleshooting

- If the value of the =org-babel-no-eval-on-ctrl-c-ctrl-c= is non-nil
  then it will result in some test failure, as there are tests which
  rely on this behavior.

debug log:

solving 9601ea7 ...
found 9601ea7 in https://git.savannah.gnu.org/cgit/emacs/org-mode.git

(*) Git path names are given by the tree(s) the blob belongs to.
    Blobs themselves have no identifier aside from the hash of its contents.^

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).