* [bug] regression tests broken @ 2011-11-11 21:29 Martyn Jago 2011-11-11 21:43 ` Eric Schulte 0 siblings, 1 reply; 8+ messages in thread From: Martyn Jago @ 2011-11-11 21:29 UTC (permalink / raw) To: emacs-orgmode Hi All automated regression tests broken since 7720786 commit http://martynjago.com:3333 Best, Martyn ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [bug] regression tests broken 2011-11-11 21:29 [bug] regression tests broken Martyn Jago @ 2011-11-11 21:43 ` Eric Schulte 2011-11-12 18:22 ` Martyn Jago 0 siblings, 1 reply; 8+ messages in thread From: Eric Schulte @ 2011-11-11 21:43 UTC (permalink / raw) To: Martyn Jago; +Cc: emacs-orgmode Hi Martyn, I have all tests passing in a branch in the Org-mode git repo (which I plan to merge into master soon). Could you try a run of this on the standard-code-block-syntax branch? As a side note, how automated is your crusecontrol.rb setup? Would you recommend this as a way to run automated tests on the Org-mode server after every git commit? Thanks -- Eric Martyn Jago <martyn.jago@btinternet.com> writes: > Hi > > All automated regression tests broken since 7720786 commit > > http://martynjago.com:3333 > > Best, Martyn > > -- Eric Schulte http://cs.unm.edu/~eschulte/ ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [bug] regression tests broken 2011-11-11 21:43 ` Eric Schulte @ 2011-11-12 18:22 ` Martyn Jago 2011-11-13 15:25 ` Martyn Jago 2011-11-13 16:04 ` Eric Schulte 0 siblings, 2 replies; 8+ messages in thread From: Martyn Jago @ 2011-11-12 18:22 UTC (permalink / raw) To: emacs-orgmode Hi Eric Eric Schulte <schulte.eric@gmail.com> writes: > Hi Martyn, > > I have all tests passing in a branch in the Org-mode git repo (which I > plan to merge into master soon). Could you try a run of this on the > standard-code-block-syntax branch? I've added the standard-code-block-syntax branch to my test server, and have configured notification so that you will be informed via email on failure. Unfortunately, the branch is currently failing, as are all other builds including my test-branch. I will try to take a look tomorrow on why three tests are failing. > As a side note, how automated is your crusecontrol.rb setup? Would you > recommend this as a way to run automated tests on the Org-mode server > after every git commit? It is completely autonomous, git-pulling, cleaning, building, building docs, and running tests within 10 minutes or so of a fresh commit to the Org repository. Builds are serialized. I would certainly recommend it as an easy way of setting up a test server, since everything you need is built-in, such as email notification (notifying the list is of course a possibility), RSS, and rapid backtrace investigation etc. I have come across other gnu projects that use cruisecontrolrb. Your server would need Ruby installed (I use v1.9.2), but installation should be straight-forward since I have already written the scripts, which is fairly simple anyway. Hope that helps Best Martyn > Thanks -- Eric > > Martyn Jago <martyn.jago@btinternet.com> writes: > >> Hi >> >> All automated regression tests broken since 7720786 commit >> >> http://martynjago.com:3333 >> >> Best, Martyn >> >> ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [bug] regression tests broken 2011-11-12 18:22 ` Martyn Jago @ 2011-11-13 15:25 ` Martyn Jago 2011-11-13 16:08 ` Martyn Jago 2011-11-13 16:15 ` Eric Schulte 2011-11-13 16:04 ` Eric Schulte 1 sibling, 2 replies; 8+ messages in thread From: Martyn Jago @ 2011-11-13 15:25 UTC (permalink / raw) To: emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 1416 bytes --] Martyn Jago <martyn.jago@btinternet.com> writes: Hi Eric [...] Further to the problem of tests failing under sandboxed conditions unexpectedly, I've made a few observations which I want to pass on. * ID Currently the ID searching command `org-id-find' has global scope to find ID's all over the host system. Since developers frequently have > 1 Orgs on their system, this can lead to confusion. For instance, the test `ob-C/table' in in test-ob-C.el appears to contain an orphan ID, which fails, but may well pass on the devs system, since the ID probably exists somewhere (does this test also require to be called test-ob-C/table)?. Unfortunately `org-id-find' seems to be heavily dependent on org-mode.el forms, and I suspect it would require a lot of time for me to attempt to restrict it's scope to `the current Org system in use, so I will leave it to someone with more experience. Another by-product is that sandboxed test systems are not guaranteed to pick up the _correct_ file containing the search ID (this plagued me this-morning). * `org-test-load' attempts to load symlinks (Emacs interlocking files). Currently `org-test-load' will happily attempt to load an interlocking symbolic link (such as .#test-ob-exp.el -> martyn@88-96-171-138.59787). This results in a failure to run any tests, and is particularly annoying during test development. Below is a patch that fixes this problem for me. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-Avoid-loading-and-failing-symbolic-links-interlockin.patch --] [-- Type: text/x-patch, Size: 1180 bytes --] From 04dd7358295caef0d42a3fe93f4979ccd3b5c630 Mon Sep 17 00:00:00 2001 From: Martyn Jago <martyn.jago@btinternet.com> Date: Sun, 13 Nov 2011 14:43:34 +0000 Subject: [PATCH] Avoid loading (and failing) symbolic links (interlocking files) * testing/org-test.el: During test development various interlocking files may be present in testing/lisp directory (since they are being edited by emacs). Currently org-test-load will attempt to load these and fail. --- testing/org-test.el | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/testing/org-test.el b/testing/org-test.el index 57b7252..ea8cae4 100644 --- a/testing/org-test.el +++ b/testing/org-test.el @@ -274,9 +274,10 @@ otherwise place the point at the beginning of the inserted text." (mapc (lambda (path) (if (file-directory-p path) - (rld path) + (rld path) (catch 'missing-test-dependency - (load-file path)))) + (when (string-match "^[A-Za-z].*\\.el$" path) + load-file path)))) (directory-files base 'full "^\\([^.]\\|\\.\\([^.]\\|\\..\\)\\).*\\.el$")))) (rld (expand-file-name "lisp" org-test-dir)) -- 1.7.3.4 [-- Attachment #3: Type: text/plain, Size: 154 bytes --] I'm still looking at a potential variable passing problem during html export by inline calls, and will submit something as soon as I can. Best, Martyn ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [bug] regression tests broken 2011-11-13 15:25 ` Martyn Jago @ 2011-11-13 16:08 ` Martyn Jago 2011-11-13 16:15 ` Eric Schulte 1 sibling, 0 replies; 8+ messages in thread From: Martyn Jago @ 2011-11-13 16:08 UTC (permalink / raw) To: emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 379 bytes --] Hi Eric Oops, the patch was broken - this one fixes the problem... [...] > Currently `org-test-load' will happily attempt to load an interlocking > symbolic link (such as .#test-ob-exp.el -> > martyn@88-96-171-138.59787). This results in a failure to run any tests, > and is particularly annoying during test development. Below is a patch > that fixes this problem for me. > [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: Fix-don-t-load-symlinks-Emacs-interlocking-files --] [-- Type: text/x-patch, Size: 1191 bytes --] From b868e208dd3ece7c10818ed73f2cf0dc86dac94b Mon Sep 17 00:00:00 2001 From: Martyn Jago <martyn.jago@btinternet.com> Date: Sun, 13 Nov 2011 16:03:51 +0000 Subject: [PATCH] Fix don't load symlinks (Emacs interlocking files) * testing/org-test.el: During test development various interlocking files may be present in testing/lisp directory (since they are being edited by emacs). Currently org-test-load will attempt to load these --- testing/org-test.el | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/testing/org-test.el b/testing/org-test.el index ea8cae4..7d2f7e7 100644 --- a/testing/org-test.el +++ b/testing/org-test.el @@ -276,8 +276,9 @@ otherwise place the point at the beginning of the inserted text." (if (file-directory-p path) (rld path) (catch 'missing-test-dependency - (when (string-match "^[A-Za-z].*\\.el$" path) - load-file path)))) + (when (string-match "^[A-Za-z].*\\.el$" + (file-name-nondirectory path)) + (load-file path))))) (directory-files base 'full "^\\([^.]\\|\\.\\([^.]\\|\\..\\)\\).*\\.el$")))) (rld (expand-file-name "lisp" org-test-dir)) -- 1.7.3.4 [-- Attachment #3: Type: text/plain, Size: 22 bytes --] Best, Martyn [...] ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [bug] regression tests broken 2011-11-13 15:25 ` Martyn Jago 2011-11-13 16:08 ` Martyn Jago @ 2011-11-13 16:15 ` Eric Schulte 1 sibling, 0 replies; 8+ messages in thread From: Eric Schulte @ 2011-11-13 16:15 UTC (permalink / raw) To: Martyn Jago; +Cc: emacs-orgmode Martyn Jago <martyn.jago@btinternet.com> writes: > Martyn Jago <martyn.jago@btinternet.com> writes: > Hi Eric > > [...] > > Further to the problem of tests failing under sandboxed conditions > unexpectedly, I've made a few observations which I want to pass on. > > * ID > > Currently the ID searching command `org-id-find' has global scope to > find ID's all over the host system. Since developers frequently have > 1 > Orgs on their system, this can lead to confusion. > Customizing the `org-id-locations-file' can be used to isolate ID lists between different instances of Emacs. Only the files in the `org-id-files' list should be scanned for IDs, so it shouldn't be too difficult to limit the scope of those files used in testing to only the example files in the testing directory. > > For instance, the test `ob-C/table' in in test-ob-C.el appears to > contain an orphan ID, which fails, Confirmed, I don't have this ID located on my system either. Perhaps the author of this test file can help here. Note that this test is marked as expected to fail, so it should not affect the test suite as a whole. > but may well pass on the devs system, since the ID probably exists > somewhere (does this test also require to be called test-ob-C/table)?. > No, the existing name is fine, see the ERT test match string in `org-test-run-all-tests' to see which tests will be picked up by the Org-mode test suite.. > > Unfortunately `org-id-find' seems to be heavily dependent on org-mode.el > forms, and I suspect it would require a lot of time for me to attempt to > restrict it's scope to `the current Org system in use, so I will leave > it to someone with more experience. > > Another by-product is that sandboxed test systems are not guaranteed to > pick up the _correct_ file containing the search ID (this plagued me > this-morning). > Given that all of these IDs are long hash tags I doubt there is any risk of ID duplication unless there are multiple checkouts of Org-mode being included in the files Org-mode scans for IDs. > > * `org-test-load' attempts to load symlinks (Emacs interlocking >files). > > Currently `org-test-load' will happily attempt to load an interlocking > symbolic link (such as .#test-ob-exp.el -> > martyn@88-96-171-138.59787). This results in a failure to run any tests, > and is particularly annoying during test development. Below is a patch > that fixes this problem for me. > Great, I've just applied this patch. Thanks -- Eric > > From 04dd7358295caef0d42a3fe93f4979ccd3b5c630 Mon Sep 17 00:00:00 2001 > From: Martyn Jago <martyn.jago@btinternet.com> > Date: Sun, 13 Nov 2011 14:43:34 +0000 > Subject: [PATCH] Avoid loading (and failing) symbolic links (interlocking files) > * testing/org-test.el: During test development various interlocking > files may be present in testing/lisp directory (since they are being > edited by emacs). Currently org-test-load will attempt to load these > and fail. > > --- > testing/org-test.el | 5 +++-- > 1 files changed, 3 insertions(+), 2 deletions(-) > > diff --git a/testing/org-test.el b/testing/org-test.el > index 57b7252..ea8cae4 100644 > --- a/testing/org-test.el > +++ b/testing/org-test.el > @@ -274,9 +274,10 @@ otherwise place the point at the beginning of the inserted text." > (mapc > (lambda (path) > (if (file-directory-p path) > - (rld path) > + (rld path) > (catch 'missing-test-dependency > - (load-file path)))) > + (when (string-match "^[A-Za-z].*\\.el$" path) > + load-file path)))) > (directory-files base 'full > "^\\([^.]\\|\\.\\([^.]\\|\\..\\)\\).*\\.el$")))) > (rld (expand-file-name "lisp" org-test-dir)) -- Eric Schulte http://cs.unm.edu/~eschulte/ ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [bug] regression tests broken 2011-11-12 18:22 ` Martyn Jago 2011-11-13 15:25 ` Martyn Jago @ 2011-11-13 16:04 ` Eric Schulte 2011-11-13 18:37 ` Martyn Jago 1 sibling, 1 reply; 8+ messages in thread From: Eric Schulte @ 2011-11-13 16:04 UTC (permalink / raw) To: Martyn Jago; +Cc: emacs-orgmode Martyn Jago <martyn.jago@btinternet.com> writes: > Hi Eric > > Eric Schulte <schulte.eric@gmail.com> writes: > >> Hi Martyn, >> >> I have all tests passing in a branch in the Org-mode git repo (which I >> plan to merge into master soon). Could you try a run of this on the >> standard-code-block-syntax branch? > > I've added the standard-code-block-syntax branch to my test server, and > have configured notification so that you will be informed via email on > failure. Unfortunately, the branch is currently failing, as are all > other builds including my test-branch. I will try to take a look > tomorrow on why three tests are failing. > I had to take special care that previous versions of Org-mode weren't loaded when running tests on this branch. The problem being that if some defvar forms are defined by a previous instillation of Org-mode their new values may not be overwritten with the latest versions in this branch. From looking at your test output it seems the only two tests which are failing are the most recent two related to nested code blocks. Again these are passing on my system so I'm not sure what the cause could be. > >> As a side note, how automated is your crusecontrol.rb setup? Would you >> recommend this as a way to run automated tests on the Org-mode server >> after every git commit? > > It is completely autonomous, git-pulling, cleaning, building, building > docs, and running tests within 10 minutes or so of a fresh commit to the > Org repository. Builds are serialized. > > I would certainly recommend it as an easy way of setting up a test > server, since everything you need is built-in, such as email > notification (notifying the list is of course a possibility), RSS, and > rapid backtrace investigation etc. > > I have come across other gnu projects that use cruisecontrolrb. > > Your server would need Ruby installed (I use v1.9.2), but installation > should be straight-forward since I have already written the scripts, > which is fairly simple anyway. > > Hope that helps > This sounds like a very nice setup. It would be great if you can work with me and Jason to see if we can set this up on the Org-mode server. Thanks -- Eric > > Best Martyn > >> Thanks -- Eric >> >> Martyn Jago <martyn.jago@btinternet.com> writes: >> >>> Hi >>> >>> All automated regression tests broken since 7720786 commit >>> >>> http://martynjago.com:3333 >>> >>> Best, Martyn >>> >>> > > -- Eric Schulte http://cs.unm.edu/~eschulte/ ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [bug] regression tests broken 2011-11-13 16:04 ` Eric Schulte @ 2011-11-13 18:37 ` Martyn Jago 0 siblings, 0 replies; 8+ messages in thread From: Martyn Jago @ 2011-11-13 18:37 UTC (permalink / raw) To: emacs-orgmode Hi Eric Eric Schulte <schulte.eric@gmail.com> writes: [...] > This sounds like a very nice setup. It would be great if you can work > with me and Jason to see if we can set this up on the Org-mode server. No problem. If it is a Linux server, I can work on setting up a fresh instance on Linux prior the actual deployment (currently on OSX). Best, Martyn > Thanks -- Eric > >> >> Best Martyn >> >>> Thanks -- Eric >>> >>> Martyn Jago <martyn.jago@btinternet.com> writes: >>> >>>> Hi >>>> >>>> All automated regression tests broken since 7720786 commit >>>> >>>> http://martynjago.com:3333 >>>> >>>> Best, Martyn >>>> >>>> >> >> ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2011-11-13 18:37 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2011-11-11 21:29 [bug] regression tests broken Martyn Jago 2011-11-11 21:43 ` Eric Schulte 2011-11-12 18:22 ` Martyn Jago 2011-11-13 15:25 ` Martyn Jago 2011-11-13 16:08 ` Martyn Jago 2011-11-13 16:15 ` Eric Schulte 2011-11-13 16:04 ` Eric Schulte 2011-11-13 18:37 ` Martyn Jago
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).