make test on Mac

make test on Mac

[Neuwirth Erich]

I am regularly building org mode on my Mac.
I do not pretend to bean experienced Emacs programmer, but I am will it to help if I can contribute.

This is what I get when I run
make test
on the current source.

Ran 314 tests, 307 results as expected, 7 unexpected (2012-08-01 16:06:02+0200)
5 expected failures

7 unexpected results:
   FAILED  ob-fortran/command-arguments
   FAILED  ob-fortran/fortran-var-program
   FAILED  ob-fortran/input-var
   FAILED  ob-fortran/list-var
   FAILED  ob-fortran/list-var-from-table
   FAILED  ob-fortran/preprosessor-var
   FAILED  ob-fortran/simple-program



Does it make any sense for the developers that I mail this? 
Should I send more or is this useless information?

Erich

Re: make test on Mac

[Bastien]

Hi Erich,

yes, it does make sense to report such errors as they may be related to
your architecture/installation -- and developers cannot test all possible
architectures/installations.

Neuwirth Erich <erich.neuwirth@univie.ac.at> writes:

> I am regularly building org mode on my Mac.
> I do not pretend to bean experienced Emacs programmer, but I am will it to help if I can contribute.
>
> This is what I get when I run
> make test
> on the current source.
>
> Ran 314 tests, 307 results as expected, 7 unexpected (2012-08-01 16:06:02+0200)
> 5 expected failures

I can't reproduce this on my Emacs.

GNU Emacs 24.1.50.1 (i686-pc-linux-gnu, GTK+ Version 2.24.10)
 of 2012-07-26 on myhost

> 7 unexpected results:
>    FAILED  ob-fortran/command-arguments
>    FAILED  ob-fortran/fortran-var-program
>    FAILED  ob-fortran/input-var
>    FAILED  ob-fortran/list-var
>    FAILED  ob-fortran/list-var-from-table
>    FAILED  ob-fortran/preprosessor-var
>    FAILED  ob-fortran/simple-program

Hopefully someone using a Mac can try against latest git head and run a 
make test to confirm.

> Does it make any sense for the developers that I mail this? 
> Should I send more or is this useless information?

You should test what is your version of MacOSX and Emacs.

M-x emacs-version RET

Thanks!

-- 
 Bastien

Re: make test on Mac

[Eric Schulte]

>
> I can't reproduce this on my Emacs.
>
> GNU Emacs 24.1.50.1 (i686-pc-linux-gnu, GTK+ Version 2.24.10)
>  of 2012-07-26 on myhost
>
>> 7 unexpected results:
>>    FAILED  ob-fortran/command-arguments
>>    FAILED  ob-fortran/fortran-var-program
>>    FAILED  ob-fortran/input-var
>>    FAILED  ob-fortran/list-var
>>    FAILED  ob-fortran/list-var-from-table
>>    FAILED  ob-fortran/preprosessor-var
>>    FAILED  ob-fortran/simple-program
>

Hmm, these tests should only be run if the "gfortran" executable is in
your path.  If it isn't then that is the reason for the failure, and we
should improve the `org-test-for-executable' function for Macs.

Could you paste the actual error message from one of these tests into an
e-mail?  I'm not sure if there is an easy way to run a single test and
see the results, perhaps a Makefile target for running a single test
would be useful?  In the mean time, if you can

1. start Emacs
2. load the test functionality in /testing/org-test.el
3. and run one of these tests with e.g.,
   M-x ert RET ob-fortran/command-arguments RET

that should work to see more specifics on why these tests are failing.

Thanks,

-- 
Eric Schulte
http://cs.unm.edu/~eschulte

Re: make test on Mac

[Bastien]

Eric Schulte <eric.schulte@gmx.com> writes:

> Hmm, these tests should only be run if the "gfortran" executable is in
> your path.  

So let's take them out of the test suite when gfortran is not present?

-- 
 Bastien

Re: make test on Mac

[Eric Schulte]

Bastien <bzg@gnu.org> writes:

> Eric Schulte <eric.schulte@gmx.com> writes:
>
>> Hmm, these tests should only be run if the "gfortran" executable is in
>> your path.  
>
> So let's take them out of the test suite when gfortran is not present?

That is what we do currently, or at least that is what we're supposed to
be doing currently...  From the top of test-ob-fortran.el

    ;; -*- emacs-lisp -*-
    (org-test-for-executable "gfortran")
    (unless (featurep 'ob-fortran)
      (signal 'missing-test-dependency "Support for Fortran code blocks"))

This throws a signal which inhibits the tests in the file from being
run, at least this is how it is supposed to work.

Cheers,

-- 
Eric Schulte
http://cs.unm.edu/~eschulte

Re: make test on Mac

[Bastien]

Eric Schulte <eric.schulte@gmx.com> writes:

> Bastien <bzg@gnu.org> writes:
>
>> Eric Schulte <eric.schulte@gmx.com> writes:
>>
>>> Hmm, these tests should only be run if the "gfortran" executable is in
>>> your path.  
>>
>> So let's take them out of the test suite when gfortran is not present?
>
> That is what we do currently, or at least that is what we're supposed to
> be doing currently...  From the top of test-ob-fortran.el
>
>     ;; -*- emacs-lisp -*-
>     (org-test-for-executable "gfortran")
>     (unless (featurep 'ob-fortran)
>       (signal 'missing-test-dependency "Support for Fortran code blocks"))

Maybe this should be 

    (unless (and (org-test-for-executable "gfortran") (featurep 'ob-fortran))
      (signal 'missing-test-dependency "Support for Fortran code blocks"))

?

to also ignore the test when gfortran does not exist?  Wild guess.

-- 
 Bastien

Re: make test on Mac

[Eric Schulte]

Bastien <bzg@gnu.org> writes:

> Eric Schulte <eric.schulte@gmx.com> writes:
>
>> Bastien <bzg@gnu.org> writes:
>>
>>> Eric Schulte <eric.schulte@gmx.com> writes:
>>>
>>>> Hmm, these tests should only be run if the "gfortran" executable is in
>>>> your path.  
>>>
>>> So let's take them out of the test suite when gfortran is not present?
>>
>> That is what we do currently, or at least that is what we're supposed to
>> be doing currently...  From the top of test-ob-fortran.el
>>
>>     ;; -*- emacs-lisp -*-
>>     (org-test-for-executable "gfortran")
>>     (unless (featurep 'ob-fortran)
>>       (signal 'missing-test-dependency "Support for Fortran code blocks"))
>
> Maybe this should be 
>
>     (unless (and (org-test-for-executable "gfortran") (featurep 'ob-fortran))
>       (signal 'missing-test-dependency "Support for Fortran code blocks"))
>
> ?
>
> to also ignore the test when gfortran does not exist?  Wild guess.

Nope, `org-test-for-executable' signals it's own error, but if we
changed `org-test-for-executable' to return a boolean and then used your
example above it would be more readable.

Best,

-- 
Eric Schulte
http://cs.unm.edu/~eschulte

Re: make test on Mac

[Bastien]

Eric Schulte <eric.schulte@gmx.com> writes:

> `org-test-for-executable' signals it's own error, but if we
> changed `org-test-for-executable' to return a boolean and then used your
> example above it would be more readable.

Perhaps -- what I mean is that if (org-test-for-executable "python")
signal a missing dependancy, then test-ob-python.el should not be part
of the testing suite.  

Do you see what I mean?

-- 
 Bastien

Re: make test on Mac

[Eric Schulte]

Bastien <bzg@altern.org> writes:

> Eric Schulte <eric.schulte@gmx.com> writes:
>
>> `org-test-for-executable' signals it's own error, but if we
>> changed `org-test-for-executable' to return a boolean and then used your
>> example above it would be more readable.
>
> Perhaps -- what I mean is that if (org-test-for-executable "python")
> signal a missing dependancy, then test-ob-python.el should not be part
> of the testing suite.  
>
> Do you see what I mean?

Yes, I think we are already doing exactly what you are suggesting.

Specifically, as each file of tests with external dependencies is loaded
(e.g. test-ob-python.el) the first forms evaluated in that file are

    ;; -*- emacs-lisp -*-
    (org-test-for-executable "python")
    (unless (featurep 'ob-python)
      (signal 'missing-test-dependency "Support for Python code blocks"))

Thus, if the required executable (in this case python) is not available
on the user's system, or if the user has not loaded ob-python, then a
missing-test-dependency signal will be thrown and none of the tests in
the remainder of the file will be loaded.  This ensures that only tests
for which the required executable and Babel code are available will ever
be included in the test suite.

The `org-test-load' function, which loads of all of the test files,
catches these missing-test-dependency signals and for each one it
creates a dummy test that fails but is expected to fail.  These tests
serve to indicate to the user that there are tests which are not being
run.

Does this all make sense?

Thanks,

-- 
Eric Schulte
http://cs.unm.edu/~eschulte

Re: make test on Mac

[Bastien]

Hi Eric,

Eric Schulte <eric.schulte@gmx.com> writes:

> The `org-test-load' function, which loads of all of the test files,
> catches these missing-test-dependency signals and for each one it
> creates a dummy test that fails but is expected to fail.  These tests
> serve to indicate to the user that there are tests which are not being
> run.

That's the part I overlooked.

> Does this all make sense?

100% -- thanks for the detailed explanations!

-- 
 Bastien

Re: make test on Mac

[Neuwirth Erich]

I do have gfortran
Mahler:org-mode neuwirth$ whereis gfortran
/usr/bin/gfortran

Version information
Mahler:org-mode neuwirth$ gfortran
i686-apple-darwin11-gfortran-4.2.1: no input files

And it is in emacs' exec-path
exec-path
("/usr/bin" "/bin" "/usr/sbin" "/sbin" "/Applications/Emacs.app/Contents/MacOS/bin")

I already installed Mountain Lion

Emacs version 
(emacs-version)
"GNU Emacs 24.1.1 (x86_64-apple-darwin, NS apple-appkit-1038.36)
 of 2012-06-10 on bob.porkrind.org"

I did not run the tests from within Emacs, I ran
make test
in the directory where I keep the sources
I am using git to pull the current version

Are the tests supposed to work only from within Emacs?


On Aug 1, 2012, at 4:23 PM, Bastien <bzg@gnu.org> wrote:

> Hi Erich,
> 
> yes, it does make sense to report such errors as they may be related to
> your architecture/installation -- and developers cannot test all possible
> architectures/installations.
> 
> Neuwirth Erich <erich.neuwirth@univie.ac.at> writes:
> 
>> I am regularly building org mode on my Mac.
>> I do not pretend to bean experienced Emacs programmer, but I am will it to help if I can contribute.
>> 
>> This is what I get when I run
>> make test
>> on the current source.
>> 
>> Ran 314 tests, 307 results as expected, 7 unexpected (2012-08-01 16:06:02+0200)
>> 5 expected failures
> 
> I can't reproduce this on my Emacs.
> 
> GNU Emacs 24.1.50.1 (i686-pc-linux-gnu, GTK+ Version 2.24.10)
> of 2012-07-26 on myhost
> 
>> 7 unexpected results:
>>   FAILED  ob-fortran/command-arguments
>>   FAILED  ob-fortran/fortran-var-program
>>   FAILED  ob-fortran/input-var
>>   FAILED  ob-fortran/list-var
>>   FAILED  ob-fortran/list-var-from-table
>>   FAILED  ob-fortran/preprosessor-var
>>   FAILED  ob-fortran/simple-program
> 
> Hopefully someone using a Mac can try against latest git head and run a 
> make test to confirm.
> 
>> Does it make any sense for the developers that I mail this? 
>> Should I send more or is this useless information?
> 
> You should test what is your version of MacOSX and Emacs.
> 
> M-x emacs-version RET
> 
> Thanks!
> 
> -- 
> Bastien

Re: make test on Mac

[Achim Gratz]

Neuwirth Erich <erich.neuwirth <at> univie.ac.at> writes:
> Are the tests supposed to work only from within Emacs?

No, but you might be in a better position to tell us what went wrong, as you
still haven't shown what error you are getting from the tests in question.

So could you please go into the org directory and instead of make run:

emacs -Q -L lisp -l ob-fortran testing/examples/ob-fortran-test.org

Then just run one of the examples (C-c C-c on the #+begin_src line) and tell us
what happens.  There might be additional information in the *Messages* buffer.


Regards,
Achim.

Re: make test on Mac

[Neuwirth Erich]

The private reply was not intended.
I checked.
I do (of course) have /bin/bash
and the error is produced by bash.
It seems that the test suite does not find cc1 on Mountain Lion.
cc1 in installed byXCode, but PATH does not include the directory with cc1.
So I will try to add it there and report again.
Any further suggestions?




On Aug 3, 2012, at 10:32 PM, ASSI <Stromeko@nexgo.de> wrote:

> [It looks you sent this mail to me rather than the list, so my reply is 
> private as well.  I'd prefer to keep the discussion on-list.]
> 
> On Thursday 02 August 2012, 09:05:52, Neuwirth Erich wrote:
>> i686-apple-darwin11-gfortran-4.2.1: error trying to exec 'cc1': execvp: No
>> such file or directory /bin/bash:
>> /var/folders/cl/zktxb0v51ng7jfv0mspj2w940000gn/T/babel-9230sRi/fortran-bi
>> n-9230CSz: Permission denied
>> 
>> So I see this is a problem with gcc.
> 
> Maybe, but it rather seems that actually it is expecting to find /bin/bash, 
> but comes up empty.
> 
>> But gfortran is running on my machine.
> 
> "Running" meaning what exactly?  From the above it doesn't appear it might 
> actually work when you tried to start it from the command line.
> 
> 
> Regards,
> Achim.
> -- 
> +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+
> 
> Wavetables for the Waldorf Blofeld:
> http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables
> 

Re: make test on Mac

[Achim Gratz]

Neuwirth Erich writes:
> I do (of course) have /bin/bash and the error is produced by bash.

OK, so the linebreaks have been corrupted in your original post.  You
actually got two error messages it seems:

>>> i686-apple-darwin11-gfortran-4.2.1: error trying to exec 'cc1':
>>> execvp: No such file or directory

This is the gfortran frontend telling you that it doesn't find its own
compiler backend…

>>> /bin/bash:
>>> /var/folders/cl/zktxb0v51ng7jfv0mspj2w940000gn/T/babel-9230sRi/fortran-bi
>>> n-9230CSz: Permission denied

…and this is bash telling you that no executable was produced, which is
explained by the first error message.

> It seems that the test suite does not find cc1 on Mountain Lion.

The test suite does not even look for that file, it is part of the
compiler.  This seems to be a problem with the installation of your
compiler(s), but Org can't do anything about it.  Just do

$ make TMPDIR=`pwd` TEST_NO_AUTOCLEAN=1 test
$ cd tmp-orgtest

and try to compile any of the (non-empty) *.F90 files you'll find there.

> cc1 in installed byXCode, but PATH does not include the directory with cc1.
> So I will try to add it there and report again.

No, the compiler frontends should know where their backends are, they
should not be in path (otherwise it would become impossible to use
different versions of a compiler). Have you moved the files after
installation, perhaps?

> Any further suggestions?

Reinstall XCode, maybe.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: make test on Mac

[Neuwirth Erich]

After reinstalling fortran following the instructions for th R Tools implementation
on http://www.webmo.net/support/fortran_osx.html
The fortran test examples now woe.
But I am getting another set or errors (after updating from git)

6 expected failures

10 unexpected results:
   FAILED  test-org/auto-fill-function
   FAILED  test-org/backward-element
   FAILED  test-org/comment-dwim
   FAILED  test-org/down-element
   FAILED  test-org/drag-element-backward
   FAILED  test-org/drag-element-forward
   FAILED  test-org/fill-paragraph
   FAILED  test-org/forward-element
   FAILED  test-org/mark-subtree
   FAILED  test-org/up-element

How can I get more details error messages for these problems?



On Aug 2, 2012, at 8:35 AM, Achim Gratz <Stromeko@nexgo.de> wrote:

> Neuwirth Erich <erich.neuwirth <at> univie.ac.at> writes:
>> Are the tests supposed to work only from within Emacs?
> 
> No, but you might be in a better position to tell us what went wrong, as you
> still haven't shown what error you are getting from the tests in question.
> 
> So could you please go into the org directory and instead of make run:
> 
> emacs -Q -L lisp -l ob-fortran testing/examples/ob-fortran-test.org
> 
> Then just run one of the examples (C-c C-c on the #+begin_src line) and tell us
> what happens.  There might be additional information in the *Messages* buffer.
> 
> 
> Regards,
> Achim.
> 
> 

Re: make test on Mac

[Achim Gratz]

Neuwirth Erich <erich.neuwirth <at> univie.ac.at> writes:
> 7 unexpected results:
>    FAILED  ob-fortran/command-arguments
>    FAILED  ob-fortran/fortran-var-program
>    FAILED  ob-fortran/input-var
>    FAILED  ob-fortran/list-var
>    FAILED  ob-fortran/list-var-from-table
>    FAILED  ob-fortran/preprosessor-var
>    FAILED  ob-fortran/simple-program

Do you even have a Fortran compiler installed?  If not, just remove fortran from
the list of Babel languages that get tested.  If yes, then somehow Babel fails
to use the Fortran compiler properly and we'd need to see the output of the
failing tests and/or you could try to run the test suite manually and watch for
error messages.

> Does it make any sense for the developers that I mail this? 

Sure, unless you already know for certain it is not a problem with Org... in
that case we'd ask you to take it to a more appropriate place.


Regards,
Achim.