From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sebastian Rose Subject: Re: How you can help Date: Thu, 23 Oct 2008 16:20:21 +0200 Message-ID: <87ljwfbdga.fsf@kassiopeya.MSHEIMNETZ> References: <967CE7ED-05E9-4031-9F3B-CFB826511554@alexanderonline.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Kt10j-0007Jk-7c for emacs-orgmode@gnu.org; Thu, 23 Oct 2008 10:18:01 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Kt10i-0007IL-Dz for emacs-orgmode@gnu.org; Thu, 23 Oct 2008 10:18:00 -0400 Received: from [199.232.76.173] (port=40696 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Kt10i-0007Hy-1F for emacs-orgmode@gnu.org; Thu, 23 Oct 2008 10:18:00 -0400 Received: from mail.gmx.net ([213.165.64.20]:57378) by monty-python.gnu.org with smtp (Exim 4.60) (envelope-from ) id 1Kt10h-0000ay-Ju for emacs-orgmode@gnu.org; Thu, 23 Oct 2008 10:18:00 -0400 In-Reply-To: <967CE7ED-05E9-4031-9F3B-CFB826511554@alexanderonline.org> (Ben Alexander's message of "Thu, 23 Oct 2008 13:04:48 +0100") List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Ben Alexander Cc: emacs-orgmode Org-Mode Hi Ben, I cc'ed the list. The tests I described in my email to the list are not automated. The reason for that is my lack of (e)lisp knowledge. BUT, they where easy to handle for non programmers. I think the little test will make it to the worg site this week, when all private data is removed. You could take a look at it by then. And: improve, improve, improve... :-) If you know of someone who knows how to do automated tests in elisp, or some technique, package, whatever, please post it to the list, so we all can take a look at it and comment->decide something. This is _highly_ _appreciated_. Ben Alexander writes: > Sebastian Rose wrote: >> 5. I also think of little packages for testing parts of org. > > I'm curious if you or someone else has any ideas for writing automated tests for > org-mode. I haven't the foggiest idea how someone would write a test for the > parts of org that control what is displayed on the screen. I mean, when the > bug is 'it doesn't look right' how can you tell? Believe me, my idea is the foggiest of all possible ideas ;-) (so where my ideas of JavaScript some mounth ago), so I won't be of much help, I fear. > Perhaps the git repository should have a small collection of small org- > mode files that reproduce certain bugs? If there were some examples of how to > create such a test, then perhaps bug reporters would find it much easier to > create them. YES! Exactly. Every corner case an Org-file, every bug an Org-file. _DATA_ for testing is something, everyone _can_ provide. But git later, yes, maybe. Since this would need Carsten to think and act on this, I feel Worg is a nice place to start the first expieriments. We need Carstens power for other things (when will Org-mode finaly wash my car? It's repeater-TODO, but nothing happens!!!) Basically, I'd try to keep the the testing as simple as possible, and try to get elisp programmers to help with the code. We should try to keep the hurdles for testers as humble (?) as possible, and put all that's needed to be helpfull on one page in worg. I recently discovered the very unautomated `(print object)' in the elisp reference. Not everything can be done automated, maybe, but if I would have known this stupid `print' function, I would know more about elisp and some files in org already. And it would have been faster and easier to create the two minor patches I wrote. This is, where the 'links to elisp tutorials', 'tipps and tricks', 'emacs debugger' come handy. Willingness to help is no problem, as the Org-community reveals. As for me, it's often a lack of time and/or knowledge, that prevents it. And the aim of all this is to help helping, in means of good and detailed bug reports in the first place. > I do see some confusing issues due to different configuration files. So > creating a test file might involve making sure org-mode doesn't read any > configuration (how do you do that?) and possible asking org- > mode to extract all the configuration variables it has right now and dump them > into a test file (...and how do you do that?) True. Hm - to test without configuration, we already have `emacs -q'. Dumping the variables is just a list of (print var) statements. A (quite) complete list of variables could be extracted with grep? e.g.: grep -r defvar lisp/ Once we have such a list, we could set/reset some/all variables. This will not be perfect, but could work reasonable well. And yes, it would indeed be nice, to have elisp to reliably reset emacs/org configuration, so one could do several different test in sequence. Preferably they would _always_ run and dump all errors to some file or buffer, even if one or more tests fail. I think elsip provides funtionalities to handle those errors and skip to the next test. This would be a huge step. But still, we have to test with different _DATA_ too! The test I described is just a quick hack I did to do the testing, while Carsten was bug-hunting on the other end of my internet connection. The XHTML-export test was much easier to do, then some other tests we might need. In the end, it was a test, automated or not. And I had to do it, because I'm one out of a few who use org to maintain a web-site (locally in my case) and export an entire directory tree with lots of _DATA_, use org-info.js, `#+SETUPFILE' lines, etc. , I believe. This became obvious in this case. I was the one who even noticed the bug, which means, no one else was using recursive XHTML export for a while, or their setup didn't reveal that bug. So clearly I was the one to provide some support for this very part of the system. I can't rely on the assumption, that some maintainer has all possible setups at hand all the time (maybe this was possible years ago). While the testing of the HTML-export was quite simple, it requires a lot of _DATA_. Namly files and directories + different setups, per file setups, #+SETUPFILE, images, with/without org-info.js, extra styles set or not... to have a realistic testing environment (the test didn't have all of these...). A summary could be, that it's nice to provide different setups for maintainers and testers. - *Easy to install* (unpack and done), - *easy to use* ('emacs -q' and click a few [[elisp:]] links to set everything up) - and *easy to download* once they are removed again (a central place on Worg). - *Corner cases*, like the 'empty line before first heading bug' in the LaTeX exporter recently. Regards, -- Sebastian Rose, EMMA STIL - mediendesign, Niemeyerstr.6, 30449 Hannover Tel.: +49 (0)511 - 36 58 472 Fax: +49 (0)1805 - 233633 - 11044 mobil: +49 (0)173 - 83 93 417 Email: s.roseATemma-stil.de, sebastian_roseATgmx.de Http: www.emma-stil.de