* How you can help
@ 2008-10-23 7:35 Carsten Dominik
2008-10-23 8:12 ` Manish
` (2 more replies)
0 siblings, 3 replies; 25+ messages in thread
From: Carsten Dominik @ 2008-10-23 7:35 UTC (permalink / raw)
To: emacs-orgmode List
Hi Org users,
I need to get control over the time I spent on developing
Org-mode. Recently, I have again worked too hard on it, spending
more time than I should, in order to get 6.9 and 6.10 out and to
seize the chance to get the best possible version into Emacs
23.1.
However, this is getting out of control, and I am now putting
myself a hard limit of 1 hour per day, clocked by Org-mode, which
will clearly cut in on my development speed and posting rate.
Here is how you can help me to make the most of that one hour.
1. If you report a bug, please try to do as much work as you can
yourself. Before you send it, think if you have collected all
the information you can. There are great examples of good bug
reports on the list already. The best bug reports are, of
course, those that are accompanied by a patch.
2. If you are reading the list and see bug reports, consider
putting in 10 minutes, trying to reproduce this problem
yourself, and maybe adding information that might be useful
for me to track down that bug. Again, this is already
happening, I am only trying to encourage this type of
behavior.
3. I have recently spent much time on fixing bugs in parts of Org
that I did not write myself, so this is taking much more time
than usually. If you know Emacs Lisp, and I know there are a
number of excellent Lisp programmers on this group, consider
"adopting" one of the following subsystems. By "adoption", I
mean that you make it your mission to deeply understand this
part of Org, so that *you* will be in the position to fix
bugs, taking that off my shoulders.
- org-publish.el. I think the biggest bugs are out now, but I
am sure this system can be improved quite a bit. If you are
that Lisp programmer trying to take up this task, consider
teaming up with Sebastian Rose who is a great guy, has quite
some understanding of that system and good ideas about it,
but just is not enough of a Lisp programmer to really take
that on.
- org-export-latex.el. This is a tough one because you need
to know both LaTeX and Lisp. And the code is complex, in
part because Org-mode allows to write LaTeX is a relatively
lazy way. Bastien has done a great great job capturing this
into an exporter. However, there are still problems and
bugs, some came up recently on the list. And Bastien
currently does not have the time to contribute consistently.
As a result, I have started to fix some bugs, but this is
really eating too much of my time. This subsystem can also
use feature additions, for example better handling of image
insertion, maybe with captions. I have ideas about this, so
talk to me if you want to help out.
4. Try to answer as many messages on the mailing list as you can.
I have been trying hard to make sure that each and every
reasonable question on the list (and this is really the only
kind we get in this amazing community) will be answered.
Doing this still takes a significant fraction of my Org-mode
development time. I will clearly spend less time on this in
the future, in this way also giving others more time to
answer. Again, there are already quite a few people who
regularly *answer* questions, and all I am trying to do here
is encouraging this activity.
5. Help developing the Org-mode FAQ by adding useful information
to it yourself. All you need to do is to get acces to Worg,
which will help getting to know git in the process. Worg is
meant to be user edited, so please go wild and add information
and links at will.
Thanks to you all, for your understanding and help.
- Carsten
^ permalink raw reply [flat|nested] 25+ messages in thread* Re: How you can help 2008-10-23 7:35 How you can help Carsten Dominik @ 2008-10-23 8:12 ` Manish 2008-10-23 9:24 ` Sebastian Rose 2008-10-23 15:23 ` Russell Adams 2 siblings, 0 replies; 25+ messages in thread From: Manish @ 2008-10-23 8:12 UTC (permalink / raw) To: Carsten Dominik; +Cc: emacs-orgmode List On Thu, Oct 23, 2008 at 1:05 PM, Carsten Dominik wrote: > Hi Org users, > > I need to get control over the time I spent on developing > Org-mode. Recently, I have again worked too hard on it, spending > more time than I should, in order to get 6.9 and 6.10 out and to > seize the chance to get the best possible version into Emacs > 23.1. > > However, this is getting out of control, and I am now putting > myself a hard limit of 1 hour per day, clocked by Org-mode, which > will clearly cut in on my development speed and posting rate. > > Here is how you can help me to make the most of that one hour. > > 1. If you report a bug, please try to do as much work as you can > yourself. Before you send it, think if you have collected all > the information you can. There are great examples of good bug > reports on the list already. The best bug reports are, of > course, those that are accompanied by a patch. > > 2. If you are reading the list and see bug reports, consider > putting in 10 minutes, trying to reproduce this problem > yourself, and maybe adding information that might be useful > for me to track down that bug. Again, this is already > happening, I am only trying to encourage this type of > behavior. > > 3. I have recently spent much time on fixing bugs in parts of Org > that I did not write myself, so this is taking much more time > than usually. If you know Emacs Lisp, and I know there are a > number of excellent Lisp programmers on this group, consider > "adopting" one of the following subsystems. By "adoption", I > mean that you make it your mission to deeply understand this > part of Org, so that *you* will be in the position to fix > bugs, taking that off my shoulders. > > - org-publish.el. I think the biggest bugs are out now, but I > am sure this system can be improved quite a bit. If you are > that Lisp programmer trying to take up this task, consider > teaming up with Sebastian Rose who is a great guy, has quite > some understanding of that system and good ideas about it, > but just is not enough of a Lisp programmer to really take > that on. > > - org-export-latex.el. This is a tough one because you need > to know both LaTeX and Lisp. And the code is complex, in > part because Org-mode allows to write LaTeX is a relatively > lazy way. Bastien has done a great great job capturing this > into an exporter. However, there are still problems and > bugs, some came up recently on the list. And Bastien > currently does not have the time to contribute consistently. > As a result, I have started to fix some bugs, but this is > really eating too much of my time. This subsystem can also > use feature additions, for example better handling of image > insertion, maybe with captions. I have ideas about this, so > talk to me if you want to help out. > > 4. Try to answer as many messages on the mailing list as you can. > I have been trying hard to make sure that each and every > reasonable question on the list (and this is really the only > kind we get in this amazing community) will be answered. > Doing this still takes a significant fraction of my Org-mode > development time. I will clearly spend less time on this in > the future, in this way also giving others more time to > answer. Again, there are already quite a few people who > regularly *answer* questions, and all I am trying to do here > is encouraging this activity. > > 5. Help developing the Org-mode FAQ by adding useful information > to it yourself. All you need to do is to get acces to Worg, > which will help getting to know git in the process. Worg is > meant to be user edited, so please go wild and add information > and links at will. > > Thanks to you all, for your understanding and help. It is quite a challenge to try out and understand numerous features and facilities already in Org and the pace at which the new changes were coming was unreal and super-human. The backlog of things I wanted to try, understand and absorb in my workflow was consistently increasing. This is not a complaint; I am having so much fun. *BUT* I do not want my favourite developer of my favourite package in my favourite editor to burn out. Please do take it easy. Unfortunately, I can not help with #3 (adopting a subsystem) since I am Lisp-challenged but with rest I will. Sincere thanks for all your time, effort and sacrifices. -- Manish ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: How you can help 2008-10-23 7:35 How you can help Carsten Dominik 2008-10-23 8:12 ` Manish @ 2008-10-23 9:24 ` Sebastian Rose 2008-10-23 10:28 ` Sebastian Rose 2008-10-23 15:23 ` Russell Adams 2 siblings, 1 reply; 25+ messages in thread From: Sebastian Rose @ 2008-10-23 9:24 UTC (permalink / raw) Cc: emacs-orgmode List Should we put a page on worg by this name (see subject of this thread)? We could show, how to turn on debugging, write a good bug report, link to good elisp tutorials and describe how to use the elisp debugger in emacs for simple debugging. If no one stops me, I'll do it these days. As or me, a page like that would be helpfull. I'll be back for report - maybe one of the lisp freaks could add some details later on? Regards, Sebastian Carsten Dominik wrote: > Hi Org users, > > I need to get control over the time I spent on developing > Org-mode. Recently, I have again worked too hard on it, spending > more time than I should, in order to get 6.9 and 6.10 out and to > seize the chance to get the best possible version into Emacs > 23.1. > > However, this is getting out of control, and I am now putting > myself a hard limit of 1 hour per day, clocked by Org-mode, which > will clearly cut in on my development speed and posting rate. > > Here is how you can help me to make the most of that one hour. > > 1. If you report a bug, please try to do as much work as you can > yourself. Before you send it, think if you have collected all > the information you can. There are great examples of good bug > reports on the list already. The best bug reports are, of > course, those that are accompanied by a patch. > > 2. If you are reading the list and see bug reports, consider > putting in 10 minutes, trying to reproduce this problem > yourself, and maybe adding information that might be useful > for me to track down that bug. Again, this is already > happening, I am only trying to encourage this type of > behavior. > > 3. I have recently spent much time on fixing bugs in parts of Org > that I did not write myself, so this is taking much more time > than usually. If you know Emacs Lisp, and I know there are a > number of excellent Lisp programmers on this group, consider > "adopting" one of the following subsystems. By "adoption", I > mean that you make it your mission to deeply understand this > part of Org, so that *you* will be in the position to fix > bugs, taking that off my shoulders. > > - org-publish.el. I think the biggest bugs are out now, but I > am sure this system can be improved quite a bit. If you are > that Lisp programmer trying to take up this task, consider > teaming up with Sebastian Rose who is a great guy, has quite > some understanding of that system and good ideas about it, > but just is not enough of a Lisp programmer to really take > that on. > > - org-export-latex.el. This is a tough one because you need > to know both LaTeX and Lisp. And the code is complex, in > part because Org-mode allows to write LaTeX is a relatively > lazy way. Bastien has done a great great job capturing this > into an exporter. However, there are still problems and > bugs, some came up recently on the list. And Bastien > currently does not have the time to contribute consistently. > As a result, I have started to fix some bugs, but this is > really eating too much of my time. This subsystem can also > use feature additions, for example better handling of image > insertion, maybe with captions. I have ideas about this, so > talk to me if you want to help out. > > 4. Try to answer as many messages on the mailing list as you can. > I have been trying hard to make sure that each and every > reasonable question on the list (and this is really the only > kind we get in this amazing community) will be answered. > Doing this still takes a significant fraction of my Org-mode > development time. I will clearly spend less time on this in > the future, in this way also giving others more time to > answer. Again, there are already quite a few people who > regularly *answer* questions, and all I am trying to do here > is encouraging this activity. > > 5. Help developing the Org-mode FAQ by adding useful information > to it yourself. All you need to do is to get acces to Worg, > which will help getting to know git in the process. Worg is > meant to be user edited, so please go wild and add information > and links at will. > > Thanks to you all, for your understanding and help. > > - Carsten > > > > _______________________________________________ > Emacs-orgmode mailing list > Remember: use `Reply All' to send replies to the list. > Emacs-orgmode@gnu.org > http://lists.gnu.org/mailman/listinfo/emacs-orgmode > ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: How you can help 2008-10-23 9:24 ` Sebastian Rose @ 2008-10-23 10:28 ` Sebastian Rose 0 siblings, 0 replies; 25+ messages in thread From: Sebastian Rose @ 2008-10-23 10:28 UTC (permalink / raw) Cc: emacs-orgmode List Sebastian Rose wrote: > Should we put a page on worg by this name (see subject of this > thread)? We could show, how to turn on debugging, write a good bug > report, link to good elisp tutorials and describe how to use the elisp > debugger in emacs for simple debugging. > > If no one stops me, I'll do it these days. As or me, a page like that > would be helpfull. I'll be back for report - maybe one of the lisp > freaks could add some details later on? Hm - there is a link to the 'Feedback' section of the manual (http://orgmode.org/org.html#Feedback) on http://orgmode.org/worg/org-contribute.php but 1. It's hard to find. One has find worg (http://orgmode.org/worg/), and then click 'How to contribute to Org?' to get to http://orgmode.org/worg/org-contribute.php where the link is. I think I wouldn't follow the link 'How to contribute to Org', if I was to report a bug (maybe it's my bad english, but I lots of people whos english is even worse). 2. While the 'Feedback' section in the manual is as good as the rest of it, it's short and does, for good reasons, not provide more information, than absolutely needed. Suggestions: 1. on Worgs startpage (http://orgmode.org/worg/): add an extra section (as the FIRST one) 'How you can help'. 2. Under that headline, put the link to the 'Feedback' section of the manual. 3. Add more links to one or more pages for tips on debugging etc. 4. Add subdirectory for those debugging tips. We could also add things like coding conventions, notes about the APIs in org etc. to that directory. ....Please comment... 5. I also think of little packages for testing parts of org. Example XHTML-export: When the HTML-exporter didn't work lately, I set up a little directory or testing. On top of that tree I had an Org-file with [[elisp:]] links to click on, like [[elisp:(setq debug-on-error t)]] or [[elisp:(debug-on-entry 'org-export-org-to)]] and one for loading a setup.el in the same directory (primaryly to adjust org-publish-project-alist - in principle, everything could be setup in that file, but needed two different setups for these tests). Now I can just open that Org-file, click on two or three links and get a backtrace, if something is wrong. It's so simple to test the exporter this way, that it takes 30 seconds or even less (on failure :-) With simple trees like that (eventually downloadable as *.tar.bz2) one may test a module with all the little corner cases, which might not be in the Org-files the tester uses for his daily work and without risk. If we encounter a new corner case, we could just add it to the test packages Org-files and document it in the packages index.org (or README.org) to prevent a regression. When working on a module, one could programm against those tests. Creating and maintaining those test packages is a simple thing to do. All it requires is knowledge of USING org and just a tiny little extra. Hence I suppose this work should be done by the community/worgers. It will be much more fun to add tested patches (and more secure to create one)! Regards, Sebastian > Carsten Dominik wrote: >> Hi Org users, >> >> I need to get control over the time I spent on developing >> Org-mode. Recently, I have again worked too hard on it, spending >> more time than I should, in order to get 6.9 and 6.10 out and to >> seize the chance to get the best possible version into Emacs >> 23.1. >> >> However, this is getting out of control, and I am now putting >> myself a hard limit of 1 hour per day, clocked by Org-mode, which >> will clearly cut in on my development speed and posting rate. >> >> Here is how you can help me to make the most of that one hour. >> >> 1. If you report a bug, please try to do as much work as you can >> yourself. Before you send it, think if you have collected all >> the information you can. There are great examples of good bug >> reports on the list already. The best bug reports are, of >> course, those that are accompanied by a patch. >> >> 2. If you are reading the list and see bug reports, consider >> putting in 10 minutes, trying to reproduce this problem >> yourself, and maybe adding information that might be useful >> for me to track down that bug. Again, this is already >> happening, I am only trying to encourage this type of >> behavior. >> >> 3. I have recently spent much time on fixing bugs in parts of Org >> that I did not write myself, so this is taking much more time >> than usually. If you know Emacs Lisp, and I know there are a >> number of excellent Lisp programmers on this group, consider >> "adopting" one of the following subsystems. By "adoption", I >> mean that you make it your mission to deeply understand this >> part of Org, so that *you* will be in the position to fix >> bugs, taking that off my shoulders. >> >> - org-publish.el. I think the biggest bugs are out now, but I >> am sure this system can be improved quite a bit. If you are >> that Lisp programmer trying to take up this task, consider >> teaming up with Sebastian Rose who is a great guy, has quite >> some understanding of that system and good ideas about it, >> but just is not enough of a Lisp programmer to really take >> that on. >> >> - org-export-latex.el. This is a tough one because you need >> to know both LaTeX and Lisp. And the code is complex, in >> part because Org-mode allows to write LaTeX is a relatively >> lazy way. Bastien has done a great great job capturing this >> into an exporter. However, there are still problems and >> bugs, some came up recently on the list. And Bastien >> currently does not have the time to contribute consistently. >> As a result, I have started to fix some bugs, but this is >> really eating too much of my time. This subsystem can also >> use feature additions, for example better handling of image >> insertion, maybe with captions. I have ideas about this, so >> talk to me if you want to help out. >> >> 4. Try to answer as many messages on the mailing list as you can. >> I have been trying hard to make sure that each and every >> reasonable question on the list (and this is really the only >> kind we get in this amazing community) will be answered. >> Doing this still takes a significant fraction of my Org-mode >> development time. I will clearly spend less time on this in >> the future, in this way also giving others more time to >> answer. Again, there are already quite a few people who >> regularly *answer* questions, and all I am trying to do here >> is encouraging this activity. >> >> 5. Help developing the Org-mode FAQ by adding useful information >> to it yourself. All you need to do is to get acces to Worg, >> which will help getting to know git in the process. Worg is >> meant to be user edited, so please go wild and add information >> and links at will. >> >> Thanks to you all, for your understanding and help. >> >> - Carsten >> >> >> >> _______________________________________________ >> Emacs-orgmode mailing list >> Remember: use `Reply All' to send replies to the list. >> Emacs-orgmode@gnu.org >> http://lists.gnu.org/mailman/listinfo/emacs-orgmode >> > > > > _______________________________________________ > Emacs-orgmode mailing list > Remember: use `Reply All' to send replies to the list. > Emacs-orgmode@gnu.org > http://lists.gnu.org/mailman/listinfo/emacs-orgmode > ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: How you can help 2008-10-23 7:35 How you can help Carsten Dominik 2008-10-23 8:12 ` Manish 2008-10-23 9:24 ` Sebastian Rose @ 2008-10-23 15:23 ` Russell Adams 2 siblings, 0 replies; 25+ messages in thread From: Russell Adams @ 2008-10-23 15:23 UTC (permalink / raw) To: emacs-orgmode On Thu, Oct 23, 2008 at 09:35:46AM +0200, Carsten Dominik wrote: > Hi Org users, > > I need to get control over the time I spent on developing > Org-mode. Recently, I have again worked too hard on it, spending > more time than I should, in order to get 6.9 and 6.10 out and to > seize the chance to get the best possible version into Emacs > 23.1. > > However, this is getting out of control, and I am now putting > myself a hard limit of 1 hour per day, clocked by Org-mode, which > will clearly cut in on my development speed and posting rate. Carsten, you need a break sometime too. ;] > - org-export-latex.el. This is a tough one because you need > to know both LaTeX and Lisp. And the code is complex, in > part because Org-mode allows to write LaTeX is a relatively > lazy way. Bastien has done a great great job capturing this > into an exporter. However, there are still problems and > bugs, some came up recently on the list. And Bastien > currently does not have the time to contribute consistently. > As a result, I have started to fix some bugs, but this is > really eating too much of my time. This subsystem can also > use feature additions, for example better handling of image > insertion, maybe with captions. I have ideas about this, so > talk to me if you want to help out. As you've discovered I'm willing to test latex issues, but I don't know Lisp. Whoever picks this up can continue to use me for testing. > Thanks to you all, for your understanding and help. > > - Carsten You still rock! ------------------------------------------------------------------ Russell Adams RLAdams@AdamsInfoServ.com PGP Key ID: 0x1160DCB3 http://www.adamsinfoserv.com/ Fingerprint: 1723 D8CA 4280 1EC9 557F 66E8 1154 E018 1160 DCB3 ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: How you can help
@ 2008-10-23 12:04 Ben Alexander
2008-10-23 13:43 ` Bernt Hansen
` (2 more replies)
0 siblings, 3 replies; 25+ messages in thread
From: Ben Alexander @ 2008-10-23 12:04 UTC (permalink / raw)
To: sebastian_rose; +Cc: emacs-orgmode
Sebastian Rose wrote:
> 5. I also think of little packages for testing parts of org.
I'm curious if you or someone else has any ideas for writing automated
tests for org-mode. I haven't the foggiest idea how someone would
write a test for the parts of org that control what is displayed on
the screen. I mean, when the bug is 'it doesn't look right' how can
you tell?
Perhaps the git repository should have a small collection of small org-
mode files that reproduce certain bugs? If there were some examples
of how to create such a test, then perhaps bug reporters would find it
much easier to create them.
I do see some confusing issues due to different configuration files.
So creating a test file might involve making sure org-mode doesn't
read any configuration (how do you do that?) and possible asking org-
mode to extract all the configuration variables it has right now and
dump them into a test file (...and how do you do that?)
^ permalink raw reply [flat|nested] 25+ messages in thread* Re: How you can help 2008-10-23 12:04 Ben Alexander @ 2008-10-23 13:43 ` Bernt Hansen 2008-10-23 15:04 ` Sebastian Rose 2008-10-23 14:20 ` Sebastian Rose 2008-10-23 15:08 ` Sebastian Rose 2 siblings, 1 reply; 25+ messages in thread From: Bernt Hansen @ 2008-10-23 13:43 UTC (permalink / raw) To: Ben Alexander; +Cc: emacs-orgmode Ben Alexander <bva@alexanderonline.org> writes: > Sebastian Rose wrote: >> 5. I also think of little packages for testing parts of org. > > I'm curious if you or someone else has any ideas for writing automated > tests for org-mode. I haven't the foggiest idea how someone would > write a test for the parts of org that control what is displayed on > the screen. I mean, when the bug is 'it doesn't look right' how can > you tell? > > Perhaps the git repository should have a small collection of small > org- > mode files that reproduce certain bugs? If there were some examples > of how to create such a test, then perhaps bug reporters would find it > much easier to create them. > > I do see some confusing issues due to different configuration files. > So creating a test file might involve making sure org-mode doesn't > read any configuration (how do you do that?) and possible asking org- > mode to extract all the configuration variables it has right now and > dump them into a test file (...and how do you do that?) Running a minimal emacs should suppress custom config files: emacs -q -l yourtest.el Some kind of regression testing framework would be awesome. Org-mode is large enough that this is almost a necessity to keep things stable and bug-free. Maybe something can be put together from the git testing framework and use of emacs -batch to process test org files and verify the output is as expected (with diff or some other tool). -Bernt ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: How you can help 2008-10-23 13:43 ` Bernt Hansen @ 2008-10-23 15:04 ` Sebastian Rose 2008-10-23 16:19 ` Bernt Hansen 0 siblings, 1 reply; 25+ messages in thread From: Sebastian Rose @ 2008-10-23 15:04 UTC (permalink / raw) To: Bernt Hansen; +Cc: Ben Alexander, emacs-orgmode Bernt Hansen <bernt@norang.ca> writes: > Running a minimal emacs should suppress custom config files: > emacs -q -l yourtest.el Added this one to the Clippboard section on new org-tests/index.org in Worg.git. (this section will be temporary...) > Some kind of regression testing framework would be awesome. Org-mode is > large enough that this is almost a necessity to keep things stable and > bug-free. It's big and feature-RICH. > Maybe something can be put together from the git testing framework and > use of emacs -batch to process test org files and verify the output is > as expected (with diff or some other tool). Hey, diff is a good idea!! I didn't take the verification of the output into account yet :-) I just pushed a change of Worgs start page, and added a directory 'org-tests'. I've placed an index.org there, which now is just a collection of ideas (I'm on my day job, so I can't really work on it now). Don't know how often the git repo is published. Bernt and Ben, are you 'worgers' allready? Do you think it makes sense to add snippets and ideas to the new page in Worg? I think while the list great to exchange ideas, it's good to have a place, where all those ideas are destilled to one-liners. Cheers, -- Sebastian Rose, EMMA STIL - mediendesign, Niemeyerstr.6, 30449 Hannover Tel.: +49 (0)511 - 36 58 472 Fax: +49 (0)1805 - 233633 - 11044 mobil: +49 (0)173 - 83 93 417 Email: s.rose@emma-stil.de, sebastian_rose@gmx.de Http: www.emma-stil.de ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: How you can help 2008-10-23 15:04 ` Sebastian Rose @ 2008-10-23 16:19 ` Bernt Hansen 0 siblings, 0 replies; 25+ messages in thread From: Bernt Hansen @ 2008-10-23 16:19 UTC (permalink / raw) To: Sebastian Rose; +Cc: Ben Alexander, emacs-orgmode Sebastian Rose <sebastian_rose@gmx.de> writes: > Don't know how often the git repo is published. > Bernt and Ben, are you 'worgers' allready? I think it's published hourly (maybe) and yes I have access to Worg. -Bernt ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: How you can help 2008-10-23 12:04 Ben Alexander 2008-10-23 13:43 ` Bernt Hansen @ 2008-10-23 14:20 ` Sebastian Rose 2008-10-23 14:50 ` Manish 2008-10-23 14:55 ` Ben Alexander 2008-10-23 15:08 ` Sebastian Rose 2 siblings, 2 replies; 25+ messages in thread From: Sebastian Rose @ 2008-10-23 14:20 UTC (permalink / raw) To: Ben Alexander; +Cc: emacs-orgmode Org-Mode Hi Ben, I cc'ed the list. The tests I described in my email to the list are not automated. The reason for that is my lack of (e)lisp knowledge. BUT, they where easy to handle for non programmers. I think the little test will make it to the worg site this week, when all private data is removed. You could take a look at it by then. And: improve, improve, improve... :-) If you know of someone who knows how to do automated tests in elisp, or some technique, package, whatever, please post it to the list, so we all can take a look at it and comment->decide something. This is _highly_ _appreciated_. Ben Alexander <bva@alexanderonline.org> writes: > Sebastian Rose wrote: >> 5. I also think of little packages for testing parts of org. > > I'm curious if you or someone else has any ideas for writing automated tests for > org-mode. I haven't the foggiest idea how someone would write a test for the > parts of org that control what is displayed on the screen. I mean, when the > bug is 'it doesn't look right' how can you tell? Believe me, my idea is the foggiest of all possible ideas ;-) (so where my ideas of JavaScript some mounth ago), so I won't be of much help, I fear. > Perhaps the git repository should have a small collection of small org- > mode files that reproduce certain bugs? If there were some examples of how to > create such a test, then perhaps bug reporters would find it much easier to > create them. YES! Exactly. Every corner case an Org-file, every bug an Org-file. _DATA_ for testing is something, everyone _can_ provide. But git later, yes, maybe. Since this would need Carsten to think and act on this, I feel Worg is a nice place to start the first expieriments. We need Carstens power for other things (when will Org-mode finaly wash my car? It's repeater-TODO, but nothing happens!!!) Basically, I'd try to keep the the testing as simple as possible, and try to get elisp programmers to help with the code. We should try to keep the hurdles for testers as humble (?) as possible, and put all that's needed to be helpfull on one page in worg. I recently discovered the very unautomated `(print object)' in the elisp reference. Not everything can be done automated, maybe, but if I would have known this stupid `print' function, I would know more about elisp and some files in org already. And it would have been faster and easier to create the two minor patches I wrote. This is, where the 'links to elisp tutorials', 'tipps and tricks', 'emacs debugger' come handy. Willingness to help is no problem, as the Org-community reveals. As for me, it's often a lack of time and/or knowledge, that prevents it. And the aim of all this is to help helping, in means of good and detailed bug reports in the first place. > I do see some confusing issues due to different configuration files. So > creating a test file might involve making sure org-mode doesn't read any > configuration (how do you do that?) and possible asking org- > mode to extract all the configuration variables it has right now and dump them > into a test file (...and how do you do that?) True. Hm - to test without configuration, we already have `emacs -q'. Dumping the variables is just a list of (print var) statements. A (quite) complete list of variables could be extracted with grep? e.g.: grep -r defvar lisp/ Once we have such a list, we could set/reset some/all variables. This will not be perfect, but could work reasonable well. And yes, it would indeed be nice, to have elisp to reliably reset emacs/org configuration, so one could do several different test in sequence. Preferably they would _always_ run and dump all errors to some file or buffer, even if one or more tests fail. I think elsip provides funtionalities to handle those errors and skip to the next test. This would be a huge step. But still, we have to test with different _DATA_ too! The test I described is just a quick hack I did to do the testing, while Carsten was bug-hunting on the other end of my internet connection. The XHTML-export test was much easier to do, then some other tests we might need. In the end, it was a test, automated or not. And I had to do it, because I'm one out of a few who use org to maintain a web-site (locally in my case) and export an entire directory tree with lots of _DATA_, use org-info.js, `#+SETUPFILE' lines, etc. , I believe. This became obvious in this case. I was the one who even noticed the bug, which means, no one else was using recursive XHTML export for a while, or their setup didn't reveal that bug. So clearly I was the one to provide some support for this very part of the system. I can't rely on the assumption, that some maintainer has all possible setups at hand all the time (maybe this was possible years ago). While the testing of the HTML-export was quite simple, it requires a lot of _DATA_. Namly files and directories + different setups, per file setups, #+SETUPFILE, images, with/without org-info.js, extra styles set or not... to have a realistic testing environment (the test didn't have all of these...). A summary could be, that it's nice to provide different setups for maintainers and testers. - *Easy to install* (unpack and done), - *easy to use* ('emacs -q' and click a few [[elisp:]] links to set everything up) - and *easy to download* once they are removed again (a central place on Worg). - *Corner cases*, like the 'empty line before first heading bug' in the LaTeX exporter recently. Regards, -- Sebastian Rose, EMMA STIL - mediendesign, Niemeyerstr.6, 30449 Hannover Tel.: +49 (0)511 - 36 58 472 Fax: +49 (0)1805 - 233633 - 11044 mobil: +49 (0)173 - 83 93 417 Email: s.roseATemma-stil.de, sebastian_roseATgmx.de Http: www.emma-stil.de ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: How you can help 2008-10-23 14:20 ` Sebastian Rose @ 2008-10-23 14:50 ` Manish 2008-10-23 15:46 ` Eric Schulte 2008-10-23 14:55 ` Ben Alexander 1 sibling, 1 reply; 25+ messages in thread From: Manish @ 2008-10-23 14:50 UTC (permalink / raw) To: Sebastian Rose; +Cc: Ben Alexander, emacs-orgmode Org-Mode On Thu, Oct 23, 2008 at 7:50 PM, Sebastian Rose wrote: [snip] > > If you know of someone who knows how to do automated tests in > elisp, or some technique, package, whatever, please post it to the > list, so we all can take a look at it and comment->decide > something. This is _highly_ _appreciated_. We could start here: http://www.emacswiki.org/emacs-fr/UnitTesting#toc2 -- Manish ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: How you can help 2008-10-23 14:50 ` Manish @ 2008-10-23 15:46 ` Eric Schulte 2008-10-23 16:18 ` Avdi Grimm 0 siblings, 1 reply; 25+ messages in thread From: Eric Schulte @ 2008-10-23 15:46 UTC (permalink / raw) To: Manish; +Cc: Ben Alexander, emacs-orgmode Org-Mode Hi, I'd recommend ert.el, it is actively maintained, and I have it on good authority that this the tool of choice for elisp unit testing. http://github.com/ohler/ert/tree/master ,----[from ert.el] | ;;; Commentary: | | ;; ERT is a tool for automated testing in Emacs Lisp. Its main | ;; features are facilities for defining and running test cases and | ;; reporting the results as well as for debugging test failures | ;; interactively. | ;; | ;; The main entry points are `ert-deftest', which is similar to | ;; `defun' but defines a test, and `ert-run-tests-interactively', | ;; which runs tests and offers an interactive interface for inspecting | ;; results and debugging. There is also `ert-run-tests-batch' for | ;; non-interactive use. | ;; | ;; The body of `ert-deftest' forms resembles a function body, but the | ;; additional operators `should', `should-not' and `should-error' are | ;; available. `should' is similar to cl's `assert', but signals a | ;; different error when its condition is violated that is caught and | ;; processed by ERT. In addition, it analyzes its argument form and | ;; records information that helps debugging (`assert' tries to do | ;; something similar when its second argument SHOW-ARGS is true, but | ;; `should' is more sophisticated). For information on `should-not' | ;; and `should-error', see their docstrings. `---- Manish <mailtomanish.sharma@gmail.com> writes: > On Thu, Oct 23, 2008 at 7:50 PM, Sebastian Rose wrote: > [snip] > > > > If you know of someone who knows how to do automated tests in > > elisp, or some technique, package, whatever, please post it to the > > list, so we all can take a look at it and comment->decide > > something. This is _highly_ _appreciated_. > > We could start here: > http://www.emacswiki.org/emacs-fr/UnitTesting#toc2 > > -- Manish > > > _______________________________________________ > Emacs-orgmode mailing list > Remember: use `Reply All' to send replies to the list. > Emacs-orgmode@gnu.org > http://lists.gnu.org/mailman/listinfo/emacs-orgmode ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: How you can help 2008-10-23 15:46 ` Eric Schulte @ 2008-10-23 16:18 ` Avdi Grimm 0 siblings, 0 replies; 25+ messages in thread From: Avdi Grimm @ 2008-10-23 16:18 UTC (permalink / raw) To: emacs-orgmode Org-Mode I've been considering buckling down and doing some more ELisp coding, which would mean org-mode coding because I've become ridiculously dependent on org-mode. If I do, I would only do it in a test-first way, because that's the only way I'll write code anymore; so it would definitely be good to know that unit tests will be be accepted into the codebase. Personally, if I did this I think I would like to use Emacs Lisp Expectations: http://www.emacswiki.org/cgi-bin/emacs/EmacsLispExpectations I'm a big fan of Jay Field's work, I like the way the original Expectations for Ruby is structured, and I like the looks of the EL Expectations API. I'm open to other libraries though - so long as SOME unit-testing standard it accepted. -- Avdi Home: http://avdi.org Developer Blog: http://avdi.org/devblog/ Twitter: http://twitter.com/avdi Journal: http://avdi.livejournal.com ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: How you can help 2008-10-23 14:20 ` Sebastian Rose 2008-10-23 14:50 ` Manish @ 2008-10-23 14:55 ` Ben Alexander 2008-10-23 16:26 ` Sebastian Rose 1 sibling, 1 reply; 25+ messages in thread From: Ben Alexander @ 2008-10-23 14:55 UTC (permalink / raw) To: Sebastian Rose; +Cc: emacs-orgmode Org-Mode Well, I was just looking at http://www.emacswiki.org/emacs-fr/UnitTesting Unfortunately for me, I can't tell if Emacs comes with any builtin framework already, so I downloaded one of the many options listed on that page to my local site-lisp directory: http://www.wanglianghome.org/svn/test/test.el The personal issue I have is that I'm on a Mac, using Aquamacs, and the command line version of emacs is a different binary, so there might be trouble when a test passes at the command line, but fails where it matters to me. I don't even make sure that org-mode is up to date at the command line. I thought it wasn't, but I just checked and now it is. Plus, I don't really understand internals of emacs (like basic internals: I understand point and mark, buffer and file, but not transient mark, indirect buffer, symbols vs strings, window vs tab vs frame) The tutorial I'd need to write a test is one which lays out code I could copy and paste to do the following * setup the test environment - create a test directory - create a sample test.org file - put the cursor in a particular place * run the command we need to test - hit the 'TAB' key, or C-c C-c (some folks might need to be reminded how to find out exactly what command is actually being run when you hit a keystroke. And some of me might need to be told what lisp-code to use when the keystroke runs different commands at different places in a file) - reformat a table - clock in/out - create the agenda - export .html .ics .dvi file * How do we specify the correct result??? - check that the headline folded properly. What's the lisp code for getting the folded string as displayed? - check that the cursor is where it should be? especially when the cursor is near elipses... - check that the agenda is built properly. What's the lisp code for getting the agenda as a string? - check that the exported files are correct. Maybe the right suggestion is to run the export on two different files, so the test can focus on the 'diff' between them. That way different people who run the same test on different hosts can get the same result. More experienced folks must suggest whether it's better to have the expectation of a test be specified as a string in the test code or as a separate file in the test directory. It might depend on context, so feel free to lay out three or four cases. If even one kind of test for each of these 'cases' exists, hopefully we can encourage people to report bugs with the test. I mean, if people are already going through the effort to create the small sample org file to demonstrate their bug, and it's only a short (obvious) step to translating their english description of what should happen into equivalent lisp code.... that seems like a big temptation for someone to do it. Plus it might help them convince themselves that it really is a bug, and not that they are going crazy. I gotta move on for the day here, but I'll keep thinking a bit about this. On 2008-Oct-23, at 15:20, Sebastian Rose wrote: > Hi Ben, > > I cc'ed the list. > > > The tests I described in my email to the list are not automated. The > reason for that is my lack of (e)lisp knowledge. > > BUT, they where easy to handle for non programmers. I think the little > test will make it to the worg site this week, when all private data is > removed. You could take a look at it by then. And: improve, improve, > improve... :-) > > If you know of someone who knows how to do automated tests in elisp, > or > some technique, package, whatever, please post it to the list, so we > all > can take a look at it and comment->decide something. This is _highly_ > _appreciated_. > > > Ben Alexander <bva@alexanderonline.org> writes: >> Sebastian Rose wrote: >>> 5. I also think of little packages for testing parts of org. >> >> I'm curious if you or someone else has any ideas for writing >> automated tests for >> org-mode. I haven't the foggiest idea how someone would write a >> test for the >> parts of org that control what is displayed on the screen. I >> mean, when the >> bug is 'it doesn't look right' how can you tell? > > Believe me, my idea is the foggiest of all possible ideas ;-) (so > where > my ideas of JavaScript some mounth ago), so I won't be of much help, I > fear. > > >> Perhaps the git repository should have a small collection of small >> org- >> mode files that reproduce certain bugs? If there were some >> examples of how to >> create such a test, then perhaps bug reporters would find it much >> easier to >> create them. > > YES! Exactly. Every corner case an Org-file, every bug an Org-file. > _DATA_ for testing is something, everyone _can_ provide. > > But git later, yes, maybe. > > Since this would need Carsten to think and act on this, I feel Worg > is a > nice place to start the first expieriments. We need Carstens power for > other things (when will Org-mode finaly wash my car? It's repeater- > TODO, > but nothing happens!!!) > > Basically, I'd try to keep the the testing as simple as possible, and > try to get elisp programmers to help with the code. We should try to > keep the hurdles for testers as humble (?) as possible, and put all > that's needed to be helpfull on one page in worg. > > I recently discovered the very unautomated `(print object)' in the > elisp > reference. Not everything can be done automated, maybe, but if I would > have known this stupid `print' function, I would know more about elisp > and some files in org already. And it would have been faster and > easier > to create the two minor patches I wrote. This is, where the 'links to > elisp tutorials', 'tipps and tricks', 'emacs debugger' come handy. > > Willingness to help is no problem, as the Org-community reveals. As > for > me, it's often a lack of time and/or knowledge, that prevents it. > > And the aim of all this is to help helping, in means of good and > detailed bug reports in the first place. > > > > >> I do see some confusing issues due to different configuration >> files. So >> creating a test file might involve making sure org-mode doesn't >> read any >> configuration (how do you do that?) and possible asking org- >> mode to extract all the configuration variables it has right now >> and dump them >> into a test file (...and how do you do that?) > > True. > > Hm - to test without configuration, we already have `emacs -q'. > Dumping the variables is just a list of (print var) statements. A > (quite) complete list of variables could be extracted with grep? > > e.g.: > > grep -r defvar lisp/ > > Once we have such a list, we could set/reset some/all variables. > > This will not be perfect, but could work reasonable well. > > And yes, it would indeed be nice, to have elisp to reliably reset > emacs/org configuration, so one could do several different test in > sequence. Preferably they would _always_ run and dump all errors to > some > file or buffer, even if one or more tests fail. > > I think elsip provides funtionalities to handle those errors and > skip to > the next test. This would be a huge step. > > > > > > > But still, we have to test with different _DATA_ too! The test I > described is just a quick hack I did to do the testing, while Carsten > was bug-hunting on the other end of my internet connection. > > The XHTML-export test was much easier to do, then some other tests we > might need. In the end, it was a test, automated or not. And I had > to do > it, because I'm one out of a few who use org to maintain a web-site > (locally in my case) and export an entire directory tree with lots of > _DATA_, use org-info.js, `#+SETUPFILE' lines, etc. , I believe. This > became obvious in this case. I was the one who even noticed the bug, > which means, no one else was using recursive XHTML export for a while, > or their setup didn't reveal that bug. > > So clearly I was the one to provide some support for this very part of > the system. I can't rely on the assumption, that some maintainer has > all > possible setups at hand all the time (maybe this was possible years > ago). > > While the testing of the HTML-export was quite simple, it requires a > lot > of _DATA_. Namly files and directories + different setups, per file > setups, #+SETUPFILE, images, with/without org-info.js, extra styles > set > or not... to have a realistic testing environment (the test didn't > have > all of these...). > > > > > A summary could be, that it's nice to provide different setups for > maintainers and testers. > > - *Easy to install* (unpack and done), > - *easy to use* ('emacs -q' and click a few [[elisp:]] links to set > everything up) > - and *easy to download* once they are removed again (a central > place on > Worg). > - *Corner cases*, like the 'empty line before first heading bug' in > the > LaTeX exporter recently. > > > > > Regards, > > > -- > Sebastian Rose, EMMA STIL - mediendesign, Niemeyerstr.6, 30449 > Hannover > > Tel.: +49 (0)511 - 36 58 472 > Fax: +49 (0)1805 - 233633 - 11044 > mobil: +49 (0)173 - 83 93 417 > Email: s.roseATemma-stil.de, sebastian_roseATgmx.de > Http: www.emma-stil.de ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: How you can help 2008-10-23 14:55 ` Ben Alexander @ 2008-10-23 16:26 ` Sebastian Rose 2008-10-23 16:42 ` Avdi Grimm [not found] ` <D43ED86C-EFD4-4BA8-8528-4F82DB11D625@alexanderonline.org> 0 siblings, 2 replies; 25+ messages in thread From: Sebastian Rose @ 2008-10-23 16:26 UTC (permalink / raw) To: Ben Alexander; +Cc: emacs-orgmode Org-Mode Ben, I added this to the new page on worg. Please tell me: is it OK for you, if I simply add your ideas there?? What do the others think? Ben Alexander <bva@alexanderonline.org> writes: > The tutorial I'd need to write a test is one which lays out code I could copy > and paste to do the following > > * setup the test environment > - create a test directory > - create a sample test.org file > - put the cursor in a particular place > > * run the command we need to test > - hit the 'TAB' key, or C-c C-c (some folks might need to be reminded how to > find out exactly what command is actually being run when you hit a keystroke. > And some of me might need to be told what lisp-code to use when the keystroke > runs different commands at different places in a file) > - reformat a table > - clock in/out > - create the agenda > - export .html .ics .dvi file > * How do we specify the correct result??? > - check that the headline folded properly. What's the lisp code for getting > the folded string as displayed? > - check that the cursor is where it should be? especially when the cursor is > near elipses... > - check that the agenda is built properly. What's the lisp code for getting > the agenda as a string? > - check that the exported files are correct. Maybe the right suggestion is to > run the export on two different files, so the test can focus on the 'diff' > between them. That way different people who run the same test on different > hosts can get the same result. Please, if you find time, add the rest somewhere there? If not, I'll take a closer look tonight and check in what's interesting (which seems to be all of it, actually). http://www.emacswiki.org/emacs-fr/UnitTesting: Does this mean to add code to Org's *.el files? This would mean a lot of work for Carsten :-( OK, once done... But: sh> grep -r defun lisp/ | wc -l 1290 Checking in/out of every function makes 2580 places to check. I can't estimate the effort, actually, because I never did Unit testing. I guess this number much higher than needed? Could anyone with testing expierience estimate? While I think automated tests are a great thing to do and I'm happy to see this reaction on the list :-), I still think we should also provide the 'simple' tests without additional dependencies (when will package management make it into emacs...). And I'd like to keep the effort and impact for Carsten as small as possible. He will have to check each and every patch because of feeling, and actually being, responsible for the code. Another question: will the code compile in emacs without the testing framework installed? Probably yes. Should we try to think of those 'simple test packages' as the more _DATA_ part of other tests? This would mean one could code an automated test without the need to create data too, since we have the data seperated. Again, I'd like to keep the hurdles low for everyone involved. When automated testing turns out to be a foolish easy and rock solid, this could change. * Three basic approaches: 1 Simple data and setup packages. 2 Code, that executes org commands and test for correct output, no changes in Org's code needed. 3 Test framework with changes to Org's code. I'd promote a combination from 1 and 2, and see, how far it goes. DATA +--- d1 +--- d2 +--- d3 TESTS +---t-a +---t-b +---t-c Now one would run test t-a over d1, d2, d3. Still a programmer could use d1 to fix a bug in d1 and finally push the fix for automated testing against d2 and d3. bug X tester verifies (is ist bug? how to reproduce...) tester adds X to dX (message -> list or programmer) programmer installs dX programmer fixes bugX programmer push (message -> list or tester) tester tests against d1, d2, d3 where tester = [human|machine] preferable human (simplicity) Should we track the dependencies between code and tests? Suppose one has worked on a.el. Shure more than one test has to run. Example: org-publish.el contains code common to LaTeX-export, HTML-export... Regards, -- Sebastian Rose, EMMA STIL - mediendesign, Niemeyerstr.6, 30449 Hannover Tel.: +49 (0)511 - 36 58 472 Fax: +49 (0)1805 - 233633 - 11044 mobil: +49 (0)173 - 83 93 417 Email: s.rose@emma-stil.de, sebastian_rose@gmx.de Http: www.emma-stil.de ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: How you can help 2008-10-23 16:26 ` Sebastian Rose @ 2008-10-23 16:42 ` Avdi Grimm 2008-10-23 17:33 ` Sebastian Rose 2008-10-24 18:33 ` Ben Alexander [not found] ` <D43ED86C-EFD4-4BA8-8528-4F82DB11D625@alexanderonline.org> 1 sibling, 2 replies; 25+ messages in thread From: Avdi Grimm @ 2008-10-23 16:42 UTC (permalink / raw) To: Sebastian Rose; +Cc: Ben Alexander, emacs-orgmode Org-Mode A few points, from someone with a decent amount of testing background: * As someone who has contributed to OS projects, the lack of a pre-existing set of regression tests in org-mode is actually *the* most significant blocker to my getting involved in org-mode development. I do all of my coding - both professional and personal - in the context of tests, and not having the existing framework means that before I can start working on new features I first need to spend time yak-shaving on testing infrastructure. * I think there's a lot of over-thinking going on here. Here's the test-first coding discipline in a nutshell: 1. Identify a problem/missing feature. 2. Write a test (possibly using a unit-testing framework to help) which will pass when the bug has been fixed or feature added. This can be as simple as calling a function and validating its return value. 3. Run the test. Verify it FAILS. 4. Write code to make the test PASS. 5. Refactor, if you introduced any code duplication in step 4. 6. Run all the tests, to make sure you didn't break anything else. 7. Commit. If someone would be so kind as to identify a small bug or feature, I would be happy to demonstrate this workflow in the form of code, time permitting. -- Avdi Home: http://avdi.org Developer Blog: http://avdi.org/devblog/ Twitter: http://twitter.com/avdi Journal: http://avdi.livejournal.com ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: How you can help 2008-10-23 16:42 ` Avdi Grimm @ 2008-10-23 17:33 ` Sebastian Rose 2008-10-23 19:10 ` Avdi Grimm 2008-10-24 21:09 ` Tom Breton (Tehom) 2008-10-24 18:33 ` Ben Alexander 1 sibling, 2 replies; 25+ messages in thread From: Sebastian Rose @ 2008-10-23 17:33 UTC (permalink / raw) To: emacs-orgmode Org-Mode "Avdi Grimm" <avdi@avdi.org> writes: > A few points, from someone with a decent amount of testing background: Jippie! Please stay with us for a few days :-D > * As someone who has contributed to OS projects, the lack of a > pre-existing set of regression tests in org-mode is actually *the* > most significant blocker to my getting involved in org-mode > development. I do all of my coding - both professional and personal - > in the context of tests, and not having the existing framework means > that before I can start working on new features I first need to spend > time yak-shaving on testing infrastructure. > > * I think there's a lot of over-thinking going on here. Here's the > test-first coding discipline in a nutshell: > 1. Identify a problem/missing feature. > 2. Write a test (possibly using a unit-testing framework to help) > which will pass when the bug has been fixed or feature added. This can > be as simple as calling a function and validating its return value. > 3. Run the test. Verify it FAILS. > 4. Write code to make the test PASS. > 5. Refactor, if you introduced any code duplication in step 4. > 6. Run all the tests, to make sure you didn't break anything else. > 7. Commit. > > If someone would be so kind as to identify a small bug or feature, I > would be happy to demonstrate this workflow in the form of code, time > permitting. Sounds great! Not shure if we can provide a bug though :-D Bug anyone? Hm - how about: 1. A not yet existent elisp file test-worg.el, that defines a function hello-worg, and a variable lang, and simply puts "Hello Worg"into the minibuffer (if (string= lang "en")) and "Hallo Worg" (if (string= lang "de")) ? Did you work with unit-testing frameworks for elisp already? Which one? Recommendations? Could say something about the effort to get started with such a framework? Can we add it without changing Org's code? If understand 2. correctly - yes? If we can, would we loose quality/speed of tests? How much? Tell us all :-) Oha - I have an appointment (concert) - back tonight! All the best, -- Sebastian Rose, EMMA STIL - mediendesign, Niemeyerstr.6, 30449 Hannover Tel.: +49 (0)511 - 36 58 472 Fax: +49 (0)1805 - 233633 - 11044 mobil: +49 (0)173 - 83 93 417 Email: s.rose@emma-stil.de, sebastian_rose@gmx.de Http: www.emma-stil.de ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: How you can help 2008-10-23 17:33 ` Sebastian Rose @ 2008-10-23 19:10 ` Avdi Grimm 2008-10-24 21:09 ` Tom Breton (Tehom) 1 sibling, 0 replies; 25+ messages in thread From: Avdi Grimm @ 2008-10-23 19:10 UTC (permalink / raw) To: Sebastian Rose; +Cc: emacs-orgmode Org-Mode On Thu, Oct 23, 2008 at 1:33 PM, Sebastian Rose <sebastian_rose@gmx.de> wrote: > Hm - how about: > > 1. A not yet existent elisp file test-worg.el, that defines a function > hello-worg, and a variable lang, and simply puts "Hello Worg"into the > minibuffer (if (string= lang "en")) and "Hallo Worg" (if (string= lang > "de")) ? I'd prefer something that would be of concrete value to someone in using org-mode - made-up scenarios tend to lead to unrealistic tests. > Did you work with unit-testing frameworks for elisp already? > Which one? I have not. I've done test-driven development in Java, C#, Ruby, and C++(!) though, and if I can do it in C++ I think I can get up to speed in any language pretty quickly ;-) > Recommendations? As I said in another comment, I'm liking the look of Emacs Lisp Expectations a *lot*. Partly because I think Jay Fields is bang on the money for how maintain good habits in your tests, and partly because it has a mocking framework to go with it. Testing without a mocking framework is less fun than testing with. However, I haven't actually used it yet, so I can't responsibly recommend it. Let's just say that the discovery of it recently was the thing that got me excited about doing some serious ELisp development. > Could say something about the effort to get started with such a > framework? I think if you have some *running* examples to crib from it shouldn't take someone who is already ELisp-savvy more than a few hours to get the hang of it. > > Can we add it without changing Org's code? If understand 2. correctly - yes? In a dynamic language like ELisp, any testing framework that requires you to change the code just to get it under test isn't worth using. No, you should not have to change org-mode's code to get started with tests. That said, developing in concert with a test suite will definitely tend to influence how you write the code. Generally speaking, developing with tests encourages smaller functions which do one thing and one thing alone and don't have a lot of non-obvious inputs and outputs. Since this is usually the direction you *want* your code to go in anyway, it's a virtuous cycle. -- Avdi Home: http://avdi.org Developer Blog: http://avdi.org/devblog/ Twitter: http://twitter.com/avdi Journal: http://avdi.livejournal.com ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: How you can help 2008-10-23 17:33 ` Sebastian Rose 2008-10-23 19:10 ` Avdi Grimm @ 2008-10-24 21:09 ` Tom Breton (Tehom) 1 sibling, 0 replies; 25+ messages in thread From: Tom Breton (Tehom) @ 2008-10-24 21:09 UTC (permalink / raw) Cc: emacs-orgmode Org-Mode > "Avdi Grimm" <avdi@avdi.org> writes: > > A few points, from someone with a decent amount of testing background: > > Jippie! Please stay with us for a few days :-D [...] As someone with also a decent amount of testing background, I'd like to answer also. > Did you work with unit-testing frameworks for elisp already? > Which one? I co-wrote regress.el (Wayne Mesard wrote the first version) and I wrote rtest. I've written some testing for allout.el before I switched over to org-mode (which is very nice, BTW, Carsten, and provides pretty much everything I wish-listed for allout mode). Of particular interest for testing org-mode, I wrote mockbuf.el, which mocks files and buffers so that tests can deal with files without changing anything permanently. > Recommendations? Hard for me to say objectively. I prefer my rtest but it's in flux right now. I wouldn't recommend the current public version, rtest 3.0 I've designed and half written a better version but I can't really ask you to wait until I get around to making a better version of rtest. There is a development version that works and does nice things; it's messy as hell right now and I'm embarrassed about that. It isn't public but I could put it up if you'll forgive its messy state, It has mockbuf, which I found indispensable in writing tests for allout.el. In fact, I mostly developed it for that, though all the allout-specific code is elsewhere. Also useful for writing tests: safe-equal.el, caselet.el, and select.cl, all in the ancillary directory. I would steer clear of frameworks that provide many of synonyms for equality tests, inequality tests, etc. You already know how to write (not (equal ...)) in elisp. Why learn the name and syntax of something like (shouldnt-be-equal A B)? Having a lot of that makes a package LOOK heftier but doesn't do anything more for you. IMO analyzing test failures should be done by inspecting the test form, not by making you write things in non-standard elisp that could be written in standard elisp. Using ert would mean requiring ewoc, which means tests wouldn't work on emacs 21. Too bad, because it has some appeal. > Could say something about the effort to get started with such a > framework? For rtest, right now you just need (a) the development version, (b) unzip it in your emacs-site-lisp or set the paths and autoloads manually if you prefer, (c) include the following in a file to be tested: (when (not (fboundp 'rtest:deftest)) (defmacro rtest:deftest (&rest dummy)) (defmacro rtest:if-avail (&rest dummy))) Then write tests as: (rtest:deftest function-name ("Docstring" (test-form) ;;Return a boolean ) ;;Repeat as wanted ) That will get you started, but really you'd want to get familiar with the stuff in mockbuf.el - which is so messy right now that I'm embarrassed about it, because it contains both old and new versions of many things in flux. Other advice: Make a directory to hold short examples of files that illustrate "before" and "after" states for the various transformations. Each unit (eg org-agenda.el, org-archive.el etc) should have its own subtree of examples. One thing I'd do early is build the following test helper functions: A macro that runs a test body inside a "let" of all the configuration options set to expected values. A function to compare vector-style strings. `equal' doesn't suffice. rtest has, in ancillary/safe-equal.el, safe-equal-compare-objects and safe-equal-compare-vectors, which IIRC suffice for this. The ability to write only the visible parts to a separate buffer was indispensable in testing allout's folding. You already have it in org-export-visible with the type that keeps the buffer, "?\ ". Then you can just compare contents. You will likely have a lot of what I call "output tests", tests of code that has the property that if the output looks right, it is right. IMO with output tests, you need to make it easy on yourself. TDD is all well and fine, but not if it makes you repeat yourself. If there is available output that looks right, capture it and use it as a base for comparison. That's not "pure" but it's far easier and it works right. If the available output isn't yet right, capture it anyways and tweak it to make it right - but this rarely happens. I automated that sort of thing for a Prolog test harness I wrote, works fine. You'd just write a test of a special type that produced and captured output. It would originally be ungraded, but you would then grade it, and it would remember the correct output and compare against it. I was planning to add the same to rtest and I partly did, but never finished it. Still on my todo list. > Can we add it without changing Org's code? If understand 2. correctly - yes? > rtest's tests like to be in the same file as the code. If you use the form above, the file still compiles and runs even when rtest is not available. It's possible to write the tests in another file, but then either tests aren't grouped together (must be run one by one) or you have to do extra work to group them. If they're in file, you can run every test in a package with `rtest:library'. The other really convenient way to run a test is rtest:defun-at-point. > If we can, would we loose quality/speed of tests? Not at all. Tom Breton (Tehom) PS: One more piece of advice: Make an auto-insert or at least a skeleton that inserts test headers, like the "(when (not (fboundp ..." stuff above. You'll be using it a fair bit, and it's real nice to have it just automatically be there when you start a new file. This advice is geared toward my way of working, though, with the test code in the same file as the functional code. ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: How you can help 2008-10-23 16:42 ` Avdi Grimm 2008-10-23 17:33 ` Sebastian Rose @ 2008-10-24 18:33 ` Ben Alexander 2008-10-24 18:44 ` Avdi Grimm 1 sibling, 1 reply; 25+ messages in thread From: Ben Alexander @ 2008-10-24 18:33 UTC (permalink / raw) To: Avdi Grimm; +Cc: emacs-orgmode Org-Mode On 2008-Oct-23, at 17:42, Avdi Grimm wrote: > > If someone would be so kind as to identify a small bug or feature, I > would be happy to demonstrate this workflow in the form of code, time > permitting. > > Ok, here's your chance. This is something that has bothered me for quite some time, but I've never been able to reliably reproduce the problem. And it's such a small issue. Sometimes, when it looks like my cursor is at the end of a headline and I hit tab, the headline does not cycle. Normally it does. It hit me that the point was not on the headline, it only looked like it. So here's the sample test.org file * First Headline Some body text * Second Headline (goto-char 17) ;that's the end of the first line (org-cycle) At this point the buffer looks like: * First Headline... * Second Headline (goto-char 25) (org-cycle) I think the buffer should look like this * First Headline Some body text * Second Headline but it still looks like: * First Headline... * Second Headline * ABOUT THIS BEHAVIOR I should be honest here. This may not be a bug. As I created this test data, it became a bit more clear that exactly where the cursor is matters a lot here. I *promise* that I've seen this behavior even when the cursor shows up before the ellipses, but in the test case I set up here, I could not make that happen. I also found that if I used the key-chord M-x goto-char <RETURN> 25, everything worked fine. The cursor stayed in front of the ellipses and the tab key worked. But when I used M-: (goto-char 25) the cursor moved to the same line as the ellipses, but after them. and the <TAB> key stopped 'working'. I finally figured this out while playing with git. I switched branches at the command line and needed to perform a 'revert-buffer'. This left the cursor before the ellipses, but unable to properly (org- cycle) using the <TAB> key. More experimenting.... I *think* the revert-buffer command tries to keep point close to the same place, and the org buffer had automatic folding. The bad thing about this for me is that I'll hit the <TAB> key four times trying to make sure it's not just an empty tree. Meanwhile, (org-cycle) has indented the first line of the body, but hasn't shown me the text I'm changing. Then I get confused about why the buffer needs saving, probably 10 minutes later when I've forgotten all about hitting the tab key. But I'm way out of my depth to try to understand the relationship between (revert-buffer) (org-cycle) the arrow keys, and all. I just hit M-< or S-<TAB> and try again. Or C-a, which sometimes works. Sometimes I grab the mouse as a last resort. * ABOUT TESTING ISSUES Just from this example, I'm eager to understand: 1. How can we differentiate between where the cursor appears and the value of point 2. How can we tell what the users sees on the screen versus what the buffer contains So the value of a testing framework is that this: if I'm going to announce to the world that I can't get something to work, (like the tab key), I'm going to make darn sure it's a real problem. I'm going to spend the time to create a small sample file, that reliable creates the problem, and I'm going to try a few different scenarios. But if I start to get lost or confused about which settings I've fiddled with or what is supposed to happen, I'll wander away from the problem and I won't submit the bug. We all lose in that scenario. But if the testing framework exists, and it is lightweight enough for novice emacs users (advanced enough to use M-x but not advanced enough to read lisp) then maybe I would have used it for this example. -Ben ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: How you can help 2008-10-24 18:33 ` Ben Alexander @ 2008-10-24 18:44 ` Avdi Grimm 2008-10-24 19:02 ` Jeff Mickey 0 siblings, 1 reply; 25+ messages in thread From: Avdi Grimm @ 2008-10-24 18:44 UTC (permalink / raw) To: Ben Alexander; +Cc: emacs-orgmode Org-Mode On Fri, Oct 24, 2008 at 2:33 PM, Ben Alexander <bva@alexanderonline.org> wrote: > Ok, here's your chance. This is something that has bothered me for quite > some time, but I've never been able to reliably reproduce the problem. And > it's such a small issue. Thanks! I'll try to take a look tonight or this weekend. > So the value of a testing framework is that this: if I'm going to announce > to the world that I can't get something to work, (like the tab key), I'm > going to make darn sure it's a real problem. I'm going to spend the time to > create a small sample file, that reliable creates the problem, and I'm going > to try a few different scenarios. But if I start to get lost or confused > about which settings I've fiddled with or what is supposed to happen, I'll > wander away from the problem and I won't submit the bug. We all lose in > that scenario. But if the testing framework exists, and it is lightweight > enough for novice emacs users (advanced enough to use M-x but not advanced > enough to read lisp) then maybe I would have used it for this example. For the record, I normally would never expect the user to submit tests. A description of the problem and some sample data is all I expect. Writing the a test to reproduce the behavior is the programmer's responsibility as far as I'm concerned. -- Avdi Home: http://avdi.org Developer Blog: http://avdi.org/devblog/ Twitter: http://twitter.com/avdi Journal: http://avdi.livejournal.com ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: How you can help 2008-10-24 18:44 ` Avdi Grimm @ 2008-10-24 19:02 ` Jeff Mickey 0 siblings, 0 replies; 25+ messages in thread From: Jeff Mickey @ 2008-10-24 19:02 UTC (permalink / raw) To: Avdi Grimm; +Cc: Ben Alexander, emacs-orgmode Org-Mode On Fri, Oct 24, 2008 at 14:44, Avdi Grimm <avdi@avdi.org> wrote: > For the record, I normally would never expect the user to submit > tests. A description of the problem and some sample data is all I > expect. Writing the a test to reproduce the behavior is the > programmer's responsibility as far as I'm concerned. That's the best part of open source software! A surprising amount of the time, the user IS a programmer, and thus is able to provide tests. And it's not that we should require anyone to provide tests before they report a bug, but they should be required to outline the steps to reproduce the behavior, so any programmer could take 5 minutes and quickly whip up a test. ps: I almost have a baby ert framework for org-mode. It doesn't do anything yet, but the design is you write a test, and add it to the org-test-hook to be run by org-test-all. -- . : [ + carpe diem totus tuus + ] : . ^ permalink raw reply [flat|nested] 25+ messages in thread
[parent not found: <D43ED86C-EFD4-4BA8-8528-4F82DB11D625@alexanderonline.org>]
* Re: How you can help [not found] ` <D43ED86C-EFD4-4BA8-8528-4F82DB11D625@alexanderonline.org> @ 2008-10-23 17:12 ` Sebastian Rose 0 siblings, 0 replies; 25+ messages in thread From: Sebastian Rose @ 2008-10-23 17:12 UTC (permalink / raw) To: Ben Alexander; +Cc: emacs-orgmode Org-Mode (cc'ing the list again) Ben Alexander <bva@alexanderonline.org> writes: > On 2008-Oct-23, at 17:26, Sebastian Rose wrote: > >> Ben, >> >> I added this to the new page on worg. Please tell me: is it OK for you, >> if I simply add your ideas there?? >> > Sure put what you want there. Great. > FWIW, I'm brand new (today!) to git and I've cloned Worg today too. But I not a > worger' -- I'm not sure I've read much of it online. So I'll look what else should be added? Hm - this thread is so busy, that I'd say I take a few hours on weekend to sum it up. >> What do the others think? >> >> >> Ben Alexander <bva@alexanderonline.org> writes: >>> way too much >>> >> >> >> Please, if you find time, add the rest somewhere there? >> If not, I'll take a closer look tonight and check in what's interesting >> (which seems to be all of it, actually). >> >> >> http://www.emacswiki.org/emacs-fr/UnitTesting: >> >> Does this mean to add code to Org's *.el files? This would mean a lot of >> work for Carsten :-( OK, once done... >> > > Uh, I hope that's not necessary. I'd hope that the tests could be written > completely independently of any org-code. If the basic idea is to launch emacs > in batch mode, then the test could redefine any function locally, in the test > code, instead of screwing around with what Carsten has. > > If every test is written this way, it'll be very slow to run all the tests. > Maybe that doesn't matter for bug-report tests? But here's where I should shut > up and listen instead of talk. Great! I hope so too!! If slow, it's still no problem to run all those tests before a release or once a day, whatever. Glad to here your interested in keeping impact on Carsten low! Cheers, -- Sebastian Rose, EMMA STIL - mediendesign, Niemeyerstr.6, 30449 Hannover Tel.: +49 (0)511 - 36 58 472 Fax: +49 (0)1805 - 233633 - 11044 mobil: +49 (0)173 - 83 93 417 Email: s.rose@emma-stil.de, sebastian_rose@gmx.de Http: www.emma-stil.de ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: How you can help 2008-10-23 12:04 Ben Alexander 2008-10-23 13:43 ` Bernt Hansen 2008-10-23 14:20 ` Sebastian Rose @ 2008-10-23 15:08 ` Sebastian Rose 2 siblings, 0 replies; 25+ messages in thread From: Sebastian Rose @ 2008-10-23 15:08 UTC (permalink / raw) To: Ben Alexander; +Cc: emacs-orgmode Org-Mode Ben Alexander <bva@alexanderonline.org> writes: > Perhaps the git repository should have a small collection of small org- > mode files that reproduce certain bugs? Ah, stupid me, Worg IS a git repo :-D -- Sebastian Rose, EMMA STIL - mediendesign, Niemeyerstr.6, 30449 Hannover Tel.: +49 (0)511 - 36 58 472 Fax: +49 (0)1805 - 233633 - 11044 mobil: +49 (0)173 - 83 93 417 Email: s.rose@emma-stil.de, sebastian_rose@gmx.de Http: www.emma-stil.de ^ permalink raw reply [flat|nested] 25+ messages in thread
[parent not found: <E1Kt27M-0008Tx-0J@box188.bluehost.com>]
* Re: How you can help [not found] <E1Kt27M-0008Tx-0J@box188.bluehost.com> @ 2008-10-23 16:11 ` Robert Goldman 0 siblings, 0 replies; 25+ messages in thread From: Robert Goldman @ 2008-10-23 16:11 UTC (permalink / raw) To: emacs-orgmode > Date: Thu, 23 Oct 2008 15:55:41 +0100 > From: Ben Alexander <bva@alexanderonline.org> > Subject: Re: [Orgmode] How you can help > To: Sebastian Rose <sebastian_rose@gmx.de> > Cc: emacs-orgmode Org-Mode <emacs-orgmode@gnu.org> > Message-ID: <752DA813-1B66-4FD1-B28E-3C23176BA13D@alexanderonline.org> > Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes > > Well, I was just looking at http://www.emacswiki.org/emacs-fr/UnitTesting > > Unfortunately for me, I can't tell if Emacs comes with any builtin > framework already, so I downloaded one of the many options listed on > that page to my local site-lisp directory: http://www.wanglianghome.org/svn/test/test.el > > The personal issue I have is that I'm on a Mac, using Aquamacs, and > the command line version of emacs is a different binary, so there > might be trouble when a test passes at the command line, but fails > where it matters to me. I don't even make sure that org-mode is up to > date at the command line. I thought it wasn't, but I just checked and > now it is. Plus, I don't really understand internals of emacs (like > basic internals: I understand point and mark, buffer and file, but not > transient mark, indirect buffer, symbols vs strings, window vs tab vs > frame) Actually, if you want, you *can* run Aquamacs from the command line, but it can be a pain to do it. I figured out how to do this when I was trying to use the Makefile for org-mode. I ended up with the following emacs command-line: EMACS=/Applications/Aquamacs\ Emacs.app/Contents/MacOS/Aquamacs\ Emacs and this line for batchmode compiling. Note that I had to augment the standard emacs command-line -q option with Aquamacs' -Q: BATCH=$(EMACS) -batch -Q -q -eval \ "(progn (add-to-list (quote load-path) (expand-file-name \"./lisp/\")) \ (add-to-list (quote load-path) \"$(lispdir)\"))" HTH, r ^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2008-10-24 21:09 UTC | newest]
Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2008-10-23 7:35 How you can help Carsten Dominik
2008-10-23 8:12 ` Manish
2008-10-23 9:24 ` Sebastian Rose
2008-10-23 10:28 ` Sebastian Rose
2008-10-23 15:23 ` Russell Adams
-- strict thread matches above, loose matches on Subject: below --
2008-10-23 12:04 Ben Alexander
2008-10-23 13:43 ` Bernt Hansen
2008-10-23 15:04 ` Sebastian Rose
2008-10-23 16:19 ` Bernt Hansen
2008-10-23 14:20 ` Sebastian Rose
2008-10-23 14:50 ` Manish
2008-10-23 15:46 ` Eric Schulte
2008-10-23 16:18 ` Avdi Grimm
2008-10-23 14:55 ` Ben Alexander
2008-10-23 16:26 ` Sebastian Rose
2008-10-23 16:42 ` Avdi Grimm
2008-10-23 17:33 ` Sebastian Rose
2008-10-23 19:10 ` Avdi Grimm
2008-10-24 21:09 ` Tom Breton (Tehom)
2008-10-24 18:33 ` Ben Alexander
2008-10-24 18:44 ` Avdi Grimm
2008-10-24 19:02 ` Jeff Mickey
[not found] ` <D43ED86C-EFD4-4BA8-8528-4F82DB11D625@alexanderonline.org>
2008-10-23 17:12 ` Sebastian Rose
2008-10-23 15:08 ` Sebastian Rose
[not found] <E1Kt27M-0008Tx-0J@box188.bluehost.com>
2008-10-23 16:11 ` Robert Goldman
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).