Org-mode as a bug tracker.

Org-mode as a bug tracker.

[Óscar Fuentes]

Right now I'm using Trac (a bug tracker and wiki combo) for bug tracking
and project documentation on my personal projects.

Looking at org-mode it seems a superb system for a this task, at least
for one-person projects:

 * Plain text.

 * No server required.

 * Can be stored on the same VCS than the source code.

 * Great for working off-line.

 * Features such as time tracking and others that Trac does not support
   or does badly with plugins.

 * It's Emacs!

The only downside is the lack of a referenced wiki system: simple links
to revisions (r1010) tickets (#245) etc.

Any reasons why this is not a good idea?

Any example of a similar org-mode usage?

-- 
Óscar

Re: Org-mode as a bug tracker.

[Óscar Fuentes]

Óscar Fuentes <ofv@wanadoo.es> writes:

> Any reasons why this is not a good idea?

Just remembered that time ago, when the bug tracker for Emacs was
discussed, Bastien proposed to use org-mode for it. I argued against
because the Emacs bug database would soon fill dozens of megabytes and
this volume does not fit the philosophy of a text-based database.

So I calculated what amount of plain text would require a replica of my
Trac system: it's about 40 MB, with a few thousands tickets. I could
partitionate this on four unequal parts, though, with the largest being
20 MB and 2000 tickets.

Is this data volume too much for org?

Another nuisance is attached files. This requires an ad-hoc mechanism
and I'm not sure I want them stored along with the source files.

Any experiences on using org for bug tracking is appreciated.

-- 
Óscar

Re: Re: Org-mode as a bug tracker.

[Eric S Fraga]

At Fri, 17 Jul 2009 18:25:11 +0200, Óscar Fuentes wrote:

[...]

> So I calculated what amount of plain text would require a replica of my
> Trac system: it's about 40 MB, with a few thousands tickets. I could
> partitionate this on four unequal parts, though, with the largest being
> 20 MB and 2000 tickets.
> 
> Is this data volume too much for org?
> 

I guess the question would be what you intend to do with the data.
For normal bug tracking, I guess inserting, searching and state (todo)
handling would be the main tasks.  These operations would be either
O(n) or O(1), where n is the size of the file, I would imagine, so 20
MB might not be too bad?

I can't give you any specific data points that fall into that
magnitude but I can say that for files 1% that big, a slow netbook has
no problems even doing agenda views which are more expensive
operations than those mentioned above.

> Another nuisance is attached files. This requires an ad-hoc mechanism
> and I'm not sure I want them stored along with the source files.

Actually, I'm glad you brought this up.  Since moving my org files to
be under version control (and boy does that make a difference to my
peace of mind!), I don't know how to deal with attachments.
Attachments, for me, are usually ephemeral and usually types of
documents I don't particularly like to deal with (typically MS Word or
PPT).

Any suggestions?

eric

Re: Org-mode as a bug tracker.

[Matthew Lundin]

Eric S Fraga <ucecesf@ucl.ac.uk> writes:

> At Fri, 17 Jul 2009 18:25:11 +0200, Óscar Fuentes wrote:
>
>> Another nuisance is attached files. This requires an ad-hoc mechanism
>> and I'm not sure I want them stored along with the source files.
>
> Actually, I'm glad you brought this up.  Since moving my org files to
> be under version control (and boy does that make a difference to my
> peace of mind!), 

Yes. I've come to conclude that version control is a *must* for org files,
given the possibility of accidentally deleting whole trees.

> I don't know how to deal with attachments.
> Attachments, for me, are usually ephemeral and usually types of
> documents I don't particularly like to deal with (typically MS Word or
> PPT).

I believe org-attach achieves this end admirably. When you use
org-attach to "attach" a document to a headline, it will deposit the
file (or, if you prefer, a link to the file) in a location defined by
the variable org-attach-directory. The file and the headline are linked
via a global id property. You can open the file from the heading to
which it is attached by calling org-attach again.

http://orgmode.org/manual/Attachments.html

Obviously, if these files are ephemeral, you can leave the
org-attach-directory outside of your version control system.

Best,
Matt

Re: Org-mode as a bug tracker.

[Eric S Fraga]

At Fri, 17 Jul 2009 13:35:29 -0500, Matthew Lundin wrote:
> 
> > I don't know how to deal with attachments.
> > Attachments, for me, are usually ephemeral and usually types of
> > documents I don't particularly like to deal with (typically MS Word or
> > PPT).
> 
> I believe org-attach achieves this end admirably. When you use
> org-attach to "attach" a document to a headline, it will deposit the
> file (or, if you prefer, a link to the file) in a location defined by
> the variable org-attach-directory. The file and the headline are linked
> via a global id property. You can open the file from the heading to
> which it is attached by calling org-attach again.
> 
> http://orgmode.org/manual/Attachments.html
> 
> Obviously, if these files are ephemeral, you can leave the
> org-attach-directory outside of your version control system.

Thanks for your input on this, Matt.  That approach (using a directory
outside my version control hierarchy) makes sense.  It sort of
emulates what I'm doing already which is to use explicit links to
files.  Time to play...

Re: Re: Org-mode as a bug tracker.

[Bastien]

Óscar Fuentes <ofv@wanadoo.es> writes:

> Óscar Fuentes <ofv@wanadoo.es> writes:
>
>> Any reasons why this is not a good idea?
>
> Just remembered that time ago, when the bug tracker for Emacs was
> discussed, Bastien proposed to use org-mode for it. 

Actually I'm glad you bring this up again, as I want to work on this
proposal again.

There are actually two ideas: 

  (1) use org-mode instead of outline-mode in Emacs internal files
      etc/TODO and admin/FOR-RELEASE.  This is pretty straightforward
      and not a big change.  As a minimal change, it improves the way we
      can navigate through these files, but it opens new possibilities
      about adding milestones, tagging tasks, keep track of the email
      which originated the task, assign them to someone, etc.

  (2) use org-mode as a collective bug tracker.

Let me dwell a bit on the second idea.

A good bug tracker for Emacs would let both users and developers easily
access (read/write) to a constantly updated bugs database.

One way to do this with Org files is the "Worg" way: share Org files
over git (or bzr) and let's people contribute to it.  However, this is
not a good solution for *users*.  Even for developers it's not usable:
people will have to pull the last version of the bug database to check
that they are not working on the same things...  too bad.

Another solution would be to take the Worg road only for publishing the
Org bug database, and take another road for writing stuff into it.  I
think a clever system combining HTML input and mail interactions could
do it:

  - A HTML form would let users fill a bug report that would be add to
    the Org bug database;

  - M-x report-emacs-bug would be sent to a machine able to extract an
    Org subtree from the email and add it to the Org bug database;

  - When a developer is taking any action on a bug (revising, closing,
    etc) he emails the updated version of the task to the system and the
    system takes care of replace the old entry by the new one.

  - Whenever X changes an entry assigned to Y, Y receives an email
    asking for permission about this changes.  If yes, then the change
    is applied to the bug database, if no it isn't.

This is the basic workflow.  Of course, permissions and other issues
could be refined but I think such a system is feasible.   

> I argued against
> because the Emacs bug database would soon fill dozens of megabytes and
> this volume does not fit the philosophy of a text-based database.

I don't think the size of the database would really be an issue for the
system above - but maybe I'm wrong on this.

> Another nuisance is attached files. This requires an ad-hoc mechanism
> and I'm not sure I want them stored along with the source files.

I guess 20MB is because there are many files attached - as Matt
mentioned you can attach files *outside* the bug database so this 
isn't really an issue.

Looking forward reading your ideas on the proposed setup above!

Thanks,

-- 
 Bastien

Re: Org-mode as a bug tracker.

[Óscar Fuentes]

Bastien <bastienguerry@googlemail.com> writes:

>>> Any reasons why this is not a good idea?
>>
>> Just remembered that time ago, when the bug tracker for Emacs was
>> discussed, Bastien proposed to use org-mode for it. 
>
> Actually I'm glad you bring this up again, as I want to work on this
> proposal again.
>
> There are actually two ideas: 
>
>   (1) use org-mode instead of outline-mode in Emacs internal files
>       etc/TODO and admin/FOR-RELEASE.  This is pretty straightforward
>       and not a big change.  As a minimal change, it improves the way we
>       can navigate through these files, but it opens new possibilities
>       about adding milestones, tagging tasks, keep track of the email
>       which originated the task, assign them to someone, etc.

Using org-mode instead of outline-mode is a no-brainer. The only
incovenient is org's complexity. A basic but effective use of org is
straightforward but its extensive documentation may seem daunting for
the occasional user. Maybe a paragraph or two at the beginning of the
file explaining what's required for adding entries and doing simple
queries would help those developers who don't know nor plan to use org
for other uses.

>   (2) use org-mode as a collective bug tracker.
>
> Let me dwell a bit on the second idea.
>
> A good bug tracker for Emacs would let both users and developers easily
> access (read/write) to a constantly updated bugs database.
>
> One way to do this with Org files is the "Worg" way: share Org files
> over git (or bzr) and let's people contribute to it.  However, this is
> not a good solution for *users*.  Even for developers it's not usable:
> people will have to pull the last version of the bug database to check
> that they are not working on the same things...  too bad.
>
> Another solution would be to take the Worg road only for publishing the
> Org bug database, and take another road for writing stuff into it.  I
> think a clever system combining HTML input and mail interactions could
> do it:
>
>   - A HTML form would let users fill a bug report that would be add to
>     the Org bug database;
>
>   - M-x report-emacs-bug would be sent to a machine able to extract an
>     Org subtree from the email and add it to the Org bug database;
>
>   - When a developer is taking any action on a bug (revising, closing,
>     etc) he emails the updated version of the task to the system and the
>     system takes care of replace the old entry by the new one.
>
>   - Whenever X changes an entry assigned to Y, Y receives an email
>     asking for permission about this changes.  If yes, then the change
>     is applied to the bug database, if no it isn't.
>
> This is the basic workflow.  Of course, permissions and other issues
> could be refined but I think such a system is feasible.   

IMAO this setup is more complex and fragile than a conventional bug
tracker. The idea may seem appealing at first for a group of veteran
emacs users (those who insist on managing the bug database via e-mail
because they refuse to use a web browser, for instance) but I'm far from
convinced about its effectiveness.

>> I argued against because the Emacs bug database would soon fill
>> dozens of megabytes and this volume does not fit the philosophy of a
>> text-based database.
>
> I don't think the size of the database would really be an issue for the
> system above - but maybe I'm wrong on this.

I'm afraid you are. Lots of emacs bug reports comprises hundreds of
lines of stack dumps, plus e-mail discussions with lots of quoted text,
etc. Org is great for notes, but is it practical for containing tens of
thousands of bug reports, some of them made of thousands of lines? And
you don't control what's on a bug report, they usually contain all sorts
of text constructs and random characters. How well it would deal with
bug reports about org's itself, containing excerpts from other org
files?  Wouldn't this confuse org?

>> Another nuisance is attached files. This requires an ad-hoc mechanism
>> and I'm not sure I want them stored along with the source files.
>
> I guess 20MB is because there are many files attached - as Matt
> mentioned you can attach files *outside* the bug database so this 
> isn't really an issue.

Nope, the 20MB is the bugs' text alone. But attached files belong to the
tickets and supposedly provide key information, so you can wipe them
away to a place where they are not distributed along with the bug
database.

I think org as a bug tracker may work very well for individual
developers or for small groups, but not for open big projects such as
emacs.

-- 
Óscar

Re: Re: Org-mode as a bug tracker.

[Bastien]

Óscar Fuentes <ofv@wanadoo.es> writes:

> Using org-mode instead of outline-mode is a no-brainer. The only
> incovenient is org's complexity. A basic but effective use of org is
> straightforward but its extensive documentation may seem daunting for
> the occasional user. Maybe a paragraph or two at the beginning of the
> file explaining what's required for adding entries and doing simple
> queries would help those developers who don't know nor plan to use org
> for other uses.

I will write a page on Worg about this.

>> This is the basic workflow.  Of course, permissions and other issues
>> could be refined but I think such a system is feasible.   
>
> IMAO this setup is more complex and fragile than a conventional bug
> tracker. The idea may seem appealing at first for a group of veteran
> emacs users (those who insist on managing the bug database via e-mail
> because they refuse to use a web browser, for instance) but I'm far from
> convinced about its effectiveness.

Aside from the scalability of Org wrt to big bug databases, I'm myself
only 50% convinced it's an effective setup.  I'd be glad to work on the
remaining 50%.

>> I don't think the size of the database would really be an issue for the
>> system above - but maybe I'm wrong on this.
>
> I'm afraid you are. Lots of emacs bug reports comprises hundreds of
> lines of stack dumps, plus e-mail discussions with lots of quoted text,
> etc. Org is great for notes, but is it practical for containing tens of
> thousands of bug reports, some of them made of thousands of lines? And
> you don't control what's on a bug report, they usually contain all sorts
> of text constructs and random characters. How well it would deal with
> bug reports about org's itself, containing excerpts from other org
> files?  Wouldn't this confuse org?

I don't know.  Org is certainly not written for that purpose.  But
cannot the dumps and discussions but attached as files?  If so, the 
Org database would only need links to these files, not the full bug
entry.

> Nope, the 20MB is the bugs' text alone. 

Gee...

> But attached files belong to the
> tickets and supposedly provide key information, so you can wipe them
> away to a place where they are not distributed along with the bug
> database.

Yes.

> I think org as a bug tracker may work very well for individual
> developers or for small groups, but not for open big projects such as
> emacs.

Yes.  In the setup I described in the previous email, no human directly
write anything in an Org buffer, everything is taken care of by scripts.
Which is kinda sick, 'cause Org is for humans.  

But still, I will continue to brainstorm on this, because if Org is so
useful for individual bug databases, there should be a clever and useful
way to *share* these individual databases and have a collective tool.

-- 
 Bastien

Re: Re: Org-mode as a bug tracker.

[JBash]

[-- Attachment #1.1: Type: text/plain, Size: 3537 bytes --]

Not sure if this helps, and I don't know the details or the mechanism, but I
believe that John Wiegley uses org-mode as a bug tracking tool for his
ledger app.

http://wiki.github.com/jwiegley/ledger

On Mon, Jul 20, 2009 at 2:18 AM, Bastien <bastienguerry@googlemail.com>wrote:

> Óscar Fuentes <ofv@wanadoo.es> writes:
>
> > Using org-mode instead of outline-mode is a no-brainer. The only
> > incovenient is org's complexity. A basic but effective use of org is
> > straightforward but its extensive documentation may seem daunting for
> > the occasional user. Maybe a paragraph or two at the beginning of the
> > file explaining what's required for adding entries and doing simple
> > queries would help those developers who don't know nor plan to use org
> > for other uses.
>
> I will write a page on Worg about this.
>
> >> This is the basic workflow.  Of course, permissions and other issues
> >> could be refined but I think such a system is feasible.
> >
> > IMAO this setup is more complex and fragile than a conventional bug
> > tracker. The idea may seem appealing at first for a group of veteran
> > emacs users (those who insist on managing the bug database via e-mail
> > because they refuse to use a web browser, for instance) but I'm far from
> > convinced about its effectiveness.
>
> Aside from the scalability of Org wrt to big bug databases, I'm myself
> only 50% convinced it's an effective setup.  I'd be glad to work on the
> remaining 50%.
>
> >> I don't think the size of the database would really be an issue for the
> >> system above - but maybe I'm wrong on this.
> >
> > I'm afraid you are. Lots of emacs bug reports comprises hundreds of
> > lines of stack dumps, plus e-mail discussions with lots of quoted text,
> > etc. Org is great for notes, but is it practical for containing tens of
> > thousands of bug reports, some of them made of thousands of lines? And
> > you don't control what's on a bug report, they usually contain all sorts
> > of text constructs and random characters. How well it would deal with
> > bug reports about org's itself, containing excerpts from other org
> > files?  Wouldn't this confuse org?
>
> I don't know.  Org is certainly not written for that purpose.  But
> cannot the dumps and discussions but attached as files?  If so, the
> Org database would only need links to these files, not the full bug
> entry.
>
> > Nope, the 20MB is the bugs' text alone.
>
> Gee...
>
> > But attached files belong to the
> > tickets and supposedly provide key information, so you can wipe them
> > away to a place where they are not distributed along with the bug
> > database.
>
> Yes.
>
> > I think org as a bug tracker may work very well for individual
> > developers or for small groups, but not for open big projects such as
> > emacs.
>
> Yes.  In the setup I described in the previous email, no human directly
> write anything in an Org buffer, everything is taken care of by scripts.
> Which is kinda sick, 'cause Org is for humans.
>
> But still, I will continue to brainstorm on this, because if Org is so
> useful for individual bug databases, there should be a clever and useful
> way to *share* these individual databases and have a collective tool.
>
> --
>  Bastien
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Remember: use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
>

[-- Attachment #1.2: Type: text/html, Size: 4606 bytes --]

[-- Attachment #2: Type: text/plain, Size: 204 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Org-mode as a bug tracker.

[Óscar Fuentes]

JBash <bashveank@gmail.com> writes:

> Not sure if this helps, and I don't know the details or the mechanism, but I
> believe that John Wiegley uses org-mode as a bug tracking tool for his
> ledger app.
>
> http://wiki.github.com/jwiegley/ledger

Yup. It's on the file named TODO of the `planning` git branch.

Thanks!

-- 
Óscar

Re: Org-mode as a bug tracker.

[Bastien]

Hi Oscar,

Óscar Fuentes <ofv@wanadoo.es> writes:

> Right now I'm using Trac (a bug tracker and wiki combo) for bug tracking
> and project documentation on my personal projects.
>
> Looking at org-mode it seems a superb system for a this task, at least
> for one-person projects:
>
>  * Plain text.
>
>  * No server required.
>
>  * Can be stored on the same VCS than the source code.
>
>  * Great for working off-line.
>
>  * Features such as time tracking and others that Trac does not support
>    or does badly with plugins.
>
>  * It's Emacs!

Yes, Org-mode matches perfectly the needs for a personal bug tracker.

I use Org for all my tasks and software bugs are just a subset of them.
Or maybe using Org makes me think of "repair the washing machine" as a
bug?  In both case, Org greatly helps tracking stuff.

> The only downside is the lack of a referenced wiki system: simple links
> to revisions (r1010) tickets (#245) etc.

I guess simple revisions can be emulated with notes: C-c C-z
But revisions themselves are not linkable then.

Or a revision can be a new subtree in this task?  Then this 
revision can have its own ID (check org-id.el in contrib/) 
and you can link to the revision.  But that's perhaps too
much structure for bug.

> Any reasons why this is not a good idea?

Not a single one :)

> Any example of a similar org-mode usage?

Maybe have a look at org-mode/ORGWEBPAGE/todo.org - but I guess Carsten
has some other secret todo.org :)

-- 
 Bastien

Re: OrgmodeOrg-mode as a bug tracker.

[Wes Hardaker]

>>>>> On Fri, 17 Jul 2009 17:34:13 +0200, Óscar Fuentes <ofv@wanadoo.es> said:

o> The only downside is the lack of a referenced wiki system: simple links
o> to revisions (r1010) tickets (#245) etc.

You can actually get around this somewhat if you're linking to, say, a
svn web server.  I do this using something like this:

  (setq org-link-abbrev-alist
   '(
     ("nsb"
     . "http://sourceforge.net/tracker/index.php?func=detail&group_id=12694&atid=112694&aid=%s")))

Thus any links like [[nsb:1234]] will link to the right web page for bug
#1234 in the Net-SNMP bug database.  SVN repo pointers, etc, can be done
just as easily.

-- 
\ Wes Hardaker                           http://pontifications.hardakers.net /
 \_____ "In the bathtub of history the truth is harder to hold than ________/
       \_______ the soap, and much more difficult to find." _______/
               \_________ -- Terry Pratchett ______________/
                         \__________________/

Re: OrgmodeOrg-mode as a bug tracker.

[Óscar Fuentes]

Wes Hardaker <wjhns209@hardakers.net> writes:

> o> The only downside is the lack of a referenced wiki system: simple links
> o> to revisions (r1010) tickets (#245) etc.
>
> You can actually get around this somewhat if you're linking to, say, a
> svn web server.  I do this using something like this:
>
>   (setq org-link-abbrev-alist
>    '(
>      ("nsb"
>      . "http://sourceforge.net/tracker/index.php?func=detail&group_id=12694&atid=112694&aid=%s")))
>
> Thus any links like [[nsb:1234]] will link to the right web page for bug
> #1234 in the Net-SNMP bug database.  SVN repo pointers, etc, can be done
> just as easily.

I was thinking along this lines for implementing links to subversion
revisions. However, links to bugs reports are not so easy, because you
must explicitly assign a unique id to each TODO. org-id.el is not very
human-friendly as it generates long ids.

I guess it is possible to fix this with some lisp customizations.  Once
the automatic generation and insertion of a simple (counter-based)
unique id for each new TODO is achieved, linking to bugs becomes easy
using the mechanism you mentioned above.

-- 
Óscar

Re: Re: OrgmodeOrg-mode as a bug tracker.

[Bastien]

Óscar Fuentes <ofv@wanadoo.es> writes:

> Wes Hardaker <wjhns209@hardakers.net> writes:
>
>> o> The only downside is the lack of a referenced wiki system: simple links
>> o> to revisions (r1010) tickets (#245) etc.
>>
>> You can actually get around this somewhat if you're linking to, say, a
>> svn web server.  I do this using something like this:
>>
>>   (setq org-link-abbrev-alist
>>    '(
>>      ("nsb"
>>      . "http://sourceforge.net/tracker/index.php?func=detail&group_id=12694&atid=112694&aid=%s")))
>>
>> Thus any links like [[nsb:1234]] will link to the right web page for bug
>> #1234 in the Net-SNMP bug database.  SVN repo pointers, etc, can be done
>> just as easily.

More generally, using Org as a front-end for other bug tracking online
tools looks more actionnable than using it as a back-end.

> I was thinking along this lines for implementing links to subversion
> revisions. However, links to bugs reports are not so easy, because you
> must explicitly assign a unique id to each TODO. org-id.el is not very
> human-friendly as it generates long ids.

Aren't those ids always hidden, either within a link or within a drawer?

> I guess it is possible to fix this with some lisp customizations.  Once
> the automatic generation and insertion of a simple (counter-based)
> unique id for each new TODO is achieved, linking to bugs becomes easy
> using the mechanism you mentioned above.

I think it's already the case.

-- 
 Bastien