Re: Re: Org-mode development

[Adam Spiers <orgmode@adamspiers.org>]

Bastien (bzg@altern.org) wrote:
> David Smith <david.daniel.smith@gmail.com> writes:
> > I couldn't wait any longer so I went ahead and set up a mercurial repository 
> > starting with the current 5.17a. I would have liked to have imported older 
> > history but it was a lot of work so hopefully Carsten can just send them to 
> > me and put them in later.

I also have a mercurial repository which goes back to 5.01, and I
believe Carsten has one with history too.  From what I understand of
mercurial, it's not possible to retroactively add history though.
I've uploaded my repository here in case anyone's interested:

  http://www.adamspiers.org/cgi-bin/hg.cgi/org-pacific/

> This is a good idea insofar that it provides a place where people can
> experiment with Org code freely.  But I'm a bit skeptical over whether
> this will really fit Org's development constraints -- of course, only
> Carsten can decide on this.
> 
> The Linus/Carsten parallel is limited.  The Linux kernel is really
> self-contained, while Org is part of Emacs.  This makes a difference.
> Letting code sneaking in Org is not only a matter of having FSF papers
> signed, but also of making sure that the code fits with general Emacs
> conventions.  This require special attention, and such attention might
> be easier to provide in a somewhat centralized development framework.
>
> > Carsten brought up the very good issue that any patches that get into
> > the official branch need to have copyright assigned to the FSF. 
> >
> > This is easy to handle: there will be a separate repository managed by
> > either Carsten or, if he doesn't want to, by me for merging patches
> > once they have FSF copyright assignment.
> 
> This is a trade-off.  With a repository you need to spend time checking
> about legal issues and - as I pointed out above - about "Emacs-ability"
> of the patches.

I acknowledge your concerns around copyright (in fact I am still
personally waiting to hear back from our company lawyers before I can
submit the FSF papers in total confidence), quality control, and risk
of incurring process overheads - but I'm not sure I agree with your
conclusion ;-)

I think the Linux/org-mode comparison is not *so* limited:

Firstly, like emacs, Linux has its own strict conventions for
determining what is acceptable code and what is not.  Upholding such
quality controls is mostly a political/social challenge independent
of any underlying SCM, and a decentralized model certainly does not
enforce any relaxation of standards.

Secondly, whilst contributions to Linux do not require copyright
assignment to the FSF, the kernel developers have other very
important issues to worry about, such as possible patent
infringement, or accidental inclusion of patches containing
proprietary code.  In fact, as a result of this, one could say that
they face a much bigger logistical challenge than FSF projects, since
every contributor retains copyright on their individual
contributions, so copyright ownership has to be tracked for every
single line of code.  Despite this, they engineered a development
process which enabled the required tracking within the decentralized
approach they desperately needed, by using specially formatted
changelog entries containing Signed-off-by: and Acked-by: headers, as
documented here:

http://git.kernel.org/?p=linux/kernel/git/stable/linux-2.6.23.y.git;a=blob_plain;f=Documentation/SubmittingPatches;hb=HEAD

This works very well because the copyright meta-data is made explicit
on a per-patch basis without the SCM implementation forcing
artificial boundaries between machines which contain "official" code
repositories and those which contain potentially contentious code
with respect to copyright etc.

Thirdly, and with respect to the risk of new processes detracting
from valuable coding time, I am not suggesting that org-mode needs to
adopt such a system, as it would almost certainly be too
heavy-weight.  But surely time needs to be spent checking on legal
and emacs conformism issues regardless of the underlying SCM?  And if
anything, a decentralised implementation could potentially save time
since:

  - It would be possible to automate merging of patches from trusted,
    copyright-safe repositories where so desired, in a similar manner
    to Linus who has the freedom to choose how much time he spends
    reviewing any given patch, ranging from careful analysis through
    to direct acceptance from his most trusted lietenants without any
    review.

  - Developers who have not signed FSF papers can still track the
    upstream branches and even provide proof-of-concept patches via
    publicly accessible repositories in order to help Carsten
    understand the issues being discussed.  (Many of my own submitted
    patches are of this nature - for example the following is
    not a particularly clean way of allowing SPACE to cancel prefix
    sub-keymaps:

      http://www.adamspiers.org/cgi-bin/hg.cgi/org-pacific/rev/8fadf3f7ddda

    but hopefully serves as a good illustration of the required
    behaviour.)

I cannot see how a centralized model could offer this flexibility, so
if anything I would argue that this is a good reason for having a
decentralized framework.

In summary, I do *not* believe that adopting a decentralized approach
would *require* Carsten to change the high-level process of
maintainership - it simply opens up some new possibilities and
provides us all with some very cool tools which would make our lives
easier (see below).

> My feeling is that there will be too much energy spent
> on coordination here -- and I guess Carsten prefers coding.

Well, it does indeed ultimately come down to what our esteemed
maintainer thinks, yes :-)

>  Note: Linus said that git spares you the cost of coordination, but he's
>  really speaking about something very specific: coordinating people thru
>  git vs. coordinating them thru cvs (i.e. handling branches with git
>  vs. handling branches with cvs.)
> 
> Again: having an external repo is a great way to experiment with code.
> And no matter whether Org is developed thru it or not, you can always
> manage the repo and submit interesting patches to the mailing list.

True.  Though having an upstream repository with the official release
history to do short-term forks from would be a huge help with this
approach (rebasing a mini-fork on a newer upstream release is a
nightmare in older tools, and trivial in hg/git).  It would also
enable things like patch bisection (binary search) to quickly locate
the source of regressions, online browsing of repository history,
etc.

> > Carsten's role of shaping and filtering features sounds exactly like
> > Linus's role in Linux and so this development model should work well
> > and scale better then what we've done in the past.
> 
> I think the current development model works okay -- until Carsten feels
> to much pressure and doesn't want to cope with it anymore :)

You could phrase that another way - it works great, but only because
Carsten is superhumanly efficient :-)  And perhaps it's unfair for us
to expect him to maintain these levels of efficiency
forever... though I wouldn't be surprised if he did anyway :-)

> The area where we need to "scale better" is that of providing tutorials,
> screencasts, FAQs, etc.  That fact that Org attracts lots of agile geeks
> shouldn't let us forget about this aspect... and I believe this is where
> a decentralized development model would *really* help.

I'm in favour of that too :-)

> > Hopefully, ease-of-use won't be a problem with mercurial.
> 
> (I've never worked with mercurial yet, but I know git a bit and I think
> the switch is not a sweat.)

Agreed, these days they are pretty similar.

> > I'll be putting my personal patches there for things like trac bugs
> > integration, ical import, taskjuggler export, html export
> > improvements, and css and javascript for the exported html pages.
> 
> Great!  I'll be checking your repo and see whether I can contribute.
> But the example you give confirms that the repo is more a place where to
> put Org-related code rather than a place where to develop Org itself.

It can be for both IMHO ;-)

> For example, I guess add-ons about javascript for the exported HTML
> pages can't live in Emacs...  but will be helpful for some people.
> 
> > Hopefully others will fork mercurial repos as well, and send patches
> > around.

Yep, anything which promotes creativity is good!

> Thanks for bringing this up again!  
> Hopefully my points are not too dim :)

Never :-)