Concerns about community contributor support

Concerns about community contributor support

[Timothy]

Dear all,

Over the last few months I have felt an increasing level of concern over
the lack of response to patches. This email is rather long, but please,
bear with me. The goal is to start a discussion on the problems this
creates, and consider short and long-term solutions.

When both community and maintainer response to new patches is lacking,
many first-time contributors are actively dissuaded from contributing
again. Furthermore, each patch represents a considerable time investment
--- particularly if it's from an individual who is new to the mailing
list / patch workflow. Org-mode is not "done" and still requires the
support of long-term contributors to keep improving, anything that
discourages them from contributing back to the community needs to be
carefully understood and resolved if we want to continue harmoniously.

Take for example Jay Bosamiya's patch from September last year [1]. It
appears to be his first submission to this mailing list, and yet there
has been absolutely no response to it. There are currently 24 other
patches listed on the updates.orgmode.org which have seen no response
from this community, some of which are from first-time contributors.
There are 36 other patches with at least two replies, but yet to be
resolved. Bastien's updates.orgmode.org is fantastic in helping prevent
contributions slip through the cracks, but it is also highlighting the
lack of community response to a significant number of patches.

This mailing list was my first experience with an email+patch based
contribution workflow. Thankfully, I received prompt and friendly
feedback and was guided through the adjustments needed so it could be
merged in a timely manner. Should my patch have been similarly ignored,
I would have been quite disheartened; it is not an overstatement to say
I would likely have written off this mailing list and not tried again.

Simply put, this is not good enough. This does a disservice to those
that have dedicated time and effort to try and better this project only
to be ignored. Not rejected, not even acknowledged, nothing.

It is imperative that this community improves our response to
contributions for the long-term health of this project. Do not take me
to be a doomsayer; I have faith that Org is going to keep on improving
regardless. However, failing to welcome and encourage contributors has a
deleterious effect on the health of the project.

I do not blame the maintainers in the slightest. As Bastien brought up
in a recent worg discussion, as time goes on we find ourselves taking on
more and more life responsibilities. Therefore it's in our best interest
to delegate some of the maintainer responsibilities to consistently
active, and supportive community members to "pass down the torch" so the
community and platform can continue to expand with grace and care.

What can the community do?

I don't know of any silver bullet, but I believe there are some steps
which could help, namely:
+ The development and publication of "reasonable expectations" which
  contributors should have when submitting a patch, and that the
  maintainers should strive to uphold (e.g. "expect a response within
  <some timeframe>").
+ A community effort/sprint to respond to those patches that have been
  seemingly abandoned
+ Onboarding of new maintainers, when reasonable and suitable candidates
  exist (I'd very willingly throw my hat in the ring for consideration).
  If it's too much work, spread it out as much as possible.

If any other ideas come to mind, please share them so we can discuss
them further.

Finally, it's not all bad.

While this discussion has called for some criticism, I don't want to
give the false impression that I think nothing is working and nobody is
supporting contributors. This is not the case at all, there are some
standout individuals one the mailing list who have been fantastic. Kudos
to you all.

My best to everyone,

Timothy

[1] https://orgmode.org/list/CAOywxZg1cBL07THLZXHBBCzm6te2vMtqnmM0w63331gybrjZuw@mail.gmail.com/
[2] https://orgmode.org/list/87h7qi2l2m.fsf@gmail.com/

Re: Concerns about community contributor support

[Thomas S. Dye]

Aloha Timothy,

As a long-time follower of this list and a devoted, if often 
ignorant or confused, user of Org mode, I'd like to give my 
perspective on your concerns, which I find genuine and IMHO 
intended to further the Org mode project.

I was drawn to Org mode when Eric Schulte and Dan Davison were 
implementing Org babel.  At the time, I had dabbled in literate 
programming and was using reproducible research practices in my 
work, so the babel project made sense to me and I was thrilled to 
find a couple of terrific programmers working on what to my mind 
was a beautiful implementation of these ideas.

I knew about Carsten Dominik from his work with RefTeX, which I 
also used in my work,
but got to know him better as the creator and maintainer of Org 
mode.  My impression of Carsten was an indefatigable worker whose 
vision of what Org mode might be kept growing as the user base 
expanded and diversified.

The mailing list was a different place back then, less technical 
and open to more noise than it is today.  It was a place that 
understood the importance of kindness for a collaboration of 
volunteers.  I think the list has done an admirable job of 
maintaining the ethos of kindness, but Org mode development is in 
a new phase that *requires* technique and is quicker to identify 
and filter out noise.  When Bastien took over as maintainer after 
Carsten exhausted himself working on Org mode (my interpretation), 
Nicolas Goaziou took over most of the coding work.  His brief was 
clearly to put the Org mode code into better order, which he did 
with astonishing efficiency and expertise (at least from my often 
ignorant and confused perspective).  His work on the syntax, 
exporter, linter, and manual represents IMHO an outstanding 
contribution to Org mode.  I'm sure there are other important 
contributions that I'm not remembering right now.  My point is 
that the project changed from one that was expanding its own 
vision of what it might be to one that was working to ensure that 
it wouldn't collapse from its own weight.

Kyle Meyer is the next step in this direction, whose efforts have 
helped tame the discussions on the Org mode list with a remarkable 
facility for threading together the various strands of discussion, 
some of which have histories that comprise several years. 
Combined with a command of git, his work has brought the 
discussion into closer contact with the development of the code 
base.

These changes mean that contributions need to be checked for 
contributions to Nicolas' project and also fit into the history of 
discussion and development.  The Org mode project now resembles a 
scholarly discipline that moves slowly and deliberately toward a 
more or less well defined goal.  The days when Carsten would bang 
out a new feature during his train ride home from work are gone.

Bastien did recently call for maintainers, though IIRC he was 
interested in ox-* and ob-* maintainers more than org-* 
maintainers.  If enough good programmers heed his call, then there 
might be some progress on the concerns you raise.  But, my sense 
is that patches to Org mode proper will continue to be adopted 
slowly and deliberately.  It would be a shame if that pace put off 
talented programmers, but my sense FWIW is that the pace might be 
necessary until the code base is fully tamed.

All the best,
Tom

Timothy <tecosaur@gmail.com> writes:

> Dear all,
>
> Over the last few months I have felt an increasing level of 
> concern over
> the lack of response to patches. This email is rather long, but 
> please,
> bear with me. The goal is to start a discussion on the problems 
> this
> creates, and consider short and long-term solutions.
>
> When both community and maintainer response to new patches is 
> lacking,
> many first-time contributors are actively dissuaded from 
> contributing
> again. Furthermore, each patch represents a considerable time 
> investment
> --- particularly if it's from an individual who is new to the 
> mailing
> list / patch workflow. Org-mode is not "done" and still requires 
> the
> support of long-term contributors to keep improving, anything 
> that
> discourages them from contributing back to the community needs 
> to be
> carefully understood and resolved if we want to continue 
> harmoniously.
>
> Take for example Jay Bosamiya's patch from September last year 
> [1]. It
> appears to be his first submission to this mailing list, and yet 
> there
> has been absolutely no response to it. There are currently 24 
> other
> patches listed on the updates.orgmode.org which have seen no 
> response
> from this community, some of which are from first-time 
> contributors.
> There are 36 other patches with at least two replies, but yet to 
> be
> resolved. Bastien's updates.orgmode.org is fantastic in helping 
> prevent
> contributions slip through the cracks, but it is also 
> highlighting the
> lack of community response to a significant number of patches.
>
> This mailing list was my first experience with an email+patch 
> based
> contribution workflow. Thankfully, I received prompt and 
> friendly
> feedback and was guided through the adjustments needed so it 
> could be
> merged in a timely manner. Should my patch have been similarly 
> ignored,
> I would have been quite disheartened; it is not an overstatement 
> to say
> I would likely have written off this mailing list and not tried 
> again.
>
> Simply put, this is not good enough. This does a disservice to 
> those
> that have dedicated time and effort to try and better this 
> project only
> to be ignored. Not rejected, not even acknowledged, nothing.
>
> It is imperative that this community improves our response to
> contributions for the long-term health of this project. Do not 
> take me
> to be a doomsayer; I have faith that Org is going to keep on 
> improving
> regardless. However, failing to welcome and encourage 
> contributors has a
> deleterious effect on the health of the project.
>
> I do not blame the maintainers in the slightest. As Bastien 
> brought up
> in a recent worg discussion, as time goes on we find ourselves 
> taking on
> more and more life responsibilities. Therefore it's in our best 
> interest
> to delegate some of the maintainer responsibilities to 
> consistently
> active, and supportive community members to "pass down the 
> torch" so the
> community and platform can continue to expand with grace and 
> care.
>
> What can the community do?
>
> I don't know of any silver bullet, but I believe there are some 
> steps
> which could help, namely:
> + The development and publication of "reasonable expectations" 
> which
>   contributors should have when submitting a patch, and that the
>   maintainers should strive to uphold (e.g. "expect a response 
>   within
>   <some timeframe>").
> + A community effort/sprint to respond to those patches that 
> have been
>   seemingly abandoned
> + Onboarding of new maintainers, when reasonable and suitable 
> candidates
>   exist (I'd very willingly throw my hat in the ring for 
>   consideration).
>   If it's too much work, spread it out as much as possible.
>
> If any other ideas come to mind, please share them so we can 
> discuss
> them further.
>
> Finally, it's not all bad.
>
> While this discussion has called for some criticism, I don't 
> want to
> give the false impression that I think nothing is working and 
> nobody is
> supporting contributors. This is not the case at all, there are 
> some
> standout individuals one the mailing list who have been 
> fantastic. Kudos
> to you all.
>
> My best to everyone,
>
> Timothy
>
> [1] 
> https://orgmode.org/list/CAOywxZg1cBL07THLZXHBBCzm6te2vMtqnmM0w63331gybrjZuw@mail.gmail.com/
> [2] https://orgmode.org/list/87h7qi2l2m.fsf@gmail.com/


--
Thomas S. Dye
https://tsdye.online/tsdye

Re: Concerns about community contributor support

[Tim Cross]

"Thomas S. Dye" <tsd@tsdye.online> writes:

> Aloha Timothy,
>
> working on Org mode (my interpretation), Nicolas Goaziou took over most of the
> coding work.  His brief was clearly to put the Org mode code into better order,
> which he did with astonishing efficiency and expertise (at least from my often
> ignorant and confused perspective).  His work on the syntax, exporter, linter,
> and manual represents IMHO an outstanding contribution to Org mode.  I'm sure
> there are other important contributions that I'm not remembering right now.  My
> point is that the project changed from one that was expanding its own vision of
> what it might be to one that was working to ensure that it wouldn't collapse
> from its own weight.
>

Totally agree. The work Nicolas has put in to consolidate, stabilise and
clarify the org code base has been critical to the long term viability
of the project.  I very much appreciate the work he has contributed and
the many times he has assisted me in understanding how things work
within org-mode.

> Kyle Meyer is the next step in this direction, whose efforts have helped tame
> the discussions on the Org mode list with a remarkable facility for threading
> together the various strands of discussion, some of which have histories that
> comprise several years. Combined with a command of git, his work has brought the
> discussion into closer contact with the development of the code base.
>

Also agree. Getting the right balance between features, stability and
maintenance is extremely challenging. This is especially so with
org-mode as it has a level of flexibility which makes for a very wide
set of use cases. Identifying what should be part of 'core' and what
would be better off as an extension or add-on is often challenging. What
may seem like a great addition/enhancement for one may be of no benefit
to another or may actually complicate their use case. It can be
challenging to understand and interpret all these different perspectives
and determining the optimal forward direction. 


> These changes mean that contributions need to be checked for contributions to
> Nicolas' project and also fit into the history of discussion and development.
> The Org mode project now resembles a scholarly discipline that moves slowly and
> deliberately toward a more or less well defined goal.  The days when Carsten
> would bang out a new feature during his train ride home from work are gone.

This is a common development path for a successful project. Kent Beck
(extreme/agile development) has been focusing a bit on this with his 3x
development model (eXplore, eXpand, eXtract). (see
https://medium.com/@kentbeck_7670/fast-slow-in-3x-explore-expand-extract-6d4c94a7539).
To some extent, we are in an expand/extract stage for org mode. Focus is
on addressing performance and extracting core functionality - new
'features' need to demonstrate a high level of 'return on investment'.
At this stage, enhancing or extending the functionality is a slower and
more cautious process which requires greater justification than was
common in the 'explore' stage.  

>
> Bastien did recently call for maintainers, though IIRC he was interested in ox-*
> and ob-* maintainers more than org-* maintainers.  If enough good programmers
> heed his call, then there might be some progress on the concerns you raise.
> But, my sense is that patches to Org mode proper will continue to be adopted
> slowly and deliberately.  It would be a shame if that pace put off talented
> programmers, but my sense FWIW is that the pace might be necessary until the
> code base is fully tamed.
>

From my perspective, I see bug fixes applied fairly quickly.
Enhancements and extensions are applied more conservatively, which I
think is the correct approach. 

I suspect the best model for moving forward is for new features and
enhancements to be initially implemented as add on contribution packages
rather than as extensions/enhancement to the core 'org-mode' package.
Such packages, if found to be popular or useful to a large number of
org-mode users could then be considered for inclusion as part of the
main org-mode package. The nature of elisp makes this type of model very
suitable as you can modify almost any aspect of org-mode via add on
packages in a consistent and stable manner and avoid impacting the core
functionality until the extension/enhancement has matured sufficiently.


-- 
Tim Cross

Re: Concerns about community contributor support

[Timothy]

Tim Cross <theophilusx@gmail.com> writes:

>> Nicolas Goaziou [...]
>
> Totally agree. The work Nicolas has put in to consolidate, stabilise and
> clarify the org code base has been critical to the long term viability
> of the project.  I very much appreciate the work he has contributed and
> the many times he has assisted me in understanding how things work
> within org-mode.

>> Kyle Meyer [...]
>
> Also agree. Getting the right balance between features, stability and
> maintenance is extremely challenging. This is especially so with
> org-mode as it has a level of flexibility which makes for a very wide
> set of use cases. Identifying what should be part of 'core' and what
> would be better off as an extension or add-on is often challenging. What
> may seem like a great addition/enhancement for one may be of no benefit
> to another or may actually complicate their use case. It can be
> challenging to understand and interpret all these different perspectives
> and determining the optimal forward direction.

Thank you for these points. They are separate to my concerns about the
lack of response to patches, but worth noting in the overall context of
the development of Org mode.

>> These changes mean that contributions need to be checked for contributions to
>> Nicolas' project and also fit into the history of discussion and development.
>> The Org mode project now resembles a scholarly discipline that moves slowly and
>> deliberately toward a more or less well defined goal.  The days when Carsten
>> would bang out a new feature during his train ride home from work are gone.
>
> This is a common development path for a successful project. Kent Beck
> (extreme/agile development) has been focusing a bit on this with his 3x
> development model (eXplore, eXpand, eXtract). (see
> https://medium.com/@kentbeck_7670/fast-slow-in-3x-explore-expand-extract-6d4c94a7539).
> To some extent, we are in an expand/extract stage for org mode. Focus is
> on addressing performance and extracting core functionality - new
> 'features' need to demonstrate a high level of 'return on investment'.
> At this stage, enhancing or extending the functionality is a slower and
> more cautious process which requires greater justification than was
> common in the 'explore' stage.

Interesting link, thanks. Org being in the expand/extract stage
certainly rings true to me from my exposure.

That said, I don't see why being in this stage of development means we
don't need to acknowledge people's attempted contributions.

> From my perspective, I see bug fixes applied fairly quickly.
> Enhancements and extensions are applied more conservatively, which I
> think is the correct approach.

I'm not sure if this is a deliberate tangent, or a miscommunication in
my original email, but how quickly patches are *applied* is not
something I mentioned at all in my original email.

My concern is centred around the lack of /response/ to people sending
in patches. Radio silence for *six months* after sending in a
contribution seems ridiculous to me.

> I suspect the best model for moving forward is for new features and
> enhancements to be initially implemented as add on contribution packages
> rather than as extensions/enhancement to the core 'org-mode' package.
> Such packages, if found to be popular or useful to a large number of
> org-mode users could then be considered for inclusion as part of the
> main org-mode package. The nature of elisp makes this type of model very
> suitable as you can modify almost any aspect of org-mode via add on
> packages in a consistent and stable manner and avoid impacting the core
> functionality until the extension/enhancement has matured sufficiently.

This is an interesting thought. Putting aside the fact that this is
somewhat tangential to points I wished to discuss, I have a number of
reservations:
+ Many patches are modifying core functionality, and would not be
  suitable as an add-on (e.g. [1], [2], [3], and more)
+ Many patches aren't even being acknowledged. Given this, I am highly
  suspicious that good additions will actually be moved into Org.
+ This feels like it could become a bit of a graveyard of ideas.
+ This complicates the development model.

--
Timothy

[1]: [PATCH] Add org-meta*-final-hook
     https://orgmode.org/list/CAOywxZg1cBL07THLZXHBBCzm6te2vMtqnmM0w63331gybrjZuw@mail.gmail.com/
[2]: [PATCH] ob-C.el: Fix a number a bugs related to table parameters
     https://orgmode.org/list/874kkqao1e.fsf@bzg.fr/
[3]: [PATCH] Fontification for inline src blocks
     https://orgmode.org/list/87pmzf4bd0.fsf@gmail.com/

Re: Concerns about community contributor support

[Tim Cross]

Timothy <tecosaur@gmail.com> writes:

> Tim Cross <theophilusx@gmail.com> writes:
>> This is a common development path for a successful project. Kent Beck
>> (extreme/agile development) has been focusing a bit on this with his 3x
>> development model (eXplore, eXpand, eXtract). (see
>> https://medium.com/@kentbeck_7670/fast-slow-in-3x-explore-expand-extract-6d4c94a7539).
>> To some extent, we are in an expand/extract stage for org mode. Focus is
>> on addressing performance and extracting core functionality - new
>> 'features' need to demonstrate a high level of 'return on investment'.
>> At this stage, enhancing or extending the functionality is a slower and
>> more cautious process which requires greater justification than was
>> common in the 'explore' stage.
>
> Interesting link, thanks. Org being in the expand/extract stage
> certainly rings true to me from my exposure.
>
> That said, I don't see why being in this stage of development means we
> don't need to acknowledge people's attempted contributions.
>

This is probably just a reflection of how a largely community driven
approach works. There isn't really anyone with a responsibility to
respond or reply to a patch within some acceptable 'time'. There is a
very limited number of people who have commit rights and who do apply
patches etc. At this time, priority appears to be given to patches which
fix existing behaviour (bug fixes) over ones which extend or modify
existing behaviour (enhancements and new features). 

I suspect the level of response something receives is a reflection of
the level of relevance or interest to others on the list. Personally, I
don't respond to messages which either

- Don't have relevance to how I use org mode. 

- I don't understand the objective or reason/rationale. 

- I don't agree or believe the idea is a good one. Sometimes I will reply
  if I feel (very subjective) the idea would have negative consequences
  in general or I think there is a simpler alternative solution.
  However, often I will take a wait and see approach before contributing
  anything. 

> My concern is centred around the lack of /response/ to people sending
> in patches. Radio silence for *six months* after sending in a
> contribution seems ridiculous to me.
>

I don't think it is ridiculous. I think it is more a reflection of how
less formal and structured approaches work. I can understand why someone
who has put in the effort to create and submit a patch would appreciate
some feedback, but I'm not sure an expectation or desire for feedback is
sufficient.

Personally, I feel that if you make a patch submission, you sometimes
need to 'sell' the patch. This is especially true if the patch is not a
basic bug fix. Having developed a patch, it is easy to become very close
to it and assume aspects of the patch or what it is proposing are self
evident. It can be challenging to put yourself in the shoes of someone
encountering the patch for the first time and consider how your
submission will come across. Simply submitting the patch and expecting
feedback may not be sufficient and followup posts will be required to
either clarify, remind or encourage feedback. 

> This is an interesting thought. Putting aside the fact that this is
> somewhat tangential to points I wished to discuss, I have a number of
> reservations:
> + Many patches are modifying core functionality, and would not be
>   suitable as an add-on (e.g. [1], [2], [3], and more)

There is no limit on what additional external 'add on' packages can do.
They can easily modify core functionality. This is one of the core
benefits/strengths of lisp languages. I can redefine a core function in
my add on package and provided my package is loaded after the core, my
redefined version of the function will be what is executed when the core
uses that function. For example, I have a heavily 'patched' version of
org mode, but I never actually change the core org code. Instead, I have
my own org configuration which redefines and modifies core functionality
which I wanted changed to suit my requirements.

I'm not totally clear about what sort of acknowledgement you want/expect
for patch submissions. I get the impression it is more than a simple
"thank you for your patch" and rather more detailed review/feedback of
your patch. For the latter case, it is likely more a reflection of
available resources - in particular, people with the necessary elisp
skill to be able to review and provide valid feedback and available time
for those limited resources. My observation, which is just one view, is
that bug fixes get priority and other patches get attention based on a
combination of community interest, available time and the 'squeaky
wheel' principal. This relates to my point about needing to 'sell' your
patch. For example, I had a quick look at some of the patches you
referenced. None of them had any relevance to my use of org mode and
none of them involved org functionality I am familiar with (from an
internals perspective), which means I am not in a position to comment.  

> + This feels like it could become a bit of a graveyard of ideas.

Yes, that is possible. However, I'm not sure that is necessarily a bad
thing. My experience with software development is that around 80% of
ideas never survive. This is partly why it is so important during the
'explore' stage of development to allow/support the rapid addition and
implementation of new ideas. Ideas which seem good on the surface can
often turn out to be less beneficial in reality or have unforeseen
negative consequences which only become evident with broader use.

It is survival of the fittest. Ideas which are really good, which
satisfy a common use case and which avoid adverse impact tend to
survive. Sometimes, this does mean a good idea fails because it wasn't
the right time, it wasn't presented clearly or understood or because it
needed more refinement or additional features/functionality which
doesn't yet exist.

> + Many patches aren't even being acknowledged. Given this, I am highly
>   suspicious that good additions will actually be moved into Org.

This depends on a lot of factors - assignment of copyright, quality of
code, type of addition, community desire for addition etc. My personal
perspective is that we should actively discourage further additions or
extensions to org mode and instead focus on making org mode itself a
core, clear and well documented and maintained mode of common basic
functionality and have external packages which take that common
functionality and wrap/customize/extend it to provide more high-level
functionality. I think it is a mistake to try and make core org-mode
everything to everyone.

I use org mode a lot - daily and it is linked into almost every workflow
I have. However, the actual org mode features I use are quite limited.
For example, I don't publish web sites with org mode or use it to
generate gnuplot graphs or code python blocks. I also don't want
additions, extensions or enhancements to publishing, generation of
gnuplot graphs or enhancements to python source block handling to impact,
alter or otherwise affect what I do. However, every time you modify the
core org-mode package, this potential exists. 

> + This complicates the development model.

I actually feel it would make things clearer. Development of org-mode
itself would focus on bug fixes api refinement and performance
improvements. Everything else would depend on add on packages, which
might adopt completely different development models. 

Like with most things, there are pros and cons. This reply is already
far too long, so I won't go into further detail.

To avoid confusion, I'm not suggesting all is fine and perfect. There is
always room for improvement. However, we need to work within the
resource constraints we have and adjust our expectations to be within
those constraints. We can probably do better with expectation management
and perhaps some more details on worg would be helpful wrt contributing,
acknowledgement and feedback. Maintaining a community takes effort,
especially a community with the diverse backgrounds, languages, cultures
and different social norms as this one. Getting the right balance is
challenging.

While it is good to raise and identify areas we may be failing in or
which need improvement, we also need to try and identify viable
solutions. Along these lines, I wonder if it would be worth having a
section on worg for non-bugfix patches where the community could provide
feedback. This could be as simple as basic "I think this is a good/bad
idea" to more structured feedback, such as code review etc. Personally,
I'd like it if we had something like gitlab (not github) which would
make it easier for people to add/review patches. However, even this
requires resources to maintain. 

--
Tim Cross

Re: Concerns about community contributor support

[David Masterson]

Tim Cross <theophilusx@gmail.com> writes:

> I suspect the best model for moving forward is for new features and
> enhancements to be initially implemented as add on contribution packages
> rather than as extensions/enhancement to the core 'org-mode' package.
> Such packages, if found to be popular or useful to a large number of
> org-mode users could then be considered for inclusion as part of the
> main org-mode package. The nature of elisp makes this type of model very
> suitable as you can modify almost any aspect of org-mode via add on
> packages in a consistent and stable manner and avoid impacting the core
> functionality until the extension/enhancement has matured
> sufficiently.

What is the current status of having a BNF (or...?) parser for Org
files?  In particular, the parser rules that could then be adopted by
tools that use Org files on other systems (iPhone, Android, ...)?  Given
the availability of parser generators (bison...), this would be good for
breaking into smartphones where Emacs doesn't run.

-- 
David Masterson

Re: Concerns about community contributor support

[Gustav Wikström]

You didn't ask me, but since I'm currently here and reading the list I might just give 2c to the topic.

My understanding is that a BNF-grammar is virtually impossible for Org. The org language is ambiguous and writing a context free grammar for it hence is not possible. For reference, see [1]. It deals with the topic, but with markdown instead of Org as the language. Both languages face the same issue.

[1]: https://roopc.net/posts/2014/markdown-cfg/ 

Kindly,
Gustav

________________________________________
From: Emacs-orgmode <emacs-orgmode-bounces+gustav=whil.se@gnu.org> on behalf of David Masterson <dsmasterson92630@outlook.com>
Sent: Monday, April 19, 2021 23:43
To: Tim Cross
Cc: emacs-orgmode@gnu.org
Subject: Re: Concerns about community contributor support

Tim Cross <theophilusx@gmail.com> writes:

> I suspect the best model for moving forward is for new features and
> enhancements to be initially implemented as add on contribution packages
> rather than as extensions/enhancement to the core 'org-mode' package.
> Such packages, if found to be popular or useful to a large number of
> org-mode users could then be considered for inclusion as part of the
> main org-mode package. The nature of elisp makes this type of model very
> suitable as you can modify almost any aspect of org-mode via add on
> packages in a consistent and stable manner and avoid impacting the core
> functionality until the extension/enhancement has matured
> sufficiently.

What is the current status of having a BNF (or...?) parser for Org
files?  In particular, the parser rules that could then be adopted by
tools that use Org files on other systems (iPhone, Android, ...)?  Given
the availability of parser generators (bison...), this would be good for
breaking into smartphones where Emacs doesn't run.

--
David Masterson

Re: Concerns about community contributor support

[David Masterson]

Gustav Wikström <gustav@whil.se> writes:

> You didn't ask me, but since I'm currently here and reading the list I
> might just give 2c to the topic.

Your 2c appreciated.  I'm looking to learn.

> My understanding is that a BNF-grammar is virtually impossible for
> Org. The org language is ambiguous and writing a context free grammar
> for it hence is not possible. For reference, see [1]. It deals with
> the topic, but with markdown instead of Org as the language. Both
> languages face the same issue.

That's a shame.  I'll look at the reference.

> [1]: https://roopc.net/posts/2014/markdown-cfg/ 

Thanks
-- 
David Masterson

Re: Concerns about community contributor support

[Tim Cross]

David Masterson <dsmasterson92630@outlook.com> writes:

> Tim Cross <theophilusx@gmail.com> writes:
>
>> I suspect the best model for moving forward is for new features and
>> enhancements to be initially implemented as add on contribution packages
>> rather than as extensions/enhancement to the core 'org-mode' package.
>> Such packages, if found to be popular or useful to a large number of
>> org-mode users could then be considered for inclusion as part of the
>> main org-mode package. The nature of elisp makes this type of model very
>> suitable as you can modify almost any aspect of org-mode via add on
>> packages in a consistent and stable manner and avoid impacting the core
>> functionality until the extension/enhancement has matured
>> sufficiently.
>
> What is the current status of having a BNF (or...?) parser for Org
> files?  In particular, the parser rules that could then be adopted by
> tools that use Org files on other systems (iPhone, Android, ...)?  Given
> the availability of parser generators (bison...), this would be good for
> breaking into smartphones where Emacs doesn't run.

While not BNF, Tom Gillespie has been working on documentation of the
org grammar as part of a broader objective to make it easier for other
external tools to parse org files. Below is a message he recently posted
to the list.

Note that I think this is a slightly different topic to the
development/maintenance of org-mode. The external packages I was
referring to are still Elisp based packages. Non-Elisp tools which can
parse and possibly edit org files would be a good thing for accessing
org files on other devices and outside of Emacs. However, such tools
will always be more limited because of the complexity of adding things
like multiple export formats, babel tangling of src blocks etc (most of
which was really only viable in Emacs because Emacs already had support
for much of that functionality - a subtle point of org mode often
overlooked is that what it primarily did was take existing Emacs
functionality and 'wrapped' it in a UI layer to provide a more
consistent and 'directed' interface to existing Emacs functionality).

 > From Tom Gillespie <tgbugs@gmail.com>

 > Dear all,
 >    Here is a draft of a formal grammar for Org mode [1]. It is still
 > in a rough state, despite quite a bit of work. However, following some
 > changes to improve performance for parsing real (big) Org files, I
 > think it is time to share it with the community so that we can start
 > to gather feedback. There are a number of opportunities that I have
 > found for simplifying the org grammar (sometimes by extending it to
 > make it more regular, and in the process adding useful features) that
 > are much easier to understand with this grammar in hand as a
 > reference. The grammar itself is implemented using Racket's #lang brag
 > (see [2] for an overview of brag's syntax). I had considered trying to
 > break it up into literate sections in an Org file, but for now decided
 > to leave it as a single file to simplify the development workflow. As
 > a result the full implementation is fairly long [3]. Comments and
 > feedback would be greatly appreciated. Best!
 > Tom
 
 > 1. https://github.com/tgbugs/laundry
 > 2. https://docs.racket-lang.org/brag/#%28part._.The_language%29
 > 3. https://github.com/tgbugs/laundry/blob/master/org-mode/parser.rkt
 
-- 
Tim Cross

Re: Concerns about community contributor support

[Tom Gillespie]

Hi Tim, David, and Gustav,
    I am fairly certain that with only a few exceptions it is possible
to specify a context free grammar for org syntax, followed by a second
pass that deals specifically with markup and a few other forms,
notably the reassembly of things like plain lists. The fact that this
is possible because most org constructs are line oriented.

Just a note that the linked parser.rkt [0] is indeed a BNF describing org
syntax in the same style as a bison/yacc grammar. One of the reasons
why I set out to work on this was precisely so that there could be a
reference that could be consulted by the community when questions
about extended org come up.

There are proposals for new syntax that appear on this list with
terrifying frequency, and they are routinely shot down or simply
ignored for good reason, however it is hard to communicate that to
enthusiastic contributors who have an immediate use case that they
want to solve and share and are unlikely to be aware of side effects.
Having a grammar where such issues can be tested empirically should
provide a significant safeguard while also making it easier for
contributors to play with the grammar and see the issues.

In all my work on the grammar I have found maybe 2 or 3 places where
the grammar could be "extended" but it isn't so much extended as it is
regularized, where some parts of org already parse a more complex
grammar while other very similar parts choose not to. Overall the cost
of not parsing certain forms in certain situations adds complexity
rather than reducing it.

The situation for contribution is also further complicated by the fact
that the elisp implementation of org mode is internally inconsistent
in its behavior with regard to the syntax, so great care has to be
taken if someone tries to make and argument based on the behavior of
one component.

All this to say that the need for a conservative approach to changes
and extensions combined with the internally inconsistent behavior of
different parts of the elisp implementation means that the
introduction of new features is extremely difficult because it is hard
to predict the consequences on other parts of org.

Overcoming this is why I started working on the grammar, because
in the absence of a formal spec for what org should do, it is very hard
to make changes to what it is currently doing without having nasty
side effects.

Best!
Tom

0. https://github.com/tgbugs/laundry/blob/next/laundry/parser.rkt note
the upcoming path change (which I will note in the original thread when
it happens).

PS I'm planning to reply to the main thread as well. My short take is
finding a dedicated and responsive maintainer that can take over from
Bastien is a high priority. The only other thing that might help is to
have some way to track outstanding and closed patches, issues, etc.
that is more accessible than trolling through years worth of posts on
this mailing list, but that is a can of worms that has already been
shot down multiple times.

Re: Concerns about community contributor support

[David Masterson]

Tom Gillespie <tgbugs@gmail.com> writes:

> Hi Tim, David, and Gustav,

Hi

>     I am fairly certain that with only a few exceptions it is possible
> to specify a context free grammar for org syntax, followed by a second
> pass that deals specifically with markup and a few other forms,
> notably the reassembly of things like plain lists. The fact that this
> is possible because most org constructs are line oriented.

What do you think about having multiple Org grammars that parse specific
parts of an Org file and treat the rest as text blobs?  The idea is that
small tools (on smartphones) could concentrate on (say) gathering and
using the info related to one of:
+ task management
+ tangled code
+ Org file options
+ etc.

> Just a note that the linked parser.rkt [0] is indeed a BNF describing org
> syntax in the same style as a bison/yacc grammar. One of the reasons
> why I set out to work on this was precisely so that there could be a
> reference that could be consulted by the community when questions
> about extended org come up.

How complete do you feel this grammar is?

> In all my work on the grammar I have found maybe 2 or 3 places where
> the grammar could be "extended" but it isn't so much extended as it is
> regularized, where some parts of org already parse a more complex
> grammar while other very similar parts choose not to. Overall the cost
> of not parsing certain forms in certain situations adds complexity
> rather than reducing it.

Hmm.

> Overcoming this is why I started working on the grammar, because
> in the absence of a formal spec for what org should do, it is very hard
> to make changes to what it is currently doing without having nasty
> side effects.

Thanks for the effort! This could lead to Org developments on non-Enacs
platforms (smartphones & tablets)

> 0. https://github.com/tgbugs/laundry/blob/next/laundry/parser.rkt note
> the upcoming path change (which I will note in the original thread when
> it happens).

I'll see if I can look at this -- it's been decades since I played with
grammars. 

-- 
David Masterson

Re: Concerns about community contributor support

[Jean Louis]

* David Masterson <dsmasterson92630@outlook.com> [2021-04-20 00:59]:
> What is the current status of having a BNF (or...?) parser for Org
> files?  In particular, the parser rules that could then be adopted by
> tools that use Org files on other systems (iPhone, Android, ...)?  Given
> the availability of parser generators (bison...), this would be good for
> breaking into smartphones where Emacs doesn't run.

Emacs run on Android, LineageOS, Replicant and GNU/Linux mobile
OS-es. I normally install it under Termux. There is also Emacs app
without Termux.

Re: Concerns about community contributor support

[David Masterson]

Jean Louis <bugs@gnu.support> writes:

> * David Masterson <dsmasterson92630@outlook.com> [2021-04-20 00:59]:
>> What is the current status of having a BNF (or...?) parser for Org
>> files?  In particular, the parser rules that could then be adopted by
>> tools that use Org files on other systems (iPhone, Android, ...)?  Given
>> the availability of parser generators (bison...), this would be good for
>> breaking into smartphones where Emacs doesn't run.
>
> Emacs run on Android, LineageOS, Replicant and GNU/Linux mobile
> OS-es. I normally install it under Termux. There is also Emacs app
> without Termux.

But not "iOS". (and I know that's a 4-letter word)

-- 
David Masterson

Re: Concerns about community contributor support

[Timothy]

Hello Thomas, good to hear from you.

Thomas S. Dye <tsd@tsdye.online> writes:

> As a long-time follower of this list and a devoted, if often ignorant or
> confused, user of Org mode, I'd like to give my perspective on your concerns,
> which I find genuine and IMHO intended to further the Org mode project.

Thank you, as a more recent addition to this mailing list I was hoping
to hear from people who have been 'around' longer than I have (~ a year,
for reference).

Ultimately, my hope for this thread is that it may lead to some degree
of improvement in the reception new patches have.

> I was drawn to Org mode when Eric Schulte and Dan Davison were implementing Org
> babel.  At the time, I had dabbled in literate programming and was using
> reproducible research practices in my work, so the babel project made sense to
> me and I was thrilled to find a couple of terrific programmers working on what
> to my mind was a beautiful implementation of these ideas.
>
> I knew about Carsten Dominik from his work with RefTeX, which I also used in my
> work,
> but got to know him better as the creator and maintainer of Org mode.  My
> impression of Carsten was an indefatigable worker whose vision of what Org mode
> might be kept growing as the user base expanded and diversified.
>
> The mailing list was a different place back then, less technical and open to
> more noise than it is today.  It was a place that understood the importance of
> kindness for a collaboration of volunteers.  I think the list has done an
> admirable job of maintaining the ethos of kindness

I also think that the tone and attitude on this mailing list has been
quite good in my experience :)

> , but Org mode development is
> in a new phase that *requires* technique and is quicker to identify and filter
> out noise.

Hmmm, what constitutes noise?

> When Bastien took over as maintainer after Carsten exhausted himself
> working on Org mode (my interpretation), Nicolas Goaziou took over most of the
> coding work.  His brief was clearly to put the Org mode code into better order,
> which he did with astonishing efficiency and expertise (at least from my often
> ignorant and confused perspective).  His work on the syntax, exporter, linter,
> and manual represents IMHO an outstanding contribution to Org mode.  I'm sure
> there are other important contributions that I'm not remembering right now.  My
> point is that the project changed from one that was expanding its own vision of
> what it might be to one that was working to ensure that it wouldn't collapse
> from its own weight.
>
> Kyle Meyer is the next step in this direction, whose efforts have helped tame
> the discussions on the Org mode list with a remarkable facility for threading
> together the various strands of discussion, some of which have histories that
> comprise several years. Combined with a command of git, his work has brought the
> discussion into closer contact with the development of the code base.

Fantastic to get a summary of what Nicolas and Kyles long-term
contributions to Org have been, thanks.

> These changes mean that contributions need to be checked for contributions to
> Nicolas' project and also fit into the history of discussion and development.
> The Org mode project now resembles a scholarly discipline that moves slowly and
> deliberately toward a more or less well defined goal.  The days when Carsten
> would bang out a new feature during his train ride home from work are gone.

I think here there may have been a minor misunderstanding
/miscommunication. Reading this paragraph I get the sense you read my
email as complaining about a delay in merging patches, however this is a
separate ---if related--- point to what I intended to raise: the
lack of /response/ to patches.

1. Were I talking about merging: a more considered development model, as
   you describe above, can certainly see a protracted merge delay.
   However, 6 months for a minor feature addition [1], and 2 months for
   a minor bug fix [2] is not justified by a more considered development
   model IMO.
2. (My main point) Even if development is slower, leaving a first-time
   contributor with /absolutely no response/, i.e. *zero* replies to
   their email *months* after they sent it (see [1] and [2] for example,
   and updates.orgmode.org for more) is not good enough IMO. We should
   be better.

> Bastien did recently call for maintainers, though IIRC he was interested in ox-*
> and ob-* maintainers more than org-* maintainers.  If enough good programmers
> heed his call, then there might be some progress on the concerns you raise.
> But, my sense is that patches to Org mode proper will continue to be adopted
> slowly and deliberately.  It would be a shame if that pace put off talented
> programmers, but my sense FWIW is that the pace might be necessary until the
> code base is fully tamed.

I'm fairly sure this was strictly for ob-* maintainers, otherwise I
would have volunteered for ox-html and ox-latex :P.

Once again, with reference to my earlier paragraph, IMO slowed
development is one thing, not responding at all to attempted
contributors for months on end another. It is the latter which I seek to
improve. I can, have, and will try to help with this myself; but I think
we would benefit from a "community effort" and a discussion on what the
best way to improve this is.

> All the best,
> Tom

[1]: https://orgmode.org/list/CAOywxZg1cBL07THLZXHBBCzm6te2vMtqnmM0w63331gybrjZuw@mail.gmail.com/
[2]: https://orgmode.org/list/CAC38U-fT22jDmOEXcjoqOODWzY61cr-Ny_YgVbo1ibreqoxjGw@mail.gmail.com/

Re: Concerns about community contributor support

[Thomas S. Dye]

Timothy <tecosaur@gmail.com> writes:

>> , but Org mode development is
>> in a new phase that *requires* technique and is quicker to 
>> identify and filter
>> out noise.
>
> Hmmm, what constitutes noise?
>
Good question.  I suppose like most words the meaning changes over 
time.  Early on, posts along the lines of "Wouldn't it be cool if 
Org mode would do this?" were given more space than they seem to 
be today.  Tim Cross points out a project trajectory that appears 
to be common and would explain why.  At a more concrete level, I 
can offer several of my posts to the list :)

>> These changes mean that contributions need to be checked for 
>> contributions to
>> Nicolas' project and also fit into the history of discussion 
>> and development.
>> The Org mode project now resembles a scholarly discipline that 
>> moves slowly and
>> deliberately toward a more or less well defined goal.  The days 
>> when Carsten
>> would bang out a new feature during his train ride home from 
>> work are gone.
>
> I think here there may have been a minor misunderstanding
> /miscommunication. Reading this paragraph I get the sense you 
> read my
> email as complaining about a delay in merging patches, however 
> this is a
> separate ---if related--- point to what I intended to raise: the
> lack of /response/ to patches.
>
> 1. Were I talking about merging: a more considered development 
> model, as
>    you describe above, can certainly see a protracted merge 
>    delay.
>    However, 6 months for a minor feature addition [1], and 2 
>    months for
>    a minor bug fix [2] is not justified by a more considered 
>    development
>    model IMO.
> 2. (My main point) Even if development is slower, leaving a 
> first-time
>    contributor with /absolutely no response/, i.e. *zero* 
>    replies to
>    their email *months* after they sent it (see [1] and [2] for 
>    example,
>    and updates.orgmode.org for more) is not good enough IMO. We 
>    should
>    be better.
>
So, something in between merging (or not) and appearing on the 
public list that Bastien keeps?

> Once again, with reference to my earlier paragraph, IMO slowed
> development is one thing, not responding at all to attempted
> contributors for months on end another. It is the latter which I 
> seek to
> improve. I can, have, and will try to help with this myself; but 
> I think
> we would benefit from a "community effort" and a discussion on 
> what the
> best way to improve this is.
>

What do you think of Tim Cross' suggestion that a way forward is 
for "new features and enhancements to be initially implemented as 
add on contribution packages rather than as extensions/enhancement 
to the core 'org-mode' package"? This sounds good to me, but I'm 
not much of a programmer and lack the ability to suss out its 
ramifications fully.  I can see how it would ease Org mode 
maintenance, though.

All the best,
Tom

--
Thomas S. Dye
https://tsdye.online/tsdye

Re: Concerns about community contributor support

[Timothy]

Thomas S. Dye <tsd@tsdye.online> writes:

>> Hmmm, what constitutes noise?
>>
> Good question.  I suppose like most words the meaning changes over time.  Early
> on, posts along the lines of "Wouldn't it be cool if Org mode would do this?"
> were given more space than they seem to be today.  Tim Cross points out a
> project trajectory that appears to be common and would explain why.  At a more
> concrete level, I can offer several of my posts to the list :)

Follow up: What should be the response to "noise", because I don't think
it should be a cold shoulder.

>> I think here there may have been a minor misunderstanding
>> /miscommunication. Reading this paragraph I get the sense you read my
>> email as complaining about a delay in merging patches, however this is a
>> separate ---if related--- point to what I intended to raise: the
>> lack of /response/ to patches.
>>
>> 1. [snip]
>> 2. [snip]
> So, something in between merging (or not) and appearing on the public list that
> Bastien keeps?

I'll try to clarify my meaning:
+ Patches are sent in
+ The /do/ appear on the public list
+ Months go by without anyone even replying to the patch

I think it's fair to expect something along the lines of: "Not sure
about this, will require further thought", "Looks good, but want to wait
for X to approve", "Not quite sure why this is a good idea, please
explain further", or one of a hundred other suitable cursory responses.

Instead, there are currently 24 patches listed on
https://updates.orgmode.org/#patches which have not received a single
response. Then, there are also a large number of other patches with 1-2
cursory replies that seem stuck in limbo (e.g. [1]), with no signs of
being merged or rejected (better than 0 replies, but still not good).

>> Once again, with reference to my earlier paragraph, IMO slowed
>> development is one thing, not responding at all to attempted
>> contributors for months on end another. It is the latter which I seek to
>> improve. I can, have, and will try to help with this myself; but I think
>> we would benefit from a "community effort" and a discussion on what the
>> best way to improve this is.
>>
>
> What do you think of Tim Cross' suggestion that a way forward is for "new
> features and enhancements to be initially implemented as add on contribution
> packages rather than as extensions/enhancement to the core 'org-mode' package"?
> This sounds good to me, but I'm not much of a programmer and lack the ability to
> suss out its ramifications fully.  I can see how it would ease Org mode
> maintenance, though.

I'm going to reply to Tim Cross' email, but a short version of my
current thoughts is:
+ This feels like a little bit of a tangent from the lack of response
+ Many patches are modifying core functionality, and would not be
  suitable as an add-on (e.g. [2], [3], [4], and more)
+ I'm concerned that good changes could quickly make there way here and
  never make their way into Org

> All the best,
> Tom

--
Timothy

[1]: [PATCH] ob-lilypond: allow user configuration of header-args
     https://orgmode.org/list/CANwLyLNCUDEqrtLe9DxB+xZg9T-DWfmfHzrPMuqcUZLZW347Tg@mail.gmail.com/
[2]: [PATCH] Add org-meta*-final-hook
     https://orgmode.org/list/CAOywxZg1cBL07THLZXHBBCzm6te2vMtqnmM0w63331gybrjZuw@mail.gmail.com/
[3]: [PATCH] ob-C.el: Fix a number a bugs related to table parameters
     https://orgmode.org/list/874kkqao1e.fsf@bzg.fr/
[4]: [PATCH] Fontification for inline src blocks
     https://orgmode.org/list/87pmzf4bd0.fsf@gmail.com/

Re: Concerns about community contributor support

[Thomas S. Dye]

Timothy <tecosaur@gmail.com> writes:

> Thomas S. Dye <tsd@tsdye.online> writes:
>
>>> Hmmm, what constitutes noise?
>>>
>> Good question.  I suppose like most words the meaning changes 
>> over time.  Early
>> on, posts along the lines of "Wouldn't it be cool if Org mode 
>> would do this?"
>> were given more space than they seem to be today.  Tim Cross 
>> points out a
>> project trajectory that appears to be common and would explain 
>> why.  At a more
>> concrete level, I can offer several of my posts to the list :)
>
> Follow up: What should be the response to "noise", because I 
> don't think
> it should be a cold shoulder.
>
Agreed.  I'm trying to put myself in the maintainers' shoes.  They 
volunteer lots of highly skilled time, which I admire greatly.  I 
can imagine a situation where a patch falls outside a maintainer's 
interest and they have difficulty finding the time to understand 
it and offer a meaningful critique.

Historical note: when Carsten took his leave and announced Bastien 
would maintain Org mode, I jumped in noisily to suggest that a 
committee approach might be better.  I couldn't imagine a world 
with two Carstens!  It appears there is a committee of sorts now, 
though I'm in the dark how it is organized.  Assuming there is a 
committee, perhaps addition of an Initial Patch Reviewer might be 
a good idea?

All the best,
Tom

--
Thomas S. Dye
https://tsdye.online/tsdye

Re: Concerns about community contributor support

[Timothy]

Thomas S. Dye <tsd@tsdye.online> writes:

>> Follow up: What should be the response to "noise", because I don't think
>> it should be a cold shoulder.
>>
> Agreed.  I'm trying to put myself in the maintainers' shoes.  They volunteer
> lots of highly skilled time, which I admire greatly.  I can imagine a situation
> where a patch falls outside a maintainer's interest and they have difficulty
> finding the time to understand it and offer a meaningful critique.
>
> Historical note: when Carsten took his leave and announced Bastien would
> maintain Org mode, I jumped in noisily to suggest that a committee approach
> might be better.  I couldn't imagine a world with two Carstens!  It appears
> there is a committee of sorts now, though I'm in the dark how it is organized.
> Assuming there is a committee, perhaps addition of an Initial Patch Reviewer
> might be a good idea?

I desperately need to head to bed, so I'll make this quick, but I think
the unclear structure doesn't help.

Often I'm at a loss as to whether a patch has been left for Bastien to
take a look at, overlooked, rejected without comment, or what.*
Crucially, in any of those cases I think the contributor deserves to
know.

I think responding to a first-time contribution with silence is more
likely to push someone away than any other effect.

Org development may be slowing down/refocusing, but I don't think that
means we should disregard new volunteered effort.

A clearly layed out structure and set of roles sounds like it could be
helpful to me.

> All the best,
> Tom

--
Timothy

* I happen to be currently feeling this with the set of patches I
  recently sent in. One was responded to and merged, another had a burst
  of replies but seems to be left in limbo/waiting for Bastien?, and
  then I have 4 others which I am just *hoping* will eventually be
  noticed / responded to. It's not a good feeling, and it's part of
  whats prompted me to send the original email.

Re: Concerns about community contributor support

[Eric S Fraga]

Hello all,

I've avoided saying anything in this discussion but not from lack of
empathy with the initial post.  Many valid points have been made in the
thread and I understand the frustrations.  My own view is that org is
now at a different stage than it was some years ago.  It is a
feature-full package which generally works well for a very *large* set
of use cases.  As a result, it is being used by many people and so is no
longer a niche product.

And, hence:

On Saturday, 17 Apr 2021 at 23:29, Thomas S. Dye wrote:
> But, my sense is that patches to Org mode proper will continue to be
> adopted slowly and deliberately.

and this is as it should be.  I *rely* on org for my work these days.  I
would not want the type of chaotic development we had in the early days,
development that would affect the stability of the package.  New
features need to be considered very carefully.

But, also, as has also been said: the "maintainers" are volunteers and
do have other things to do.  Stating that there is an expectation for
them to answer within a particular time frame is not fair.

If there is a feature *you* need that is not there, the beauty of Emacs
is that you can have that feature, if you have implemented it,
regardless of whether it is accepted in the main org package.  A large
part of my org environment is code that I have written myself to meet my
needs; my org specific config file is 3000 lines.  Some bits along the
way have migrated into org or have informed org features but I can work
the way I want to or need to regardless of whether the features are in
the main code or in my own config.

The excellent work that was done in creating version 9 (or maybe 8?) in
providing a wide range of hooks and filters means that practially
everything is customisable without requiring changes to org itself.

And this leads back to the first point: I want org to exhibit a certain
level of stability now as otherwise much of my workflow would break.  I
suspect many others have this same requirement.  And the maintainers are
very good at avoiding breakage when new features are accepted but this
takes time to evaluate the impact of those new features.

thank you,
eric

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4.4-254-g37749c

Re: Concerns about community contributor support

[Timothy]

Hi Eric,

Thanks for writing in and sharing your thoughts. I have some specific
comments that you may find below, but more generally: I get the
impression you approached this from the view of Org development and
patch merging.

In short, while I appreciate that Org development should be a considered
process and that maintainer time is limited --- I think we could improve
how well we communicate this to contributors, and maybe even our
treatment of contributions.

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> Hello all,
>
> I've avoided saying anything in this discussion but not from lack of
> empathy with the initial post.  Many valid points have been made in the
> thread and I understand the frustrations.  My own view is that org is
> now at a different stage than it was some years ago.  It is a
> feature-full package which generally works well for a very *large* set
> of use cases.  As a result, it is being used by many people and so is no
> longer a niche product.

I can't say I see how being used by a large number of people and growing
beyond a niche product means that we can't set expectations for patches
and/or responsiveness. Though I see you go in a slightly different
direction below...

> And, hence:
>
> On Saturday, 17 Apr 2021 at 23:29, Thomas S. Dye wrote:
>> But, my sense is that patches to Org mode proper will continue to be
>> adopted slowly and deliberately.
>
> and this is as it should be.  I *rely* on org for my work these days.  I
> would not want the type of chaotic development we had in the early days,
> development that would affect the stability of the package.  New
> features need to be considered very carefully.

Indeed, but a lot don't seem considered, they just seem ignored.
Particularly with a lack of communication, this can leave the
contributor wondering whether they need to resend there email, bump it,
wait for a particular maintainer to have a look at it, explain the
intent further, etc. and I don't think leaving such uncertainty to grow
is very nice.

> But, also, as has also been said: the "maintainers" are volunteers and
> do have other things to do.  Stating that there is an expectation for
> them to answer within a particular time frame is not fair.

Maybe I'm not being entirely reasonable, but I would have thought
something like "a cursory response within two months of submitting your
patch" wouldn't be too hard to uphold; particularly when a cursory
response could just be "we've been rather busy as of late, we'll get
back to you on this in the future".

> If there is a feature *you* need that is not there, the beauty of Emacs
> is that you can have that feature, if you have implemented it,
> regardless of whether it is accepted in the main org package.  A large
> part of my org environment is code that I have written myself to meet my
> needs; my org specific config file is 3000 lines.  Some bits along the
> way have migrated into org or have informed org features but I can work
> the way I want to or need to regardless of whether the features are in
> the main code or in my own config.
>
> The excellent work that was done in creating version 9 (or maybe 8?) in
> providing a wide range of hooks and filters means that practially
> everything is customisable without requiring changes to org itself.
>
> And this leads back to the first point: I want org to exhibit a certain
> level of stability now as otherwise much of my workflow would break.  I
> suspect many others have this same requirement.  And the maintainers are
> very good at avoiding breakage when new features are accepted but this
> takes time to evaluate the impact of those new features.

I too appreciate the versatility of elisp, and the way Org has been
constructed that allow it to be modified so heavily by the end user ---
I should know, I have ~4k lines of Org modification in my config.

However, this does not preclude good ideas for Org modification being
merged in. If I have a bugfix, or a very useful modification to Org,
that's great for me, but it's good to share the improvement upstream.
Isn't this a large part of the benefit of an open source development
model?

And when a patch does need to be carefully considered over a period of
months or years, surely it's good to communicate that to the author
instead of leaving them with silence?

Please let me know what your thoughts are,

Timothy.

Re: Concerns about community contributor support

[Gustav Wikström]

Hi Tim,

Another data point from me. I agree with your concerns, although they are difficult to solve! Since we're talking about voluntary work and non-paid work. And maintenance can take a lot of time and effort.

This is not the first time this topic comes up on the list. Both you and me took part of the discussion after the 9.4.2 release, that hinted at similar topics [1]. And since Bastien has been in the process of handing over the maintenance role to someone else for quite some time now, it's not strange that the issue will continue to resurface until that process is done.

What I take away from this thread is mainly one thing: Another hand raised, eager to take on community work! Since you mentioned that you're interested. The more the merrier! Open source is tough though. It's very much a meritocracy. But by doing stuff that is considered to be good, and good stuff will come back to you.

Org mode isn't "finished". And I for one hope the community can continue to surprise with new, nice functionality. Even though some are perfectly happy where it is right now. In my world this is dual property of stability and extensibility is enabled by refactoring into a stable, small(er), less entangled and functional core while also encouraging extension packages/modes. Like org-roam and the likes.

A suggestion from me, fwiw, and if you're serious about getting more involved in the community, is to see if Bastien has some time to discuss this maintenance transition, and to see if there is anywhere where you can fit into that picture. But start small. ;)

I'm very interested in hearing more on how the thoughts are going in that department - and if there are others who have similar thoughts as you in terms of maybe putting more time into the project. But maybe don't see how help out, and with what.

Personally I'm also still hoping for "Org mode maintainer/advocate/whatever" to be something that someone out there can be occupied full time with. The value of the ideas and the software surely is there to warrant it. The question is only how to make it interesting enough for people to help out with funding!

[1]: https://lists.gnu.org/archive/html/emacs-orgmode/2020-12/msg00511.html

Best,
Gustav
________________________________________
From: Emacs-orgmode <emacs-orgmode-bounces+gustav=whil.se@gnu.org> on behalf of Timothy <tecosaur@gmail.com>
Sent: Friday, April 16, 2021 20:43
To: org-mode-email
Subject: Concerns about community contributor support

Dear all,

Over the last few months I have felt an increasing level of concern over
the lack of response to patches. This email is rather long, but please,
bear with me. The goal is to start a discussion on the problems this
creates, and consider short and long-term solutions.

When both community and maintainer response to new patches is lacking,
many first-time contributors are actively dissuaded from contributing
again. Furthermore, each patch represents a considerable time investment
--- particularly if it's from an individual who is new to the mailing
list / patch workflow. Org-mode is not "done" and still requires the
support of long-term contributors to keep improving, anything that
discourages them from contributing back to the community needs to be
carefully understood and resolved if we want to continue harmoniously.

Take for example Jay Bosamiya's patch from September last year [1]. It
appears to be his first submission to this mailing list, and yet there
has been absolutely no response to it. There are currently 24 other
patches listed on the updates.orgmode.org which have seen no response
from this community, some of which are from first-time contributors.
There are 36 other patches with at least two replies, but yet to be
resolved. Bastien's updates.orgmode.org is fantastic in helping prevent
contributions slip through the cracks, but it is also highlighting the
lack of community response to a significant number of patches.

This mailing list was my first experience with an email+patch based
contribution workflow. Thankfully, I received prompt and friendly
feedback and was guided through the adjustments needed so it could be
merged in a timely manner. Should my patch have been similarly ignored,
I would have been quite disheartened; it is not an overstatement to say
I would likely have written off this mailing list and not tried again.

Simply put, this is not good enough. This does a disservice to those
that have dedicated time and effort to try and better this project only
to be ignored. Not rejected, not even acknowledged, nothing.

It is imperative that this community improves our response to
contributions for the long-term health of this project. Do not take me
to be a doomsayer; I have faith that Org is going to keep on improving
regardless. However, failing to welcome and encourage contributors has a
deleterious effect on the health of the project.

I do not blame the maintainers in the slightest. As Bastien brought up
in a recent worg discussion, as time goes on we find ourselves taking on
more and more life responsibilities. Therefore it's in our best interest
to delegate some of the maintainer responsibilities to consistently
active, and supportive community members to "pass down the torch" so the
community and platform can continue to expand with grace and care.

What can the community do?

I don't know of any silver bullet, but I believe there are some steps
which could help, namely:
+ The development and publication of "reasonable expectations" which
  contributors should have when submitting a patch, and that the
  maintainers should strive to uphold (e.g. "expect a response within
  <some timeframe>").
+ A community effort/sprint to respond to those patches that have been
  seemingly abandoned
+ Onboarding of new maintainers, when reasonable and suitable candidates
  exist (I'd very willingly throw my hat in the ring for consideration).
  If it's too much work, spread it out as much as possible.

If any other ideas come to mind, please share them so we can discuss
them further.

Finally, it's not all bad.

While this discussion has called for some criticism, I don't want to
give the false impression that I think nothing is working and nobody is
supporting contributors. This is not the case at all, there are some
standout individuals one the mailing list who have been fantastic. Kudos
to you all.

My best to everyone,

Timothy

[1] https://orgmode.org/list/CAOywxZg1cBL07THLZXHBBCzm6te2vMtqnmM0w63331gybrjZuw@mail.gmail.com/
[2] https://orgmode.org/list/87h7qi2l2m.fsf@gmail.com/

Re: Concerns about community contributor support

[Jean Louis]

It should be clear that more maintainers and developers with direct
access are needed for patches to be applied timely.

I am sure that those on emacs-devel mailing list, Emacs developers,
they could help on that, but maybe some of them do not observe this
mailing list.

Number of not applied patches does not seem to be that large.

Project leader shall have his team and delegate them to other
developers.

This situation badly contradicts to Org mode purposes which is meant
to manage patches.

Maybe there shall be a published list of developers and project
managers, so that this type of communication may be addressd
properly. As now original poster explained the problem, but I did not
see response from none of pushers, I mean those who have repository
access.

So how many people are there who have repository access?

Who is not busy but willing to apply at least 1-5 patches pending?



Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/

Re: Concerns about community contributor support

[Tim Cross]

Jean Louis <bugs@gnu.support> writes:

> It should be clear that more maintainers and developers with direct
> access are needed for patches to be applied timely.
>
> I am sure that those on emacs-devel mailing list, Emacs developers,
> they could help on that, but maybe some of them do not observe this
> mailing list.
>
> Number of not applied patches does not seem to be that large.
>
> Project leader shall have his team and delegate them to other
> developers.
>
> This situation badly contradicts to Org mode purposes which is meant
> to manage patches.
>
> Maybe there shall be a published list of developers and project
> managers, so that this type of communication may be addressd
> properly. As now original poster explained the problem, but I did not
> see response from none of pushers, I mean those who have repository
> access.
>
> So how many people are there who have repository access?
>
> Who is not busy but willing to apply at least 1-5 patches pending?
>

I'm not sure I agree with this analysis. As Timothy mentioned a few
times, the issue is NOT about the time taken to apply patches. It is
about a lack of feedback regarding the state of the patches.

Increasing the number of people with the ability to apply
patches/changes to the repository is unlikely to be helpful. It could in
fact have the reverse result, leading to increased inconsistency and
reduced stability.

I find bug fixes are applied very quickly. Enhancements and extensions
are introduced more conservatively, but I think that is a positive
rather than negative aspect of org development. For many users, org is
already very feature rich/complete.


-- 
Tim Cross

Re: Concerns about community contributor support

[Heinz Tuechler]

Tim Cross wrote on 21.04.2021 11:50:
> I find bug fixes are applied very quickly. Enhancements and extensions
> are introduced more conservatively, but I think that is a positive
> rather than negative aspect of org development. For many users, org is
> already very feature rich/complete.

This is my view as an org user too. I read this list with interest and
appreciate bug fixes. Similar to what Eric stated, much of my work flow
depends on org. Therefore my main interest is in stability and
consistency of org.

best regards,
Heinz

Re: Concerns about community contributor support

[ian martins]

[-- Attachment #1: Type: text/plain, Size: 1808 bytes --]

Timothy, thanks for raising this. I agree with everything you've said
in this thread. I think it may be a hard problem to solve, but maybe
we can start by just trying to improve. To be clear the problem I'm
talking about is that potential contributors sometimes receive no
response from the list.


Maybe it could help to normalize somewhat generic responses.  Here are
some possible situations where we might not respond to a patch
submission, followed by a potential response we might consider:

1. A patch looks useful to me, but I feel I don't know if it's a good
   idea for org in general, or maybe I think it is but I cannot apply the
   patch because (I'm not a maintainer, I don't have elisp experience, I
   haven't signed the copyright release, etc)

   "Thanks for submitting this. I'd use it.  Hopefully a maintainer
   will take a look."

2. A patch does not look useful to me, but I can see how it may be useful
   to someone else and it was posted a while ago and no one has commented
   yet.

   "Thanks for submitting this. It looks like this affects an org
   feature that not many of us use.  Sorry, but it might not get much
   attention."

3. A patch does not look useful to me, and can't imagine how it is useful
   in any context.

   "What is your use case?"


These trite responses may be seen as mailing list noise, but I don't
think so. They assure the patch submitter that their patch has been
seen and at least for (1) they signal to maintainers that the patch
would be useful to somebody.

There are volunteer maintainers for the codebase, and volunteers who
help with the mailing list.  Maybe someone who wants to help with this
could check Bark once in a while and respond to submissions that have
been missed or post them to the list to put them in front of everyone
again.

[-- Attachment #2: Type: text/html, Size: 2087 bytes --]

Re: Concerns about community contributor support

[Timothy]

Heinz Tuechler <tuechler@gmx.at> writes:

> Tim Cross wrote on 21.04.2021 11:50:
>> I find bug fixes are applied very quickly. Enhancements and extensions
>> are introduced more conservatively, but I think that is a positive
>> rather than negative aspect of org development. For many users, org is
>> already very feature rich/complete.
>
> This is my view as an org user too. I read this list with interest and
> appreciate bug fixes. Similar to what Eric stated, much of my work flow
> depends on org. Therefore my main interest is in stability and
> consistency of org.

I just want to jump in before the "Re:" becomes inaccurate, I don't
think stability and consistency has much to do with responding to
contributions, it just changes what the response may be --- which is
another matter.

--
Timothy

Re: Concerns about community contributor support

[Eric S Fraga]

On Wednesday, 21 Apr 2021 at 12:55, ian martins wrote:
> 1. A patch looks useful to me, but I feel I don't know if it's a good

[...]

>    "Thanks for submitting this. I'd use it.  Hopefully a maintainer
>    will take a look."

Ian,

I think you will find that a few of us do post answers like this every
now and again.  I know that I do.

The lack of answers to cases 2 & 3 are essentially showing a lack of
interest which is basically an (implicit) answer as well.  It may seem
dismissive but, given the volume of email some of us have to deal with
in our day job, it's just reality, I would suggest, without it meaning
to be judgemental in any way.

(now back to my day job ;-))

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4.4-254-g37749c

Re: Concerns about community contributor support

[ian martins]

[-- Attachment #1: Type: text/plain, Size: 1382 bytes --]

On Wed, Apr 21, 2021 at 9:27 AM Eric S Fraga <e.fraga@ucl.ac.uk> wrote:

>
> On Wednesday, 21 Apr 2021 at 12:55, ian martins wrote:
> > 1. A patch looks useful to me, but I feel I don't know if it's a good
>
> [...]
>
> >    "Thanks for submitting this. I'd use it.  Hopefully a maintainer
> >    will take a look."
>
> Ian,
>
> I think you will find that a few of us do post answers like this every
> now and again.  I know that I do.
>

I didn't mean to imply that you or anyone else never does this. I actually
don't do it myself, and on reflection I believe it is for the reasons I
listed above (a patch may look helpful but I don't know for sure that it'll
be good overall, or it applies to a part of the code that I've not looked
at). I thought others might have the same reasons for not responding.

The lack of answers to cases 2 & 3 are essentially showing a lack of
> interest which is basically an (implicit) answer as well.  It may seem
> dismissive but, given the volume of email some of us have to deal with
> in our day job, it's just reality, I would suggest, without it meaning
> to be judgemental in any way.
>

Lack of interest /is/ an implicit answer. The question is if that is a good
way to answer. I agree with Timothy that ignoring newcomers' first attempt
at contribution is an effective way to drive people away. Is that worth it
in order to reduce email?

[-- Attachment #2: Type: text/html, Size: 2013 bytes --]

Re: Concerns about community contributor support

[Bruce D'Arcus]

I've been thinking about this general issue over the past year, as
it's a common problem regardless of project, hosting platform, etc.

I do agree in general that situations where submission of ideas and
patches languish without attention are a problem.

But while I don't think there are any easy answers, I do think clarity
upfront can help.

For example, perhaps there needs to be a prominent statement early on
this page, which states upfront how the community vets new ideas or
patches, what someone who submits such can expect, including in terms
of timeline, and how to interpret lack of response?

https://orgmode.org/worg/org-contribute.html

Bruce

Re: Concerns about community contributor support

[Tim Cross]

"Bruce D'Arcus" <bdarcus@gmail.com> writes:

> I've been thinking about this general issue over the past year, as
> it's a common problem regardless of project, hosting platform, etc.
>
> I do agree in general that situations where submission of ideas and
> patches languish without attention are a problem.
>
> But while I don't think there are any easy answers, I do think clarity
> upfront can help.
>
> For example, perhaps there needs to be a prominent statement early on
> this page, which states upfront how the community vets new ideas or
> patches, what someone who submits such can expect, including in terms
> of timeline, and how to interpret lack of response?
>
> https://orgmode.org/worg/org-contribute.html
>

Yes, we can certainly help manage expectations, which I think is going
to be more effective. At the very least, it is a good place to start.
Perhaps it would be worthwhile if everyone interested in this issue have
a look at what is on worg about contributing and propose some additions/updates?

Responding to essentially just flag you have seen the patch message
really only adds noise and may even 'drown out' those responses which
add real 'value' to any discussion. I also doubt that asking people to
do this would actually result in any actual change of behaviour by
subscribers.

-- 
Tim Cross

Re: Concerns about community contributor support

[ian martins]

[-- Attachment #1: Type: text/plain, Size: 511 bytes --]

On Wed, Apr 21, 2021 at 3:45 PM Tim Cross <theophilusx@gmail.com> wrote:

> Responding to essentially just flag you have seen the patch message
> really only adds noise and may even 'drown out' those responses which
> add real 'value' to any discussion. I also doubt that asking people to
> do this would actually result in any actual change of behaviour by
> subscribers.
>

Timothy said there were 25 patches without response and the list goes back
six months, so we're only talking about 50 emails per year.

[-- Attachment #2: Type: text/html, Size: 851 bytes --]

Re: Concerns about community contributor support

[Tim Cross]

ian martins <ianxm@jhu.edu> writes:

> On Wed, Apr 21, 2021 at 3:45 PM Tim Cross <theophilusx@gmail.com> wrote:
>
>  Responding to essentially just flag you have seen the patch message
>  really only adds noise and may even 'drown out' those responses which
>  add real 'value' to any discussion. I also doubt that asking people to
>  do this would actually result in any actual change of behaviour by
>  subscribers.
>
> Timothy said there were 25 patches without response and the list goes back six months, so we're only talking about 50 emails per year.

That assumes there is a single 'owner' who accepts the responsibility to
respond to every patch submitted. That isn't the situation with open
source projects where people volunteer their time.

Having someone respond to the author of a patch and provide some
meaningful feedback would be great, but I don't see how that can happen
in a project which is already stretched and with limited resources.
There has already been multiple messages requesting additional help and
for volunteers willing to put in the time needed to maintain parts of
org mode. Adding to that workload isn't going to help. 

responses to patches really need to come from community members who are
sufficiently interested in the patch to examine it or make comment on
how useful they feel it would be. To some extent, if someone submits a
patch and hears nothing, either they have failed to clearly explain what
the patch does (and therefore did not tweak any interest) or it is a
patch to introduce functionality which is of low interest to community
participants.

I think you can classify patches into 3 basic types -

1. Bug fixes. Patches which do not change functionality, but address
bugs and performance bottlenecks in existing features. At present, this
type of patch seems to get applied fairly quickly.

2. Patches which extend existing functionality. This type of patch
requires significant consideration and evaluation. They can be time
consuming to assess and a call needs to be made as to whether the
additional complexity such enhancements add can justify the increased
maintenance overhead such enhancements introduce. This is particularly
challenging with org mode because org supports a diverse and sometimes
complex range of workflows. Verifying an enhancement will not have
adverse impact on some of these workflows is difficult and time
consuming. Even apparently simple and straight-forward changes can have
unexpected impact - consider for example the enhancement to support
electric indent mode. This seemed fairly straight-forward and was a
change which made org mode aligned with other Emacs modes and helps
improve consistency withing Emacs across modes. However, it has had some
adverse impact and caused some confusion because of how it interacts
with existing org behaviour and configuration settings, such as with org
indentation behaviour.

3. Patches which introduce new functionality. This type of patch also
requires considerable resources to evaluate and adds to overall
maintenance load. It is one thing to write an extension, but a
completely different thing to maintain it over time. Even assessing the
real demand for such extensions is challenging and time consuming and
represents a significant demand on volunteers time.

Asking volunteers to respond to patches of type 2 and 3 within some
nominated time period is probably unreasonable. It also runs the danger
of discouraging people from stepping up to volunteer to help maintain
parts of org. This is why I think a better approach would be to provide
more details and explanation on patch submission which can help set the
expectations for the patch submitter and provide some guidance on what
to do if they want to encourage/ask for feedback.

This is also part of why I think patches of type 3 and possibly many
type 2 patches should be initially released as separate 'add on'
packages and made available via gitlab/github/melpa by the individual
responsible for writing the patch. The author would then be able to see
how useful/popular/desired their patch is, be able to ask for feedback
and be able to get issue/bug reports to refine their work. This could be
viewed as an 'incubator' like process. If such an enhancement/extension
turns out to be very popular or demanded by the org community, it could
then be migrated into either org core or the proposed org contrib
package (by which time, it would likely be more mature and stable than
it was when initially developed). It also has the advantage of not
impacting existing org users who are not interested in the
enhancement/extension. For an org user, little is more frustrating than
an enhancement/extension which results in them having to either modify
their workflow or update their often large repository of org documents. 

If we were to provide a detailed explanation on how to contribute bug
fixes, enhancements and extensions on the worg site, contributors will
know what is required, will be able to set their expectations in -line
with how things work and have increased clarity regarding the structure
of the org mode project etc. 

I would be willing to start drafting such a page if the community
thought this would be worthwhile and be prepared to assist and assuming
those responsible for maintenance agree. What I draft would be a
starting point only and would require input to ensure it does represent
what the community and maintainers believe is the right direction to
take. 

-- 
Tim Cross

Re: Concerns about community contributor support

[Timothy]

Tim Cross <theophilusx@gmail.com> writes:

> ian martins <ianxm@jhu.edu> writes:
>
>> On Wed, Apr 21, 2021 at 3:45 PM Tim Cross <theophilusx@gmail.com> wrote:
>>
>> [Noise]
>>
>> Timothy said there were 25 patches without response and the list goes back six months, so we're only talking about 50 emails per year.

> That assumes there is a single 'owner' who accepts the responsibility to
> respond to every patch submitted. That isn't the situation with open
> source projects where people volunteer their time.
>
> Having someone respond to the author of a patch and provide some
> meaningful feedback would be great, but I don't see how that can happen
> in a project which is already stretched and with limited resources.
> There has already been multiple messages requesting additional help and
> for volunteers willing to put in the time needed to maintain parts of
> org mode. Adding to that workload isn't going to help.

As far as I know the only call for help maintaining Org has been with
babel packages. Otherwise you would have seen me volunteering :P I'd
like to do more if I get the opportunity.

> [snip]
>
> I think you can classify patches into 3 basic types -
>
> 1. [Fixes]
>
> 2. [Extending existing stuff]
>
> 3. [New stuff]
>
> Asking volunteers to respond to patches of type 2 and 3 within some
> nominated time period is probably unreasonable.

I'd like to suggest that a response can just be "We've got your patch,
it will take some time to go through and see ow it interacts with Org".

> It also runs the danger of discouraging people from stepping up to
> volunteer to help maintain parts of org.

TBH I don't see how being asked to provide the odd cursory response
would be that off-putting. 50 currently patches needing a response per
year / say 3 maintainers ~= cursory quick email every 2-4 weeks on
average just to say a patch has been seen, thanks for submitting it, and
maybe that it might take a while to be reviewed.

> This is why I think a better approach would be to provide more details
> and explanation on patch submission which can help set the
> expectations for the patch submitter and provide some guidance on what
> to do if they want to encourage/ask for feedback.

I think this would be a very good idea, I'll say a bit more below where
you mention Worg.

> This is also part of why I think patches of type 3 and possibly many
> type 2 patches should be initially released as separate 'add on'
> packages and made available via gitlab/github/melpa by the individual
> responsible for writing the patch. The author would then be able to see
> how useful/popular/desired their patch is, be able to ask for feedback
> and be able to get issue/bug reports to refine their work. This could be
> viewed as an 'incubator' like process. If such an enhancement/extension
> turns out to be very popular or demanded by the org community, it could
> then be migrated into either org core or the proposed org contrib
> package (by which time, it would likely be more mature and stable than
> it was when initially developed). It also has the advantage of not
> impacting existing org users who are not interested in the
> enhancement/extension. For an org user, little is more frustrating than
> an enhancement/extension which results in them having to either modify
> their workflow or update their often large repository of org documents.

I think volume of email replies saying "I'd like this" is a bad measure
for a few reasons. (1) I get the sense there's a fairly high degree of
tacit approval, (2) I've seen the same idea presented simply at
different times get very different responses based on how the initial
replies reframed/directed the discussion.

Additionally, if people who like it can "just use it", a patch may be
well-liked and used a lot but not have many peoples speaking in support
of it in the ML.

In other words, I think that such a system could be too fickle. I
suspect some good patches will easily "fall through the cracks" with
such a method. I can think of a several merged patches which I consider
a good idea which would not fare well under such a system.

Then there's another concern if you're modifying parts of Org's
internals --- they can be tweaked in Org, and then the overridden methods
can cause errors in a number of ways. I know this very well, as I do
this sort of thing in a few places in my config, e.g. I was affected by
a change in org--mks-read-key. Is a patch author going to be interested
in maintaining their patch in the hope that it one day gets merged with
Org? This seems like a bit of a stretch to me.

> If we were to provide a detailed explanation on how to contribute bug
> fixes, enhancements and extensions on the worg site, contributors will
> know what is required, will be able to set their expectations in -line
> with how things work and have increased clarity regarding the structure
> of the org mode project etc.
>
> I would be willing to start drafting such a page if the community
> thought this would be worthwhile and be prepared to assist and assuming
> those responsible for maintenance agree. What I draft would be a
> starting point only and would require input to ensure it does represent
> what the community and maintainers believe is the right direction to
> take.

Fantastic! I think such an entry would be a big improvement, and I hope
that such an addition would help prevent contributors from feeling
surprised/disappointed. I think a short entry on this may also be a good
idea for orgmode.org/contribute.html.

--
Timothy

Maintaining babel packages — a list of packages that need help?

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 568 bytes --]


Timothy <tecosaur@gmail.com> writes:
> As far as I know the only call for help maintaining Org has been with
> babel packages. Otherwise you would have seen me volunteering :P I'd
> like to do more if I get the opportunity.

I’m currently in the process of enabling myself to contribute. Do we
have a list of babel-packages that need maintenance? This is also
something I’m personally interested in, because I use babel a lot,
including in multi-language workflows.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: Maintaining babel packages — a list of packages that need help?

[ian martins]

[-- Attachment #1: Type: text/plain, Size: 1076 bytes --]

There's a list of languages here [1]. It looks like someone helpfully added
maintainers to the table. You could find languages without a listed
maintainer there and then check the header of the source file [2] to be
sure there's no current maintainer.

[1] https://orgmode.org/worg/org-contrib/babel/languages/index.html
[2] https://code.orgmode.org/bzg/org-mode/src/master/lisp

On Thu, Apr 22, 2021 at 1:14 AM Dr. Arne Babenhauserheide <arne_bab@web.de>
wrote:

>
> Timothy <tecosaur@gmail.com> writes:
> > As far as I know the only call for help maintaining Org has been with
> > babel packages. Otherwise you would have seen me volunteering :P I'd
> > like to do more if I get the opportunity.
>
> I’m currently in the process of enabling myself to contribute. Do we
> have a list of babel-packages that need maintenance? This is also
> something I’m personally interested in, because I use babel a lot,
> including in multi-language workflows.
>
> Best wishes,
> Arne
> --
> Unpolitisch sein
> heißt politisch sein
> ohne es zu merken
>

[-- Attachment #2: Type: text/html, Size: 1653 bytes --]

Re: Maintaining babel packages — a list of packages that need help?

[Bastien]

Hi Arne,

"Dr. Arne Babenhauserheide" <arne_bab@web.de> writes:

> I’m currently in the process of enabling myself to contribute. Do we
> have a list of babel-packages that need maintenance? This is also
> something I’m personally interested in, because I use babel a lot,
> including in multi-language workflows.

Some files in lisp/ob-*.el (in the org-mode.git repository) already
have a maintainer, Kyle recently listed them.

If a file does not have a "Maintainer:" header and you want to take
over maintainance, that's very welcome.

Thanks!

Re: Concerns about community contributor support

[ian martins]

[-- Attachment #1: Type: text/plain, Size: 2368 bytes --]

On Wed, Apr 21, 2021 at 9:49 PM Tim Cross <theophilusx@gmail.com> wrote:

>
> ian martins <ianxm@jhu.edu> writes:
>
> > On Wed, Apr 21, 2021 at 3:45 PM Tim Cross <theophilusx@gmail.com> wrote:
> >
> >  Responding to essentially just flag you have seen the patch message
> >  really only adds noise and may even 'drown out' those responses which
> >  add real 'value' to any discussion. I also doubt that asking people to
> >  do this would actually result in any actual change of behaviour by
> >  subscribers.
> >
> > Timothy said there were 25 patches without response and the list goes
> back six months, so we're only talking about 50 emails per year.
>
> That assumes there is a single 'owner' who accepts the responsibility to
> respond to every patch submitted.
>

Why does it? If some individual notices that some patch was submitted but
hasn't gotten a response, that person can send a "thanks for submitting.
probably maintainers don't use that feature." And someone else could check
Woof every three or six months, if so inclined. It doesn't have to be
anybody's job, and it doesn't have to be limited to some predesignated
group.

Having someone respond to the author of a patch and provide some
> meaningful feedback would be great, but I don't see how that can happen
> in a project which is already stretched and with limited resources.
> There has already been multiple messages requesting additional help and
> for volunteers willing to put in the time needed to maintain parts of
> org mode. Adding to that workload isn't going to help.
>

I don't see how this adds to the workload in a significant way, but also
couldn't this be the solution to that problem? Instead of ignoring
potential future contributors we could encourage and welcome them. Maybe we
don't have to be constrained by our current limit on resources.


> To some extent, if someone submits a
> patch and hears nothing, either they have failed to clearly explain what
> the patch does (and therefore did not tweak any interest) or it is a
> patch to introduce functionality which is of low interest to community
> participants.
>

But if they hear nothing, how are they to know which was the problem? And
if it was their first contact, how should they know the problem is one of
the things you listed instead of any number of reasons one can imagine that
no one is talking to them?

[-- Attachment #2: Type: text/html, Size: 3367 bytes --]

Re: Concerns about community contributor support

[Tim Cross]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> On Wednesday, 21 Apr 2021 at 12:55, ian martins wrote:
>> 1. A patch looks useful to me, but I feel I don't know if it's a good
>
> [...]
>
>>    "Thanks for submitting this. I'd use it.  Hopefully a maintainer
>>    will take a look."
>
> Ian,
>
> I think you will find that a few of us do post answers like this every
> now and again.  I know that I do.
>
> The lack of answers to cases 2 & 3 are essentially showing a lack of
> interest which is basically an (implicit) answer as well.  It may seem
> dismissive but, given the volume of email some of us have to deal with
> in our day job, it's just reality, I would suggest, without it meaning
> to be judgemental in any way.
>
> (now back to my day job ;-))

+1. If a patch is of interest to me, I usually respond. I don't respond
to patches which are of no interest or which I don't understand.

-- 
Tim Cross

Re: Concerns about community contributor support

[Bastien]

Dear Timothy,

thanks for raising this points so carefully, they are important.

I see three distinct problems:

1. The lack of response and/or follow-up when people contribute by
   sending bug reports or patches on the list.

2. The lack of maintainance on documenting the contribution process
   and the correct expectations for future contributors.

3. The inherent difficulty to move the Org codebase forward.

I won't comment on (3).  For (1) and (2), I suggest appointing a
"contributor steward" with these responsibilities:

1. Maintain updates.orgmode.org (i.e. make sure it is accurate.)
2. Maintain the documentation for contributors.
3. Help contributors enhancing their patches.
4. Try to reproduce bugs (and confirm them for updates.orgmode.org)
5. Make sure patch contributors are not left with no answer.

You started (1), which is really appreciated.

Tim and others mentioned (2), and that's certainly needed too.

(3) has historically been the role of the maintainer and of the core
contributors, but helping with this is very welcome: knowing the doc
at https://orgmode.org/worg/org-contribute.html by heart, educating
contributors on the commit messages, etc.  This all helps.

(4) is perhaps the most daunting task: I even think we could have
someone only dedicated to this very important task.  Just count the
number of times Nicolas says "I cannot reproduce this."  Each time,
there is a real bug that is *not* fixed...

(5) is not about systematically welcome patch submitters with a
message (that would be annoying) but to monitor updates.orgmode.org
and decide what to do with a patch that didn't receive feedback:
either say thanks and ping the list for why you think the patch
deserves more attention, or thanks and dismiss the patch, or another
answer.

What do you think?  Would you be willing to take this role?

If not, that's perfectly okay, I'll send a call for help.

Thanks,

-- 
 Bastien

Contributor Steward role (was Re: Concerns about community contributor support)

[Timothy]

Dear Bastien,

Thank you for your well-thought-out reply.
With regards to the question your email ends with:

> What do you think?  Would you be willing to take this role?
> If not, that's perfectly okay, I'll send a call for help.

The short answer is "yes, mostly". The long answer follows :P

I also think that regardless it would be good to put out a call asking
if anyone else is interested/willing to do this: I'm going to be busy
sometimes, a reduced workload is clearly preferable, and less should
slip through the cracks :)

Bastien <bzg@gnu.org> writes:

> Dear Timothy,
>
> thanks for raising this points so carefully, they are important.
>
> I see three distinct problems:
>
> A. The lack of response and/or follow-up when people contribute by
>    sending bug reports or patches on the list.

This is something I'm definitely happy to help with.

> B. The lack of maintainance on documenting the contribution process
>    and the correct expectations for future contributors.

I'm happy to document this, but I don't know what expectations should be
set. I think this is a matter that should be decided by consensus among
the current/active maintainers.

> C. The inherent difficulty to move the Org codebase forward.

I think there are some things that can be done to improve the structure
and system of contribution/development, but I'm waiting on external
developments and it will take a fair bit of my time to properly
implement a prototype. ETA 1-2 years.

> I won't comment on (C).  For (A) and (B), I suggest appointing a
> "contributor steward" with these responsibilities:
>
> 1. Maintain updates.orgmode.org (i.e. make sure it is accurate.)
> 2. Maintain the documentation for contributors.
> 3. Help contributors enhancing their patches.
> 4. Try to reproduce bugs (and confirm them for updates.orgmode.org)
> 5. Make sure patch contributors are not left with no answer.
>
> You started (1), which is really appreciated.

I'll try to keep this up :)

> Tim and others mentioned (2), and that's certainly needed too.

See my comment on (B) above.

> (3) has historically been the role of the maintainer and of the core
> contributors, but helping with this is very welcome: knowing the doc
> at https://orgmode.org/worg/org-contribute.html by heart, educating
> contributors on the commit messages, etc.  This all helps.

I can give this a go :)

> (4) is perhaps the most daunting task: I even think we could have
> someone only dedicated to this very important task.  Just count the
> number of times Nicolas says "I cannot reproduce this."  Each time,
> there is a real bug that is *not* fixed...

Mmmm, even if I say I'm willing to do this, I suspect this is something
I'd by nature push off enough that I'd frequently forget about this 😅.

> (5) is not about systematically welcome patch submitters with a
> message (that would be annoying) but to monitor updates.orgmode.org
> and decide what to do with a patch that didn't receive feedback:
> either say thanks and ping the list for why you think the patch
> deserves more attention, or thanks and dismiss the patch, or another
> answer.

This is how I see this two. As I indicated at the start, I'm happy to do
this, but I think a second person would help ensure that nobody slips
through the cracks.

--
Timothy

Re: Contributor Steward role (was Re: Concerns about community contributor support)

[Bastien]

Hi Timothy,

thanks for your answer and your willingness to help here, much
appreciated.

Timothy <tecosaur@gmail.com> writes:

> I also think that regardless it would be good to put out a call asking
> if anyone else is interested/willing to do this: I'm going to be busy
> sometimes, a reduced workload is clearly preferable, and less should
> slip through the cracks :)

Sure.  We can certainly have two or more contributor stewards.

> I'm happy to document this, but I don't know what expectations should be
> set. I think this is a matter that should be decided by consensus among
> the current/active maintainers.

I started something here: 
https://orgmode.org/worg/org-contribute.html#what-can-I-expect

Please adapt along your own commitment and expectations.

>> (4) is perhaps the most daunting task: I even think we could have
>> someone only dedicated to this very important task.  Just count the
>> number of times Nicolas says "I cannot reproduce this."  Each time,
>> there is a real bug that is *not* fixed...
>
> Mmmm, even if I say I'm willing to do this, I suspect this is something
> I'd by nature push off enough that I'd frequently forget about this 😅.

Sure -- also, this is time consuming and probably requires someone to
focus just on this.  I will raise a call for help for this.

> This is how I see this two. As I indicated at the start, I'm happy to do
> this, but I think a second person would help ensure that nobody slips
> through the cracks.

Great!  Thanks a lot.

Re: Concerns about community contributor support

[Tim Cross]

Bastien <bzg@gnu.org> writes:

> Dear Timothy,
>
> thanks for raising this points so carefully, they are important.
>
> I see three distinct problems:
>
> 1. The lack of response and/or follow-up when people contribute by
>    sending bug reports or patches on the list.
>
> 2. The lack of maintainance on documenting the contribution process
>    and the correct expectations for future contributors.
>
> 3. The inherent difficulty to move the Org codebase forward.
>
> I won't comment on (3).  For (1) and (2), I suggest appointing a
> "contributor steward" with these responsibilities:
>
> 1. Maintain updates.orgmode.org (i.e. make sure it is accurate.)
> 2. Maintain the documentation for contributors.
> 3. Help contributors enhancing their patches.
> 4. Try to reproduce bugs (and confirm them for updates.orgmode.org)
> 5. Make sure patch contributors are not left with no answer.
>
> You started (1), which is really appreciated.
>
> Tim and others mentioned (2), and that's certainly needed too.
>
> (3) has historically been the role of the maintainer and of the core
> contributors, but helping with this is very welcome: knowing the doc
> at https://orgmode.org/worg/org-contribute.html by heart, educating
> contributors on the commit messages, etc.  This all helps.
>
> (4) is perhaps the most daunting task: I even think we could have
> someone only dedicated to this very important task.  Just count the
> number of times Nicolas says "I cannot reproduce this."  Each time,
> there is a real bug that is *not* fixed...
>
> (5) is not about systematically welcome patch submitters with a
> message (that would be annoying) but to monitor updates.orgmode.org
> and decide what to do with a patch that didn't receive feedback:
> either say thanks and ping the list for why you think the patch
> deserves more attention, or thanks and dismiss the patch, or another
> answer.
>
> What do you think?  Would you be willing to take this role?
>
> If not, that's perfectly okay, I'll send a call for help.
>
Hi Bastien et. al.

I agree and am willing to help if I can.

One thing I do think would be helpful, particularly for documentation on
maintenance of org mode, would be a clear outline of project goals and
philosophy. Some of this would be 'concrete' statements, such as the
minimum supported Emacs version, size of contribution and requirements
to sign FSF copyright assignment paperwork, inclusion of acceptance
tests, documentation, maintenance of backwards compatibility, API
stability etc. Other parts would be more 'fuzzy' guidelines, such as
avoiding complexity and 'blow out' in options/arguments, balancing
features and maintainability, what should become part of org core and
what should be in contributions or a completely separate add on package
and what guidelines are used in assessing extensions/enhancements for
inclusion etc.

It will be challenging to define this as there are a wide diversity of
views and priorities amongst the community. However, I think it would be
an overall benefit for both the community and on-going development of
org mode. 

While I would be happy to help with this, I think the initial content at
least needs to come from current key maintainers and if possible, some
input from Carsten would be good. 

-- 
Tim Cross

Re: Concerns about community contributor support

[Bastien]

Hi Tim,

Tim Cross <theophilusx@gmail.com> writes:

> I agree and am willing to help if I can.

Great!  Would you agree to be "officially" appointed to this role,
with Timothy?  I put quotes on "officially": when moving toward the
new maintainance team, I'd like to list maintainers and their roles,
including the "contributor stewards".  If you prefer not to publicly
appear but still work on this, that's perfectly okay too, of course.

> One thing I do think would be helpful, particularly for documentation on
> maintenance of org mode, would be a clear outline of project goals and
> philosophy. Some of this would be 'concrete' statements, such as the
> minimum supported Emacs version, size of contribution and requirements
> to sign FSF copyright assignment paperwork, inclusion of acceptance
> tests, documentation, maintenance of backwards compatibility, API
> stability etc. Other parts would be more 'fuzzy' guidelines, such as
> avoiding complexity and 'blow out' in options/arguments, balancing
> features and maintainability, what should become part of org core and
> what should be in contributions or a completely separate add on package
> and what guidelines are used in assessing extensions/enhancements for
> inclusion etc.
>
> It will be challenging to define this as there are a wide diversity of
> views and priorities amongst the community. However, I think it would be
> an overall benefit for both the community and on-going development of
> org mode. 
>
> While I would be happy to help with this, I think the initial content at
> least needs to come from current key maintainers and if possible, some
> input from Carsten would be good. 

Fully agreed.  Input from old timers will be precious, too.  I will
work on this as we are transitioning to the new maintainance team.

Also, I commit to achieve this transition by August, 1st.  This may
seem a bit far away, but there is really no rush here, and I want to
ensure we have a smooth transition.

Re: Concerns about community contributor support

[Tim Cross]

Bastien <bzg@gnu.org> writes:

> Hi Tim,
>
> Tim Cross <theophilusx@gmail.com> writes:
>
>> I agree and am willing to help if I can.
>
> Great!  Would you agree to be "officially" appointed to this role,
> with Timothy?  I put quotes on "officially": when moving toward the
> new maintainance team, I'd like to list maintainers and their roles,
> including the "contributor stewards".  If you prefer not to publicly
> appear but still work on this, that's perfectly okay too, of course.
>

Yes, but with some caveats. As may have already been picked up by some,
I'm extremely conservative regarding the addition of new features and
enhancements to org mode. I have considerable concern regarding both the
performance and maintainability of the mode and worry about the
increasing complexity in options, performance impact from increasingly
complex font locking just to have more 'eye candy' at the cost of
sluggish behaviour for large files and striking the right balance
between the 'richness' of org file functionality and consistency in
exporters or source block handling. I really would hate to see org mode
end up like MS word with 90% of users only using 10% of the
functionality with 90% unused functionality just becoming a burden to
maintainers.

For these reasons, I'm probably not the best person to assist with the
review and guidance for patches aimed at adding/extending functionality.
However, I would be happy to

- Assist in trying to reproduce and confirm bugs
- Assist in extracting additional information and providing
  guidance/clarification for bug reports 
- Assist with documentation
- Provide some guidance on patches, particularly bug fixes.

My time availability can vary greatly depending on work. Also, as a
blind user, my ability to reproduce bugs can sometimes be hampered by
the necessity of running Emacspeak in conjunction with org mode (they
can sometimes interfere with each other). However, apart from that, more
than happy to help where I can, so if your happy, sign me up!

>> One thing I do think would be helpful, particularly for documentation on
>> maintenance of org mode, would be a clear outline of project goals and
>> philosophy. Some of this would be 'concrete' statements, such as the
>> minimum supported Emacs version, size of contribution and requirements
>> to sign FSF copyright assignment paperwork, inclusion of acceptance
>> tests, documentation, maintenance of backwards compatibility, API
>> stability etc. Other parts would be more 'fuzzy' guidelines, such as
>> avoiding complexity and 'blow out' in options/arguments, balancing
>> features and maintainability, what should become part of org core and
>> what should be in contributions or a completely separate add on package
>> and what guidelines are used in assessing extensions/enhancements for
>> inclusion etc.
>>
>> It will be challenging to define this as there are a wide diversity of
>> views and priorities amongst the community. However, I think it would be
>> an overall benefit for both the community and on-going development of
>> org mode. 
>>
>> While I would be happy to help with this, I think the initial content at
>> least needs to come from current key maintainers and if possible, some
>> input from Carsten would be good. 
>
> Fully agreed.  Input from old timers will be precious, too.  I will
> work on this as we are transitioning to the new maintainance team.
>
> Also, I commit to achieve this transition by August, 1st.  This may
> seem a bit far away, but there is really no rush here, and I want to
> ensure we have a smooth transition.

All your efforts greatly appreciated. It is a lot of effort and herding
cats often takes a lot longer then anticipated! 

-- 
Tim Cross

Re: Concerns about community contributor support

[Bastien]

Hi Tim,

thank you very much for your detailed answer.

Tim Cross <theophilusx@gmail.com> writes:

> Yes, but with some caveats. 

Thank you!

> For these reasons, I'm probably not the best person to assist with the
> review and guidance for patches aimed at adding/extending functionality.

The role of a "contributor steward" is to address the issues Timothy
originally raised about contributors not receiving answers when they
take the time to send a patch or a bug report, it is not about making
technical decisions.  Of course, your advice as a long-time user on
these decisions is always very welcome.

> However, I would be happy to
>
> - Assist in trying to reproduce and confirm bugs
> - Assist in extracting additional information and providing
>   guidance/clarification for bug reports 
> - Assist with documentation
> - Provide some guidance on patches, particularly bug fixes.

That's exactly what is badly needed to discharge core maintainers from
routine tasks and to let them focus on technical stuff.

This has basically been my role in my early years of maintainership:
making sure to keep this list as welcoming as possible, to continue to
attract new people with new ideas (and new bug report).  This is a
somewhat undervalued role, but a deeply necessary one -- so again,
thanks for helping here.

> My time availability can vary greatly depending on work. 

That's no problem.  We have a team of Tim's :)

> Also, as a
> blind user, my ability to reproduce bugs can sometimes be hampered by
> the necessity of running Emacspeak in conjunction with org mode (they
> can sometimes interfere with each other). However, apart from that, more
> than happy to help where I can, so if your happy, sign me up!

If the task becomes too much for you, you can always say that you
won't be available for a while.

Thanks!

Re: Concerns about community contributor support

[Tim Cross]

Bastien <bzg@gnu.org> writes:

> Hi Tim,
>
> thank you very much for your detailed answer.
>
> Tim Cross <theophilusx@gmail.com> writes:
>
>> Yes, but with some caveats. 
>
> Thank you!
>
>> For these reasons, I'm probably not the best person to assist with the
>> review and guidance for patches aimed at adding/extending functionality.
>
> The role of a "contributor steward" is to address the issues Timothy
> originally raised about contributors not receiving answers when they
> take the time to send a patch or a bug report, it is not about making
> technical decisions.  Of course, your advice as a long-time user on
> these decisions is always very welcome.
>
>> However, I would be happy to
>>
>> - Assist in trying to reproduce and confirm bugs
>> - Assist in extracting additional information and providing
>>   guidance/clarification for bug reports 
>> - Assist with documentation
>> - Provide some guidance on patches, particularly bug fixes.
>
> That's exactly what is badly needed to discharge core maintainers from
> routine tasks and to let them focus on technical stuff.
>
> This has basically been my role in my early years of maintainership:
> making sure to keep this list as welcoming as possible, to continue to
> attract new people with new ideas (and new bug report).  This is a
> somewhat undervalued role, but a deeply necessary one -- so again,
> thanks for helping here.
>
>> My time availability can vary greatly depending on work. 
>
> That's no problem.  We have a team of Tim's :)
>
>> Also, as a
>> blind user, my ability to reproduce bugs can sometimes be hampered by
>> the necessity of running Emacspeak in conjunction with org mode (they
>> can sometimes interfere with each other). However, apart from that, more
>> than happy to help where I can, so if your happy, sign me up!
>
> If the task becomes too much for you, you can always say that you
> won't be available for a while.
>
> Thanks!

OK, consider me 'singed up". Today I will see if I can re-jig my mail
system to make it easier to track and respond. As Timothy seems to be
doing a fine job 'catching up' with what is already on updates.orgmode,
I'll leave that alone and will focus on responding to new bug reports,
patches etc which come through the list.

I'll also setup a 'clean' virtual image so that I have a vanilla, clean
Emacs config I can use for evaluating/reproducing bugs (current emacs
stable 27.2). .

What is our position with bugs which can only be reproduced in the
current development version of Emacs? I'd expect we track them, but
focus more on Emacs stable given that dev is a moving target and any
bugs may be resolved by later changes in Emacs?

Finally, is it also necessary to monitor org bugs reported to the emacs
bug tracker or are they sent on to the emacs-orgmode list via some other
process? (I currently do monitor emacs-devel but not the bug tracker). 

-- 
Tim Cross

Re: Concerns about community contributor support

[Bastien]

Tim Cross <theophilusx@gmail.com> writes:

> OK, consider me 'singed up".

Done, thanks again.

> What is our position with bugs which can only be reproduced in the
> current development version of Emacs? 

They are extremely rare, so don't worry about this too much.

> I'd expect we track them, but focus more on Emacs stable given that
> dev is a moving target and any bugs may be resolved by later changes
> in Emacs?

Yes, I would do that too.

> Finally, is it also necessary to monitor org bugs reported to the emacs
> bug tracker or are they sent on to the emacs-orgmode list via some other
> process? (I currently do monitor emacs-devel but not the bug tracker). 

Yes, it is useful to monitor these bugs too, and they are also sent to
the org-mode list, so you can focus on this list.

Help with reproducing bugs reported on this list (was: Concerns about community contributor support)

[Bastien]

Hi all,

we are looking for someone to take charge of a very important task:
reproducing bugs reported on this list.

Bug reports generally starts with "Bug: " in the email subject, even
if some emails directly sent to the list may omit this keyword.

Once you confirm a bug, you would need to reply to it and to add a
header to your email like this:

  X-Woof-Bug: confirmed

And your job is done.

This will make the bug appear on https://updates.orgmode.org which
makes it easy for contributors to know that their time is correctly
used when they want to contribute with fixes.

This is a very important task: anyone can contribute to it with the
same process, but having someone in charge would certain help.

This human "bug validator" will join the contributor stewards, as
discussed here: https://orgmode.org/list/87v98b0yvm.fsf@gnu.org/

Thanks!

PS: Please don't use this thread to discuss bug reporting tools and
processes: we need human power more than a change of tooling.

Re: Help with reproducing bugs reported on this list

[Bastien]

Bastien <bzg@gnu.org> writes:

> we are looking for someone to take charge of a very important task:
> reproducing bugs reported on this list.

I'm happy to close this call for help, as John (cc'ed) kindly
volunteered to help with this task.

Thanks John!

Of course, everyone is still welcome to try reproducing bugs,
as this is a critical tasks for maintainance.

Re: Concerns about community contributor support

[Nick Savage]

As a new contributor, I wanted to add my two cents. I've submitted a 
minor amount of patches (somewhere between 1 and 3, I can't remember 
exactly), and I feel that the other problems you raise, primarily the 
first one, are obstacles towards that though. Patches like that are 
obviously minor, since they aren't fixing bugs, so they're low priority. 
That drags out the process though, and it's a little discouraging to 
submit patches that don't receive responses, and make me less eager to 
submit more.For example, I submitted a tiny refactoring patch last month 
that hasn't received an answer yet. I'm not looking for special 
treatment of course, and think that review of patches is super important 
(especially for a new contributor like me)

I think that the solutions you raise are great starts. I feel like the 
documentation that I've used has been pretty accurate, but it could 
always be better, and someone who is the point person for dealing with 
people like me would be a good idea. I would not like to volunteer for 
that myself. I don't feel comfortable enough with my abilities, or my 
experience with the community, to able to help others or give reasonable 
answers for patches other than something like "yes, I've seen this". I 
will try to help out more with verifying bugs though.

On a side note, I'd like some guidance though on whether or not the 
community is interested in a refactoring project (done in pieces of 
course). I'm wondering if I should be attempting to submit minor (or 
larger) patches moving things around to make it easier to maintain, if 
there is a way to go about this (since submitting a bunch at a time, 
breaking them up, etc), certain things I should focus on or ignore, etc. 
I haven't done much since that mostly because work has consumed my free 
time, but I'm going to have a significant amount more free time starting 
in the next week or so, so I'm looking for somewhere to direct my energies!

On 4/25/21 12:30 AM, Bastien wrote:
> Dear Timothy,
>
> thanks for raising this points so carefully, they are important.
>
> I see three distinct problems:
>
> 1. The lack of response and/or follow-up when people contribute by
>     sending bug reports or patches on the list.
>
> 2. The lack of maintainance on documenting the contribution process
>     and the correct expectations for future contributors.
>
> 3. The inherent difficulty to move the Org codebase forward.
>
> I won't comment on (3).  For (1) and (2), I suggest appointing a
> "contributor steward" with these responsibilities:
>
> 1. Maintain updates.orgmode.org (i.e. make sure it is accurate.)
> 2. Maintain the documentation for contributors.
> 3. Help contributors enhancing their patches.
> 4. Try to reproduce bugs (and confirm them for updates.orgmode.org)
> 5. Make sure patch contributors are not left with no answer.
>
> You started (1), which is really appreciated.
>
> Tim and others mentioned (2), and that's certainly needed too.
>
> (3) has historically been the role of the maintainer and of the core
> contributors, but helping with this is very welcome: knowing the doc
> at https://orgmode.org/worg/org-contribute.html by heart, educating
> contributors on the commit messages, etc.  This all helps.
>
> (4) is perhaps the most daunting task: I even think we could have
> someone only dedicated to this very important task.  Just count the
> number of times Nicolas says "I cannot reproduce this."  Each time,
> there is a real bug that is *not* fixed...
>
> (5) is not about systematically welcome patch submitters with a
> message (that would be annoying) but to monitor updates.orgmode.org
> and decide what to do with a patch that didn't receive feedback:
> either say thanks and ping the list for why you think the patch
> deserves more attention, or thanks and dismiss the patch, or another
> answer.
>
> What do you think?  Would you be willing to take this role?
>
> If not, that's perfectly okay, I'll send a call for help.
>
> Thanks,
>

Re: Concerns about community contributor support

[Bastien]

Hi Nick,

Nick Savage <nick@nicksavage.ca> writes:

> On a side note, I'd like some guidance though on whether or not the
> community is interested in a refactoring project (done in pieces of
> course). I'm wondering if I should be attempting to submit minor (or
> larger) patches moving things around to make it easier to maintain, if
> there is a way to go about this (since submitting a bunch at a time,
> breaking them up, etc), certain things I should focus on or ignore,
> etc. I haven't done much since that mostly because work has consumed
> my free time, but I'm going to have a significant amount more free
> time starting in the next week or so, so I'm looking for somewhere to
> direct my energies!

The way to go for structural changes is to work on them in a separate
branch of yours, then discuss the change on the list by suggesting to
test your branch.  Once the branch is stable enough, you can submit it
as a patch or a list of patches.

See https://orgmode.org/worg/org-contribute.html#org4a93bd0 and let
us know if that's clear enough - otherwise please enhance this page.

Thanks!

Re: Concerns about community contributor support

[D]

Hi Timothy,

I was quite weary to bring up this point, but given the sheer volume of
patch-related exchanges in recent memory I feel that it may be worth
bringing up as I have not yet seen it discussed (if I overlooked it, my
apologies): I don't think the core problem can (or maybe should) be
solved by sheer manpower alone.  I would argue that the issue may be
partially infrastructural in nature.  This ML is incredibly active, with
a lot of different people surely following it for different reasons.
However, I am not entirely convinced that having patches, bug reports
and general discussions all in one place is necessarily a net positive
for people wanting to contribute to the project (once a project hits a
certain size, at least).  It lacks a certain separation of concerns that
clearly informed the design of forges like GitHub (having separate
"Issues" and "PRs").

I may be alone with this opinion, given manually organizing is a
personal weakness of mine, and something I prefer to avoid at all costs,
but I think having a separate list for patches would greatly improve
their visibility without having to resort to band-aids like automated
subject-based filtering schemes which are thwarted by mundane human
error ("[PTACH]").  On the other hand, were I completely alone with this
opinion, then we wouldn't have dozens of emacs-* mailing lists already.

TL;DR: Maybe we could improve the visibility of patches by having a
dedicated mailing list for them? This would also allow for a greater
deal of automation in the way we deal with patches.

Cheers,
D.

Re: Concerns about community contributor support

[Bastien]

Hi,

D <d.williams@posteo.net> writes:

> TL;DR: Maybe we could improve the visibility of patches by having a
> dedicated mailing list for them? This would also allow for a greater
> deal of automation in the way we deal with patches.

We do have a dedicated information channel for patches:
https://updates.orgmode.org/#patches

You can subscribe to it with this RSS feed:
https://updates.orgmode.org/feed/patches

Once subscribed, you will only receive the patches, nothing else.

You can track all updates via the gwene.org.orgmode.updates newsgroup
on news.gmane.io.

This idea is precisely to help people organize their contributions to
Org, whether they want to help, fix confirmed bugs or review patches.

Of course, https://updates.orgmode.org is in alpha and we can still
improve it a lot.  In particular, I plan to let it track unconfirmed
bugs too, to help with bug triage, and to provide woof.el to ease
interaction with this tool directly from within Emacs.

Ideas are welcome: https://github.com/bzg/woof/issues

In general, I find it good to have a central communication place for
the community, where newcomers can learn from more experienced users
and I've always resisted to the urge of having e.g. org-users@ and
org-devel@ mailing lists, as some projects have.  Nowadays, users who
don't want to mix with Org's development can interact on many other
places (SO, reddit and others).

-- 
 Bastien

Re: Concerns about community contributor support

[D]

Hi Bastien, Hi Ihort

> https://updates.orgmode.org

whoops, I completely overlooked that.

> Org, whether they want to help, fix confirmed bugs or review patches.
> 
> Of course, https://updates.orgmode.org is in alpha and we can still
> improve it a lot.  In particular, I plan to let it track unconfirmed
> bugs too, to help with bug triage, and to provide woof.el to ease
> interaction with this tool directly from within Emacs.
> 
> Ideas are welcome: https://github.com/bzg/woof/issues

I'll look into it, thank you!

> In general, I find it good to have a central communication place for
> the community, where newcomers can learn from more experienced users
> and I've always resisted to the urge of having e.g. org-users@ and
> org-devel@ mailing lists, as some projects have.  Nowadays, users who
> don't want to mix with Org's development can interact on many other
> places (SO, reddit and others).

I'm not sure sure whether it is about mixing with development
personally, but I think my point is largely addressed by
updates.orgmode.org.

Cheers,
D.

Re: Concerns about community contributor support

[Ihor Radchenko]

D <d.williams@posteo.net> writes:
> TL;DR: Maybe we could improve the visibility of patches by having a
> dedicated mailing list for them? This would also allow for a greater
> deal of automation in the way we deal with patches.

What about https://updates.orgmode.org/? Or do you refer to something
else?

Best,
Ihor