Modularizing Org as GSoC project (was: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library))

Modularizing Org as GSoC project (was: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library))

[Ihor Radchenko]

arthur miller <arthur.miller@live.com> writes:

>> ...
>>Are there any specific todo items there that you view as more suitable
>>for people with limited experience in Emacs codebase?
>
> I am not sure if this is something, I hope you don't mind me asking, but could a
> work to modularize Org, be an appropriate subject?

Yes, it can be.

RMS previously asked us to do exactly this:
https://orgmode.org/list/E1kIPh1-0001Lu-Rg@fencepost.gnu.org

Also, see previous discussion on re-integrating org-capture into remember.el:
https://yhetil.org/emacs-devel/87bka7c2e0.fsf@localhost
Similar for org-protocol.el.

> For example turn displaying pretty text (bold, italics etc), pretty links,
> tables, dates, and perhaps some other stuff into, from Org-mode, independent,
> minor modes that could be used in other parts of Emacs and more independently of
> Org mode. I think both Org-mode and Hyperbole, and perhaps some other libraries
> (button.el, help-mode, info), could use some minor mode that works with
> links. For us users, it would mean less cruft loaded into Emacs, if those big
> players could share some code.
>
> I haven't done much research on this, just something I had in my head for a long
> time.

I would like to hear in more details what you have in mind. As is, your
proposal appears to be for multiple projects. Or am I mistaken?

For links in particular, Hyperbole already implements the described
minor mode, or what am I missing?

For tables, dates, and other markup, it is not clear what you want to
achieve. Do you want Org markup to be displayed in non-Org buffers?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Sv: Modularizing Org as GSoC project (was: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library))

[arthur miller]

[-- Attachment #1: Type: text/plain, Size: 6662 bytes --]

>>> ...
>>>Are there any specific todo items there that you view as more suitable
>>>for people with limited experience in Emacs codebase?
>>
>> I am not sure if this is something, I hope you don't mind me asking, but
>> could a
>> work to modularize Org, be an appropriate subject?
>
>Yes, it can be.
>
>RMS previously asked us to do exactly this:
>https://orgmode.org/list/E1kIPh1-0001Lu-Rg@fencepost.gnu.org
>
>Also, see previous discussion on re-integrating org-capture into remember.el:
>https://yhetil.org/emacs-devel/87bka7c2e0.fsf@localhost
>Similar for org-protocol.el.
>
>> For example turn displaying pretty text (bold, italics etc), pretty links,
>> tables, dates, and perhaps some other stuff into, from Org-mode, independent,
>> minor modes that could be used in other parts of Emacs and more independently
>> of
>> Org mode. I think both Org-mode and Hyperbole, and perhaps some other
>> libraries
>> (button.el, help-mode, info), could use some minor mode that works with
>> links. For us users, it would mean less cruft loaded into Emacs, if those big
>> players could share some code.
>>
>> I haven't done much research on this, just something I had in my head for a
>> long
>> time.
>
>I would like to hear in more details what you have in mind. As is, your
>proposal appears to be for multiple projects. Or am I mistaken?
>
>For links in particular, Hyperbole already implements the described
>minor mode, or what am I missing?
>
>For tables, dates, and other markup, it is not clear what you want to
>achieve.

To be honest I am not sure myself :)

>         Do you want Org markup to be displayed in non-Org buffers?

Well, part of it. Or more to make some parts of org-markup configurable, and
usable as minor modes so they can be easier used outside of org-mode.

For example about links, there could be a mode "text-link-mode" or
"pretty-links-mode" or something, that understands what a link description is,
and what a link itself is. The minor mode would have some mean to parse
description and link parts, and when on it would do what org-toggle-link-display
does. For example org uses angle brackets for link desc and url, whereas
markdown uses angle brackets and parenthesis. Thus link-mode could/should be
enough customizable so that modes could be clients of this minor mode, as well
as for user to be able to setup a regex or set a function that recognizes some
custom syntax for descriptions and links. Also a minor mode can come with a key
map and some actions, for example to follow link, to insert a link etc. I think
of org-links, but a bit more generalized and usable without org-mode
itself. Org-mode could use those under the hood.

Similarly for italics, bolds, underlines, etc. Those could be slightly
generalized and taken out into minor mode. Clients could setup their own
start/end markers and use them to enrich the text on the display instead of
perhaps defining own faces and regexes. I don't know how useful and desirable it
might be in other modes, perhaps in comments in programming languages, or
similar. Just a thought.

org-timestamp interactiveity could be usable elsewhere than just in org
mode. For example I can insert  org-timestamp in any mode, but it does work to
use C-left/right to change the date. It could be refactored into small tiny
minor mode timestamp-mode or something, that comes with tis mode map and enable
this interactive stuff.

ascii tables or org tables started as its own mode but got consumed by org. They
are still usable outside of org mode. I can create table with org-table-create
and I can align table with org-table-align, but by default I don't have this
functionality bound in some keymap if I am not in org-mode. Perhaps I am just
not familiar with it. But this could be a minor mode also.

I guess org-capture is a candidate, and probably some others.

By refactoring out those things, org-mode loose nothing, since everything is
still available in Emacs, and everything could be still used in org-mode under
the hoos, but it would be easier to use those things in other major modes. It
might also be easier to work with org-mode if it is less monolithic.

I am sorry if it is too scatchy, and uninformed, it is just some thoughts I had,
but I was never really motivated to do those things myself.
________________________________
Från: Ihor Radchenko <yantar92@posteo.net>
Skickat: den 20 augusti 2024 19:40
Till: arthur miller <arthur.miller@live.com>
Kopia: emacs-devel@gnu.org <emacs-devel@gnu.org>; emacs-orgmode@gnu.org <emacs-orgmode@gnu.org>
Ämne: Modularizing Org as GSoC project (was: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library))

arthur miller <arthur.miller@live.com> writes:

>> ...
>>Are there any specific todo items there that you view as more suitable
>>for people with limited experience in Emacs codebase?
>
> I am not sure if this is something, I hope you don't mind me asking, but could a
> work to modularize Org, be an appropriate subject?

Yes, it can be.

RMS previously asked us to do exactly this:
https://orgmode.org/list/E1kIPh1-0001Lu-Rg@fencepost.gnu.org

Also, see previous discussion on re-integrating org-capture into remember.el:
https://yhetil.org/emacs-devel/87bka7c2e0.fsf@localhost
Similar for org-protocol.el.

> For example turn displaying pretty text (bold, italics etc), pretty links,
> tables, dates, and perhaps some other stuff into, from Org-mode, independent,
> minor modes that could be used in other parts of Emacs and more independently of
> Org mode. I think both Org-mode and Hyperbole, and perhaps some other libraries
> (button.el, help-mode, info), could use some minor mode that works with
> links. For us users, it would mean less cruft loaded into Emacs, if those big
> players could share some code.
>
> I haven't done much research on this, just something I had in my head for a long
> time.

I would like to hear in more details what you have in mind. As is, your
proposal appears to be for multiple projects. Or am I mistaken?

For links in particular, Hyperbole already implements the described
minor mode, or what am I missing?

For tables, dates, and other markup, it is not clear what you want to
achieve. Do you want Org markup to be displayed in non-Org buffers?

--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

[-- Attachment #2: Type: text/html, Size: 17302 bytes --]

Re: Sv: Modularizing Org as GSoC project (was: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library))

[Ihor Radchenko]

arthur miller <arthur.miller@live.com> writes:

>>         Do you want Org markup to be displayed in non-Org buffers?
>
> Well, part of it. Or more to make some parts of org-markup configurable, and
> usable as minor modes so they can be easier used outside of org-mode.

This is not the direction Org mode project is going. We do the reverse -
Org markup is getting more stable, with the aim to simplify developing
third-party parsers and eventually registering Org syntax in IANA MIME db.

However, stable Org markup does not mean that we cannot implement Org
mode features outside Org mode. Another general direction we are going to
is using parser APIs across Org mode. The idea is to encapsulate the
fine details of Org markup into the parser, only leaving some very basic
structures like recursiveness of markup objects exposed. In theory, one
may eventually just replace the parser with, say, Markdown parser (with
appropriate API), and get Org mode features in Markdown (well... except
that Markdown has fewer kinds of syntax and certain things do not exist
there)

> For example about links, there could be a mode "text-link-mode" or
> "pretty-links-mode" or something, that understands what a link description is,
> and what a link itself is. The minor mode would have some mean to parse
> description and link parts, and when on it would do what org-toggle-link-display
> does. For example org uses angle brackets for link desc and url, whereas
> markdown uses angle brackets and parenthesis. Thus link-mode could/should be
> enough customizable so that modes could be clients of this minor mode, as well
> as for user to be able to setup a regex or set a function that recognizes some
> custom syntax for descriptions and links. Also a minor mode can come with a key
> map and some actions, for example to follow link, to insert a link etc. I think
> of org-links, but a bit more generalized and usable without org-mode
> itself. Org-mode could use those under the hood.

I am not sure if many places (except Markdown) have a notion of link +
description. And the rest is supported by, say, Hyperbole.

What might be more interesting is generalizing Org previews: Org can
preview LaTeX and images, but we can think of it more generally - any
kind of text object might have alternative (possibly image)
representation. See https://list.orgmode.org/orgmode/87edbhljr7.fsf@hyperspace/

> Similarly for italics, bolds, underlines, etc. Those could be slightly
> generalized and taken out into minor mode. Clients could setup their own
> start/end markers and use them to enrich the text on the display instead of
> perhaps defining own faces and regexes. I don't know how useful and desirable it
> might be in other modes, perhaps in comments in programming languages, or
> similar. Just a thought.

You are describing font-lock here. Users can already add custom
fontification in buffers by supplying regexps + face to be used for
specific text fragments. What is the novelty?

> org-timestamp interactiveity could be usable elsewhere than just in org
> mode. For example I can insert  org-timestamp in any mode, but it does work to
> use C-left/right to change the date. It could be refactored into small tiny
> minor mode timestamp-mode or something, that comes with tis mode map and enable
> this interactive stuff.

This idea does sound useful.

> ascii tables or org tables started as its own mode but got consumed by org. They
> are still usable outside of org mode. I can create table with org-table-create
> and I can align table with org-table-align, but by default I don't have this
> functionality bound in some keymap if I am not in org-mode. Perhaps I am just
> not familiar with it. But this could be a minor mode also.

See orgtbl-mode.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Sv: Sv: Modularizing Org as GSoC project (was: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library))

[arthur miller]

[-- Attachment #1: Type: text/plain, Size: 10007 bytes --]

>>>         Do you want Org markup to be displayed in non-Org buffers?
>>
>> Well, part of it. Or more to make some parts of org-markup configurable, and
>> usable as minor modes so they can be easier used outside of org-mode.
>
>This is not the direction Org mode project is going. We do the reverse -
>Org markup is getting more stable, with the aim to simplify developing
>third-party parsers and eventually registering Org syntax in IANA MIME db.

I am not subscribed to mailing lists due to lots of trafic going on there, so I
am not overly introduced in directions in which org goes, so forgive me if I am
not so familiar with the latest features.

>However, stable Org markup does not mean that we cannot implement Org
>mode features outside Org mode. Another general direction we are going to
>is using parser APIs across Org mode. The idea is to encapsulate the
>fine details of Org markup into the parser, only leaving some very basic
>structures like recursiveness of markup objects exposed. In theory, one
>may eventually just replace the parser with, say, Markdown parser (with
>appropriate API), and get Org mode features in Markdown (well... except
>that Markdown has fewer kinds of syntax and certain things do not exist
>there)

Well, yes something like that; make features available outside of org-mode, and
make org-mode use those features instead of having everything in a monolithical
library.

>> For example about links, there could be a mode "text-link-mode" or
>> "pretty-links-mode" or something, that understands what a link description is,
>> and what a link itself is. The minor mode would have some mean to parse
>> description and link parts, and when on it would do what
>> org-toggle-link-display
>> does. For example org uses angle brackets for link desc and url, whereas
>> markdown uses angle brackets and parenthesis. Thus link-mode could/should be
>> enough customizable so that modes could be clients of this minor mode, as well
>> as for user to be able to setup a regex or set a function that recognizes some
>> custom syntax for descriptions and links. Also a minor mode can come with a
>> key
>> map and some actions, for example to follow link, to insert a link etc. I
>> think
>> of org-links, but a bit more generalized and usable without org-mode
>> itself. Org-mode could use those under the hood.
>
>I am not sure if many places (except Markdown) have a notion of link +
>description. And the rest is supported by, say, Hyperbole.

The idea was that if such a minor mode has a list of regexes it recognizes,
neither org-, md- or hyperbole would need to keep their code for this, but could
relly on users to just (desc-links-mode +1). If I had this as global minor mode,
I could also enable it while writing mail and use say md syntax and have pretty
links, which admittedly would come out ugly on the other end, but that is an
agreement between the reader and the writer. In other words org links or md
links would be available in any format and up to users where they want to use them.

>What might be more interesting is generalizing Org previews: Org can
>preview LaTeX and images, but we can think of it more generally - any
>kind of text object might have alternative (possibly image)
>representation. See https://list.orgmode.org/orgmode/87edbhljr7.fsf@hyperspace/
>
>> Similarly for italics, bolds, underlines, etc. Those could be slightly
>> generalized and taken out into minor mode. Clients could setup their own
>> start/end markers and use them to enrich the text on the display instead of
>> perhaps defining own faces and regexes. I don't know how useful and desirable
>> it
>> might be in other modes, perhaps in comments in programming languages, or
>> similar. Just a thought.
>
>You are describing font-lock here. Users can already add custom
>fontification in buffers by supplying regexps + face to be used for
>specific text fragments. What is the novelty?

Not a novelty at all :). The same as for links. A user could (rich-text-mode
+1), and have org-markup (or their own) available in any mode of their choice. I
had font-lock and thingatpt in my mind, but I didn't want to talk about the
implementation. After all, org uses font-lock under the hood for this anyway (or
am I too unfamiliar now? :)). It is just a convenience. In all of this cases, I
am thinking of those small minor modes be a tiny small "frameworks", where a
user can setup like very few regexes, or whatever is needed for a parser, and
have a "customized" syntax instead of going on through writing their own minor
mode.

>> org-timestamp interactiveity could be usable elsewhere than just in org
>> mode. For example I can insert  org-timestamp in any mode, but it does work to
>> use C-left/right to change the date. It could be refactored into small tiny
>> minor mode timestamp-mode or something, that comes with tis mode map and
>> enable
>> this interactive stuff.
>
>This idea does sound useful.
>
>> ascii tables or org tables started as its own mode but got consumed by org.
>> They
>> are still usable outside of org mode. I can create table with org-table-create
>> and I can align table with org-table-align, but by default I don't have this
>> functionality bound in some keymap if I am not in org-mode. Perhaps I am just
>> not familiar with it. But this could be a minor mode also.
>
>See orgtbl-mode.

I see. Thanks. Something like that for other features.
________________________________
Från: Ihor Radchenko <yantar92@posteo.net>
Skickat: den 22 augusti 2024 14:23
Till: arthur miller <arthur.miller@live.com>
Kopia: emacs-devel@gnu.org <emacs-devel@gnu.org>; emacs-orgmode@gnu.org <emacs-orgmode@gnu.org>
Ämne: Re: Sv: Modularizing Org as GSoC project (was: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library))

arthur miller <arthur.miller@live.com> writes:

>>         Do you want Org markup to be displayed in non-Org buffers?
>
> Well, part of it. Or more to make some parts of org-markup configurable, and
> usable as minor modes so they can be easier used outside of org-mode.

This is not the direction Org mode project is going. We do the reverse -
Org markup is getting more stable, with the aim to simplify developing
third-party parsers and eventually registering Org syntax in IANA MIME db.

However, stable Org markup does not mean that we cannot implement Org
mode features outside Org mode. Another general direction we are going to
is using parser APIs across Org mode. The idea is to encapsulate the
fine details of Org markup into the parser, only leaving some very basic
structures like recursiveness of markup objects exposed. In theory, one
may eventually just replace the parser with, say, Markdown parser (with
appropriate API), and get Org mode features in Markdown (well... except
that Markdown has fewer kinds of syntax and certain things do not exist
there)

> For example about links, there could be a mode "text-link-mode" or
> "pretty-links-mode" or something, that understands what a link description is,
> and what a link itself is. The minor mode would have some mean to parse
> description and link parts, and when on it would do what org-toggle-link-display
> does. For example org uses angle brackets for link desc and url, whereas
> markdown uses angle brackets and parenthesis. Thus link-mode could/should be
> enough customizable so that modes could be clients of this minor mode, as well
> as for user to be able to setup a regex or set a function that recognizes some
> custom syntax for descriptions and links. Also a minor mode can come with a key
> map and some actions, for example to follow link, to insert a link etc. I think
> of org-links, but a bit more generalized and usable without org-mode
> itself. Org-mode could use those under the hood.

I am not sure if many places (except Markdown) have a notion of link +
description. And the rest is supported by, say, Hyperbole.

What might be more interesting is generalizing Org previews: Org can
preview LaTeX and images, but we can think of it more generally - any
kind of text object might have alternative (possibly image)
representation. See https://list.orgmode.org/orgmode/87edbhljr7.fsf@hyperspace/

> Similarly for italics, bolds, underlines, etc. Those could be slightly
> generalized and taken out into minor mode. Clients could setup their own
> start/end markers and use them to enrich the text on the display instead of
> perhaps defining own faces and regexes. I don't know how useful and desirable it
> might be in other modes, perhaps in comments in programming languages, or
> similar. Just a thought.

You are describing font-lock here. Users can already add custom
fontification in buffers by supplying regexps + face to be used for
specific text fragments. What is the novelty?

> org-timestamp interactiveity could be usable elsewhere than just in org
> mode. For example I can insert  org-timestamp in any mode, but it does work to
> use C-left/right to change the date. It could be refactored into small tiny
> minor mode timestamp-mode or something, that comes with tis mode map and enable
> this interactive stuff.

This idea does sound useful.

> ascii tables or org tables started as its own mode but got consumed by org. They
> are still usable outside of org mode. I can create table with org-table-create
> and I can align table with org-table-align, but by default I don't have this
> functionality bound in some keymap if I am not in org-mode. Perhaps I am just
> not familiar with it. But this could be a minor mode also.

See orgtbl-mode.

--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

[-- Attachment #2: Type: text/html, Size: 22272 bytes --]

Re: Sv: Sv: Modularizing Org as GSoC project (was: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library))

[Ihor Radchenko]

arthur miller <arthur.miller@live.com> writes:

>>> Well, part of it. Or more to make some parts of org-markup configurable, and
>>> usable as minor modes so they can be easier used outside of org-mode.
>>
>>This is not the direction Org mode project is going. We do the reverse -
>>Org markup is getting more stable, with the aim to simplify developing
>>third-party parsers and eventually registering Org syntax in IANA MIME db.
>
> I am not subscribed to mailing lists due to lots of trafic going on there, so I
> am not overly introduced in directions in which org goes, so forgive me if I am
> not so familiar with the latest features.

Sorry, I did not intend my comment to sound personal. I just explained
why configurable markup is not something I see as a good idea.

> Well, yes something like that; make features available outside of org-mode, and
> make org-mode use those features instead of having everything in a monolithical
> library.
> ...

To conclude, we may use these ideas for GSoC projects, but maybe
separately. I doubt that "modularizing" in general would be a reasonable
task to finish within summer.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Sv: Sv: Sv: Modularizing Org as GSoC project (was: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library))

[arthur miller]

[-- Attachment #1: Type: text/plain, Size: 2859 bytes --]

>arthur miller <arthur.miller@live.com> writes:
>
>>>> Well, part of it. Or more to make some parts of org-markup configurable, and
>>>> usable as minor modes so they can be easier used outside of org-mode.
>>>
>>>This is not the direction Org mode project is going. We do the reverse -
>>>Org markup is getting more stable, with the aim to simplify developing
>>>third-party parsers and eventually registering Org syntax in IANA MIME db.
>>
>> I am not subscribed to mailing lists due to lots of trafic going on there, so
>> I
>> am not overly introduced in directions in which org goes, so forgive me if I
>> am
>> not so familiar with the latest features.
>
>Sorry, I did not intend my comment to sound personal. I just explained
>why configurable markup is not something I see as a good idea.

No worries; I didn't took it personally at all; I was just letting you
know I am not very up-to date with latest development, and where Org is
going, if I suggest something that is diagonally opposite to Org development.
________________________________
Från: Ihor Radchenko <yantar92@posteo.net>
Skickat: den 31 augusti 2024 14:20
Till: arthur miller <arthur.miller@live.com>
Kopia: emacs-devel@gnu.org <emacs-devel@gnu.org>; emacs-orgmode@gnu.org <emacs-orgmode@gnu.org>
Ämne: Re: Sv: Sv: Modularizing Org as GSoC project (was: New Emacs features via Google Summer of Code (or other similar stipend schemes) (was: as for Calc and the math library))

arthur miller <arthur.miller@live.com> writes:

>>> Well, part of it. Or more to make some parts of org-markup configurable, and
>>> usable as minor modes so they can be easier used outside of org-mode.
>>
>>This is not the direction Org mode project is going. We do the reverse -
>>Org markup is getting more stable, with the aim to simplify developing
>>third-party parsers and eventually registering Org syntax in IANA MIME db.
>
> I am not subscribed to mailing lists due to lots of trafic going on there, so I
> am not overly introduced in directions in which org goes, so forgive me if I am
> not so familiar with the latest features.

Sorry, I did not intend my comment to sound personal. I just explained
why configurable markup is not something I see as a good idea.

> Well, yes something like that; make features available outside of org-mode, and
> make org-mode use those features instead of having everything in a monolithical
> library.
> ...

To conclude, we may use these ideas for GSoC projects, but maybe
separately. I doubt that "modularizing" in general would be a reasonable
task to finish within summer.

--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

[-- Attachment #2: Type: text/html, Size: 6182 bytes --]