Enhancing the HTML exporter: Create a new backend or contribute to the upstream

Enhancing the HTML exporter: Create a new backend or contribute to the upstream

[Sébastien Gendre]

[-- Attachment #1: Type: text/plain, Size: 2038 bytes --]


TL;DR: Plan to add features to HTML export, do I add them in the
       built-in HTML exporter (as upstream contribution) ? Or as a
       downloadable derived backend ? Which on do you prefer ?


Hello,

I plan to use Org-publish to create a website from Org-mode files. And I
would have some features that are not provided by the built-in HTML
exporter.

To add theses features, my first plan was to customize an org-publish
project. Using a custom preamble function, a custom completion function
and some new export options added to "org-export-options-alist".

It's still a work in progress, but the more feature I add, the more I
think I need to add them in an export backend.

So, my second plan is to create an export backend, derived from the
built-in HTML exporter.

But maybe you would prefer that I add these new features as a
contribution to the built-in HTML exporter ?

Here is what I plan to add, on each generated webpage, compared to what
the built-in HTML exporter already provide:

- A side panel containing:
  - A site web name and/or logo
  - A search field for an local search engine
  - A main navigation menu, built from a dedicated org-mode file
- A more "modern" look
- More special blocks available like:
  - A question/answer bloc, where the answer is hidden
  - Important, warning and tip blocks
  - A generic hide-show bloc
- Tab to select which content to see
- Bibliography on a dedicated webpage when using org-publish
- A button to download the Org-mode file source of a webpage
- Possibility to set the home page, when there is no index.org
  
As say above, I started it as a custom org-publish
project. You can found my work in progress here:
https://framagit.org/SebGen/org-documentation-template

For now, the search engine use the software Pagefind:
https://pagefind.app

And the menu is made from an org-mode file where first level heading
become menu item and links are simple org-mode links.


Best regards

-------
Gendre Sébastien

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 849 bytes --]

Re: Enhancing the HTML exporter: Create a new backend or contribute to the upstream

[Sébastien Gendre]

[-- Attachment #1: Type: text/plain, Size: 2601 bytes --]


Some precision about the time:

I'm actually finishing the last work I have to do for the school, so I
will be very busy until December.

Until them, I can only dedicate my breaks to the enhancement of the HTML exporter.



Sébastien Gendre <seb@k-7.ch> writes:

> [[PGP Signed Part:Good signature from B586F7C77239E29E Sébastien Gendre <seb@k-7.ch> (trust ultimate) created at 2024-08-28T03:29:14+0200 using RSA]]
>
> TL;DR: Plan to add features to HTML export, do I add them in the
>        built-in HTML exporter (as upstream contribution) ? Or as a
>        downloadable derived backend ? Which on do you prefer ?
>
>
> Hello,
>
> I plan to use Org-publish to create a website from Org-mode files. And I
> would have some features that are not provided by the built-in HTML
> exporter.
>
> To add theses features, my first plan was to customize an org-publish
> project. Using a custom preamble function, a custom completion function
> and some new export options added to "org-export-options-alist".
>
> It's still a work in progress, but the more feature I add, the more I
> think I need to add them in an export backend.
>
> So, my second plan is to create an export backend, derived from the
> built-in HTML exporter.
>
> But maybe you would prefer that I add these new features as a
> contribution to the built-in HTML exporter ?
>
> Here is what I plan to add, on each generated webpage, compared to what
> the built-in HTML exporter already provide:
>
> - A side panel containing:
>   - A site web name and/or logo
>   - A search field for an local search engine
>   - A main navigation menu, built from a dedicated org-mode file
> - A more "modern" look
> - More special blocks available like:
>   - A question/answer bloc, where the answer is hidden
>   - Important, warning and tip blocks
>   - A generic hide-show bloc
> - Tab to select which content to see
> - Bibliography on a dedicated webpage when using org-publish
> - A button to download the Org-mode file source of a webpage
> - Possibility to set the home page, when there is no index.org
>   
> As say above, I started it as a custom org-publish
> project. You can found my work in progress here:
> https://framagit.org/SebGen/org-documentation-template
>
> For now, the search engine use the software Pagefind:
> https://pagefind.app
>
> And the menu is made from an org-mode file where first level heading
> become menu item and links are simple org-mode links.
>
>
> Best regards
>
> -------
> Gendre Sébastien
>
> [[End of PGP Signed Part]]

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 849 bytes --]

Re: Enhancing the HTML exporter: Create a new backend or contribute to the upstream

[Ihor Radchenko]

Sébastien Gendre <seb@k-7.ch> writes:

> Here is what I plan to add, on each generated webpage, compared to what
> the built-in HTML exporter already provide:
>
> - A side panel containing:

Some kind of side/top panel would make sense, I think.

>   - A site web name and/or logo

+1

>   - A search field for an local search engine

+1, but I would like to know more details - will it be something you
want to ship with Org mode, as JS?

> For now, the search engine use the software Pagefind:
> https://pagefind.app

It is a very young project (2022). Any alternatives? Preferably,
well-established.

>   - A main navigation menu, built from a dedicated org-mode file

Maybe, but that's probably a feature for org-publish.
Maybe we can somehow integrate it with TOC functionality.

More generally, some kind of side (or not side) panel sounds
reasonable. What to put inside is less important as long as we allow
customizability.

> - A more "modern" look

That can be anything, so I need more details to say anything.

> - More special blocks available like:
>   - A question/answer bloc, where the answer is hidden
>   - Important, warning and tip blocks
>   - A generic hide-show bloc

Are you referring to extending the default CSS? Something else?

> - Tab to select which content to see

May you elaborate?

> - Bibliography on a dedicated webpage when using org-publish

Probably fits within "multipage export" feature we are discussing now
https://list.orgmode.org/orgmode/ZoUdiTfbYqzPwTiX@orm-t14s/T/#u

> - A button to download the Org-mode file source of a webpage

On top panel? It might be useful as default top/side panel settings in
org-publish. Not sure.

> - Possibility to set the home page, when there is no index.org

May you elaborate?
   
> And the menu is made from an org-mode file where first level heading
> become menu item and links are simple org-mode links.

This kind of idea should be discussed in more details.
I see it as a way to define special HTML markup from Org markup, but it
is a question how to implement such a feature.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Enhancing the HTML exporter: Create a new backend or contribute to the upstream

[Sébastien Gendre]

[-- Attachment #1: Type: text/plain, Size: 8514 bytes --]


Hello. Thank you for your reply, I add mines after the quotes.


Ihor Radchenko <yantar92@posteo.net> writes:

>>   - A search field for an local search engine
>
> +1, but I would like to know more details - will it be something you
> want to ship with Org mode, as JS?

For now, I have done successful testes with Pagefind. When you execute
the "pagefind" binary on the root of the generated website, it will do
the indexation of the HTML pages and add a subfolder named "pagefind/".

This subfolder contain all the JS, CSS and a data needed to add a
functioning search field to each webpage.

To add the search field, you only need to add this on the webpages:

    <link href="/pagefind/pagefind-ui.css" rel="stylesheet">
    <script src="/pagefind/pagefind-ui.js"></script>
    <div id="search"></div>
    <script>
        window.addEventListener('DOMContentLoaded', (event) => {
            new PagefindUI({ element: "#search", showSubResults: true });
        });
    </script>

For readers who want more information about Pagefind, here is its
documentation:

https://pagefind.app/docs/


>> For now, the search engine use the software Pagefind:
>> https://pagefind.app
>
> It is a very young project (2022). Any alternatives? Preferably,
> well-established.

I can search for another project.

I don't know how many work it would be needed to develop a search motor
specifically for Org-mode. But doing the indexing on Org-mode files could
let the user control the indexation of each page and section directly
from them. With buffer settings and heading properties.


>>   - A main navigation menu, built from a dedicated org-mode file
>
> Maybe, but that's probably a feature for org-publish.
> Maybe we can somehow integrate it with TOC functionality.

I was thinking of the main navigation menu to navigate in the website,
between web pages, while the TOC is for navigate on one page.

Integrate the 2 is a good idea, it can help building one unified menu on
the side panel. But separate the 2 let more flexibility to users who
want to customize the exported web page. And, maybe, it can help user to
navigate too. I'm not an UX expert.

For example, on Jupyterbook and Sphink generated webpages, the website
navigation menu is on the left and the page table of content is on the
right. (On desktop)


>> - A more "modern" look
>
> That can be anything, so I need more details to say anything.

My inspiration for "modern" look are:

* Jupyterbook: https://jupyterbook.org

* The Furo theme for Sphinx: https://github.com/pradyunsg/furo


>> - More special blocks available like:
>>   - A question/answer bloc, where the answer is hidden
>>   - Important, warning and tip blocks
>>   - A generic hide-show bloc
>
> Are you referring to extending the default CSS? Something else?

At first, I was thinking of extending the CSS theme. But after
reflection, I think it would be better to introduce new kind of Org-mode
block. So we can manage their exportation to other format, like LaTeX.

My inspiration for the 2 last kind of blocks are from "Special Content
Blocks", from Jupyterbook:
https://jupyterbook.org/en/stable/content/content-blocks.html

For the question/answer, the idea is to facilitate the creation of
exercises. When a user write a lesson on a subject and want to give the
readers some practice. It would be nice to have a way to indicate which
text is the question and which text is the answer.

On the export, a question and its answer are rendered inside a frame,
with a question number. The question can have sub-questions, like 2.a,
2.b, etc.

And in case of HTML export, the answer could be hidden and shown when
user click on a button "show reply".

But, maybe this question/answer block is a big feature and need to be
kept for the future.


>> - Tab to select which content to see
>
> May you elaborate?

My inspiration are the "Tab Content" from Jupyterbook:
https://jupyterbook.org/en/stable/content/components.html#tab-content

A use case could be in a documentation, when the writer want to share a
same example in multiple languages.

But, as for the previous blocks, it maybe need a new Org-mode element.
So we can also manage it when export to LaTeX or other formats.


>> - Bibliography on a dedicated webpage when using org-publish
>
> Probably fits within "multipage export" feature we are discussing now
> https://list.orgmode.org/orgmode/ZoUdiTfbYqzPwTiX@orm-t14s/T/#u

I will take a look.


>> - A button to download the Org-mode file source of a webpage
>
> On top panel? It might be useful as default top/side panel settings in
> org-publish. Not sure.

I was more thinking to put this button at the top of the content, as
it's the source of the webpage. While the top/side panel is more related
to the website itself.

But it make sense to have it as an org-publish setting. For example, a
":html-pulish-source". When set as "t", the HTML publish function copy
the Org-mode source beside the generated HTML file. And include a link
in the HTML file to the copied Org-mode file.

Or is it better to use the "org-publish-attachment" function to copy the
Org-mode files ?


>> - Possibility to set the home page, when there is no index.org
>
> May you elaborate?

Today, if you publish multiple Org-mode file and one of them are named
"index.org", this file will be exported to "index.html". And it will
become the default web page when someone visit your website without
specifying a path.

But if you don't want to create a file named "index.org", but define
another document as the website home page, it would be nice to have the
possibility to define it.

For example: The user set an Org-publish option ":html-home-document" to
"Introduction.org". After doing the publishing, an "index.html" file is
automatically created on the output folder and this file contain a
redirection to "Introduction.html". (If no index.org already exist)


>> And the menu is made from an org-mode file where first level heading
>> become menu item and links are simple org-mode links.
>
> This kind of idea should be discussed in more details.
> I see it as a way to define special HTML markup from Org markup, but it
> is a question how to implement such a feature.

I agree. I started with an Org-mode file to do some tests, but it
should be discussed in more details.

From what I understand, the need is to have a simple way to define a
website menu that:

  * Can have 2 levels

  * Each entry can be a link to an exported Org document or any raw link

  * We should avoid link on a menu entry who contain a sub menu,
    otherwise it become difficult to use on a touchscreen



While we talk about the "org-html-publish-to-html":

I use a lot the Org-mode attachment function for files I want to use
inside a webpage. Images, archives, sounds, videos, etc. In a lot of
formats. And I manage them separately from the general static files
(CSS, JS, etc).

On the Org-mode side:

  1. I attach a file to the section I want to use it, with the copy or
     move action

  2. I create an "attachment" link to the file


For the static files, like CSS, JS and website logo:

  1. I create a "static/" folder, separated from the folder where the
     Org-mode files are written


On the Org-publish projects

  1. I define a project that use "org-html-publish-to-html" to export
     each Org-mode document to a web page

  2. I define a second project that use "org-publish-attachment" on the
     same folder, with all the used extension, only to publish the files
     who are inside the "data/" subfolder

  3. I define a third project that also use "org-publish-attachment" but
     for the static files


It feel strange to do the publication of attached file on a separate
project while "org-html-publish-to-html" already manage a part of the
attachment management by taking care of the "attachment" links. And also
to use a publish function named "org-publish-attachment" to publish
static files.

What do you think about adding an option in "org-html-publish-to-html"
to copy each file attached to and linked from an Org-mode heading ?
Independently of its extension. But maybe we need to rename the function
"org-publish-attachment" to "org-publish-static-files" to avoid
confusion.



Best regards

-------
Gendre Sébastien

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 849 bytes --]

Re: Enhancing the HTML exporter: Create a new backend or contribute to the upstream

[Ihor Radchenko]

Before I continue with my replies, I'd like to add one general comment.

You are suggesting a number of new diverse features.
Ideally, it is better to discuss and add them separately.
Especially, when we move ahead and discuss the concrete implementations
or patches.

I will also provide some links to existing discussions on similar
features. It is a good idea to continue those discussions, so that we do
not spread things across the mailing list too much.

Sébastien Gendre <seb@k-7.ch> writes:

>>>   - A search field for an local search engine
>>
>> +1, but I would like to know more details - will it be something you
>> want to ship with Org mode, as JS?
>
> For now, I have done successful testes with Pagefind. When you execute
> the "pagefind" binary on the root of the generated website, it will do
> the indexation of the HTML pages ....
>
> This subfolder contain all the JS, CSS and a data needed to add a
> functioning search field to each webpage. ...
>
>>> For now, the search engine use the software Pagefind:
>>> https://pagefind.app
>>
>> It is a very young project (2022). Any alternatives? Preferably,
>> well-established.
>
> I can search for another project.
> 
> I don't know how many work it would be needed to develop a search motor
> specifically for Org-mode. But doing the indexing on Org-mode files could
> let the user control the indexation of each page and section directly
> from them. With buffer settings and heading properties.

Let me clarify.
Unless an indexer is very simple, we do not really want it in Org - it
will put a lot of extra load on maintainers.

On the other side, we do not want to tie things to some project that
may fade out in 10 years into the future.

The way I see search engine support in Org mode is either:

1. Using some really established project that we can expect to last for
   many years.

2. Implementing pluggable search support where users can choose which
   indexer/searcher to use

(2) will be the best.

So, may you please search across available search engines and see what
they have in common. Then, we can work out some infrastructure that is
generic enough to plug a custom engine.

Then, pagefind can be the default (it is MIT license - GPL compatible),
unless someone proposes a better alternative.

>>>   - A main navigation menu, built from a dedicated org-mode file
>>
>> Maybe, but that's probably a feature for org-publish.
>> Maybe we can somehow integrate it with TOC functionality.
>
> I was thinking of the main navigation menu to navigate in the website,
> between web pages, while the TOC is for navigate on one page.
>
> Integrate the 2 is a good idea, it can help building one unified menu on
> the side panel. But separate the 2 let more flexibility to users who
> want to customize the exported web page. And, maybe, it can help user to
> navigate too. I'm not an UX expert.
>
> For example, on Jupyterbook and Sphink generated webpages, the website
> navigation menu is on the left and the page table of content is on the
> right. (On desktop)

I see. I do not mind a site-wide navigation menu.

I think that we may do it in the following way:

1. Provide an option for users to specify the html source of such menu

2. Provide an option to generate global site menu from org-publish, or
   from a specified Org file

I'd like to provide html source approach because I saw some people using
python scripts to generate such global menus. It would be nice to
support such (existing) use case.

>>> - A more "modern" look
>>
>> That can be anything, so I need more details to say anything.
>
> My inspiration for "modern" look are:
>
> * Jupyterbook: https://jupyterbook.org
>
> * The Furo theme for Sphinx: https://github.com/pradyunsg/furo

Sorry, but that's not what I am asking about.
I do not care much about specifics of the color scheme and so.
I am looking to know what specific visual features you want to add.

From a quick glance, there is a toggle between dark/ligh themes. What else?

>>> - More special blocks available like:
>>>   - A question/answer bloc, where the answer is hidden
>>>   - Important, warning and tip blocks
>>>   - A generic hide-show bloc
>>
>> Are you referring to extending the default CSS? Something else?
>
> At first, I was thinking of extending the CSS theme. But after
> reflection, I think it would be better to introduce new kind of Org-mode
> block. So we can manage their exportation to other format, like LaTeX.
>
> My inspiration for the 2 last kind of blocks are from "Special Content
> Blocks", from Jupyterbook:
> https://jupyterbook.org/en/stable/content/content-blocks.html

Maybe something like
https://list.orgmode.org/orgmode/8B86B788-3B46-4879-ABDB-3AD04A7EC0CE@gmail.com/
?
If yes, we may continue discussion there.

> For the question/answer, the idea is to facilitate the creation of
> exercises. When a user write a lesson on a subject and want to give the
> readers some practice. It would be nice to have a way to indicate which
> text is the question and which text is the answer.
>
> On the export, a question and its answer are rendered inside a frame,
> with a question number. The question can have sub-questions, like 2.a,
> 2.b, etc.

> And in case of HTML export, the answer could be hidden and shown when
> user click on a button "show reply".

In HTML5 export, ox-html does support details and summary special
blocks. Will it be good enough?

See https://developer.mozilla.org/en-US/docs/Web/HTML/Element/summary

>>> - Tab to select which content to see
>>
>> May you elaborate?
>
> My inspiration are the "Tab Content" from Jupyterbook:
> https://jupyterbook.org/en/stable/content/components.html#tab-content
>
> A use case could be in a documentation, when the writer want to share a
> same example in multiple languages.
>
> But, as for the previous blocks, it maybe need a new Org-mode element.
> So we can also manage it when export to LaTeX or other formats.

Why do we need a new element here? Maybe just an export option for, say, list?

>>> - A button to download the Org-mode file source of a webpage
>>
>> On top panel? It might be useful as default top/side panel settings in
>> org-publish. Not sure.
>
> I was more thinking to put this button at the top of the content, as
> it's the source of the webpage. While the top/side panel is more related
> to the website itself.
>
> But it make sense to have it as an org-publish setting. For example, a
> ":html-pulish-source". When set as "t", the HTML publish function copy
> the Org-mode source beside the generated HTML file. And include a link
> in the HTML file to the copied Org-mode file.
>
> Or is it better to use the "org-publish-attachment" function to copy the
> Org-mode files ?

:htlm-publish-source sounds reasonable.

>>> - Possibility to set the home page, when there is no index.org
>>
>> May you elaborate?
>
> Today, if you publish multiple Org-mode file and one of them are named
> "index.org", this file will be exported to "index.html". And it will
> become the default web page when someone visit your website without
> specifying a path.
>
> But if you don't want to create a file named "index.org", but define
> another document as the website home page, it would be nice to have the
> possibility to define it.
>
> For example: The user set an Org-publish option ":html-home-document" to
> "Introduction.org". After doing the publishing, an "index.html" file is
> automatically created on the output folder and this file contain a
> redirection to "Introduction.html". (If no index.org already exist)

You can simply specify #+EXPORT_FILE_NAME for your desired index.html
file, even when the Org source has different name.

For redirection, it can be done my multiple means, not necessary via
dedicated redirect page.

I feel that what you are suggesting here is way too niche as a general
upstream feature.

> While we talk about the "org-html-publish-to-html":
>
> I use a lot the Org-mode attachment function for files I want to use
> inside a webpage. Images, archives, sounds, videos, etc. In a lot of
> formats. And I manage them separately from the general static files
> (CSS, JS, etc).
>
> On the Org-mode side:
>
>   1. I attach a file to the section I want to use it, with the copy or
>      move action
>
>   2. I create an "attachment" link to the file
>
>
> For the static files, like CSS, JS and website logo:
>
>   1. I create a "static/" folder, separated from the folder where the
>      Org-mode files are written
>
>
> On the Org-publish projects
>
>   1. I define a project that use "org-html-publish-to-html" to export
>      each Org-mode document to a web page
>
>   2. I define a second project that use "org-publish-attachment" on the
>      same folder, with all the used extension, only to publish the files
>      who are inside the "data/" subfolder
>
>   3. I define a third project that also use "org-publish-attachment" but
>      for the static files
>
> It feel strange to do the publication of attached file on a separate
> project while "org-html-publish-to-html" already manage a part of the
> attachment management by taking care of the "attachment" links. And also
> to use a publish function named "org-publish-attachment" to publish
> static files.
>
> What do you think about adding an option in "org-html-publish-to-html"
> to copy each file attached to and linked from an Org-mode heading ?
> Independently of its extension. But maybe we need to rename the function
> "org-publish-attachment" to "org-publish-static-files" to avoid
> confusion.

That might be useful, yes.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Enhancing the HTML exporter: Create a new backend or contribute to the upstream

[Max Nikulin]

On 07/09/2024 18:53, Ihor Radchenko wrote:
> Then, pagefind can be the default (it is MIT license - GPL compatible),

It might be more tricky:

<https://yhetil.org/emacs-devel/861q671idt.fsf@gnu.org>
emacs-devel. Re: [Nicolas Graves] [PATCH v6 01/10] rde: emacs: Start 
emacs in --daemon mode, with shepherd and pid-file. Sun, 12 May 2024 
12:36:46 +0300
> Nicolas Graves:
>> The code is given as MIT-0, hence also the two different licenses for
>> the two functions sd_notify and sd_is_socket. Not an expert on licenses
>> either, but with a proper flag about what this function's license is, I
>> guess it should be fine, since other projects also do that.

Eli Zaretskii:
> The license is only half of the problem.  Every non-trivial
> contribution to Emacs must have its copyright assigned to the FSF,
> because the FSF is in charge of protecting the Emacs sources,
> something that only the copyright holder can do, at least in some
> countries.  You will need to assign the copyright as well (a
> relatively simple procedure of filling a form and emailing it), but if
> the code is not yours, you cannot assign its copyright.

Re: Enhancing the HTML exporter: Create a new backend or contribute to the upstream

[Max Nikulin]

On 08/09/2024 21:46, Max Nikulin wrote:
> On 07/09/2024 18:53, Ihor Radchenko wrote:
>> Then, pagefind can be the default (it is MIT license - GPL compatible),
> 
> It might be more tricky:

Sorry for the noise. Of course, if you are not going to include any 
pagefind code into Org then it is not an issue.

Re: Enhancing the HTML exporter: Create a new backend or contribute to the upstream

[Orm Finnendahl]

Hi,

 that makes no sense to me whatsoever: Post processing is already
possible and built into org-export. pagefind is an external product
with its own binaries, not written in elisp nor being by any means
connected to emacs and compiling index files on generated HTML files
is exactly that: A post process.

The javascript needed and all processing scripts can easily be
included in the header, so I don't see any point in this, except
writing a tutorial, how to integrate pagefind into someone's HTML
output with the means already available with the existing backend.

And that's not even contemplating, why someone would want to throw a
multipage site search indexer onto single page HTML output which
doesn't work on static files opened from local disks ;-)

--
Orm

Am Sonntag, den 08. September 2024 um 22:55:01 Uhr (+0700) schrieb Max Nikulin:
> On 08/09/2024 21:46, Max Nikulin wrote:
> > On 07/09/2024 18:53, Ihor Radchenko wrote:
> > > Then, pagefind can be the default (it is MIT license - GPL compatible),
> > 
> > It might be more tricky:
> 
> Sorry for the noise. Of course, if you are not going to include any pagefind
> code into Org then it is not an issue.
> 
> 
> 

Re: Enhancing the HTML exporter: Create a new backend or contribute to the upstream

[Ihor Radchenko]

Orm Finnendahl <orm.finnendahl@selma.hfmdk-frankfurt.de> writes:

> The javascript needed and all processing scripts can easily be
> included in the header, so I don't see any point in this, except
> writing a tutorial, how to integrate pagefind into someone's HTML
> output with the means already available with the existing backend.
>
> And that's not even contemplating, why someone would want to throw a
> multipage site search indexer onto single page HTML output which
> doesn't work on static files opened from local disks ;-)

I agree that including indexer is just a question of adding specific
javascript.

But I think that it would be useful to provide some default toggle to
include such a tool without having to know the details of what js to
include and where. Just as a simple user option.

It should probably work for single page as well. I do not see why not.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Enhancing the HTML exporter: Create a new backend or contribute to the upstream

[Orm Finnendahl]

Am Sonntag, den 08. September 2024 um 18:42:51 Uhr (+0000) schrieb
Ihor Radchenko:
> I agree that including indexer is just a question of adding specific
> javascript.
> 
> But I think that it would be useful to provide some default toggle to
> include such a tool without having to know the details of what js to
> include and where. Just as a simple user option.
>

 whatever. The js is already included in the pagefind distribution, so
it is a simple

#+HTML_HEAD: <script src="./pagefind/pagefind-ui.js"></script>

in the org file and the searchbar html in the preamble (or wherever).

> It should probably work for single page as well. I do not see why
> not.

 sure it works. I just question the raison d'etre, when single page
search is already integrated into webbrowsers. But as always there
will be people arguing that it is necessary to have a search bar with
pop up results integrated into the page and of course there is nothing
wrong with that. I use pagefind myself, but the site I'm working on
(built with the multipage exporter BTW) currently contains more than
400 files where the browser search can't help.

--
Orm

Re: Enhancing the HTML exporter: Create a new backend or contribute to the upstream

[Ihor Radchenko]

Orm Finnendahl <orm.finnendahl@selma.hfmdk-frankfurt.de> writes:

>> It should probably work for single page as well. I do not see why
>> not.
>
>  sure it works. I just question the raison d'etre, when single page
> search is already integrated into webbrowsers. But as always there
> will be people arguing that it is necessary to have a search bar with
> pop up results integrated into the page and of course there is nothing
> wrong with that. I use pagefind myself, but the site I'm working on
> (built with the multipage exporter BTW) currently contains more than
> 400 files where the browser search can't help.

Agree. So, having search functionality probably makes more sense within
ox-publish, where we may also run post-processing to generate search
terms or whatever extra work is needed to make the indexer work.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Enhancing the HTML exporter: Create a new backend or contribute to the upstream

[Sébastien Gendre]

[-- Attachment #1: Type: text/plain, Size: 1196 bytes --]

Hello,

I have replied to your message, and the previous one, on the new thread
dedicated to the search engine feature, at "Remarks #6":

https://list.orgmode.org/878quf1ji0.fsf@k-7.ch/T/#m2425dcfc7665e88aefcbd81800d2d83aef98cd5b

Best regards

-------
Gendre Sébastien


Ihor Radchenko <yantar92@posteo.net> writes:

> Orm Finnendahl <orm.finnendahl@selma.hfmdk-frankfurt.de> writes:
>
>>> It should probably work for single page as well. I do not see why
>>> not.
>>
>>  sure it works. I just question the raison d'etre, when single page
>> search is already integrated into webbrowsers. But as always there
>> will be people arguing that it is necessary to have a search bar with
>> pop up results integrated into the page and of course there is nothing
>> wrong with that. I use pagefind myself, but the site I'm working on
>> (built with the multipage exporter BTW) currently contains more than
>> 400 files where the browser search can't help.
>
> Agree. So, having search functionality probably makes more sense within
> ox-publish, where we may also run post-processing to generate search
> terms or whatever extra work is needed to make the indexer work.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 849 bytes --]

Re: Enhancing the HTML exporter: Create a new backend or contribute to the upstream

[Sébastien Gendre]

[-- Attachment #1: Type: text/plain, Size: 1547 bytes --]

Hello,

I have replied to your message, and the previous one, on the new thread
dedicated to the search engine feature, at "Remarks #5":

https://list.orgmode.org/878quf1ji0.fsf@k-7.ch/T/#m2425dcfc7665e88aefcbd81800d2d83aef98cd5b

Best regards

-------
Gendre Sébastien


Orm Finnendahl <orm.finnendahl@selma.hfmdk-frankfurt.de> writes:

> Am Sonntag, den 08. September 2024 um 18:42:51 Uhr (+0000) schrieb
> Ihor Radchenko:
>> I agree that including indexer is just a question of adding specific
>> javascript.
>> 
>> But I think that it would be useful to provide some default toggle to
>> include such a tool without having to know the details of what js to
>> include and where. Just as a simple user option.
>>
>
>  whatever. The js is already included in the pagefind distribution, so
> it is a simple
>
> #+HTML_HEAD: <script src="./pagefind/pagefind-ui.js"></script>
>
> in the org file and the searchbar html in the preamble (or wherever).
>
>> It should probably work for single page as well. I do not see why
>> not.
>
>  sure it works. I just question the raison d'etre, when single page
> search is already integrated into webbrowsers. But as always there
> will be people arguing that it is necessary to have a search bar with
> pop up results integrated into the page and of course there is nothing
> wrong with that. I use pagefind myself, but the site I'm working on
> (built with the multipage exporter BTW) currently contains more than
> 400 files where the browser search can't help.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 849 bytes --]

Re: Enhancing the HTML exporter: Create a new backend or contribute to the upstream

[Sébastien Gendre]

[-- Attachment #1: Type: text/plain, Size: 1254 bytes --]

Hello,

I have replied to your message, and the previous one, on the new thread
dedicated to the search engine feature, at "Remarks #4":

https://list.orgmode.org/878quf1ji0.fsf@k-7.ch/T/#m2425dcfc7665e88aefcbd81800d2d83aef98cd5b

Best regards

-------
Gendre Sébastien

Ihor Radchenko <yantar92@posteo.net> writes:

> Orm Finnendahl <orm.finnendahl@selma.hfmdk-frankfurt.de> writes:
>
>> The javascript needed and all processing scripts can easily be
>> included in the header, so I don't see any point in this, except
>> writing a tutorial, how to integrate pagefind into someone's HTML
>> output with the means already available with the existing backend.
>>
>> And that's not even contemplating, why someone would want to throw a
>> multipage site search indexer onto single page HTML output which
>> doesn't work on static files opened from local disks ;-)
>
> I agree that including indexer is just a question of adding specific
> javascript.
>
> But I think that it would be useful to provide some default toggle to
> include such a tool without having to know the details of what js to
> include and where. Just as a simple user option.
>
> It should probably work for single page as well. I do not see why not.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 849 bytes --]

Re: Enhancing the HTML exporter: Create a new backend or contribute to the upstream

[Sébastien Gendre]

[-- Attachment #1: Type: text/plain, Size: 1163 bytes --]

Hello,

I have replied to your message, and the previous one, on the new thread
dedicated to the search engine feature, at "Remarks #3":

https://list.orgmode.org/878quf1ji0.fsf@k-7.ch/T/#m2425dcfc7665e88aefcbd81800d2d83aef98cd5b

Best regards

-------
Gendre Sébastien



Orm Finnendahl <orm.finnendahl@selma.hfmdk-frankfurt.de> writes:

> Hi,
>
>  that makes no sense to me whatsoever: Post processing is already
> possible and built into org-export. pagefind is an external product
> with its own binaries, not written in elisp nor being by any means
> connected to emacs and compiling index files on generated HTML files
> is exactly that: A post process.
>
> The javascript needed and all processing scripts can easily be
> included in the header, so I don't see any point in this, except
> writing a tutorial, how to integrate pagefind into someone's HTML
> output with the means already available with the existing backend.
>
> And that's not even contemplating, why someone would want to throw a
> multipage site search indexer onto single page HTML output which
> doesn't work on static files opened from local disks ;-)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 849 bytes --]

Re: Enhancing the HTML exporter: Create a new backend or contribute to the upstream

[Sébastien Gendre]

[-- Attachment #1: Type: text/plain, Size: 656 bytes --]


Hello,

I have replied to your message, and the previous one, on the new thread
dedicated to the search engine feature, at "Remarks #2":

https://list.orgmode.org/878quf1ji0.fsf@k-7.ch/T/#m2425dcfc7665e88aefcbd81800d2d83aef98cd5b

Best regards

-------
Gendre Sébastien



Max Nikulin <manikulin@gmail.com> writes:

> On 08/09/2024 21:46, Max Nikulin wrote:
>> On 07/09/2024 18:53, Ihor Radchenko wrote:
>>> Then, pagefind can be the default (it is MIT license - GPL compatible),
>> It might be more tricky:
>
> Sorry for the noise. Of course, if you are not going to include any
> pagefind code into Org then it is not an issue.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 849 bytes --]

Re: Enhancing the HTML exporter: Create a new backend or contribute to the upstream

[Sébastien Gendre]

[-- Attachment #1: Type: text/plain, Size: 2008 bytes --]

Ihor Radchenko <yantar92@posteo.net> writes:

> Before I continue with my replies, I'd like to add one general comment.
>
> You are suggesting a number of new diverse features.
> Ideally, it is better to discuss and add them separately.
> Especially, when we move ahead and discuss the concrete implementations
> or patches.
>
> I will also provide some links to existing discussions on similar
> features. It is a good idea to continue those discussions, so that we do
> not spread things across the mailing list too much.
>
> Sébastien Gendre <seb@k-7.ch> writes:

It's a good idea.

I have created a first thread to discuss the local search engine here:
https://list.orgmode.org/87a5ev31ul.fsf@k-7.ch/T/#u



> Let me clarify.
> Unless an indexer is very simple, we do not really want it in Org - it
> will put a lot of extra load on maintainers.
>
> On the other side, we do not want to tie things to some project that
> may fade out in 10 years into the future.
>
> The way I see search engine support in Org mode is either:
>
> 1. Using some really established project that we can expect to last for
>    many years.
>
> 2. Implementing pluggable search support where users can choose which
>    indexer/searcher to use
>
> (2) will be the best.
>
> So, may you please search across available search engines and see what
> they have in common. Then, we can work out some infrastructure that is
> generic enough to plug a custom engine.
>
> Then, pagefind can be the default (it is MIT license - GPL compatible),
> unless someone proposes a better alternative.

I have replied to this part on my first reply to the thread about the
search engine, at "Remarks #1":
https://list.orgmode.org/878quf1ji0.fsf@k-7.ch/T/#m2425dcfc7665e88aefcbd81800d2d83aef98cd5b


For the other parts, I have take notes of your replies and will reply on
the specific threads.

Thank you very much for all your replies. :)


Best regards

-------
Gendre Sébastien

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 849 bytes --]