From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id oKiDGkGIGGdZYAEAe85BDQ:P1 (envelope-from ) for ; Wed, 23 Oct 2024 05:23:13 +0000 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2.migadu.com with LMTPS id oKiDGkGIGGdZYAEAe85BDQ (envelope-from ) for ; Wed, 23 Oct 2024 07:23:13 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1729660993; a=rsa-sha256; cv=none; b=QJJP0OBKLIBLngzQgMAAZTo4EAXBy4lo4uUNrMAg+f1JyvbbaPTEIY4WexcB7SOc7FH4yy 9P4KUqx6HUA/t7MKFgj2GLX4MDFspE+yRufvzZCb2g4yG9r77SKIdCDxdIUOXOkgDXl8/a yJRRaeNPfFUklSX6JqEE3EjldUjj1A87dDFnCHp3lNFY2AzlAecq2P+HqiRf/AtHrwwp31 eJgsZd1x92vpUz411L3RSjvd9sg+uAcxlvnaWRHxBweo4IVf+BlwkMGEVF+2wUYYziEJ5U 5HbuZLU40y8/yTCuqqSK9w1rBMc6ZIlpKc0VEMSO9TXskZSs2NH7nd/IOaYc+w== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1729660993; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=WsX87b43LbfZfuMyCCdrAf7dBp7+T+rmu9HpmOHIWkQ=; b=V0YJaRh5iii2EiMcHTCdt/4xse02o374MmwFN9nd5CCtYloluEf1b1pxeToUgdRzZ9pCcU /N5DBqVK9VxxVZv4Dqrct9uHX7ckg0tWRDDqD4psBaNARQS6aiN1/79H3NvBT5lwe90Cip qSSWn41FAjSBlyU7Drymf7MwcvpnEO3hFfixM0w/AZFZwJehEd0Pso0aKiVzZdvU3D7F9u X6u4fF8jg0EFDNZWAysAZV57ACbZtY1Ll+wJVv+QSMWpoGRquZHlEgGPwkrF/HiL1peOIk o57TegdC6Fu9GU/xgabaf/ab7OEnX0+0uDqrtdlUWKwR8Kjce0rQaQxkQxLuPg== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 2D31F1A263 for ; Wed, 23 Oct 2024 07:23:12 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t3TpF-0006Qr-NY; Wed, 23 Oct 2024 01:22:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t3TpB-0006Qc-OM for emacs-orgmode@gnu.org; Wed, 23 Oct 2024 01:22:26 -0400 Received: from 96-100-31-185.ftth.cust.kwaoo.net ([185.31.100.96] helo=k-7.ch) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t3Tp7-000297-Rt for emacs-orgmode@gnu.org; Wed, 23 Oct 2024 01:22:24 -0400 Received: from van (_gateway [192.168.1.1]) (Authenticated sender: seb) by k-7.ch (Postfix) with ESMTPSA id 4C9802049B for ; Wed, 23 Oct 2024 07:22:16 +0200 (CEST) From: =?utf-8?Q?S=C3=A9bastien_Gendre?= To: emacs-orgmode@gnu.org Subject: Re: Feature discussion: Search field and local search engine In-Reply-To: <87a5ev31ul.fsf@k-7.ch> (=?utf-8?Q?=22S=C3=A9bastien?= Gendre"'s message of "Wed, 23 Oct 2024 06:00:34 +0200") References: <87a5ev31ul.fsf@k-7.ch> User-Agent: mu4e 1.12.1; emacs 29.4 Date: Wed, 23 Oct 2024 07:22:15 +0200 Message-ID: <878quf1ji0.fsf@k-7.ch> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Received-SPF: pass client-ip=185.31.100.96; envelope-from=seb@k-7.ch; helo=k-7.ch X-Spam_score_int: -8 X-Spam_score: -0.9 X-Spam_bar: / X-Spam_report: (-0.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, RDNS_DYNAMIC=0.982, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, TVD_RCVD_IP=0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -5.64 X-Spam-Score: -5.64 X-Migadu-Queue-Id: 2D31F1A263 X-Migadu-Scanner: mx10.migadu.com X-TUID: ZPu4inuDbLE9 --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Here is some quote of the previous discussion, with my reply below. * Remarks #1, by Ihor Radchenko at Sat, 07 Sep 2024 11:53 Ihor Radchenko writes: > > S=C3=A9bastien Gendre writes: > > I can search for another project. > >=20 > > I don't know how many work it would be needed to develop a search motor > > specifically for Org-mode. But doing the indexing on Org-mode files cou= ld > > let the user control the indexation of each page and section directly > > from them. With buffer settings and heading properties. >=20 > Let me clarify. > Unless an indexer is very simple, we do not really want it in Org - it > will put a lot of extra load on maintainers. >=20 > On the other side, we do not want to tie things to some project that > may fade out in 10 years into the future. >=20 > The way I see search engine support in Org mode is either: >=20 > 1. Using some really established project that we can expect to last for > many years. >=20 > 2. Implementing pluggable search support where users can choose which > indexer/searcher to use >=20 > (2) will be the best. >=20 > So, may you please search across available search engines and see what > they have in common. Then, we can work out some infrastructure that is > generic enough to plug a custom engine. >=20 > Then, pagefind can be the default (it is MIT license - GPL compatible), > unless someone proposes a better alternative. Yes, you are right, a pluggable search support is the best choice. I include it in the summary of this discussion, on the first message of this thread. I will continue my search of other search engine and do a summary of what they have in common. I will publish it on this thread, attached to the first message. * Remarks #2, from Max Nikulin at Sun, 8 Sep 2024 21:46 and 22:55 Max Nikulin writes: > On 07/09/2024 18:53, Ihor Radchenko wrote: > > Then, pagefind can be the default (it is MIT license - GPL compatible), >=20 > It might be more tricky: >=20 > > emacs-devel. Re: [Nicolas Graves] [PATCH v6 01/10] rde: emacs: Start=20 > emacs in --daemon mode, with shepherd and pid-file. Sun, 12 May 2024=20 > 12:36:46 +0300 > > Nicolas Graves: > >> The code is given as MIT-0, hence also the two different licenses for > >> the two functions sd_notify and sd_is_socket. Not an expert on licenses > >> either, but with a proper flag about what this function's license is, I > >> guess it should be fine, since other projects also do that. >=20 > Eli Zaretskii: > > The license is only half of the problem. Every non-trivial > > contribution to Emacs must have its copyright assigned to the FSF, > > because the FSF is in charge of protecting the Emacs sources, > > something that only the copyright holder can do, at least in some > > countries. You will need to assign the copyright as well (a > > relatively simple procedure of filling a form and emailing it), but if > > the code is not yours, you cannot assign its copyright. Max Nikulin writes: > On 08/09/2024 21:46, Max Nikulin wrote: > > On 07/09/2024 18:53, Ihor Radchenko wrote: > >> Then, pagefind can be the default (it is MIT license - GPL compatible), > >=20 > > It might be more tricky: >=20 > Sorry for the noise. Of course, if you are not going to include any=20 > pagefind code into Org then it is not an issue. If we use PageFind, isn't possible to not include it with org-mode but have an Elisp function that download it=C2=A0? Like what Elpy do with its P= ython dependencies=C2=A0? * Remarks #3, by Orm Finnendahl, at Sun, 8 Sep 2024 20:36 Orm Finnendahl writes: > that makes no sense to me whatsoever: Post processing is already > possible and built into org-export. pagefind is an external product > with its own binaries, not written in elisp nor being by any means > connected to emacs and compiling index files on generated HTML files > is exactly that: A post process. >=20 > The javascript needed and all processing scripts can easily be > included in the header, so I don't see any point in this, except > writing a tutorial, how to integrate pagefind into someone's HTML > output with the means already available with the existing backend. It is already possible to add a search field and do PageFind indexation and JS/CSS installation by set custom HTML in preamble and a custom post-processing function. That what I have started to do. But: It require the user to write custom HTML, custom Elisp function, understand how PageFind work, etc. The feature I suggest is to let the user having this search engine on its website by simply set an org-publish option to "t". A local search engine don't seems to be a niche feature. You have it with Jupyter Book [1] or Read The Docs documentations [2]. As Org-mode is a fantastic tool to write online documentation, having local search engine easy to setup is a good feature. It's also very useful for blogs to let visitor search information in old posts. If this feature is simple to use, it let the user concentrate on writing the content. And regarding the inclusion, or not, of another software: We can have an Elisp function that download PageFind=C2=A0? If not, we can display a messa= ge asking user to install it on its own if Emacs doesn't found PageFind on $PATH. Orm Finnendahl writes: > And that's not even contemplating, why someone would want to throw a > multipage site search indexer onto single page HTML output which > doesn't work on static files opened from local disks ;-) For the single HTML=C2=A0export, indexing is not necessary. But the inclusi= on of a search field could be if the page have been generated separately from the org-publish=C2=A0? For the last part about files opened from local disks, what do you mean=C2= =A0? * Remarks #4, by Ihor Radchenko at 08 Sep 2024 18:42 Ihor Radchenko writes: > Orm Finnendahl writes: >=20 > > The javascript needed and all processing scripts can easily be > > included in the header, so I don't see any point in this, except > > writing a tutorial, how to integrate pagefind into someone's HTML > > output with the means already available with the existing backend. > > > > And that's not even contemplating, why someone would want to throw a > > multipage site search indexer onto single page HTML output which > > doesn't work on static files opened from local disks ;-) >=20 > I agree that including indexer is just a question of adding specific > javascript. >=20 > But I think that it would be useful to provide some default toggle to > include such a tool without having to know the details of what js to > include and where. Just as a simple user option. >=20 > It should probably work for single page as well. I do not see why not. I agree with Ihor, the goal is to have a simple user option to enable this feature. * Remarks #5, by Orm Finnendahl at 8 Sep 2024 21:25 Orm Finnendahl writes: > Am Sonntag, den 08. September 2024 um 18:42:51 Uhr (+0000) schrieb > Ihor Radchenko: > > I agree that including indexer is just a question of adding specific > > javascript. > >=20 > > But I think that it would be useful to provide some default toggle to > > include such a tool without having to know the details of what js to > > include and where. Just as a simple user option. > > >=20 > whatever. The js is already included in the pagefind distribution, so > it is a simple >=20 > #+HTML_HEAD: >=20 > in the org file and the searchbar html in the preamble (or wherever). This would require the user to write custom HTML code and also write a custom Elisp function to launch the indexation and installation of JS/CSS. And the user would also need to understand how PageFind work. It's a lot of steps and require different skills. But if we provide a simple option to enable in org-publish, this could be very simple for the user to have a search engine. Of course, we are not gonna support every use case of search engine, only the most simple and useful one. And the pluggable system let to user the possibility to hack it the way she or he want. Orm Finnendahl writes: > > It should probably work for single page as well. I do not see why > > not. >=20 > sure it works. I just question the raison d'etre, when single page > search is already integrated into webbrowsers. But as always there > will be people arguing that it is necessary to have a search bar with > pop up results integrated into the page and of course there is nothing > wrong with that. I use pagefind myself, but the site I'm working on > (built with the multipage exporter BTW) currently contains more than > 400 files where the browser search can't help. A multi-pages search engine is of course useful the most with multi-pages publication. Having it for a singe page, I will see it as useful only if you export one HTML file that you plan to include in an already existing multi-files website. But it's maybe to specific for us to support it on a single page export. * Remarks #6, by Ihor Radchenko at Mon, 09 Sep 2024 16:40 Ihor Radchenko writes: > Orm Finnendahl writes: >=20 > >> It should probably work for single page as well. I do not see why > >> not. > > > > sure it works. I just question the raison d'etre, when single page > > search is already integrated into webbrowsers. But as always there > > will be people arguing that it is necessary to have a search bar with > > pop up results integrated into the page and of course there is nothing > > wrong with that. I use pagefind myself, but the site I'm working on > > (built with the multipage exporter BTW) currently contains more than > > 400 files where the browser search can't help. >=20 > Agree. So, having search functionality probably makes more sense within > ox-publish, where we may also run post-processing to generate search > terms or whatever extra work is needed to make the indexer work. I 100% agree with you. * EOF That it, I hope to have forgotten no one. Best regards =2D------ Gendre S=C3=A9bastien [1] https://jupyterbook.org/ [2] Example here: https://slixmpp.readthedocs.io --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQI/BAEBCAApFiEEaA9vw9ypVj1kP0tAtYb3x3I54p4FAmcYiAcLHHNlYkBrLTcu Y2gACgkQtYb3x3I54p6GaA/8DhBNqaGXD4UL8XDi+8JlXel8EQSwb1Kr4vfOxhY8 Kfm4k+qOKYOHJQOm8s4YTAB3VPb+VdBoX9ae8Pw8ZOlJGIlCT+qb7lOtRcMcBgwL sq/Gnf/q82UPxtEDBP94wzMlAbY9k+Ipz15S1r56BSZTOzMok9nYP+9ro5KqJWJi q9/immUICznYIRcHEU8/odRpxT0vxd9Ib+2JCbX8gdzY6ELn653UtCmUOia7DO62 HMixtlJ/2q3KIx33C8JmiUz1HSYglRBxX8iwu9YqiBrUcEiqC06TKnKqOHpwn1rC HIz5PSZh3HBdUPDf0nh0Vjx/aOrS+4E1ZLTkfX1SGogVZQPPQtLRu5UaJuLNMFJG quBsvk7L1JlzuDaAZA6DXX394r6Ct7kR4/NZP6jyBwNPiOL9qPU68W5sVy09SdVb FtpPnRfJBoMntX58AswGzExUiowa9qSwgJ16zzrblt7n0tpyVIDnqdQ6CryIv+K3 rahftziuZZmheF1i59y6ju5wSO4ORXILlPDAJ3NRI0+ylbB6gx0cZMtUuCypQK26 EwamH2pP4gEcFFHJ03Y0YXVvOgP2DHLU/73mcTJkNkzoLkmMrfy8vc2pYBTT+ef2 hJr01OKaoESBrF6P9hQcCz6MDKCWsBDkcgatvXTBZ73luQrqNYz8sMlW7JNdV13t aIA= =M74U -----END PGP SIGNATURE----- --=-=-=--