From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id gIniNl5mPGO4PQAAbAwnHQ (envelope-from ) for ; Tue, 04 Oct 2022 18:59:11 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id vzoFNl5mPGO68wAAG6o9tA (envelope-from ) for ; Tue, 04 Oct 2022 18:59:10 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 791A137096 for ; Tue, 4 Oct 2022 18:59:10 +0200 (CEST) Received: from localhost ([::1]:58956 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oflG7-0005fn-Hg for larch@yhetil.org; Tue, 04 Oct 2022 12:59:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48768) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ofkqq-0007MT-Et for emacs-orgmode@gnu.org; Tue, 04 Oct 2022 12:33:01 -0400 Received: from ciao.gmane.io ([116.202.254.214]:41446) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ofkqo-0004qO-CM for emacs-orgmode@gnu.org; Tue, 04 Oct 2022 12:33:00 -0400 Received: from list by ciao.gmane.io with local (Exim 4.92) (envelope-from ) id 1ofkqm-0009KH-8Q for emacs-orgmode@gnu.org; Tue, 04 Oct 2022 18:32:56 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: emacs-orgmode@gnu.org From: Max Nikulin Subject: Re: [HELP] Fwd: Org format as a new standard source format for GNU manuals Date: Tue, 4 Oct 2022 23:32:47 +0700 Message-ID: References: <87bkqx4jyg.fsf@localhost> <878rm02pc1.fsf@localhost> <87ill3st88.fsf@localhost> <87h70m252s.fsf@localhost> <87r0zpy14r.fsf@localhost> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.11.0 Content-Language: en-US In-Reply-To: <87r0zpy14r.fsf@localhost> Received-SPF: pass client-ip=116.202.254.214; envelope-from=geo-emacs-orgmode@m.gmane-mx.org; helo=ciao.gmane.io X-Spam_score_int: 4 X-Spam_score: 0.4 X-Spam_bar: / X-Spam_report: (0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_ADSP_CUSTOM_MED=0.001, FORGED_GMAIL_RCVD=1, FORGED_MUA_MOZILLA=2.309, FREEMAIL_FORGED_FROMDOMAIN=0.25, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.25, NICE_REPLY_A=-2.449, NML_ADSP_CUSTOM_MED=0.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1664902750; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=GFd7coT017TlJXdWoSl20TcAbWy9zQt41+Iry5KTB4c=; b=V6pDsV4kqVp8AeXplKFNNkN03lo/s6LdcEKvLHoSagNyjuYShaZFVzXW1sHH/DLjnzzX5W VeBq0HtDeYVD51bw3DYBpjVADJWNKYeHV7M/evZSn9r6gl7h81XikWg4TjKBjjfqIr3lio G8Pk/Fm6K5ZLbJUgTbRt6qBUIvcv6LvYLSkS6Qw8ogRN3zQk9oxMC8Tg6ZQB1+FEyxx/lj oSB6Mywyo2cEQ3B4upYO//EUQBN04GvqXEevMaNbrEVDIfQNaSAnv+Ypqvww4tjKskcFBK OvYeg9Zhd4XR4DFTylK0SMS6tn48pPfetaj95fwtqcmdWwVQ4c7Xs3b7g2PvrQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1664902750; a=rsa-sha256; cv=none; b=EVote2MsK/AxzrQ0h4IMLnZ6H99bzL5HtiNhGGEItkV/E8Xvs1cq7Z/OGUCntUkQGOPWtl +RQbMI8+umaQE4koX6vEEYfiIM2N1OdRWFvaMI1wdjAH7U/idcsMg8QB3jSadCSODDucQX C0M18WMty7pWp8CItpI/UJfL1li7j9c04Q6v5ynKAj2NCz5bCOqPeMnj53S+4jODuDRdOs sXTOvT4utCiH5lxUnMgVRB0Owl0KCTGVRoo3hcpYW8ylfqKXJN7SERSHIfxtMBhO8Go+p6 oFbzv6Kn0eHRdBoLX2jh6DjK++dp4Ya+YxRY1AhVTuQh9cnLM1m+3RKNJAcsQw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: 4.34 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 791A137096 X-Spam-Score: 4.34 X-Migadu-Scanner: scn1.migadu.com X-TUID: Nv8vzyemw4rO It seems I completely failed trying to express my idea. Instead of extending Org grammar (syntax), I suggest to change behavior of source blocks during export. In addition to current :results options, "ast" may be added. Its effect is that instead of adding text to export buffer that is parsed as Org markup, it causes insertion of a branch of syntax tree into original parse results. I admit, during export it may be necessary to iterate over source blocks one more time at a later stage. Such source blocks should return "Org Syntax Tree", a simplified variant of org-element. It allows to change implementation details and e.g. to use vectors instead of lists for attributes in org-element. A converter from Org Syntax Tree to org-element should be implemented. Certainly such format may be used directly as src_ost{(code (:class var) "language")} inline snippets or as #+begin_src ost (code nil ("libtree-{sitter}-" (code (:class var) "\"language\"") "." (code (:class var) "ext"))) #+end_src block-level elements. However I expect that it is the last resort option when there is no way to express desired construct in some other way. I think, more convenient org-babel backends may be created to parse TeX-like (texinfo-like) or SGML-like (XML-like) syntax into Org Syntax Tree hierarchy. The essential idea is that outside of source blocks usual lightweight markup is used. Source blocks however have just a few special characters ([\{}], [@{}], or [&<>], etc.) to reduce issues with escaping for regular text or verbatim-like commands. Some comments are inline. On 03/10/2022 11:36, Ihor Radchenko wrote: > Max Nikulin writes: > >> On 02/10/2022 11:59, Ihor Radchenko wrote: >> >> If you are asking how to represent such construct without introducing >> custom elements then (it may be e.g. :type, not :class) parsed AST >> should be like >> >> (code nil ("libtree-{sitter}-" >> (code (:class var) "\"language\"") >> "." >> (code (:class var) "ext"))) > > This is not much different from @name[nil]{} idea, but > more verbose. > > Also, more importantly, I strongly dislike the need to wrap the text > into "". You will have to escape \". And it will force third-party > parsers to re-implement Elisp sexp reader. By this example I was trying to show how to express @var, @samp, @file without introducing of new custom objects. I do not see any problem with verbosity of such format, it may be used for really special cases only, while some more convenient markup is used for more simple cases. >> If there was some syntax for object attributes then simple cases would >> be like >> >> [[attr:(:class var)]]~language~ > > I do not like this idea. It will require non-trivial changes in Org > parser and fontification. > > Using dedicated object properties or at least inheriting properties from > :parent is the style we employ more commonly across the code: > > @var{language} > or > @code[:class var]{language} > or > @attr[:class var]{~language~} I do not mind to have some "span" object to assign attributes to its direct children. I used link-like prefix object just because a proof of concept may be tried with no changes in Org. It does not require support of nested objects. There is no existing syntax for such "span" objects, but perhaps it is not necessary and source blocks should be used instead for special needs. >> I have no idea concerning particular markup that can be used inside >> source blocks. It might be LaTeX-like commands as discussed in the >> sibling subthread or HTML (XML) based syntax that is more verbose than >> TeX-like notation. >> >> By convention, the dynamic library >> for src_alt{\code[class=var]{language}} is >> >> src_alt{\code{libtree-\{sitter\}-\code[class=var]{"language"}.\code[class=var]{ext}}}, >> where src_alt{\code[class=var]{ext}} is the >> system-specific extension for dynamic libraries. > > I am against the idea of LaTeX-like commands. It will clash with > latex-fragment object type. > https://orgmode.org/worg/dev/org-syntax.html#LaTeX_Fragments > >> or >> >> By convention, the dynamic library for >> src_alt{language} is >> src_alt{libtree-{sitter}-> class="var">"language".ext}, >> where src_alt{ext} is the >> system-specific extension for dynamic libraries. > > This style will indeed make things easier for the parser. But I find it > too verbose for practical usage. This is why I instead proposed the idea > with variable number of brackets: @code{{can have } inside}}. Texinfo is TeX with \ replaced by @. Just another character has the category starting command. The important point is that while Org markup uses a lot of special characters (*/_+[]...) this flexible markup should use just a few ones. I do not see any obstacles to try texinfo-like markup. Source blocks allow to have several languages. >> Hypothetical "alt" babel language has default :results ast :export >> results header arguments to inject AST bypassing Org markup stage. > > The problem with src block emitting AST is clashing with the way src > blocks work during export. What `org-export-as' does is replacing/adding > src block output into the actual Org buffer text before the parsing is > done. > > Handling direct AST sexps will require a rewrite on how babel > integration with export works. Yes, it will. I am evaluating feasibility of such change instead of extending of Org syntax for custom elements.