From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0.migadu.com ([2001:41d0:303:5f26::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id KMUaDH9ZjGWEaQEAkFu2QA (envelope-from ) for ; Wed, 27 Dec 2023 18:06:07 +0100 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0.migadu.com with LMTPS id WApBB39ZjGXOnAAAqHPOHw (envelope-from ) for ; Wed, 27 Dec 2023 18:06:07 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=rn8eoP+S; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1703696766; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=/91oPm0PI7R00UTOMTF8FgbiFehu8Escm8A0uPTFY60=; b=um9gGh03j26gpLjgcotrwg8iyzU9/UHRh5Ycbmb8c5C3IjXwcW9eMcpG1xucFA964w5+S4 SPCzMCatDZ5wU0jc5Y2xudpXL0P6ioDZZqizzUi/nh9+IVZfvNCIm67B7irrQSK4FAK4J9 y26YiCzoJjAP2LkJx4KWnHdI3O2ByKC/FbP/kTTp7/baOHaRK2bFuvaHy1Yf+qa+PUyV8K AjXJ8EaPX9wH81JtOEwZqqkbJt/9BHzkGMruVpYmhHsdZtQ0Mz8PuOinvp9ZY1jHDwau7O Ino3uIbKGfaZ2MTw/OcijYBo00IY3UjxCPSCG9a+HGFRW176yxLQWURvlrAQ9A== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=rn8eoP+S; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.net ARC-Seal: i=1; s=key1; d=yhetil.org; t=1703696766; a=rsa-sha256; cv=none; b=XVAulO8qalJVzqKBDpU9wScQbaoNMqJuUrgx2rjd/wYxBY2mTvRaPD7w4LeqDTmcZzzIrG +DkkRK692Qu6xlhYXJ7h8N3YJUEuLmfqa/JvA0+DhzdMbZkpehLyTMTH3iEcPShtTgo6+T Zph8Yp0jo9Nt/H5SpfaWddzzceBFk20Qq58XBtKN9cjPEFPr6jj2QBGp/kRDakKuoaEtke xa/EEP3mdTUp6iqYETYfMqkqe8wRN5Ji5M0TQjLoVeG0vfsQwQ3l0Ihnn920d7hlxluTRt S+RdV6aZljoRuuerCXLqqB19EM/qnDcv0hxPgAjX8VuVU4omofStXy+VecZPzg== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id A7F7421DC6 for ; Wed, 27 Dec 2023 18:06:06 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rIXLB-00032R-0h; Wed, 27 Dec 2023 12:05:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rIXL7-0002ya-Ro for emacs-orgmode@gnu.org; Wed, 27 Dec 2023 12:05:06 -0500 Received: from mout02.posteo.de ([185.67.36.66]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rIXL4-0003ZE-8v for emacs-orgmode@gnu.org; Wed, 27 Dec 2023 12:05:05 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id B6CF2240103 for ; Wed, 27 Dec 2023 18:04:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1703696699; bh=eq2JPxBtEUj+S6AyqJzU5pdJGX1Bx1KvgXg9u2IIfoo=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=rn8eoP+S3DojSZMRFm2EW7gVGoPMj/pXloDzyIJohe9bC6AKetgCSdhjfFO62v1c4 R8qnDMGxqbaD1dq8eSk4FfOH7YckG7jYpP5wakB1ur1kR3TaP/Jr7PgellGn03R4y0 h7NClXDGTYY7L37xsc4VYXXRKqSRv66s5Xzr6YggzkuriK8YYoXRYIsYC5gldWfVph v9i3hL0p1zS8o+g2e+B3pIQEB9ljAQlDM7TZJd1O+3v5JKZzX8hnB4c6dijOIn4e3u Jf/zU0XzdUDx9LCdbPPusigmvY80LhDAN6exyfpFvHL17hphQ1GArA31/9FHTBiuVG pvQPrqHln/WwQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4T0dL263RSz6tvm; Wed, 27 Dec 2023 18:04:58 +0100 (CET) From: Ihor Radchenko To: Matt Cc: emacs-orgmode , "Thomas S. Dye" , Karthik Chikmagalur Subject: [PATCH v3] org-manual: Describe export process flow In-Reply-To: <18cabcd4180.110e4d2ac424365.1929818377821430813@excalamus.com> References: <87wmt1dp1c.fsf@localhost> <87le9fep69.fsf@localhost> <18cabcd4180.110e4d2ac424365.1929818377821430813@excalamus.com> Date: Wed, 27 Dec 2023 17:08:04 +0000 Message-ID: <87frznefpn.fsf@localhost> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=185.67.36.66; envelope-from=yantar92@posteo.net; helo=mout02.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -9.46 X-Spam-Score: -9.46 X-Migadu-Queue-Id: A7F7421DC6 X-Migadu-Scanner: mx10.migadu.com X-TUID: /PdZYKqZeKR2 --=-=-= Content-Type: text/plain Matt writes: > > Thank you all for the feedback! > > Thanks for taking the time to write and revise the patches. > > You didn't explicitly ask for more feedback, so I hope the following feedback on the new patches is in order. Of course. Any patch posted on the mailing list is open to feedback. Especially mine, since I can push it directly if I think that the patch is in a good shape. > For v2-0002-doc-org-manual.org-Describe-export-flow.patch here are > some quibbles: Thanks for proofreading! I accepted all but one suggestion on grammar. The new iteration of the patch is attached. > > - Headings according to =SELECT_TAGS= and =EXCLUDE_TAGS= export > > keywords, and =task=, =inline=, =arch= export options (see > > [[*Export Settings]]); > > Remove comma after "keywords": > > "Headings according to =SELECT_TAGS= and =EXCLUDE_TAGS= export > keywords and =task=, =inline=, =arch= export options (see [[*Export > Settings]]);" Here, "export keywords" and "export options" are two different groups. I wanted to indicate this and went with ";": - Headings according to =SELECT_TAGS= and =EXCLUDE_TAGS= export keywords; =task=, =inline=, =arch= export options (see [[*Export Settings]]); > Checking ox.el, I see that the commentary says, > > ";; See for > ;; more information." > > Maybe also let maintainers know about this manual section: > > "See: > > " AFAIU, you are asking to add it to WORG page. However, it is too early. The online Org manual that we can link to is only for stable Org version. So, only after the next release. For now, I added a link to the WORG page to "Extending an existing backend" section. It should be a good reference for people writing non-trivial derived backends. --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=v3-0001-doc-org-manual.org-Fix-some-obsolete-variable-nam.patch >From 6f6a1a190724176da7373e3d25f9df0bf60dbac9 Mon Sep 17 00:00:00 2001 Message-ID: <6f6a1a190724176da7373e3d25f9df0bf60dbac9.1703696392.git.yantar92@posteo.net> From: Ihor Radchenko Date: Wed, 27 Dec 2023 14:23:29 +0100 Subject: [PATCH v3 1/3] doc/org-manual.org: Fix some obsolete variable names * doc/org-manual.org (Export hooks): Use the new `org-export-before-processing-functions' and `org-export-before-parsing-functions' instead of their obsolete aliases. --- doc/org-manual.org | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/doc/org-manual.org b/doc/org-manual.org index ff1b9cffb..377706ee7 100644 --- a/doc/org-manual.org +++ b/doc/org-manual.org @@ -16397,12 +16397,14 @@ *** Export hooks :END: #+vindex: org-export-before-processing-hook +#+vindex: org-export-before-processing-functions #+vindex: org-export-before-parsing-hook The export process executes two hooks before the actual exporting -begins. The first hook, ~org-export-before-processing-hook~, runs -before any expansions of macros, Babel code, and include keywords in -the buffer. The second hook, ~org-export-before-parsing-hook~, runs -before the buffer is parsed. +begins. The first hook, ~org-export-before-processing-functions~, +runs before any expansions of macros, Babel code, and include keywords +in the buffer. The second hook, +~org-export-before-parsing-functions~, runs before the buffer is +parsed. Functions added to these hooks are called with a single argument: the export backend actually used, as a symbol. You may use them for @@ -16421,7 +16423,7 @@ *** Export hooks ;; the docstring of `org-map-entries' for details. (setq org-map-continue-from (point))))) -(add-hook 'org-export-before-parsing-hook #'my-headline-removal) +(add-hook 'org-export-before-parsing-functions #'my-headline-removal) #+end_src *** Filters -- 2.42.0 --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=v3-0002-doc-org-manual.org-Describe-export-flow.patch >From 786e3e78d4c6846188477452a529132aba76149f Mon Sep 17 00:00:00 2001 Message-ID: <786e3e78d4c6846188477452a529132aba76149f.1703696392.git.yantar92@posteo.net> In-Reply-To: <6f6a1a190724176da7373e3d25f9df0bf60dbac9.1703696392.git.yantar92@posteo.net> References: <6f6a1a190724176da7373e3d25f9df0bf60dbac9.1703696392.git.yantar92@posteo.net> From: Ihor Radchenko Date: Tue, 26 Dec 2023 15:15:23 +0100 Subject: [PATCH v3 2/3] doc/org-manual.org: Describe export flow * doc/org-manual.org (Summary of the export process): Explain how the export process is handled in Org mode. --- doc/org-manual.org | 148 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 148 insertions(+) diff --git a/doc/org-manual.org b/doc/org-manual.org index 377706ee7..66a582eae 100644 --- a/doc/org-manual.org +++ b/doc/org-manual.org @@ -16505,6 +16505,154 @@ *** Defining filters for individual files ,#+END_SRC #+end_example +*** Summary of the export process +:PROPERTIES: +:UNNUMBERED: notoc +:END: + +Org mode export is a multi-step process that works on a temporary copy +of the buffer. The export process consists of 4 major steps: + +1. Process the temporary copy, making necessary changes to the buffer + text; + +2. Parse the buffer, converting plain Org markup into an abstract + syntax tree (AST); + +3. Convert the AST to text, as prescribed by the selected export + backend; + +4. Post-process the resulting exported text. + + +#+texinfo: @noindent +Process temporary copy of the source Org buffer [fn::Unless +otherwise specified, each step of the export process only operates on +the accessible portion of the buffer. When subtree export is selected +(see [[*The Export Dispatcher]]), the buffer is narrowed to the body of +the selected subtree, so that the rest of the buffer text, except +export keywords, does not contribute to the export output.]: + +1. Execute ~org-export-before-processing-functions~ (see [[*Export hooks]]); + +2. Expand =#+include= keywords in the whole buffer (see + [[*Include Files]]); + +3. Remove commented subtrees in the whole buffer (see [[*Comment + Lines]]); + +4. Replace macros in the whole buffer (see [[*Macro Replacement]]); + +5. When ~org-export-use-babel~ is non-nil (default), process code + blocks: + + - Leave code blocks inside archived subtrees (see [[*Internal + archiving]]) as is; + + - Evaluate all the other code blocks according to code block + headers (see [[*Limit code block evaluation]]); + + - Remove code, results of evaluation, both, or neither according + to =:exports= header argument (see [[*Exporting Code Blocks]]). + + +#+texinfo: @noindent +Parse the temporary buffer, creating AST: + +1. Execute ~org-export-before-parsing-functions~ (see [[*Export hooks]]). + The hook functions may still modify the buffer; + +2. Calculate export option values according to subtree-specific export + settings, in-buffer keywords, =#+BIND= keywords, and buffer-local + and global customization. The whole buffer is considered; + +3. Determine contributing bibliographies and record them into export + options (see [[*Citations]]). The whole buffer is considered; + +4. Execute ~org-export-filter-options-functions~; + +5. Parse the accessible portion of the temporary buffer to generate an + AST. The AST is a nested list of lists representing Org syntax + elements (see [[https://orgmode.org/worg/dev/org-element-api.html][Org Element API]] for more details): + + : (org-data ... + : (heading + : (section + : (paragraph (plain-text) (bold (plain-text)))) + : (heading) + : (heading (section ...)))) + + Past this point, modifications to the temporary buffer no longer + affect the export; Org export works only with the AST; + +6. Remove elements that are not exported from the AST: + + - Headings according to =SELECT_TAGS= and =EXCLUDE_TAGS= export + keywords; =task=, =inline=, =arch= export options (see + [[*Export Settings]]); + + - Comments; + + - Clocks, drawers, fixed-width environments, footnotes, LaTeX + environments and fragments, node properties, planning lines, + property drawers, statistics cookies, timestamps, etc according + to =#+OPTIONS= keyword (see [[*Export Settings]]); + + - Table rows containing width and alignment markers (see [[*Column + Width and Alignment]]); + + - Table columns containing recalc marks (see [[*Advanced features]]). + +7. Expand environment variables in file link AST nodes according to + the =expand-links= export option (see [[*Export Settings]]); + +8. Execute ~org-export-filter-parse-tree-functions~. These + functions can modify the AST by side effects; + +9. Replace citation AST nodes and =#+print_bibliography= keyword AST + nodes as prescribed by the selected citation export processor + (see [[*Citation export processors]]). + + +#+texinfo: @noindent +Convert the AST to text by traversing the AST nodes, depth-first: + +1. Convert the leaf nodes (without children) to text as prescribed + by "transcoders" in the selected export backend + [fn:: See transcoders and ~:translate-alist~ in the docstrings + of ~org-export-define-backend~ and ~org-export-define-derived-backend~.]; + +2. Pass the converted nodes through the corresponding export + filters (see [[*Filters]]); + +3. Concatenate all the converted child nodes to produce parent + node contents; + +4. Convert the nodes with children to text, passing the nodes + themselves and their contents to the corresponding transcoders + and then to the export filters (see [[*Filters]]). + + +#+texinfo: @noindent +Post-process the exported text: + + 1. Post-process the converted AST, as prescribed by the export + backend. [fn:: See ~inner-template~ in the docstring of ~org-export-define-backend~.] + This step usually adds generated content (like Table of Contents) + to the exported text; + + 2. Execute ~org-export-filter-body-functions~; + + 3. Unless body-only export is selected (see [[*The Export Dispatcher]]), + add the necessary metadata to the final document, as prescribed + by the export backend. Examples: Document author/title; HTML + headers/footers; LaTeX preamble; + + 4. Add bibliography metadata, as prescribed by the citation export + processor; + + 5. Execute ~org-export-filter-final-output-functions~. + *** Extending an existing backend :PROPERTIES: :UNNUMBERED: notoc -- 2.42.0 --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=v3-0003-doc-org-manual.org-Add-link-to-WORG-export-refere.patch >From 66d2b5f68e0a4c62be3ed65a8d06300c0c2c5083 Mon Sep 17 00:00:00 2001 Message-ID: <66d2b5f68e0a4c62be3ed65a8d06300c0c2c5083.1703696392.git.yantar92@posteo.net> In-Reply-To: <6f6a1a190724176da7373e3d25f9df0bf60dbac9.1703696392.git.yantar92@posteo.net> References: <6f6a1a190724176da7373e3d25f9df0bf60dbac9.1703696392.git.yantar92@posteo.net> From: Ihor Radchenko Date: Wed, 27 Dec 2023 17:58:48 +0100 Subject: [PATCH v3 3/3] doc/org-manual.org: Add link to WORG export reference page * doc/org-manual.org (Extending an existing backend): Link to WORG page describing more details about writing export backends. --- doc/org-manual.org | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/doc/org-manual.org b/doc/org-manual.org index 66a582eae..bb1bad673 100644 --- a/doc/org-manual.org +++ b/doc/org-manual.org @@ -16709,7 +16709,9 @@ *** Extending an existing backend Further steps to consider would be an interactive function, self-installing an item in the export dispatcher menu, and other -user-friendly improvements. +user-friendly improvements. See + for more +details. ** Export Region :PROPERTIES: -- 2.42.0 --=-=-= Content-Type: text/plain -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at --=-=-=--