From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:1008:1e59::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id UCFWCx1kjWXGtQAAkFu2QA (envelope-from ) for ; Thu, 28 Dec 2023 13:03:41 +0100 Received: from aspmx1.migadu.com ([2001:41d0:303:e16b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id 2EdWBx1kjWVruQAA62LTzQ (envelope-from ) for ; Thu, 28 Dec 2023 13:03:41 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=VDH2DPKa; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1703765020; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=nLS/BEUFQzk+EhC1D54o+V3YTQBk7ut7sQCmET8raM8=; b=mvTCTRXkPZXDOLe3mfwhR4sQplAhkLaqp5SldAvADzUUMWtv5CyCwXv6UgpRUGB89q+Cu/ QCL3WgM6bx+3HxlJWFN/rfeJ2z3EhsoInwaYekP+aWTo8cQCEzXy1r6waLRTYSX0cpbRQP GcL0XiO79w35cKogLjHcI7so6asVNEFQIk75ShjtaA3Ogu/mtcfNIRt+zabDETyo3JYTkC eGiGSBReo5zDw+DB1wVIkDqlD0qOe8B/ZIXjXD9QrmdybEE5TCCf2y02GUVPWkCYQZnyxQ JG/pyhn1IgP9bK5+/0+Jl2OYadQOq8z27MS8JpTzUuPqpoxrIP9H3LGUvDb7Qw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=VDH2DPKa; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.net ARC-Seal: i=1; s=key1; d=yhetil.org; t=1703765020; a=rsa-sha256; cv=none; b=OL8B15VaxzzyP3xaxPCrnBRtBM6ihsReIhSfln6ncC1zXrYUDdw9NAs7leFbgp48x7duKA cSXIWlLsrlEbxyHW8yAGm9OR7fmQIGsq+dw9l2wAfEC6G0ULfoTmQUw9INrsSYvNiRgYgj Z57/4ifjwGLXl6a6seGkE/5ygKpkmItw0F0rUy5fs0kTHYjL2hC2grRQRbV0IZu2HJrQGQ ZgTV6MJq7vARfAlLmMSRabuhB+xJ50ilFExlRmfXFwBcCEtEHtu/4CI9umM3y05K2ODITw pXTsflNqFMWdXHnqjrNXyu+iGKDDhS4i6zXHGPTC/7qwrc221K+JOCFkXOwUrg== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id B9D8E40E95 for ; Thu, 28 Dec 2023 13:03:40 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rIp6J-0006Lo-Ve; Thu, 28 Dec 2023 07:02:59 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rIp6I-0006LY-Lc for emacs-orgmode@gnu.org; Thu, 28 Dec 2023 07:02:58 -0500 Received: from mout02.posteo.de ([185.67.36.66]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rIp6D-0006Sr-Qa for emacs-orgmode@gnu.org; Thu, 28 Dec 2023 07:02:58 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id B0C83240101 for ; Thu, 28 Dec 2023 13:02:50 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1703764970; bh=KFvfc+HZp6l6q5n2AfTtNj44bva6UQ/zv/uFvDhBK1Y=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=VDH2DPKalJx0Ud0rZ9bdi6ku89wobOw1qageaHSZG3aWrq+bKJgV6c3K3rg1N0tXg omuYvdDR2dE6OEYYMa3mSlqj7uKe/ZuPqrqFYw8TFFh5NpmBGCBqKPUICE3VL/9bR5 iou60j/A/INOJVrPeHt8TKBjqJvlGHIewKQM8TIV80ogvUmMDaxaJcaJ0tKSrpAllX aThLDwUbt5l3lz93BgZFpDyNL6zAG27D3/gdV3UVkrugB95v8gBPkZJI0HRWdzszm3 Xjm9xu4yPiCJIyn2Wb74UzM4lp7lJigBFdRLdMcCgjqKfNcakUIWo0C+K+FoG++Iu3 GknoSE3xzKclA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4T16Zy1sb7z6trs; Thu, 28 Dec 2023 13:02:50 +0100 (CET) From: Ihor Radchenko To: Matt Cc: emacs-orgmode Subject: [PATCH v4] org-manual: Describe export process flow In-Reply-To: <18cac710817.e615cdf1468737.687007161508493949@excalamus.com> References: <87wmt1dp1c.fsf@localhost> <87le9fep69.fsf@localhost> <18cabcd4180.110e4d2ac424365.1929818377821430813@excalamus.com> <87frznefpn.fsf@localhost> <18cac710817.e615cdf1468737.687007161508493949@excalamus.com> Date: Thu, 28 Dec 2023 12:06:02 +0000 Message-ID: <875y0ijzv9.fsf@localhost> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=185.67.36.66; envelope-from=yantar92@posteo.net; helo=mout02.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Scanner: mx13.migadu.com X-Migadu-Spam-Score: -7.13 X-Spam-Score: -7.13 X-Migadu-Queue-Id: B9D8E40E95 X-TUID: YI7PinwNeeBs --=-=-= Content-Type: text/plain Matt writes: > I was making a different point. I was trying to say that it may be a > good idea for the ox.el commentary section to specifically reference > the new manual section (next to where it references Worg). Makes sense. See the attached. I added reference to the manual in `org-export-as' docstring and made it explicit that the manual section is related to `org-export-as' by adding #+findex entry. --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=v4-0001-doc-org-manual.org-Fix-some-obsolete-variable-nam.patch >From b17d57c1afeb160783da36647f56462ee67c3558 Mon Sep 17 00:00:00 2001 Message-ID: From: Ihor Radchenko Date: Wed, 27 Dec 2023 14:23:29 +0100 Subject: [PATCH v4 1/5] doc/org-manual.org: Fix some obsolete variable names * doc/org-manual.org (Export hooks): Use the new `org-export-before-processing-functions' and `org-export-before-parsing-functions' instead of their obsolete aliases. --- doc/org-manual.org | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/doc/org-manual.org b/doc/org-manual.org index ff1b9cffb..377706ee7 100644 --- a/doc/org-manual.org +++ b/doc/org-manual.org @@ -16397,12 +16397,14 @@ *** Export hooks :END: #+vindex: org-export-before-processing-hook +#+vindex: org-export-before-processing-functions #+vindex: org-export-before-parsing-hook The export process executes two hooks before the actual exporting -begins. The first hook, ~org-export-before-processing-hook~, runs -before any expansions of macros, Babel code, and include keywords in -the buffer. The second hook, ~org-export-before-parsing-hook~, runs -before the buffer is parsed. +begins. The first hook, ~org-export-before-processing-functions~, +runs before any expansions of macros, Babel code, and include keywords +in the buffer. The second hook, +~org-export-before-parsing-functions~, runs before the buffer is +parsed. Functions added to these hooks are called with a single argument: the export backend actually used, as a symbol. You may use them for @@ -16421,7 +16423,7 @@ *** Export hooks ;; the docstring of `org-map-entries' for details. (setq org-map-continue-from (point))))) -(add-hook 'org-export-before-parsing-hook #'my-headline-removal) +(add-hook 'org-export-before-parsing-functions #'my-headline-removal) #+end_src *** Filters -- 2.42.0 --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=v4-0002-doc-org-manual.org-Describe-export-flow.patch >From b195caf687793af228a2ea9a484f2fb8cb87ddab Mon Sep 17 00:00:00 2001 Message-ID: In-Reply-To: References: From: Ihor Radchenko Date: Tue, 26 Dec 2023 15:15:23 +0100 Subject: [PATCH v4 2/5] doc/org-manual.org: Describe export flow * doc/org-manual.org (Summary of the export process): Explain how the export process is handled in Org mode. --- doc/org-manual.org | 148 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 148 insertions(+) diff --git a/doc/org-manual.org b/doc/org-manual.org index 377706ee7..66a582eae 100644 --- a/doc/org-manual.org +++ b/doc/org-manual.org @@ -16505,6 +16505,154 @@ *** Defining filters for individual files ,#+END_SRC #+end_example +*** Summary of the export process +:PROPERTIES: +:UNNUMBERED: notoc +:END: + +Org mode export is a multi-step process that works on a temporary copy +of the buffer. The export process consists of 4 major steps: + +1. Process the temporary copy, making necessary changes to the buffer + text; + +2. Parse the buffer, converting plain Org markup into an abstract + syntax tree (AST); + +3. Convert the AST to text, as prescribed by the selected export + backend; + +4. Post-process the resulting exported text. + + +#+texinfo: @noindent +Process temporary copy of the source Org buffer [fn::Unless +otherwise specified, each step of the export process only operates on +the accessible portion of the buffer. When subtree export is selected +(see [[*The Export Dispatcher]]), the buffer is narrowed to the body of +the selected subtree, so that the rest of the buffer text, except +export keywords, does not contribute to the export output.]: + +1. Execute ~org-export-before-processing-functions~ (see [[*Export hooks]]); + +2. Expand =#+include= keywords in the whole buffer (see + [[*Include Files]]); + +3. Remove commented subtrees in the whole buffer (see [[*Comment + Lines]]); + +4. Replace macros in the whole buffer (see [[*Macro Replacement]]); + +5. When ~org-export-use-babel~ is non-nil (default), process code + blocks: + + - Leave code blocks inside archived subtrees (see [[*Internal + archiving]]) as is; + + - Evaluate all the other code blocks according to code block + headers (see [[*Limit code block evaluation]]); + + - Remove code, results of evaluation, both, or neither according + to =:exports= header argument (see [[*Exporting Code Blocks]]). + + +#+texinfo: @noindent +Parse the temporary buffer, creating AST: + +1. Execute ~org-export-before-parsing-functions~ (see [[*Export hooks]]). + The hook functions may still modify the buffer; + +2. Calculate export option values according to subtree-specific export + settings, in-buffer keywords, =#+BIND= keywords, and buffer-local + and global customization. The whole buffer is considered; + +3. Determine contributing bibliographies and record them into export + options (see [[*Citations]]). The whole buffer is considered; + +4. Execute ~org-export-filter-options-functions~; + +5. Parse the accessible portion of the temporary buffer to generate an + AST. The AST is a nested list of lists representing Org syntax + elements (see [[https://orgmode.org/worg/dev/org-element-api.html][Org Element API]] for more details): + + : (org-data ... + : (heading + : (section + : (paragraph (plain-text) (bold (plain-text)))) + : (heading) + : (heading (section ...)))) + + Past this point, modifications to the temporary buffer no longer + affect the export; Org export works only with the AST; + +6. Remove elements that are not exported from the AST: + + - Headings according to =SELECT_TAGS= and =EXCLUDE_TAGS= export + keywords; =task=, =inline=, =arch= export options (see + [[*Export Settings]]); + + - Comments; + + - Clocks, drawers, fixed-width environments, footnotes, LaTeX + environments and fragments, node properties, planning lines, + property drawers, statistics cookies, timestamps, etc according + to =#+OPTIONS= keyword (see [[*Export Settings]]); + + - Table rows containing width and alignment markers (see [[*Column + Width and Alignment]]); + + - Table columns containing recalc marks (see [[*Advanced features]]). + +7. Expand environment variables in file link AST nodes according to + the =expand-links= export option (see [[*Export Settings]]); + +8. Execute ~org-export-filter-parse-tree-functions~. These + functions can modify the AST by side effects; + +9. Replace citation AST nodes and =#+print_bibliography= keyword AST + nodes as prescribed by the selected citation export processor + (see [[*Citation export processors]]). + + +#+texinfo: @noindent +Convert the AST to text by traversing the AST nodes, depth-first: + +1. Convert the leaf nodes (without children) to text as prescribed + by "transcoders" in the selected export backend + [fn:: See transcoders and ~:translate-alist~ in the docstrings + of ~org-export-define-backend~ and ~org-export-define-derived-backend~.]; + +2. Pass the converted nodes through the corresponding export + filters (see [[*Filters]]); + +3. Concatenate all the converted child nodes to produce parent + node contents; + +4. Convert the nodes with children to text, passing the nodes + themselves and their contents to the corresponding transcoders + and then to the export filters (see [[*Filters]]). + + +#+texinfo: @noindent +Post-process the exported text: + + 1. Post-process the converted AST, as prescribed by the export + backend. [fn:: See ~inner-template~ in the docstring of ~org-export-define-backend~.] + This step usually adds generated content (like Table of Contents) + to the exported text; + + 2. Execute ~org-export-filter-body-functions~; + + 3. Unless body-only export is selected (see [[*The Export Dispatcher]]), + add the necessary metadata to the final document, as prescribed + by the export backend. Examples: Document author/title; HTML + headers/footers; LaTeX preamble; + + 4. Add bibliography metadata, as prescribed by the citation export + processor; + + 5. Execute ~org-export-filter-final-output-functions~. + *** Extending an existing backend :PROPERTIES: :UNNUMBERED: notoc -- 2.42.0 --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=v4-0003-doc-org-manual.org-Add-link-to-WORG-export-refere.patch >From bbd6f3613d3d8ca1196c258ee1fcc9ec4df1d9a1 Mon Sep 17 00:00:00 2001 Message-ID: In-Reply-To: References: From: Ihor Radchenko Date: Wed, 27 Dec 2023 17:58:48 +0100 Subject: [PATCH v4 3/5] doc/org-manual.org: Add link to WORG export reference page * doc/org-manual.org (Extending an existing backend): Link to WORG page describing more details about writing export backends. --- doc/org-manual.org | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/doc/org-manual.org b/doc/org-manual.org index 66a582eae..bb1bad673 100644 --- a/doc/org-manual.org +++ b/doc/org-manual.org @@ -16709,7 +16709,9 @@ *** Extending an existing backend Further steps to consider would be an interactive function, self-installing an item in the export dispatcher menu, and other -user-friendly improvements. +user-friendly improvements. See + for more +details. ** Export Region :PROPERTIES: -- 2.42.0 --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=v4-0004-org-export-as-Add-reference-to-export-flow-in-the.patch >From 5ad0e8d6e05a312960ea006958f986097441cd5a Mon Sep 17 00:00:00 2001 Message-ID: <5ad0e8d6e05a312960ea006958f986097441cd5a.1703765065.git.yantar92@posteo.net> In-Reply-To: References: From: Ihor Radchenko Date: Thu, 28 Dec 2023 12:59:22 +0100 Subject: [PATCH v4 4/5] org-export-as: Add reference to export flow in the manual * lisp/ox.el (org-export-as): Update docstring. --- lisp/ox.el | 3 +++ 1 file changed, 3 insertions(+) diff --git a/lisp/ox.el b/lisp/ox.el index 46994f0e2..2cf364227 100644 --- a/lisp/ox.el +++ b/lisp/ox.el @@ -2923,6 +2923,9 @@ (defun org-export-as (backend &optional subtreep visible-only body-only ext-plist) "Transcode current Org buffer into BACKEND code. +See info node `(org)Advanced Export Configuration' for the details of +the transcoding process. + BACKEND is either an export backend, as returned by, e.g., `org-export-create-backend', or a symbol referring to a registered backend. -- 2.42.0 --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=v4-0005-doc-org-manual.org-Summary-of-the-export-process-.patch >From 1c89b2e32816c0ad95d344aa6b05a2ad65b2d172 Mon Sep 17 00:00:00 2001 Message-ID: <1c89b2e32816c0ad95d344aa6b05a2ad65b2d172.1703765065.git.yantar92@posteo.net> In-Reply-To: References: From: Ihor Radchenko Date: Thu, 28 Dec 2023 13:04:06 +0100 Subject: [PATCH v4 5/5] doc/org-manual.org (Summary of the export process): Reference `org-export-as' --- doc/org-manual.org | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/org-manual.org b/doc/org-manual.org index bb1bad673..de2194868 100644 --- a/doc/org-manual.org +++ b/doc/org-manual.org @@ -16510,6 +16510,7 @@ *** Summary of the export process :UNNUMBERED: notoc :END: +#+findex: org-export-as Org mode export is a multi-step process that works on a temporary copy of the buffer. The export process consists of 4 major steps: -- 2.42.0 --=-=-= Content-Type: text/plain -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at --=-=-=--