From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0.migadu.com ([2001:41d0:303:e16b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id kJP4BmPVoGafEwAAqHPOHw:P1 (envelope-from ) for ; Wed, 24 Jul 2024 10:20:19 +0000 Received: from aspmx1.migadu.com ([2001:41d0:303:e16b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0.migadu.com with LMTPS id kJP4BmPVoGafEwAAqHPOHw (envelope-from ) for ; Wed, 24 Jul 2024 12:20:19 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=RBtKkmP2; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1721816419; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=Wrp10PqzIRfDQRKPYx4T6tdusvh33JV1JwcUmesqW9w=; b=KpszGY3C7BZZCCaG067BIeF/LEL/X6r/NtCTaioUnsOQQ6W7SqBjHdpgtGST51CMDASMzW jwP1+BDawqlTadd9XHjZOnvOJqxgKcUwb9tYL9we8Dhr2va4T4aHrl5NIbaRtlxRPLpUDf ZrHucsCxwP9h7xqR4WdLKGh+GiqPXkbxxTiZl3L6Za49W4sZLam8a9KqmYXl1HFAPszJeq N4mNOGsytemEc2xMOcINcCht96rhPz/ddqOKBxprtdcOw9TJZHzS+FEFYhiwu2cWcNI7fP gQbIMIQ0o2EZt89a4Qfkf0vosZ4Ra13oZvmW8L2MuAhjYys0YzcZiJCxU+fkkQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=RBtKkmP2; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.net ARC-Seal: i=1; s=key1; d=yhetil.org; t=1721816419; a=rsa-sha256; cv=none; b=MJpcYngfG4gjveF4k9DOYuJE42xaEabTMVCRHVxrYcBaeytPhW+1O3bLqFfR4GApkVlkfS kQrD/3Q3jMWYuLUEnNulblnGUT3UQMnWGUUGDCuKY3qPJNxQY8EaSVyu/V4lecW32pQQYy bnXHxrBBLnTPEcvaVlblO9noJeO1/qc8wzOoFuTu4TLfywTGZUf4E/QLTScdAkI9ft5C1y GbAq6J6mkfATpRnAhfuLNYDc754zoS5fjGV5rwoUzIZ5SyUu+fUEBXFQeGp0oIyIpyGalE NgaVKhSSmzeF9Wjno3rxK1KRa++kfFhFuj4U/IGSOOOqgnZSQATxIP7DJX6OIA== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 9614956F16 for ; Wed, 24 Jul 2024 12:20:18 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sWZ5J-0003k4-G4; Wed, 24 Jul 2024 06:19:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sWZ5G-0003jn-Uk for emacs-orgmode@gnu.org; Wed, 24 Jul 2024 06:18:59 -0400 Received: from mout01.posteo.de ([185.67.36.65]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sWZ5D-0003mw-EQ for emacs-orgmode@gnu.org; Wed, 24 Jul 2024 06:18:58 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 87170240028 for ; Wed, 24 Jul 2024 12:18:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1721816330; bh=OsmYUhw8LhcNrUT7Dje6auEx0mpdABT7BprwlQGK1J0=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type: From; b=RBtKkmP20zXknB7R3KEcn0R0/jXsK4+mkYjEiu9nruk8dFpBAo4z9WKw7i8yL1Zxv ItSuTUsHxq6cc73Qtp0TaYhp3HQPfmAujCExv/9rCPpIl6C4F2lfSNBcCjF5iGTDvZ ZpDbHZZ0QHA8si8ro8uVUipJdF4Bm15Z0ObI6sHI/qRn940DihcVfu+phARE/SNgDZ 6kuWXxzQhMWe2TZVDJzjRKd8LiS7/q6xdsGxAfW6L/JgLBie91QlPsXvanfLH+Knbz mwqVfqSBN9zcQNYo3D4PemdRbRuR7fpVy/bjMi24WyBDGphWEB4NWaolrdw+92Zs1o A+lEoDum/kiXg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4WTVNT2p8Hz6tvp; Wed, 24 Jul 2024 12:18:49 +0200 (CEST) From: Ihor Radchenko To: Orm Finnendahl Cc: Org mailing list Subject: Re: multipage html output In-Reply-To: References: <87r0c05com.fsf@localhost> <87wmlp38gr.fsf@localhost> <874j8gz9qh.fsf@localhost> <87bk2o2o2m.fsf@localhost> <87sew011c6.fsf@localhost> Date: Wed, 24 Jul 2024 10:20:16 +0000 Message-ID: <87frrzdrbz.fsf@localhost> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=185.67.36.65; envelope-from=yantar92@posteo.net; helo=mout01.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Queue-Id: 9614956F16 X-Migadu-Scanner: mx13.migadu.com X-Migadu-Spam-Score: -9.54 X-Spam-Score: -9.54 X-TUID: C/Z8zVNUccDD --=-=-= Content-Type: text/plain Orm Finnendahl writes: > To recapitulate: In my code, org-export-as calls process-multipage in > the backend. This function: > > - collects and adds information necessary for org-multipage to do its > job, splitting the document into different parts, etc. and > > - then calls org-export-data on the subtrees and exports each returned > string to an individual file. > > - It finally issues a done string and executes a browser open/visit > file or simply exits nil. Currently, org-export-as does the following: 1. Compute global export attributes, according to the selected export backend 2. Copy original buffer into working copy 3. Process and parse the copy, generating AST 4. Do the actual export You plugged your multipage processing into (4), but what it actually does involves (3), (4), and also a new kind of post-processing. I do not think that it is a good design from the point of view of ox.el. I prefer to reuse or extend the existing mechanisms if at all possible - this makes new features less confusing for users and backend developers. > - collects and adds information necessary for org-multipage to do its > job, splitting the document into different parts, etc. and What you describe here is more or less what :filter-parse-tree filters do - they can rearrange the parse tree before passing it to the transcoders. Why not reusing it for multipage export? > - then calls org-export-data on the subtrees and exports each returned > string to an individual file. And you simply call `org-export-transcode-page' for this, followed by writing the returned string to file. The first part can fit within `org-export-as', but writing to file is going a step beyond, duplicating what `org-export-to-file' does. > - It finally issues a done string and executes a browser open/visit > file or simply exits nil. ... which again steps beyond `org-export-as' scope - post-processing is currently done as a part of `org-export-to-file'/`org-export-to-buffer'. ---- Let me propose the following changes to ox.el: 1. org-data will be transcoded using `org-export-transcode-org-data', which can be overridden by setting org-data transcoders in the individual backends. 2. org-export-as will understand transcoded output to be a list of strings and will transfer INFO plist as text property in the return values 3. org-export-to-file will make use of the text properties to retrieve the file name to write. This way, export backend itself can assign the file names where each exporter string should go. I believe that my changes should allow you to implement multipage export in the following way: 1. You can use :filter-parse-tree in ox-html backend to replace the original (org-data ...) AST with a list of ((org-page ...) (org-page ...) ...) pseudo-elements and populate INFO channel with auxiliary information you now compute in `org-html-process-multipage' 2. You can define org-page transcoder to render individual pages as needed 3. You can assign :output-file text property to the returned org-page strings and use org-export-to-file to generate the multipage output on disk 4. You can handle opening exported files by augmenting POST-PROCESS argument in `org-html-export-to-multipage-html' and calling `org-export-file' instead of `org-export-as'. The tentative patches (against Org mode main branch) implementing my changes are attached. --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0001-ox-Factor-out-org-data-transcoding-into-dedicated-ov.patch >From 540c8ef21c26df79cf48f58afb4e88130985e2f7 Mon Sep 17 00:00:00 2001 Message-ID: <540c8ef21c26df79cf48f58afb4e88130985e2f7.1721815865.git.yantar92@posteo.net> From: Ihor Radchenko Date: Wed, 24 Jul 2024 11:40:57 +0200 Subject: [PATCH 1/3] ox: Factor out org-data transcoding into dedicated overrideable transcoder * lisp/ox.el (org-export-transcode-org-data): New function serving as the default transcoder for org-data export. (org-export-transcoder): Use `org-export-transcode-org-data' when no org-data transcoder is defined. (org-export-as): Rely upon org-data transcoder to do its job. --- lisp/ox.el | 55 +++++++++++++++++++++++++++++------------------------- 1 file changed, 30 insertions(+), 25 deletions(-) diff --git a/lisp/ox.el b/lisp/ox.el index fbd9bb0df..bdee71082 100644 --- a/lisp/ox.el +++ b/lisp/ox.el @@ -1883,9 +1883,11 @@ (defun org-export-transcoder (blob info) INFO is a plist containing export directives." (let ((type (org-element-type blob))) ;; Return contents only for complete parse trees. - (if (eq type 'org-data) (lambda (_datum contents _info) contents) - (let ((transcoder (cdr (assq type (plist-get info :translate-alist))))) - (and (functionp transcoder) transcoder))))) + (let ((transcoder (cdr (assq type (plist-get info :translate-alist))))) + (cond + ((functionp transcoder) transcoder) + ;; Use default org-data transcoder unless specified. + ((eq type 'org-data) #'org-export-transcode-org-data))))) (defun org-export--keep-spaces (data info) "Non-nil, when post-blank spaces after removing DATA should be preserved. @@ -3004,31 +3006,34 @@ (defun org-export-as backend info subtreep visible-only ext-plist)) ;; Eventually transcode TREE. Wrap the resulting string into ;; a template. - (let* ((body (org-element-normalize-string - (or (org-export-data (plist-get info :parse-tree) info) - ""))) - (inner-template (cdr (assq 'inner-template - (plist-get info :translate-alist)))) - (full-body (org-export-filter-apply-functions - (plist-get info :filter-body) - (if (not (functionp inner-template)) body - (funcall inner-template body info)) - info)) - (template (cdr (assq 'template - (plist-get info :translate-alist)))) - (output - (if (or (not (functionp template)) body-only) full-body - (funcall template full-body info)))) + (let ((output + (or (org-export-data (plist-get info :parse-tree) info) + ""))) ;; Call citation export finalizer. (when (plist-get info :with-cite-processors) (setq output (org-cite-finalize-export output info))) - ;; Remove all text properties since they cannot be - ;; retrieved from an external process. Finally call - ;; final-output filter and return result. - (org-no-properties - (org-export-filter-apply-functions - (plist-get info :filter-final-output) - output info))))))))) + (let ((filters (plist-get info :filter-final-output))) + ;; Remove all text properties since they cannot be + ;; retrieved from an external process. Finally call + ;; final-output filter and return result. + (org-no-properties + (org-export-filter-apply-functions filters output info)))))))))) + +(defun org-export-transcode-org-data (_ body info) + "Transcode `org-data' node with BODY. Return transcoded string. +INFO is the communication channel plist." + (let* ((inner-template (cdr (assq 'inner-template + (plist-get info :translate-alist)))) + (full-body (org-export-filter-apply-functions + (plist-get info :filter-body) + (if (not (functionp inner-template)) body + (funcall inner-template body info)) + info)) + (template (cdr (assq 'template + (plist-get info :translate-alist)))) + (body-only (memq 'body-only (plist-get info :export-options)))) + (if (or (not (functionp template)) body-only) full-body + (funcall template full-body info)))) (defun org-export--annotate-info (backend info &optional subtreep visible-only ext-plist) "Annotate the INFO plist according to the BACKEND. -- 2.45.2 --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0002-org-export-as-Allow-the-return-value-to-be-a-list-of.patch >From 1b0b331f92abc1ca7e04f71fe7ff60da57c719b8 Mon Sep 17 00:00:00 2001 Message-ID: <1b0b331f92abc1ca7e04f71fe7ff60da57c719b8.1721815865.git.yantar92@posteo.net> In-Reply-To: <540c8ef21c26df79cf48f58afb4e88130985e2f7.1721815865.git.yantar92@posteo.net> References: <540c8ef21c26df79cf48f58afb4e88130985e2f7.1721815865.git.yantar92@posteo.net> From: Ihor Radchenko Date: Wed, 24 Jul 2024 11:51:21 +0200 Subject: [PATCH 2/3] org-export-as: Allow the return value to be a list of strings; add INFO * lisp/ox.el (org-export-as): Allow the transcoders to return list of strings and return it. When returning a string, put INFO plist as text property. Do not remove text properties assigned by the transcoders. (org-export-data): Document that list of strings may be returned. --- lisp/ox.el | 28 ++++++++++++++++++++-------- 1 file changed, 20 insertions(+), 8 deletions(-) diff --git a/lisp/ox.el b/lisp/ox.el index bdee71082..a76b3b353 100644 --- a/lisp/ox.el +++ b/lisp/ox.el @@ -1930,7 +1930,7 @@ (defun org-export-data (data info) The `:filter-parse-tree' filters are not applied. -Return a string." +Return a string or a list of strings." (or (gethash data (plist-get info :exported-data)) ;; Handle broken links according to ;; `org-export-with-broken-links'. @@ -2969,7 +2969,9 @@ (defun org-export-as with external parameters overriding Org default settings, but still inferior to file-local settings. -Return code as a string." +Return code as a string or a list of strings. +The returned strings will have their `org-export-info' property set to +export information channel." (when (symbolp backend) (setq backend (org-export-get-backend backend))) (org-export-barf-if-invalid-backend backend) (org-fold-core-ignore-modifications @@ -3009,15 +3011,25 @@ (defun org-export-as (let ((output (or (org-export-data (plist-get info :parse-tree) info) ""))) + (setq output (ensure-list output)) ;; Call citation export finalizer. (when (plist-get info :with-cite-processors) - (setq output (org-cite-finalize-export output info))) + (setq output + (mapcar + (lambda (o) (org-cite-finalize-export o info)) + output))) (let ((filters (plist-get info :filter-final-output))) - ;; Remove all text properties since they cannot be - ;; retrieved from an external process. Finally call - ;; final-output filter and return result. - (org-no-properties - (org-export-filter-apply-functions filters output info)))))))))) + ;; Call final-output filter and return result. + (setq output + (mapcar + (lambda (o) (org-export-filter-apply-functions filters o info)) + output))) + ;; Apply org-export-info property. + (setq output + (mapcar + (lambda (o) (org-add-props o nil 'org-export-info info)) + output)) + (if (length= output 1) (car output) output)))))))) (defun org-export-transcode-org-data (_ body info) "Transcode `org-data' node with BODY. Return transcoded string. -- 2.45.2 --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0003-org-export-to-file-Derive-file-name-to-write-from-ex.patch >From 6fa2efadd229a667fba1b18aecc9d1ead5f284ac Mon Sep 17 00:00:00 2001 Message-ID: <6fa2efadd229a667fba1b18aecc9d1ead5f284ac.1721815865.git.yantar92@posteo.net> In-Reply-To: <540c8ef21c26df79cf48f58afb4e88130985e2f7.1721815865.git.yantar92@posteo.net> References: <540c8ef21c26df79cf48f58afb4e88130985e2f7.1721815865.git.yantar92@posteo.net> From: Ihor Radchenko Date: Wed, 24 Jul 2024 12:09:36 +0200 Subject: [PATCH 3/3] org-export-to-file: Derive file name to write from export output * lisp/ox.el (org-export--write-output): New helper function performing writing an export output or a list of outputs to file. It derives the file name from :output-file property in the output string or INFO plist stored in the output string. (org-export-to-file): Handle export output being a list of strings. Use `org-export--write-output'. --- lisp/ox.el | 61 ++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 38 insertions(+), 23 deletions(-) diff --git a/lisp/ox.el b/lisp/ox.el index a76b3b353..d78c04998 100644 --- a/lisp/ox.el +++ b/lisp/ox.el @@ -6830,6 +6830,31 @@ (defun org-latex-export-as-latex (switch-to-buffer-other-window buffer)) buffer))) +(defun org-export--write-output (output encoding) + "Write OUTPUT to file with ENCODING. +OUTPUT may be a string or a list of strings. +The target file is retrieved from :output-file OUTPUT property or +:output-file property in plist stored in `org-export-info' property of +each string. + +Return the file name or a list of file names." + (if (listp output) (mapcar #'org-export--write-output output) + (let ((file (or + (get-text-property 0 :output-file output) + (plist-get + (get-text-property 0 'org-export-info output) + :output-file)))) + (with-temp-buffer + (insert output) + ;; Ensure final newline. This is what was done + ;; historically, when we used `write-file'. + ;; Note that adding a newline is only safe for + ;; non-binary data. + (unless (bolp) (insert "\n")) + (let ((coding-system-for-write encoding)) + (write-region nil nil file)) + file)))) + ;;;###autoload (defun org-export-to-file (backend file &optional async subtreep visible-only body-only ext-plist @@ -6878,33 +6903,23 @@ (defun org-latex-export-to-latex `(let ((output (org-export-as ',backend ,subtreep ,visible-only ,body-only - ',ext-plist))) - (with-temp-buffer - (insert output) - ;; Ensure final newline. This is what was done - ;; historically, when we used `write-file'. - ;; Note that adding a newline is only safe for - ;; non-binary data. - (unless (bolp) (insert "\n")) - (let ((coding-system-for-write ',encoding)) - (write-region nil nil ,file))) - (or (ignore-errors (funcall ',post-process ,file)) ,file))) + ',ext-plist)) + file) + (setq file (org-export--write-output output ',encoding)) + (let ((post (lambda (f) (or (ignore-errors (funcall ',post-process f)) f)))) + (if (listp file) (mapcar post file) (funcall post file))))) (let ((output (org-export-as - backend subtreep visible-only body-only ext-plist))) - (with-temp-buffer - (insert output) - ;; Ensure final newline. This is what was done - ;; historically, when we used `write-file'. - ;; Note that adding a newline is only safe for - ;; non-binary data. - (unless (bolp) (insert "\n")) - (let ((coding-system-for-write encoding)) - (write-region nil nil file))) + backend subtreep visible-only body-only ext-plist)) + file) + (setq file (org-export--write-output output encoding)) (when (and (org-export--copy-to-kill-ring-p) (org-string-nw-p output)) (org-kill-new output)) ;; Get proper return value. - (or (and (functionp post-process) (funcall post-process file)) - file)))))) + (let ((post (lambda (f) + (or (and (functionp post-process) + (funcall post-process f)) + f)))) + (if (listp file) (mapcar post file) (funcall post file)))))))) (defun org-export-output-file-name (extension &optional subtreep pub-dir) "Return output file's name according to buffer specifications. -- 2.45.2 --=-=-= Content-Type: text/plain -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at --=-=-=--