From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id cJc4AaHOTGTWXQEASxT56A (envelope-from ) for ; Sat, 29 Apr 2023 10:00:33 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id AKQ4AKHOTGTdawAAG6o9tA (envelope-from ) for ; Sat, 29 Apr 2023 10:00:33 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 9489538657 for ; Sat, 29 Apr 2023 10:00:27 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1psfUW-0002Ai-2G; Sat, 29 Apr 2023 03:59:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psfUU-0002AZ-JG for emacs-orgmode@gnu.org; Sat, 29 Apr 2023 03:59:34 -0400 Received: from mout02.posteo.de ([185.67.36.66]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1psfUP-0005BO-QB for emacs-orgmode@gnu.org; Sat, 29 Apr 2023 03:59:33 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 303DD24011D for ; Sat, 29 Apr 2023 09:59:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1682755166; bh=u4QEGKIoAWQyCoV0lGpr+2Wif4y4zXfkvZtwCZ8U9dI=; h=From:To:Cc:Subject:Date:From; b=g9CYlfOvaNcYVO1LCnWefrhyXSRHVkdRW+ylVC2c0hzYvA5Flx9RjFkBr0t/UxeoY NPpAYK8s72OlrQKnR2IToFqUikzc0owh1zmfZPWmcUwvupUSEHNQCwynnQ/vmyUVv7 9dHboIb6bH8AN3l5e4knp0r4rXE4VyuAfFuutrOZNY4UwHZElrLu96LDIVL+2IrXDQ Df/81HDthsKroJJY7vgStR1yVQBWpesPiS5+O6NhxFY25TPL1KIrL7l5Zc/UtoE/yP DIvbutxHRfln0PYqGlSr6CH7nnqHiWTO3yxY1PLDr6ZLCCcVsTso3BBTy7YEU1YJlL Ljk5+ORHP2iTA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4Q7hhF3sY0z6txV; Sat, 29 Apr 2023 09:59:25 +0200 (CEST) From: Ihor Radchenko To: Brett Viren Cc: emacs-orgmode@gnu.org Subject: Re: Dealing with inter-org links vs #+include In-Reply-To: <87ttx0cacc.fsf@bnl.gov> References: <87ttx0cacc.fsf@bnl.gov> Date: Sat, 29 Apr 2023 08:02:30 +0000 Message-ID: <87o7n73yll.fsf@localhost> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=185.67.36.66; envelope-from=yantar92@posteo.net; helo=mout02.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Seal: i=1; s=key1; d=yhetil.org; t=1682755228; a=rsa-sha256; cv=none; b=APs4g9WmYV+tfTB7MW8ZHZoap3Sx2xGZlbkJHQU6t4VONY+ea7CTMEGM31wAqdD2u2htzZ eX/K1tA26CqRLXMm2+AkQ0/O2dY/m+DtjkdmQQJiTqNvtTkXeN4k34s9KsWQ/BJwLbMUt0 lW60y7rtzcYwajiGYf+NLJYs/2DZ+GOYDA35UyKdzGg0M4APA4kShnMvo9JFWEUSnymQix bExepUfLXti09O2/8U8Nnwcac8sgihsc/6SeaW5KtN0rDcdl2O9qwkjsejLkUOPHHjOwb3 DsU/24TmJ/bvPEUtFRCQJyAtm9V+hvpMg9NefOxu7ArsF2Z/kgkfv5PKjFyvbQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=g9CYlfOv; dmarc=pass (policy=none) header.from=posteo.net; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1682755228; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=q+hR/40a2WhZYnriyc8Llik2mj7ADodHNK/e4TpWzVQ=; b=LAaqLUyj1vcrHbnrtYNHjtz/amDsw/FFXmS0ooZeOkKV7UmZc9TeyyH3a4ErjxyyQyO7RF SKllB4UGMUh6hkiPPiHWRmNH5zv2NZdiH1YDccD4zYiOnHNPyu4kkgpRx7cyKom0tJnQXO ISUqh5vfVE48g+oHLKLRmYC77uYJH50xUNQsVZF5XCWTv7Lyma9Eb/doKhxrlwqlA+KjD+ dXWW1LkJor0OBhJvjMh0NTVfuKl0FYgk8SRkGvq3g7lcngcPpJCkZeNtqI0U0i4jrEOhe1 G0HrEko5FIDiv85fMEadI0oAfwSQNHuJrVLTeyoijbbRAH3amT4+0GRX3uX24w== X-Migadu-Scanner: scn1.migadu.com Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=g9CYlfOv; dmarc=pass (policy=none) header.from=posteo.net; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -6.19 X-Spam-Score: -6.19 X-Migadu-Queue-Id: 9489538657 X-TUID: +lvcymImollB --=-=-= Content-Type: text/plain Brett Viren writes: > I want what may be two conflicting things: > > - Produce a monolithic HTML export of an main org file that #+include:'s > other org files. > > - Have links in the other org files that are valid both when the > monolith is exported to HTML and when visiting the individual org > files in Emacs. I can see the rationale, but it is tricky. > === a.org === > * A > > This is A. > === b.org === > * B > > This is B. Links: > - [[file:a.org::*A][A by file with headline]]. > - [[A][A by headline only]]. > > === main.org === > ... > * Includes > > #+include: a.org > #+include: b.org In this scenario, it indeed makes sense to replace links to a.org and b.org with internal links. However, someone who is exporting multiple files, including a.org/b.org may actually want to keep links to a.org/b.org. In particular, when only portions of a.org/b.org are included (like a single src block or certain lines). > * Links > > - A :: [[file:a.org]] > > - B :: [[file:b.org]] And this is actually a scenario where it also makes more sense to keep the links to a.org/b.org. If we follow your suggestion, these would be > - A :: [[file:main.org]] > > - B :: [[file:main.org]] which would look confusing. For reference, I am attaching a patch that blindly converts links to included files into links to the top-level includer. However, the patch will fail for partially included files where the portions the links are referencing are not actually included. It will also replace A:: and B:: links in the above example. So, it is not suitable for inclusion in Org - we need some better idea how to handle other possible scenarios. Suggestions welcome! (For now, I do not see how we can handle the described scenarios cleanly) --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=inter-links.patch diff --git a/lisp/ox.el b/lisp/ox.el index 480c484d4..1d6f4dff5 100644 --- a/lisp/ox.el +++ b/lisp/ox.el @@ -3392,7 +3392,17 @@ (defun org-export-expand-include-keyword (&optional included dir footnotes inclu (goto-char (point-max)) (maphash (lambda (k v) (insert (format "\n[fn:%s] %s\n" k v))) - footnotes)))))))))))) + footnotes)))))))))) + ;; Replace all the links to included files with links + ;; to top-level includer. + (unless included + (org-with-wide-buffer + (org-export--map-links + (lambda (link) + (org-export--update-included-link-file + (buffer-file-name (buffer-base-buffer)) + (hash-table-keys file-prefix) + link))))))) (defun org-export-parse-include-value (value &optional dir) "Extract the various parameters from #+include: VALUE. @@ -3606,15 +3616,17 @@ (defun org-export--inclusion-absolute-lines (file location only-contents lines) (while (< (point) end) (cl-incf counter) (forward-line)) counter)))))))) -(defun org-export--update-included-link (file-dir includer-dir) +(defun org-export--update-included-link (file-dir includer-dir &optional link) "Update relative file name of link at point, if possible. FILE-DIR is the directory of the file being included. INCLUDER-DIR is the directory of the file where the inclusion is going to happen. +Optional argument LINK, when non-nil, holds the link object. + Move point after the link." - (let* ((link (org-element-link-parser)) + (let* ((link (or link (org-element-link-parser))) (path (org-element-property :path link))) (if (or (not (string= "file" (org-element-property :type link))) (file-remote-p path) @@ -3633,6 +3645,80 @@ (defun org-export--update-included-link (file-dir includer-dir) (org-element-property :end link)) (insert (org-element-interpret-data new-link)))))) +(defun org-export--update-included-link-file (new-file files &optional link) + "Replace file link paths to FILES with NEW-FILE. + +Absolute paths are replaced with NEW-FILE. Relative paths are +replaced with NEW-FILE relative to `default-directory'. + +FILES are assumed to be absolute paths. + +Optional argument LINK, when non-nil, holds the link object. + +Move point after the link." + (let* ((link (or link (org-element-link-parser))) + (path (org-element-property :path link))) + (if (or (not (string= "file" (org-element-property :type link))) + (not (member (expand-file-name path) files))) + (goto-char (org-element-property :end link)) + (let ((new-path + (if (file-name-absolute-p path) + new-file (file-relative-name new-file))) + (new-link (org-element-copy link))) + (if (equal new-path path) + (goto-char (org-element-property :end link)) + (if (not (org-element-property :search-option link)) + (org-element-put-property new-link :path new-path) + ;; Internal link to included file. Do not keep file + ;; type as `org-export-resolve-link' will look into the + ;; original file version without INCLUDE keywords + ;; expanded. + (org-element-put-property new-link :type "fuzzy") + (org-element-put-property + new-link :path + (org-element-property :search-option link))) + (when (org-element-property :contents-begin link) + (org-element-adopt new-link + (buffer-substring + (org-element-property :contents-begin link) + (org-element-property :contents-end link)))) + (delete-region (org-element-property :begin link) + (org-element-property :end link)) + (insert (org-element-interpret-data new-link))))))) + +(defun org-export--map-links (func) + "Apply FUNC on every link in accessible part of current Org buffer. +FUNC is called with a single argument - link object, with point at the +beginning of a link. + +Also look for links within link's description. Org doesn't support +such construct, but `org-export-insert-image-links' may activate them." + (save-excursion + (goto-char (point-min)) + (let ((regexp (concat org-link-plain-re "\\|" org-link-angle-re))) + (while (re-search-forward org-link-any-re nil t) + (let ((link (save-excursion + (forward-char -1) + (save-match-data (org-element-context))))) + (when (org-element-type-p link 'link) + ;; Look for file links within link's description. + ;; Org doesn't support such construct, but + ;; `org-export-insert-image-links' may activate + ;; them. + (let ((contents-begin + (org-element-property :contents-begin link)) + (begin (org-element-property :begin link))) + (when contents-begin + (save-excursion + (goto-char (org-element-property :contents-end link)) + (while (re-search-backward regexp contents-begin t) + (save-match-data + (funcall func (org-element-link-parser))) + (goto-char (match-beginning 0))))) + ;; Update current link, if necessary. + (goto-char begin) + (funcall func link)))))))) + (defun org-export--prepare-file-contents (file &optional lines ind minlevel id footnotes includer) "Prepare contents of FILE for inclusion and return it as a string. @@ -3682,35 +3768,12 @@ (defun org-export--prepare-file-contents (let ((file-dir (file-name-directory file)) (includer-dir (file-name-directory includer))) (unless (file-equal-p file-dir includer-dir) - (goto-char (point-min)) (unless (eq major-mode 'org-mode) (let ((org-inhibit-startup t)) (org-mode))) ;set regexps - (let ((regexp (concat org-link-plain-re "\\|" org-link-angle-re))) - (while (re-search-forward org-link-any-re nil t) - (let ((link (save-excursion - (forward-char -1) - (save-match-data (org-element-context))))) - (when (org-element-type-p link 'link) - ;; Look for file links within link's description. - ;; Org doesn't support such construct, but - ;; `org-export-insert-image-links' may activate - ;; them. - (let ((contents-begin - (org-element-property :contents-begin link)) - (begin (org-element-property :begin link))) - (when contents-begin - (save-excursion - (goto-char (org-element-property :contents-end link)) - (while (re-search-backward regexp contents-begin t) - (save-match-data - (org-export--update-included-link - file-dir includer-dir)) - (goto-char (match-beginning 0))))) - ;; Update current link, if necessary. - (when (string= "file" (org-element-property :type link)) - (goto-char begin) - (org-export--update-included-link - file-dir includer-dir)))))))))) + (org-export--map-links + (lambda (link) + (org-export--update-included-link + file-dir includer-dir link)))))) ;; Remove blank lines at beginning and end of contents. The logic ;; behind that removal is that blank lines around include keyword ;; override blank lines in included file. --=-=-= Content-Type: text/plain -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at --=-=-=--