From mboxrd@z Thu Jan 1 00:00:00 1970 From: Rasmus Subject: Re: [patch, ox] #+INCLUDE resolves links Date: Sun, 28 Sep 2014 21:32:08 +0200 Message-ID: <87d2af1qyv.fsf@gmx.us> References: <87k34x6bjd.fsf@gmx.us> <87lhpdurfh.fsf@gmx.us> <87bnq984hd.fsf@nicolasgoaziou.fr> <87bnq5zzp7.fsf@gmx.us> <87oau4ems5.fsf@nicolasgoaziou.fr> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:34356) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XYKCl-0008Aa-Mp for emacs-orgmode@gnu.org; Sun, 28 Sep 2014 15:32:28 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XYKCg-0007Ub-M0 for emacs-orgmode@gnu.org; Sun, 28 Sep 2014 15:32:23 -0400 Received: from mout.gmx.net ([212.227.15.15]:65178) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XYKCg-0007Rr-CL for emacs-orgmode@gnu.org; Sun, 28 Sep 2014 15:32:18 -0400 Received: from W530 ([46.166.186.235]) by mail.gmx.com (mrgmx003) with ESMTPSA (Nemesis) id 0MVdfD-1XlAuL30lr-00Z0OR for ; Sun, 28 Sep 2014 21:32:11 +0200 In-Reply-To: <87oau4ems5.fsf@nicolasgoaziou.fr> (Nicolas Goaziou's message of "Wed, 24 Sep 2014 23:22:50 +0200") List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: emacs-orgmode@gnu.org --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, Thanks for the comments. I hope I addressed the previous comments and did not introduce new reasons bugs. I added tests. Comments on comments follow. Nicolas Goaziou writes: >> Okay, I hope I got it now. It's a rather forgiving regexp in terms of >> mistakes. Is that OK? > > Please no ":only-contents yes", ":only-contents true", ":only-contents > of_course!" in the regexp. If :only-contents is followed by anything but > nil or another keyword, its value is non-nil. See below. Good catch; I added explicit support for =C2=A1?[oO][fF][-_]?[cC][Oo][uU][rR][sS][\.!=C2=A1]? in the regexp! =C2=A1Gotta catch 'em all! > The sentence is not complete. Also, it should be something like "If you > set @code{:only-contents} property to a non-nil value, only...". >> [...] > This is not true anymore about the drawers. This should be merged with > the previous phrase to avoid duplicating "@code{:only-contents}" (e.g., > only the contents of the matched element are inserted, without any > planning line or property drawer). Fixed this and other documentation bugs =E2=80=94 hopefully. Let me know if it's clear. >> + (let ((matched (save-match-data >> + (org-split-string >> + (org-remove-double-quotes (match-string 1 value)) "::")))) > > There's no reason to use `org-split-string' here since you only want to > match the last "::". You can use the same regexp used by > "org-element.el", i.e. > > (when (string-match "::\\(.*\\)\\'" value) > (setq location (match-string 1 value) > value (replace-match "" nil nil value))) Custom_ID is very flexible. I've use a similar regexp. >> + (only-contents >> + (and (string-match >> + ":only-?contents?[[:space:]]*\"?\\(t\\|true\\|yes\\)?\"?" >> + value) >> + (prog1 (and (match-string 1 value) t) >> + (setq value (replace-match "" nil nil value))))) > > (only-contents > (and (string-match ":only-contents +\\([^: \r\t\n]\\S-*\\)" value) > (org-not-nil (match-string 1 value)))) I have removed flexibility in speling. >> + (narrow-to-region >> + (org-element-property >> + (if only-contents :contents-begin :begin) (org-element-at-point)) >> + (org-element-property (if only-contents :contents-end :end) >> (org-element-at-point)))) > > (let ((element (org-element-at-point))) > (let ((contents-beg > (and only-contents > (org-element-property :contents-begin element)))) > (narrow-to-region > (or contents-beg (org-element-property :begin element)) > (org-element-property (if contents-beg :contents-end :end) element= )))) Just out of curiosity, what is an example of a element that can be named and does not have a :contents-begin? >> + (when only-contents >> + ;; skip drawers and property-drawers >> + ;; these are removed as needed in `org-export--prepare-file-conte= nts' >> + ;; TODO: How to actually do this? Only line numbers are send to >> + ;; `org-export--prepare-file-contents'. split in two? >> + (goto-char (point-min)) >> + (org-skip-whitespace) >> + (beginning-of-line) >> + (let ((element (org-element-at-point))) >> + (while (memq (org-element-type element) '(drawer property-drawer)) >> + (goto-char (org-element-property :end element)) >> + (setq element (org-element-at-point))))) > > Regular drawers are not expected to be skipped. Also, the following > should be better > > (when (and only-contents > (memq (org-element-type element) '(headline inlinetask))) > (goto-char (point-min)) > (when (org-looking-at-p org-planning-line-re) (forward-line)) > (when (looking-at org-property-drawer-re) (goto-char (match-end 0))) > (unless (bolp) (forward-line))) > > This should be obviously included within the previous `let'. Okay, there's a lot of improvements in that suggestion. However, it misses this case which created using only "official" shortcuts * head SCHEDULED: <2014-09-28 sun> :LOGBOOK: - Note taken on [2014-09-28 sat 12:21] \\ a drawer :END: :PROPERTIES: :CUSTOM_ID: h :END: The patch handles something like this now cf. the last test. >> + (apply (lambda (beg end) (format "%s-%s" beg end)) >> + ;; `line-number-at-pos' returns the narrowed line-number >> + (mapcar 'line-number-at-pos (prog1 (list (point-min) (point-max)) >> (widen)))))) > > This is inefficient because `line-number-at-pos' will start counting > twice from line 1. > > (goto-char beg) > (widen) > (let ((start-line (line-number-at-pos))) > (format "%d-%d" > start-line > (+ start-line > (let ((c 0)) (while (< (point) end) (incf c) (forward-line= )) c)))) > > I didn't check, there may an off-by-one error. Anyway, all this needs > tests. Fine with me. It's a bit less elegant IMO, but you are right. I had to do it slightly differently since the line number needs to appear irrespective of whether lines are included in the call initially. That being said, I could have very well overlooked some obvioues way of doing it. =E2=80=94Rasmus --=20 Hooray! --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0001-ox-Allow-file-links-with-INCLUDE-keyword.patch >From 0c0b4a4b9ea0db7ce3f8c08c9a2947db85c54470 Mon Sep 17 00:00:00 2001 From: Rasmus Date: Sun, 28 Sep 2014 21:05:17 +0200 Subject: [PATCH] ox: Allow file-links with #+INCLUDE-keyword * org.el (org-edit-special): Handle file-links for INCLUDE. * ox.el (org-export--prepare-file-contents): Handle links and add option no-heading. * ox.el (org-export-expand-include-keyword): Resolve headline links and add option :only-contents. * orgguide.texi (Include files) org.texi (Include files): Updated. * testing/examples/include.org: New examples. * test-ox.el (test-org-export/expand-include): New tests. --- doc/org.texi | 18 +++++++ doc/orgguide.texi | 9 +++- lisp/org.el | 8 +-- lisp/ox.el | 117 +++++++++++++++++++++++++++++++++++++++---- testing/examples/include.org | 25 +++++++++ testing/lisp/test-ox.el | 57 ++++++++++++++++++++- 6 files changed, 218 insertions(+), 16 deletions(-) diff --git a/doc/org.texi b/doc/org.texi index 7d98d51..5a9c0c5 100644 --- a/doc/org.texi +++ b/doc/org.texi @@ -10008,6 +10008,24 @@ to use the obvious defaults. #+INCLUDE: "~/.emacs" :lines "10-" @r{Include lines from 10 to EOF} @end example +Finally, you may use a file-link, see @ref{search option in file links}, to +extract an object as matched by @code{org-link-search}@footnote{Note that +@code{org-link-search-must-match-exact-headline} is locally bound to non-nil. +Therefore, @code{org-link-search} only matches headlines and named +elements.}. If the @code{:only-contents} property is non-nil, only the +contents of the requested element will be included, omitting any +property-drawers, planning-lines, attributes, captions etc. The +@code{:lines} keyword operates locally with respect to the requested element. +Some examples: + +@example +#+INCLUDE: "./paper.org::#theory" :only-contents t + @r{Include the body of the heading with the custom id @code{theory}} +#+INCLUDE: "./paper.org::mytable" @r{Include named element.} +#+INCLUDE: "./paper.org::*conclusion" :lines 1-20 + @r{Include the first 20 lines of the headline named conclusion.} +@end example + @table @kbd @kindex C-c ' @item C-c ' diff --git a/doc/orgguide.texi b/doc/orgguide.texi index ca8e052..4feeaca 100644 --- a/doc/orgguide.texi +++ b/doc/orgguide.texi @@ -2264,8 +2264,13 @@ include your @file{.emacs} file, you could use: The optional second and third parameter are the markup (i.e., @samp{example} or @samp{src}), and, if the markup is @samp{src}, the language for formatting the contents. The markup is optional, if it is not given, the text will be -assumed to be in Org mode format and will be processed normally. @kbd{C-c '} -will visit the included file. +assumed to be in Org mode format and will be processed normally. File-links +will be interpreted as well: +@smallexample +#+INCLUDE: "./otherfile.org::#my_custom_id" :only-contents t +@end smallexample +@noindent +@kbd{C-c '} will visit the included file. @node Embedded @LaTeX{}, , Include files, Markup @section Embedded @LaTeX{} diff --git a/lisp/org.el b/lisp/org.el index b09e72d..c70e44a 100755 --- a/lisp/org.el +++ b/lisp/org.el @@ -20522,9 +20522,11 @@ Otherwise, return a user error." session params)))))) (keyword (if (member (org-element-property :key element) '("INCLUDE" "SETUPFILE")) - (find-file-other-window - (org-remove-double-quotes - (car (org-split-string (org-element-property :value element))))) + (org-open-link-from-string + (format "[[%s]]" + (expand-file-name + (org-remove-double-quotes + (car (org-split-string (org-element-property :value element))))))) (user-error "No special environment to edit here"))) (table (if (eq (org-element-property :type element) 'table.el) diff --git a/lisp/ox.el b/lisp/ox.el index 59091fc..ed1024b 100644 --- a/lisp/ox.el +++ b/lisp/ox.el @@ -3321,13 +3321,25 @@ paths." ;; Extract arguments from keyword's value. (let* ((value (org-element-property :value element)) (ind (org-get-indentation)) + location (file (and (string-match "^\\(\".+?\"\\|\\S-+\\)\\(?:\\s-+\\|$\\)" value) - (prog1 (expand-file-name - (org-remove-double-quotes - (match-string 1 value)) - dir) + (prog1 + (save-match-data + (let ((matched (match-string 1 value))) + (when (string-match "\\(::\\(.*?\\)\\)\"?\\'" matched) + (setq location (match-string 2 matched)) + (setq matched + (replace-match "" nil nil matched 1))) + (expand-file-name + (org-remove-double-quotes + matched) + dir))) (setq value (replace-match "" nil nil value))))) + (only-contents + (and (string-match ":only-contents +\\([^: \r\t\n]\\S-*\\)" value) + (prog1 (org-not-nil (match-string 1 value)) + (setq value (replace-match "" nil nil value))))) (lines (and (string-match ":lines +\"\\(\\(?:[0-9]+\\)?-\\(?:[0-9]+\\)?\\)\"" @@ -3387,17 +3399,88 @@ paths." (t (insert (with-temp-buffer - (let ((org-inhibit-startup t)) (org-mode)) - (insert - (org-export--prepare-file-contents - file lines ind minlevel - (or (gethash file file-prefix) - (puthash file (incf current-prefix) file-prefix)))) + (let ((org-inhibit-startup t) + (lines + (if location + (org-export--inclusion-absolute-lines + file location only-contents lines) + lines))) + (org-mode) + (insert + (org-export--prepare-file-contents + file lines ind minlevel + (or (gethash file file-prefix) + (puthash file (incf current-prefix) file-prefix))))) (org-export-expand-include-keyword (cons (list file lines) included) (file-name-directory file)) (buffer-string))))))))))))) +(defun org-export--inclusion-absolute-lines (file location only-contents lines) + "Resolve absolute lines for an included file with file-link. + +FILE is string file-name of the file to include. LOCATION is a +string name within FILE to be included (located via +`org-link-search'). If ONLY-CONTENTS is non-nil only the +contents of the named element will be included, as determined +Org-Element. If LINES is non-nil only those lines are included. + +Return a string of lines to be included in the format expected by +`org-export--prepare-file-contents'." + (with-temp-buffer + (insert-file-contents file) + (unless (eq major-mode 'org-mode) + (let ((org-inhibit-startup t)) (org-mode))) + (condition-case err + ;; Enforce consistent search. + (let ((org-link-search-must-match-exact-headline t)) + (org-link-search location)) + (error + (error (format "%s for %s::%s" (error-message-string err) file location)))) + (let* ((element (org-element-at-point)) + (contents-begin + (and only-contents (org-element-property :contents-begin element)))) + (narrow-to-region + (or contents-begin (org-element-property :begin element)) + (org-element-property (if contents-begin :contents-end :end) element)) + (when (and only-contents + (memq (org-element-type element) '(headline inlinetask))) + ;; skip planning line and property-drawer. If a normal drawer + ;; precedes a property-drawer both will be included. + ;; Remaining property-drawers are removed as needed in + ;; `org-export--prepare-file-contents' + (goto-char (point-min)) + (when (org-looking-at-p org-planning-line-re) (forward-line)) + (when (looking-at org-property-drawer-re) (goto-char (match-end 0))) + (unless (bolp) (forward-line)) + (narrow-to-region (point) (point-max)))) + (when lines + (org-skip-whitespace) + (beginning-of-line) + (let* ((lines (split-string lines "-")) + (lbeg (string-to-number (car lines))) + (lend (string-to-number (cadr lines))) + (beg (if (zerop lbeg) (point-min) + (goto-char (point-min)) + (forward-line (1- lbeg)) + (point))) + (end (if (zerop lend) (point-max) + (goto-char beg) + (forward-line (1- lend)) + (point)))) + (narrow-to-region beg end))) + (let ((end (point-max))) + (goto-char (point-min)) + (widen) + (let ((start-line (line-number-at-pos))) + (format "%d-%d" + start-line + (save-excursion + (+ start-line + (let ((counter 0)) + (while (< (point) end) (incf counter) (forward-line)) + counter)))))))) + (defun org-export--prepare-file-contents (file &optional lines ind minlevel id) "Prepare the contents of FILE for inclusion and return them as a string. @@ -3444,6 +3527,20 @@ with footnotes is included in a document." (skip-chars-backward " \r\t\n") (forward-line) (delete-region (point) (point-max)) + ;; Remove property-drawers after drawers. + (when (or ind minlevel) + (unless (eq major-mode 'org-mode) + (let ((org-inhibit-startup t)) (org-mode))) + (goto-char (point-min)) + (when (looking-at org-drawer-regexp) + (goto-char (match-end 0)) + (search-forward-regexp org-drawer-regexp) + (forward-line 1) + (beginning-of-line)) + (when (looking-at org-property-drawer-re) + (delete-region (match-beginning 0) (match-end 0)) + (beginning-of-line)) + (delete-region (point) (save-excursion (and (org-skip-whitespace) (point))))) ;; If IND is set, preserve indentation of include keyword until ;; the first headline encountered. (when ind diff --git a/testing/examples/include.org b/testing/examples/include.org index 186facb..c04c942 100644 --- a/testing/examples/include.org +++ b/testing/examples/include.org @@ -8,3 +8,28 @@ Small Org file with an include keyword. * Heading body + +* Another heading +:PROPERTIES: +:CUSTOM_ID: ah +:END: +1 +2 +3 + +* A headline with a table +:PROPERTIES: +:CUSTOM_ID: ht +:END: +#+CAPTION: a table +#+NAME: tbl +| 1 | + +* drawer-headline +:LOGBOOK: +drawer +:END: +:PROPERTIES: +:CUSTOM_ID: dh +:END: +content diff --git a/testing/lisp/test-ox.el b/testing/lisp/test-ox.el index 4af3510..4706ec2 100644 --- a/testing/lisp/test-ox.el +++ b/testing/lisp/test-ox.el @@ -918,7 +918,62 @@ Footnotes[fn:1], [fn:test] and [fn:inline:anonymous footnote]. (org-export-expand-include-keyword) (org-element-map (org-element-parse-buffer) 'footnote-reference - (lambda (ref) (org-element-property :label ref)))))))))))) + (lambda (ref) (org-element-property :label ref))))))))))) + ;; If only-contents is non-nil only include contents of element + (should + (equal + "body\n" + (org-test-with-temp-text + (format "#+INCLUDE: \"%s/examples/include.org::*Heading\" :only-contents t" org-test-dir) + (org-export-expand-include-keyword) + (buffer-string)))) + ;; Headings can be included via CUSTOM_ID + (should + (org-test-with-temp-text + (format "#+INCLUDE: \"%s/examples/include.org::#ah\"" org-test-dir) + (org-export-expand-include-keyword) + (goto-char (point-min)) + (looking-at "* Another heading"))) + ;; Named objects can be included + (should + (equal + "| 1 |\n" + (org-test-with-temp-text + (format "#+INCLUDE: \"%s/examples/include.org::tbl\" :only-contents t" org-test-dir) + (org-export-expand-include-keyword) + (buffer-string)))) + ;; Including non-existing elements should result in an error + (should-error + (org-test-with-temp-text + (format "#+INCLUDE: \"%s/examples/include.org::*non-existing heading\"" org-test-dir) + (org-export-expand-include-keyword))) + ;; Lines work relatively to an included element + (should + (equal + "2\n3\n" + (org-test-with-temp-text + (format "#+INCLUDE: \"%s/examples/include.org::#ah\" :only-contents t :lines \"2-3\"" org-test-dir) + (org-export-expand-include-keyword) + (buffer-string)))) + ;; Properties should be dropped from headlines + (should + (equal + (org-test-with-temp-text + (format "#+INCLUDE: \"%s/examples/include.org::#ht\" :only-contents t" org-test-dir) + (org-export-expand-include-keyword) + (buffer-string)) + (org-test-with-temp-text + (format "#+INCLUDE: \"%s/examples/include.org::tbl\"" org-test-dir) + (org-export-expand-include-keyword) + (buffer-string)))) + ;; Properties should be dropped, drawers should not be + (should + (equal + ":LOGBOOK:\ndrawer\n:END:\ncontent\n" + (org-test-with-temp-text + (format "#+INCLUDE: \"%s/examples/include.org::#dh\" :only-contents t" org-test-dir) + (org-export-expand-include-keyword) + (buffer-string))))) (ert-deftest test-org-export/expand-macro () "Test macro expansion in an Org buffer." -- 2.1.1 --=-=-=--