Re: [patch, ox] #+INCLUDE resolves links

[Rasmus <rasmus@gmx.us>]

[-- Attachment #1: Type: text/plain, Size: 1558 bytes --]

Hi,

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> It isn't very important, but you forgot full stops at the end of
> comments in the test file.

Fixed. 

>> I want to discuss one more important potential issue before having the
>> patch applied.  Currently, location is ignored if the included part is
>> not an env (line 3381) and not a block (3392).  I'm not sure this is
>> right.  I could do one of the following:
>>
>>    1. Nothing (current state)
>>    2. Throw an error if location and env or block are combined.
>>    3. Try to use location even if block is set.  Recall, though, that
>>       location is resolved using org-mode. 
>>    4. Let location be a general regexp if env or block is non-nil.
>>       But then we are breaking with the org file-link idea.
>>    5. Make location work for org files when env or block, otherwise
>>       throw an error.
>>
>> WDYT?
>
> I think option 1 is perfect. If a block with org contents is needed, one
> can always do
>
>   #+begin_center
>   #+include: "file.org::*headline"
>   #+end_center
>
> Block and environments are really meant for literal insertion, where
> locations do not apply.

Great!  Less work.  

We can always tune it later as necessary.

>> +		 (only-contents
>> + (and (string-match ":only-contents *?\\([^: \r\t\n]\\S-*\\)?"
>> value)
>
> Is the shy *? necessary?

No!  Only the * is necessary to catch an implicit nil.  Thanks.

Feel free to commit the attached patch if there's nothing else.

—Rasmus 

-- 
Hooray!

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ox-Allow-file-links-with-INCLUDE-keyword.patch --]
[-- Type: text/x-diff, Size: 12764 bytes --]

From 35fdda4c4fe160bee2963687d368104febb2b627 Mon Sep 17 00:00:00 2001
From: Rasmus <rasmus@gmx.us>
Date: Sun, 28 Sep 2014 21:05:17 +0200
Subject: [PATCH] ox: Allow file-links with #+INCLUDE-keyword

* org.el (org-edit-special): Handle file-links for INCLUDE.
* ox.el (org-export--prepare-file-contents): Handle links and
add option no-heading.
* ox.el (org-export-expand-include-keyword): Resolve headline
links and add option :only-contents.
* orgguide.texi (Include files)
org.texi (Include files): Updated.
* testing/examples/include.org: New examples.
* test-ox.el (test-org-export/expand-include): New tests.
---
 doc/org.texi                 |  17 +++++++
 doc/orgguide.texi            |   9 +++-
 lisp/org.el                  |   9 ++--
 lisp/ox.el                   | 117 +++++++++++++++++++++++++++++++++++++++----
 testing/examples/include.org |  25 +++++++++
 testing/lisp/test-ox.el      |  59 +++++++++++++++++++++-
 6 files changed, 220 insertions(+), 16 deletions(-)

diff --git a/doc/org.texi b/doc/org.texi
index 7d98d51..296e289 100644
--- a/doc/org.texi
+++ b/doc/org.texi
@@ -10008,6 +10008,23 @@ to use the obvious defaults.
 #+INCLUDE: "~/.emacs" :lines "10-"    @r{Include lines from 10 to EOF}
 @end example
 
+Finally, you may use a file-link, see @ref{search option in file links}, to
+extract an object as matched by @code{org-link-search}@footnote{Note that
+@code{org-link-search-must-match-exact-headline} is locally bound to non-nil.
+Therefore, @code{org-link-search} only matches headlines and named
+elements.}.  If the @code{:only-contents} property is non-nil, only the
+contents of the requested element will be included, omitting properties
+drawer and planning-line if present.  The @code{:lines} keyword
+operates locally with respect to the requested element.  Some examples:
+
+@example
+#+INCLUDE: "./paper.org::#theory" :only-contents t
+   @r{Include the body of the heading with the custom id @code{theory}}
+#+INCLUDE: "./paper.org::mytable"  @r{Include named element.}
+#+INCLUDE: "./paper.org::*conclusion" :lines 1-20
+   @r{Include the first 20 lines of the headline named conclusion.}
+@end example
+
 @table @kbd
 @kindex C-c '
 @item C-c '
diff --git a/doc/orgguide.texi b/doc/orgguide.texi
index ca8e052..4feeaca 100644
--- a/doc/orgguide.texi
+++ b/doc/orgguide.texi
@@ -2264,8 +2264,13 @@ include your @file{.emacs} file, you could use:
 The optional second and third parameter are the markup (i.e., @samp{example}
 or @samp{src}), and, if the markup is @samp{src}, the language for formatting
 the contents.  The markup is optional, if it is not given, the text will be
-assumed to be in Org mode format and will be processed normally. @kbd{C-c '}
-will visit the included file.
+assumed to be in Org mode format and will be processed normally.  File-links
+will be interpreted as well:
+@smallexample
+#+INCLUDE: "./otherfile.org::#my_custom_id" :only-contents t
+@end smallexample
+@noindent
+@kbd{C-c '} will visit the included file.
 
 @node Embedded @LaTeX{},  , Include files, Markup
 @section Embedded @LaTeX{}
diff --git a/lisp/org.el b/lisp/org.el
index 9815eb4..0910296 100755
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -20524,9 +20524,12 @@ Otherwise, return a user error."
 		       session params))))))
       (keyword
        (if (member (org-element-property :key element) '("INCLUDE" "SETUPFILE"))
-           (find-file-other-window
-            (org-remove-double-quotes
-             (car (org-split-string (org-element-property :value element)))))
+           (org-open-link-from-string
+	    (format "[[%s]]"
+		    (expand-file-name
+		     (org-remove-double-quotes
+		      (car (org-split-string
+			    (org-element-property :value element)))))))
          (user-error "No special environment to edit here")))
       (table
        (if (eq (org-element-property :type element) 'table.el)
diff --git a/lisp/ox.el b/lisp/ox.el
index 90c623e..06eb8eb 100644
--- a/lisp/ox.el
+++ b/lisp/ox.el
@@ -3325,13 +3325,25 @@ paths."
 	  ;; Extract arguments from keyword's value.
 	  (let* ((value (org-element-property :value element))
 		 (ind (org-get-indentation))
+		 location
 		 (file (and (string-match
 			     "^\\(\".+?\"\\|\\S-+\\)\\(?:\\s-+\\|$\\)" value)
-			    (prog1 (expand-file-name
-				    (org-remove-double-quotes
-				     (match-string 1 value))
-				    dir)
+			    (prog1
+				(save-match-data
+				  (let ((matched (match-string 1 value)))
+				    (when (string-match "\\(::\\(.*?\\)\\)\"?\\'" matched)
+				      (setq location (match-string 2 matched))
+				      (setq matched
+					    (replace-match "" nil nil  matched 1)))
+				    (expand-file-name
+				     (org-remove-double-quotes
+				      matched)
+				     dir)))
 			      (setq value (replace-match "" nil nil value)))))
+		 (only-contents
+		  (and (string-match ":only-contents *\\([^: \r\t\n]\\S-*\\)?" value)
+		       (prog1 (org-not-nil (match-string 1 value))
+			 (setq value (replace-match "" nil nil value)))))
 		 (lines
 		  (and (string-match
 			":lines +\"\\(\\(?:[0-9]+\\)?-\\(?:[0-9]+\\)?\\)\""
@@ -3391,17 +3403,88 @@ paths."
 	       (t
 		(insert
 		 (with-temp-buffer
-		   (let ((org-inhibit-startup t)) (org-mode))
-		   (insert
-		    (org-export--prepare-file-contents
-		     file lines ind minlevel
-		     (or (gethash file file-prefix)
-			 (puthash file (incf current-prefix) file-prefix))))
+		   (let ((org-inhibit-startup t)
+			 (lines
+			  (if location
+			      (org-export--inclusion-absolute-lines
+			       file  location only-contents lines)
+			    lines)))
+		     (org-mode)
+		     (insert
+		      (org-export--prepare-file-contents
+		       file lines ind minlevel
+		       (or (gethash file file-prefix)
+			   (puthash file (incf current-prefix) file-prefix)))))
 		   (org-export-expand-include-keyword
 		    (cons (list file lines) included)
 		    (file-name-directory file))
 		   (buffer-string)))))))))))))
 
+(defun org-export--inclusion-absolute-lines (file location only-contents lines)
+  "Resolve absolute lines for an included file with file-link.
+
+FILE is string file-name of the file to include.  LOCATION is a
+string name within FILE to be included (located via
+`org-link-search').  If ONLY-CONTENTS is non-nil only the
+contents of the named element will be included, as determined
+Org-Element.  If LINES is non-nil only those lines are included.
+
+Return a string of lines to be included in the format expected by
+`org-export--prepare-file-contents'."
+  (with-temp-buffer
+    (insert-file-contents file)
+    (unless (eq major-mode 'org-mode)
+      (let ((org-inhibit-startup t)) (org-mode)))
+    (condition-case err
+	;; Enforce consistent search.
+	(let ((org-link-search-must-match-exact-headline t))
+	  (org-link-search location))
+      (error
+       (error (format "%s for %s::%s" (error-message-string err) file location))))
+    (let* ((element (org-element-at-point))
+	   (contents-begin
+	    (and only-contents (org-element-property :contents-begin element))))
+      (narrow-to-region
+       (or contents-begin (org-element-property :begin element))
+       (org-element-property (if contents-begin :contents-end :end) element))
+      (when (and only-contents
+		 (memq (org-element-type element) '(headline inlinetask)))
+	;; Skip planning line and property-drawer.  If a normal drawer
+	;; precedes a property-drawer both will be included.
+	;; Remaining property-drawers are removed as needed in
+	;; `org-export--prepare-file-contents'.
+	(goto-char (point-min))
+	(when (org-looking-at-p org-planning-line-re) (forward-line))
+	(when (looking-at org-property-drawer-re) (goto-char (match-end 0)))
+	(unless (bolp) (forward-line))
+	(narrow-to-region (point) (point-max))))
+    (when lines
+      (org-skip-whitespace)
+      (beginning-of-line)
+      (let* ((lines (split-string lines "-"))
+	     (lbeg (string-to-number (car lines)))
+	     (lend (string-to-number (cadr lines)))
+	     (beg (if (zerop lbeg) (point-min)
+		    (goto-char (point-min))
+		    (forward-line (1- lbeg))
+		    (point)))
+	     (end (if (zerop lend) (point-max)
+		    (goto-char beg)
+		    (forward-line (1-  lend))
+		    (point))))
+	(narrow-to-region beg end)))
+    (let ((end (point-max)))
+      (goto-char (point-min))
+      (widen)
+      (let ((start-line (line-number-at-pos)))
+	(format "%d-%d"
+		start-line
+		(save-excursion
+		  (+ start-line
+		     (let ((counter 0))
+		       (while (< (point) end) (incf counter) (forward-line))
+		       counter))))))))
+
 (defun org-export--prepare-file-contents (file &optional lines ind minlevel id)
   "Prepare the contents of FILE for inclusion and return them as a string.
 
@@ -3448,6 +3531,20 @@ with footnotes is included in a document."
     (skip-chars-backward " \r\t\n")
     (forward-line)
     (delete-region (point) (point-max))
+    ;; Remove property-drawers after drawers.
+    (when (or ind minlevel)
+      (unless (eq major-mode 'org-mode)
+    	(let ((org-inhibit-startup t)) (org-mode)))
+      (goto-char (point-min))
+      (when (looking-at org-drawer-regexp)
+	(goto-char (match-end 0))
+	(search-forward-regexp org-drawer-regexp)
+	(forward-line 1)
+	(beginning-of-line))
+      (when (looking-at org-property-drawer-re)
+	(delete-region (match-beginning 0) (match-end 0))
+	(beginning-of-line))
+      (delete-region (point) (save-excursion (and (org-skip-whitespace) (point)))))
     ;; If IND is set, preserve indentation of include keyword until
     ;; the first headline encountered.
     (when ind
diff --git a/testing/examples/include.org b/testing/examples/include.org
index 186facb..c04c942 100644
--- a/testing/examples/include.org
+++ b/testing/examples/include.org
@@ -8,3 +8,28 @@ Small Org file with an include keyword.
 
 * Heading
 body
+
+* Another heading
+:PROPERTIES:
+:CUSTOM_ID: ah
+:END:
+1
+2
+3
+
+* A headline with a table
+:PROPERTIES:
+:CUSTOM_ID: ht
+:END:
+#+CAPTION: a table
+#+NAME: tbl
+| 1 |
+
+* drawer-headline
+:LOGBOOK:
+drawer
+:END:
+:PROPERTIES:
+:CUSTOM_ID: dh
+:END:
+content
diff --git a/testing/lisp/test-ox.el b/testing/lisp/test-ox.el
index 4af3510..915a5a6 100644
--- a/testing/lisp/test-ox.el
+++ b/testing/lisp/test-ox.el
@@ -918,7 +918,64 @@ Footnotes[fn:1], [fn:test] and [fn:inline:anonymous footnote].
 		(org-export-expand-include-keyword)
 		(org-element-map (org-element-parse-buffer)
 		    'footnote-reference
-		  (lambda (ref) (org-element-property :label ref))))))))))))
+		  (lambda (ref) (org-element-property :label ref)))))))))))
+  ;; If only-contents is non-nil only include contents of element.
+  (should
+   (equal
+    "body\n"
+    (org-test-with-temp-text
+     (concat
+      (format "#+INCLUDE: \"%s/examples/include.org::*Heading\" " org-test-dir)
+      ":only-contents t")
+      (org-export-expand-include-keyword)
+      (buffer-string))))
+  ;; Headings can be included via CUSTOM_ID.
+  (should
+   (org-test-with-temp-text
+	(format "#+INCLUDE: \"%s/examples/include.org::#ah\"" org-test-dir)
+     (org-export-expand-include-keyword)
+     (goto-char (point-min))
+     (looking-at "* Another heading")))
+  ;; Named objects can be included.
+  (should
+   (equal
+    "| 1 |\n"
+    (org-test-with-temp-text
+	(format "#+INCLUDE: \"%s/examples/include.org::tbl\" :only-contents t" org-test-dir)
+      (org-export-expand-include-keyword)
+      (buffer-string))))
+  ;; Including non-existing elements should result in an error.
+  (should-error
+   (org-test-with-temp-text
+	(format "#+INCLUDE: \"%s/examples/include.org::*non-existing heading\"" org-test-dir)
+     (org-export-expand-include-keyword)))
+  ;; Lines work relatively to an included element.
+  (should
+   (equal
+    "2\n3\n"
+    (org-test-with-temp-text
+	(format "#+INCLUDE: \"%s/examples/include.org::#ah\" :only-contents t :lines \"2-3\"" org-test-dir)
+      (org-export-expand-include-keyword)
+      (buffer-string))))
+  ;; Properties should be dropped from headlines.
+  (should
+   (equal
+    (org-test-with-temp-text
+	(format "#+INCLUDE: \"%s/examples/include.org::#ht\" :only-contents t" org-test-dir)
+      (org-export-expand-include-keyword)
+      (buffer-string))
+    (org-test-with-temp-text
+	(format "#+INCLUDE: \"%s/examples/include.org::tbl\"" org-test-dir)
+      (org-export-expand-include-keyword)
+      (buffer-string))))
+  ;; Properties should be dropped, drawers should not be.
+  (should
+   (equal
+    ":LOGBOOK:\ndrawer\n:END:\ncontent\n"
+    (org-test-with-temp-text
+	(format "#+INCLUDE: \"%s/examples/include.org::#dh\" :only-contents t" org-test-dir)
+      (org-export-expand-include-keyword)
+      (buffer-string)))))
 
 (ert-deftest test-org-export/expand-macro ()
   "Test macro expansion in an Org buffer."
-- 
2.1.1