Re: [patch, ox] #+INCLUDE resolves links

[Rasmus <rasmus@gmx.us>]

[-- Attachment #1: Type: text/plain, Size: 2422 bytes --]

Hi,

Changes are one sentence in the documentations, casing, and I changed
the regexp so that :only-contents is valid (it's nil).

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Rasmus <rasmus@gmx.us> writes:
>
>> It's two extra words and maybe it's helpful to make the concept clear
>> to people unfamilar with org-element.  The statement "contents of the
>> requested element", while technically clear, is only barely
>> comprehensible for the uninvited.
>>
>> However, it's only tweo extra words, so I'm happy to remove them if
>> you have strong feeling about this.
>
> I have no strong feeling. Though, I admit the "etc" these two words
> imply annoys me. Anyway, you will be the final judge.

Is it better now?

>> I don't know what you mean with regular.  Anyway, my reasoning is that
>> this INCLUDE-command
>>
>>         #+INCLUDE: "/tmp/test0.org" :my-random-prop "foo"
>>
>> Will give me something like:
>>
>> \#+\textsc{begin}\(_{\text{my}}\)-random-prop
>> \section{test0}
>> \label{sec-2}
>> 1
>> \#+\textsc{end}\(_{\text{my}}\)-random-prop
>>
>> In casual testing similar stuff happened when I did not remove the
>> match.  It could have been it was a bit too casual and that I
>> misspelling the property.  I will test this properly later.
>
> OK. I wait for your report.

Block sucks up whatever remains, so the match must be removed:

(block (and (string-match "\\<\\(\\S-+\\)\\>" value)
			     (match-string 1 value)))


I want to discuss one more important potential issue before having the
patch applied.  Currently, location is ignored if the included part is
not an env (line 3381) and not a block (3392).  I'm not sure this is
right.  I could do one of the following:

   1. Nothing (current state)
   2. Throw an error if location and env or block are combined.
   3. Try to use location even if block is set.  Recall, though, that
      location is resolved using org-mode. 
   4. Let location be a general regexp if env or block is non-nil.
      But then we are breaking with the org file-link idea.
   5. Make location work for org files when env or block, otherwise
      throw an error.

WDYT?

Less important.  Should the <I "speedkey" (I don't know if that's the
right term) prompt for a location, or change the cursor position to
after the filename?

—Rasmus

-- 
Governments should be afraid of their people

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ox-Allow-file-links-with-INCLUDE-keyword.patch --]
[-- Type: text/x-diff, Size: 12757 bytes --]

From 44e20962a3c16e1f79a2d5bbc8420f00f93db637 Mon Sep 17 00:00:00 2001
From: Rasmus <rasmus@gmx.us>
Date: Sun, 28 Sep 2014 21:05:17 +0200
Subject: [PATCH] ox: Allow file-links with #+INCLUDE-keyword

* org.el (org-edit-special): Handle file-links for INCLUDE.
* ox.el (org-export--prepare-file-contents): Handle links and
add option no-heading.
* ox.el (org-export-expand-include-keyword): Resolve headline
links and add option :only-contents.
* orgguide.texi (Include files)
org.texi (Include files): Updated.
* testing/examples/include.org: New examples.
* test-ox.el (test-org-export/expand-include): New tests.
---
 doc/org.texi                 |  17 +++++++
 doc/orgguide.texi            |   9 +++-
 lisp/org.el                  |   9 ++--
 lisp/ox.el                   | 117 +++++++++++++++++++++++++++++++++++++++----
 testing/examples/include.org |  25 +++++++++
 testing/lisp/test-ox.el      |  59 +++++++++++++++++++++-
 6 files changed, 220 insertions(+), 16 deletions(-)

diff --git a/doc/org.texi b/doc/org.texi
index 7d98d51..296e289 100644
--- a/doc/org.texi
+++ b/doc/org.texi
@@ -10008,6 +10008,23 @@ to use the obvious defaults.
 #+INCLUDE: "~/.emacs" :lines "10-"    @r{Include lines from 10 to EOF}
 @end example
 
+Finally, you may use a file-link, see @ref{search option in file links}, to
+extract an object as matched by @code{org-link-search}@footnote{Note that
+@code{org-link-search-must-match-exact-headline} is locally bound to non-nil.
+Therefore, @code{org-link-search} only matches headlines and named
+elements.}.  If the @code{:only-contents} property is non-nil, only the
+contents of the requested element will be included, omitting properties
+drawer and planning-line if present.  The @code{:lines} keyword
+operates locally with respect to the requested element.  Some examples:
+
+@example
+#+INCLUDE: "./paper.org::#theory" :only-contents t
+   @r{Include the body of the heading with the custom id @code{theory}}
+#+INCLUDE: "./paper.org::mytable"  @r{Include named element.}
+#+INCLUDE: "./paper.org::*conclusion" :lines 1-20
+   @r{Include the first 20 lines of the headline named conclusion.}
+@end example
+
 @table @kbd
 @kindex C-c '
 @item C-c '
diff --git a/doc/orgguide.texi b/doc/orgguide.texi
index ca8e052..4feeaca 100644
--- a/doc/orgguide.texi
+++ b/doc/orgguide.texi
@@ -2264,8 +2264,13 @@ include your @file{.emacs} file, you could use:
 The optional second and third parameter are the markup (i.e., @samp{example}
 or @samp{src}), and, if the markup is @samp{src}, the language for formatting
 the contents.  The markup is optional, if it is not given, the text will be
-assumed to be in Org mode format and will be processed normally. @kbd{C-c '}
-will visit the included file.
+assumed to be in Org mode format and will be processed normally.  File-links
+will be interpreted as well:
+@smallexample
+#+INCLUDE: "./otherfile.org::#my_custom_id" :only-contents t
+@end smallexample
+@noindent
+@kbd{C-c '} will visit the included file.
 
 @node Embedded @LaTeX{},  , Include files, Markup
 @section Embedded @LaTeX{}
diff --git a/lisp/org.el b/lisp/org.el
index 9815eb4..0910296 100755
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -20524,9 +20524,12 @@ Otherwise, return a user error."
 		       session params))))))
       (keyword
        (if (member (org-element-property :key element) '("INCLUDE" "SETUPFILE"))
-           (find-file-other-window
-            (org-remove-double-quotes
-             (car (org-split-string (org-element-property :value element)))))
+           (org-open-link-from-string
+	    (format "[[%s]]"
+		    (expand-file-name
+		     (org-remove-double-quotes
+		      (car (org-split-string
+			    (org-element-property :value element)))))))
          (user-error "No special environment to edit here")))
       (table
        (if (eq (org-element-property :type element) 'table.el)
diff --git a/lisp/ox.el b/lisp/ox.el
index 90c623e..dc36c70 100644
--- a/lisp/ox.el
+++ b/lisp/ox.el
@@ -3325,13 +3325,25 @@ paths."
 	  ;; Extract arguments from keyword's value.
 	  (let* ((value (org-element-property :value element))
 		 (ind (org-get-indentation))
+		 location
 		 (file (and (string-match
 			     "^\\(\".+?\"\\|\\S-+\\)\\(?:\\s-+\\|$\\)" value)
-			    (prog1 (expand-file-name
-				    (org-remove-double-quotes
-				     (match-string 1 value))
-				    dir)
+			    (prog1
+				(save-match-data
+				  (let ((matched (match-string 1 value)))
+				    (when (string-match "\\(::\\(.*?\\)\\)\"?\\'" matched)
+				      (setq location (match-string 2 matched))
+				      (setq matched
+					    (replace-match "" nil nil  matched 1)))
+				    (expand-file-name
+				     (org-remove-double-quotes
+				      matched)
+				     dir)))
 			      (setq value (replace-match "" nil nil value)))))
+		 (only-contents
+		  (and (string-match ":only-contents *?\\([^: \r\t\n]\\S-*\\)?" value)
+		       (prog1 (org-not-nil (match-string 1 value))
+			 (setq value (replace-match "" nil nil value)))))
 		 (lines
 		  (and (string-match
 			":lines +\"\\(\\(?:[0-9]+\\)?-\\(?:[0-9]+\\)?\\)\""
@@ -3391,17 +3403,88 @@ paths."
 	       (t
 		(insert
 		 (with-temp-buffer
-		   (let ((org-inhibit-startup t)) (org-mode))
-		   (insert
-		    (org-export--prepare-file-contents
-		     file lines ind minlevel
-		     (or (gethash file file-prefix)
-			 (puthash file (incf current-prefix) file-prefix))))
+		   (let ((org-inhibit-startup t)
+			 (lines
+			  (if location
+			      (org-export--inclusion-absolute-lines
+			       file  location only-contents lines)
+			    lines)))
+		     (org-mode)
+		     (insert
+		      (org-export--prepare-file-contents
+		       file lines ind minlevel
+		       (or (gethash file file-prefix)
+			   (puthash file (incf current-prefix) file-prefix)))))
 		   (org-export-expand-include-keyword
 		    (cons (list file lines) included)
 		    (file-name-directory file))
 		   (buffer-string)))))))))))))
 
+(defun org-export--inclusion-absolute-lines (file location only-contents lines)
+  "Resolve absolute lines for an included file with file-link.
+
+FILE is string file-name of the file to include.  LOCATION is a
+string name within FILE to be included (located via
+`org-link-search').  If ONLY-CONTENTS is non-nil only the
+contents of the named element will be included, as determined
+Org-Element.  If LINES is non-nil only those lines are included.
+
+Return a string of lines to be included in the format expected by
+`org-export--prepare-file-contents'."
+  (with-temp-buffer
+    (insert-file-contents file)
+    (unless (eq major-mode 'org-mode)
+      (let ((org-inhibit-startup t)) (org-mode)))
+    (condition-case err
+	;; Enforce consistent search.
+	(let ((org-link-search-must-match-exact-headline t))
+	  (org-link-search location))
+      (error
+       (error (format "%s for %s::%s" (error-message-string err) file location))))
+    (let* ((element (org-element-at-point))
+	   (contents-begin
+	    (and only-contents (org-element-property :contents-begin element))))
+      (narrow-to-region
+       (or contents-begin (org-element-property :begin element))
+       (org-element-property (if contents-begin :contents-end :end) element))
+      (when (and only-contents
+		 (memq (org-element-type element) '(headline inlinetask)))
+	;; Skip planning line and property-drawer.  If a normal drawer
+	;; precedes a property-drawer both will be included.
+	;; Remaining property-drawers are removed as needed in
+	;; `org-export--prepare-file-contents'
+	(goto-char (point-min))
+	(when (org-looking-at-p org-planning-line-re) (forward-line))
+	(when (looking-at org-property-drawer-re) (goto-char (match-end 0)))
+	(unless (bolp) (forward-line))
+	(narrow-to-region (point) (point-max))))
+    (when lines
+      (org-skip-whitespace)
+      (beginning-of-line)
+      (let* ((lines (split-string lines "-"))
+	     (lbeg (string-to-number (car lines)))
+	     (lend (string-to-number (cadr lines)))
+	     (beg (if (zerop lbeg) (point-min)
+		    (goto-char (point-min))
+		    (forward-line (1- lbeg))
+		    (point)))
+	     (end (if (zerop lend) (point-max)
+		    (goto-char beg)
+		    (forward-line (1-  lend))
+		    (point))))
+	(narrow-to-region beg end)))
+    (let ((end (point-max)))
+      (goto-char (point-min))
+      (widen)
+      (let ((start-line (line-number-at-pos)))
+	(format "%d-%d"
+		start-line
+		(save-excursion
+		  (+ start-line
+		     (let ((counter 0))
+		       (while (< (point) end) (incf counter) (forward-line))
+		       counter))))))))
+
 (defun org-export--prepare-file-contents (file &optional lines ind minlevel id)
   "Prepare the contents of FILE for inclusion and return them as a string.
 
@@ -3448,6 +3531,20 @@ with footnotes is included in a document."
     (skip-chars-backward " \r\t\n")
     (forward-line)
     (delete-region (point) (point-max))
+    ;; Remove property-drawers after drawers.
+    (when (or ind minlevel)
+      (unless (eq major-mode 'org-mode)
+    	(let ((org-inhibit-startup t)) (org-mode)))
+      (goto-char (point-min))
+      (when (looking-at org-drawer-regexp)
+	(goto-char (match-end 0))
+	(search-forward-regexp org-drawer-regexp)
+	(forward-line 1)
+	(beginning-of-line))
+      (when (looking-at org-property-drawer-re)
+	(delete-region (match-beginning 0) (match-end 0))
+	(beginning-of-line))
+      (delete-region (point) (save-excursion (and (org-skip-whitespace) (point)))))
     ;; If IND is set, preserve indentation of include keyword until
     ;; the first headline encountered.
     (when ind
diff --git a/testing/examples/include.org b/testing/examples/include.org
index 186facb..c04c942 100644
--- a/testing/examples/include.org
+++ b/testing/examples/include.org
@@ -8,3 +8,28 @@ Small Org file with an include keyword.
 
 * Heading
 body
+
+* Another heading
+:PROPERTIES:
+:CUSTOM_ID: ah
+:END:
+1
+2
+3
+
+* A headline with a table
+:PROPERTIES:
+:CUSTOM_ID: ht
+:END:
+#+CAPTION: a table
+#+NAME: tbl
+| 1 |
+
+* drawer-headline
+:LOGBOOK:
+drawer
+:END:
+:PROPERTIES:
+:CUSTOM_ID: dh
+:END:
+content
diff --git a/testing/lisp/test-ox.el b/testing/lisp/test-ox.el
index 4af3510..0b7df02 100644
--- a/testing/lisp/test-ox.el
+++ b/testing/lisp/test-ox.el
@@ -918,7 +918,64 @@ Footnotes[fn:1], [fn:test] and [fn:inline:anonymous footnote].
 		(org-export-expand-include-keyword)
 		(org-element-map (org-element-parse-buffer)
 		    'footnote-reference
-		  (lambda (ref) (org-element-property :label ref))))))))))))
+		  (lambda (ref) (org-element-property :label ref)))))))))))
+  ;; If only-contents is non-nil only include contents of element
+  (should
+   (equal
+    "body\n"
+    (org-test-with-temp-text
+     (concat
+      (format "#+INCLUDE: \"%s/examples/include.org::*Heading\" " org-test-dir)
+      ":only-contents t")
+      (org-export-expand-include-keyword)
+      (buffer-string))))
+  ;; Headings can be included via CUSTOM_ID
+  (should
+   (org-test-with-temp-text
+	(format "#+INCLUDE: \"%s/examples/include.org::#ah\"" org-test-dir)
+     (org-export-expand-include-keyword)
+     (goto-char (point-min))
+     (looking-at "* Another heading")))
+  ;; Named objects can be included
+  (should
+   (equal
+    "| 1 |\n"
+    (org-test-with-temp-text
+	(format "#+INCLUDE: \"%s/examples/include.org::tbl\" :only-contents t" org-test-dir)
+      (org-export-expand-include-keyword)
+      (buffer-string))))
+  ;; Including non-existing elements should result in an error
+  (should-error
+   (org-test-with-temp-text
+	(format "#+INCLUDE: \"%s/examples/include.org::*non-existing heading\"" org-test-dir)
+     (org-export-expand-include-keyword)))
+  ;; Lines work relatively to an included element
+  (should
+   (equal
+    "2\n3\n"
+    (org-test-with-temp-text
+	(format "#+INCLUDE: \"%s/examples/include.org::#ah\" :only-contents t :lines \"2-3\"" org-test-dir)
+      (org-export-expand-include-keyword)
+      (buffer-string))))
+  ;; Properties should be dropped from headlines
+  (should
+   (equal
+    (org-test-with-temp-text
+	(format "#+INCLUDE: \"%s/examples/include.org::#ht\" :only-contents t" org-test-dir)
+      (org-export-expand-include-keyword)
+      (buffer-string))
+    (org-test-with-temp-text
+	(format "#+INCLUDE: \"%s/examples/include.org::tbl\"" org-test-dir)
+      (org-export-expand-include-keyword)
+      (buffer-string))))
+  ;; Properties should be dropped, drawers should not be
+  (should
+   (equal
+    ":LOGBOOK:\ndrawer\n:END:\ncontent\n"
+    (org-test-with-temp-text
+	(format "#+INCLUDE: \"%s/examples/include.org::#dh\" :only-contents t" org-test-dir)
+      (org-export-expand-include-keyword)
+      (buffer-string)))))
 
 (ert-deftest test-org-export/expand-macro ()
   "Test macro expansion in an Org buffer."
-- 
2.1.1