From mboxrd@z Thu Jan 1 00:00:00 1970 From: Rasmus Subject: Re: [patch, ox] #+INCLUDE resolves links Date: Wed, 24 Sep 2014 01:25:56 +0200 Message-ID: <87bnq5zzp7.fsf@gmx.us> References: <87k34x6bjd.fsf@gmx.us> <87lhpdurfh.fsf@gmx.us> <87bnq984hd.fsf@nicolasgoaziou.fr> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:35768) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XWZb4-0008CA-FD for emacs-orgmode@gnu.org; Tue, 23 Sep 2014 19:34:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XWZay-0007lK-Ly for emacs-orgmode@gnu.org; Tue, 23 Sep 2014 19:34:14 -0400 Received: from mout.gmx.net ([212.227.17.20]:61602) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XWZay-0007ki-Bj for emacs-orgmode@gnu.org; Tue, 23 Sep 2014 19:34:08 -0400 Received: from W530 ([108.61.76.10]) by mail.gmx.com (mrgmx103) with ESMTPSA (Nemesis) id 0M7Y9j-1YSoxj39z5-00xITL for ; Wed, 24 Sep 2014 01:26:01 +0200 In-Reply-To: <87bnq984hd.fsf@nicolasgoaziou.fr> (Nicolas Goaziou's message of "Sun, 21 Sep 2014 15:53:18 +0200") List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: emacs-orgmode@gnu.org --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, Okay, I hope I got a better patch here. I do =E2=80=94 for sure =E2=80=94 = present another dreadfully long email! Let's start. Nicolas Goaziou writes: >>> #+INCLUDE: file.org::#custom_id :noheadline :lines "3-" > > Is it `:only-contents' or `:no-headline'? Also ":kwd1 :kbd2 value" is > usually a shortcut for ":kwd1 nil :kbd2 value" (at least in export > attributes). Your example is thus confusing, you should include the > expected value. > > #+INCLUDE: "file.org::#custom_id" :only-contents t :lines "3-" Well it's only-contents now. Why? It's more precise in terms of the org-element terminology and it makes more sense when you include, say, a table. >> +elements.}. If the keyword @code{:only-contents} is used, only the >> contents >> +of the element in included. For headlines, drawers and properties > ^^ > >> +assumed to be in Org mode format and will be processed normally. >> File-links >> +will be interpret as well: > ^^^^^^^^^ Sorry about that. Fixed. >> ;;; ox.el --- Generic Export Engine for Org Mode >> - >> ;; Copyright (C) 2012-2014 Free Software Foundation, Inc. > > You can remove this chunk. As above. [I should somehow disable whitespace cleanup when in source repo= s]. >> + (only-contents >> + (and (string-match >> + ":\\(only-?contents?[[:space:]]*\\(?:'t\\|true\\|yes\\)?\\)" >> value) > > This should be ":only-contents t" or ":only-contents nil". > ":only-contents" alone can be tolerated as a shortcut for > ":only-contents nil", but that's all. Okay, I hope I got it now. It's a rather forgiving regexp in terms of mistakes. Is that OK? >> + (prog1 t >> + (setq value (replace-match "" nil nil value))))) > > Since `replace-match' cannot return nil here, you can remove I did it in another way in the new patch now since to allow for nil values. > (prog1 t ...) > > wrapper. If you insist on ONLY-CONTENTS being t, then No, it's changed. >> + (org-export--prepare-file-contents file location only-contents >> lines)))) > > Couldn't location, only-contents and lines be merged into a single > argument? At the moment, you are either short-circuiting or breaking > guard against circular inclusions (which relies on a combination of > file-name and lines). I try to do that now. So the arguments of `org-export--prepare-file-contents' are now unchanged compared to master. Note that lonely property drawers are now unconditionally discard if they are in the beginning of the buffer =E2=80=94 including just after an initial drawer. > IOW, each include keyword could be defined as a triplet of file name, > beginning and ending global positions. You could implement a helper > function to translate FILE LOCATION and ONLY-CONTENTS into this > triplet, > which would then be passed to `org-export--prepare-file-contents'. I just pass a string of line numbers like before. >> -(defun org-export--prepare-file-contents (file &optional lines ind minl= evel id) >> +(defun org-export--prepare-file-contents (file &optional location only-= contents lines ind minlevel id) >> "Prepare the contents of FILE for inclusion and return them as a stri= ng. >> >> +When optional argument LOCATION is a string the matching element >> +identified using `org-link-search' is returned. Note that >> +`org-link-search-must-match-exact-headline' is locally set to >> +non-nil. When ONLY-CONTENTS is non-nil only the contents of the >> +matched element in included. If LOCATION is a headline and >> +ONLY-CONTENTS is non-nil, drawers and property-drawers >> +immediately following the first headline are also removed. >> + >> When optional argument LINES is a string specifying a range of >> lines, include only those lines. >> >> @@ -3420,6 +3437,26 @@ This is useful to avoid conflicts when more than = one Org file >> with footnotes is included in a document." >> (with-temp-buffer >> (insert-file-contents file) >> + (org-mode) > > You cannot enforce `org-mode' as the current major mode since you can > include other file types. > >> + (when location >> + (condition-case err >> + ;; enforce consistency in search. >> + (let ((org-link-search-must-match-exact-headline t)) >> + (org-link-search location)) >> + ;; helpful error messages >> + (error (error (format "%s for %s::%s" >> + (error-message-string err) file location)))) >> + (narrow-to-region >> + (org-element-property >> + (if only-contents :contents-begin :begin) (org-element-at-point)) >> + (org-element-property (if only-contents :contents-end :end) >> (org-element-at-point))) >> + ;; get rid of drawers and properties >> + (when only-contents >> + (let ((element (org-element-at-point))) >> + (while (member (org-element-type element) '(drawer >> property-drawer)) >> + (delete-region (org-element-property :begin element) >> + (org-element-property :end element)) >> + (setq element (org-element-at-point)))))) > > This could be handled when building the triplet. However, please do > not > skip drawers (property drawers are fine), as you cannot tell what the > contents are. OK. >> (when lines >> (let* ((lines (split-string lines "-")) >> (lbeg (string-to-number (car lines))) >> @@ -3495,7 +3532,7 @@ with footnotes is included in a document." >> (org-element-normalize-string (buffer-string)))) >> >> (defun org-export-execute-babel-code () >> - "Execute every Babel code in the visible part of current buffer." >> + "ExecUte every Babel code in the visible part of current buffer." > > You can remove this chunk too. Sorry. Nicolas Goaziou writes: > Rasmus writes: > >>> You cannot enforce `org-mode' as the current major mode since you can >>> include other file types. >> >> But then I can't use org-element-at-point: >> >> (with-temp-buffer >> (text-mode) (insert "* test\nmy txt") (goto-char (point-min)) (org-ele= ment-at-point)) >> >> (with-temp-buffer >> (org-mode) (insert "* test\nmy txt") (goto-char (point-min)) (org-elem= ent-at-point)) > > `org-export--prepare-file-contents' is not called with the same number > of arguments when it is an Org file. You can activate `org-mode' if you > are in this situation (look for (when ind ...) and (when minlevel ...) > in the function). I hope it's OK now. In the new `org-export--inclusion-absolute-lines' I trigger org-mode when a location is given (e.g. file::#h3). This should be OK as only org files should be specified with this sort of link (I believe) and further it's currently only called in the end of `org-export-expand-include-keyword' when we know it's an org file. It doesn't make sense to apply this to source files, right? I will document the functions better before a final patch. Thanks, Rasmus -- Don't panic!!! --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0001-ox-Allow-file-links-with-INCLUDE-keyword.patch >From 29892a30446c58c42736b426d7385783d73579be Mon Sep 17 00:00:00 2001 From: Rasmus Date: Tue, 23 Sep 2014 22:16:45 +0200 Subject: [PATCH] ox: Allow file-links with #+INCLUDE-keyword * org.el (org-edit-special): Handle file-links for INCLUDE. * ox.el (org-export--prepare-file-contents): Handle links and add option no-heading. * ox.el (org-export-expand-include-keyword): Resolve headline links and add option :only-contents. * orgguide.texi (Include files) org.texi (Include files): Updated. --- doc/org.texi | 18 +++++++++ doc/orgguide.texi | 9 ++++- lisp/org.el | 8 ++-- lisp/ox.el | 115 ++++++++++++++++++++++++++++++++++++++++++++++-------- 4 files changed, 129 insertions(+), 21 deletions(-) diff --git a/doc/org.texi b/doc/org.texi index 7d98d51..c3dd1c1 100644 --- a/doc/org.texi +++ b/doc/org.texi @@ -10008,6 +10008,24 @@ to use the obvious defaults. #+INCLUDE: "~/.emacs" :lines "10-" @r{Include lines from 10 to EOF} @end example +Finally, you may use a file-link, see @ref{External links}, to extract an +object as matched by @code{org-link-search}@footnote{Note that +@code{org-link-search-must-match-exact-headline} is locally bound to non-nil. +Therefore, @code{org-link-search} only matches headlines and named +elements.}. If the keyword @code{:only-contents} is used, only the contents +of the included element. For headlines, drawers and properties +immediately following the headline will not be included when using +@code{:only-contents}. The @code{:lines} keyword is local to the +element in question. Some examples: + +@example +#+INCLUDE: "./paper.org::#theory" :only-contents + @r{Include the body of the heading with the custom id @code{theory}} +#+INCLUDE: "./paper.org::mytable" @r{Include tabel with name and caption.} +#+INCLUDE: "./paper.org::*conclusion" :lines 1-20 + @r{Include the first 20 lines of the headline named conclusion.} +@end example + @table @kbd @kindex C-c ' @item C-c ' diff --git a/doc/orgguide.texi b/doc/orgguide.texi index ca8e052..8f0098e 100644 --- a/doc/orgguide.texi +++ b/doc/orgguide.texi @@ -2264,8 +2264,13 @@ include your @file{.emacs} file, you could use: The optional second and third parameter are the markup (i.e., @samp{example} or @samp{src}), and, if the markup is @samp{src}, the language for formatting the contents. The markup is optional, if it is not given, the text will be -assumed to be in Org mode format and will be processed normally. @kbd{C-c '} -will visit the included file. +assumed to be in Org mode format and will be processed normally. File-links +will be interpreted as well: +@smallexample +#+INCLUDE: "./otherfile.org::#my_custom_id" :no-contents +@end smallexample +@noindent +@kbd{C-c '} will visit the included file. @node Embedded @LaTeX{}, , Include files, Markup @section Embedded @LaTeX{} diff --git a/lisp/org.el b/lisp/org.el index 4ffe1e8..86a1bf9 100755 --- a/lisp/org.el +++ b/lisp/org.el @@ -20520,9 +20520,11 @@ Otherwise, return a user error." session params)))))) (keyword (if (member (org-element-property :key element) '("INCLUDE" "SETUPFILE")) - (find-file-other-window - (org-remove-double-quotes - (car (org-split-string (org-element-property :value element))))) + (org-open-link-from-string + (format "[[%s]]" + (expand-file-name + (org-remove-double-quotes + (car (org-split-string (org-element-property :value element))))))) (user-error "No special environment to edit here"))) (table (if (eq (org-element-property :type element) 'table.el) diff --git a/lisp/ox.el b/lisp/ox.el index f01f951..122e62a 100644 --- a/lisp/ox.el +++ b/lisp/ox.el @@ -3321,13 +3321,24 @@ paths." ;; Extract arguments from keyword's value. (let* ((value (org-element-property :value element)) (ind (org-get-indentation)) + location (file (and (string-match "^\\(\".+?\"\\|\\S-+\\)\\(?:\\s-+\\|$\\)" value) - (prog1 (expand-file-name - (org-remove-double-quotes - (match-string 1 value)) - dir) - (setq value (replace-match "" nil nil value))))) + (let ((matched (save-match-data + (org-split-string + (org-remove-double-quotes (match-string 1 value)) "::")))) + (setq location (car-safe (cdr-safe matched))) + (prog1 (expand-file-name + (car matched) + dir) + (setq value (replace-match "" nil nil value)))))) + + (only-contents + (and (string-match + ":only-?contents?[[:space:]]*\"?\\(t\\|true\\|yes\\)?\"?" + value) + (prog1 (and (match-string 1 value) t) + (setq value (replace-match "" nil nil value))))) (lines (and (string-match ":lines +\"\\(\\(?:[0-9]+\\)?-\\(?:[0-9]+\\)?\\)\"" @@ -3370,34 +3381,92 @@ paths." (insert (let ((ind-str (make-string ind ? )) (arg-str (if (stringp src-args) - (format " %s" src-args) - "")) + (format " %s" src-args) + "")) (contents (org-escape-code-in-string - (org-export--prepare-file-contents file lines)))) + (org-export--prepare-file-contents file nil nil lines)))) (format "%s#+BEGIN_%s%s\n%s%s#+END_%s\n" ind-str block arg-str contents ind-str block)))) ((stringp block) (insert (let ((ind-str (make-string ind ? )) (contents - (org-export--prepare-file-contents file lines))) + (org-export--prepare-file-contents file lines))) (format "%s#+BEGIN_%s\n%s%s#+END_%s\n" ind-str block contents ind-str block)))) (t (insert (with-temp-buffer - (let ((org-inhibit-startup t)) (org-mode)) - (insert - (org-export--prepare-file-contents - file lines ind minlevel - (or (gethash file file-prefix) - (puthash file (incf current-prefix) file-prefix)))) + (let ((org-inhibit-startup t) + (lines (org-export--inclusion-absolute-lines + file location only-contents lines))) + (org-mode) + (insert + (org-export--prepare-file-contents + file lines ind minlevel + (or (gethash file file-prefix) + (puthash file (incf current-prefix) file-prefix))))) (org-export-expand-include-keyword (cons (list file lines) included) (file-name-directory file)) (buffer-string))))))))))))) +(defun org-export--inclusion-absolute-lines (file &optional location only-contents lines) + "Resolve absolute lines for an INCLUDEd file given arguments. + +Inputs are the FILE to be included, the LOCATION in FILE, whether +to ONLY-CONTENTS should be included and lines. + +Return absolute lines as a string." + (with-temp-buffer + (insert-file-contents file) + (when location + ;; locations are only defined for org files so + ;; OK to start org-mode. + (condition-case err + ;; enforce consistency in search. + (let ((org-link-search-must-match-exact-headline t)) + (org-link-search location)) + ;; helpful error messages + (error + (error (format "%s for %s::%s" (error-message-string err) file location)))) + (org-mode) + (narrow-to-region + (org-element-property + (if only-contents :contents-begin :begin) (org-element-at-point)) + (org-element-property (if only-contents :contents-end :end) (org-element-at-point)))) + (when only-contents + ;; skip drawers and property-drawers + ;; these are removed as needed in `org-export--prepare-file-contents' + ;; TODO: How to actually do this? Only line numbers are send to + ;; `org-export--prepare-file-contents'. split in two? + (goto-char (point-min)) + (org-skip-whitespace) + (beginning-of-line) + (let ((element (org-element-at-point))) + (while (memq (org-element-type element) '(drawer property-drawer)) + (goto-char (org-element-property :end element)) + (setq element (org-element-at-point))))) + (when lines + (org-skip-whitespace) + (beginning-of-line) + (let* ((lines (split-string lines "-")) + (lbeg (string-to-number (car lines))) + (lend (string-to-number (cadr lines))) + (beg (if (zerop lbeg) (point-min) + (goto-char (point-min)) + (forward-line (1- lbeg)) + (point))) + (end (if (zerop lend) (point-max) + (goto-char (point-min)) + (forward-line (1- lend)) + (point)))) + (narrow-to-region beg end))) + (apply (lambda (beg end) (format "%s-%s" beg end)) + ;; `line-number-at-pos' returns the narrowed line-number + (mapcar 'line-number-at-pos (prog1 (list (point-min) (point-max)) (widen)))))) + (defun org-export--prepare-file-contents (file &optional lines ind minlevel id) "Prepare the contents of FILE for inclusion and return them as a string. @@ -3419,7 +3488,8 @@ each footnote definition and reference if FILE is an Org file. This is useful to avoid conflicts when more than one Org file with footnotes is included in a document." (with-temp-buffer - (insert-file-contents file) + (switch-to-buffer-other-window (current-buffer)) + (insert-file-contents file) (when lines (let* ((lines (split-string lines "-")) (lbeg (string-to-number (car lines))) @@ -3449,6 +3519,19 @@ with footnotes is included in a document." (when ind (unless (eq major-mode 'org-mode) (let ((org-inhibit-startup t)) (org-mode))) + ;; make sure that there is not an immediate "lonely" + ;; property-drawer. Normal drawers are OK but property-drawers + ;; may follow normal drawers. + (goto-char (point-min)) + (let ((element (org-element-at-point))) + (while (memq (org-element-type element) '(drawer property-drawer)) + ;; entering here the first element is not a heading so it's + ;; safe to get rid of property-drawers. + (if (eq (org-element-type element) 'property-drawer) + (delete-region (org-element-property :begin element) + (org-element-property :end element)) + (goto-char (org-element-property :end element))) + (setq element (org-element-at-point)))) (goto-char (point-min)) (let ((ind-str (make-string ind ? ))) (while (not (or (eobp) (looking-at org-outline-regexp-bol))) -- 2.1.0 --=-=-=--