From: Kaushal Modi <kaushal.modi@gmail.com>
To: Nicolas Goaziou <mail@nicolasgoaziou.fr>
Cc: emacs-org list <emacs-orgmode@gnu.org>
Subject: Re: Allow #+SETUPFILE to point to an URL for the org file
Date: Fri, 09 Jun 2017 16:59:43 +0000 [thread overview]
Message-ID: <CAFyQvY00s0kGq2BymT6vd7we7sKC=KSM_=ogCN=HeaNT+EHOYw@mail.gmail.com> (raw)
In-Reply-To: <87o9ud77s2.fsf@nicolasgoaziou.fr>
[-- Attachment #1.1: Type: text/plain, Size: 2007 bytes --]
I have attached the updated patch, rebased to master.
On Sun, May 28, 2017 at 3:36 AM Nicolas Goaziou <mail@nicolasgoaziou.fr>
wrote:
> This is a good idea. We could even display a read-only buffer with the
> contents of the document, using new `org-file-contents'.
>
> Anyway, this can be done in a different patch.
>
Thanks. I was on the fence about whether I should open the cached file in a
read-only buffer.. but then the function was named `org-edit-special'.. the
"edit" part. But if you are fine, I can work on that in a different commit.
> > +indirectly through a file included using @samp{#+SETUPFILE:
> filename/URL}
>
> filename/URL > file name or URL
>
> > +@item #+SETUPFILE: file/URL
>
> file/URL > file name or URL
>
> > +reset the temporary file cache. Org also parses and loads the file/URL
>
> file/URL > document
>
> > +during normal exporting process. Org parses the contents of this
> > file/URL as
>
> file/URL document
>
> > +buffer for these lines and update the information. Also reset the org
> file
>
> org > Org
>
Done.
> > +(defvar ffap-url-regexp) ;Silence byte-compiler
>
> I think this can go at the beginning of the "org.el".
>
Done.
> > + ;; Reset the cache of files downloaded by `org-file-contents'.
> > + (clrhash org--file-cache))
>
> Note that we can still implement a non-interactive
> `org-reset-setupfile-cache' function and call it here instead. (clrhash
> org--file-cache) may be a bit low level at this point.
>
Done, except that I named the function org-reset-file-cache as the file
cache can be updated by org-file-contents function in general for any file.
> Otherwise, LGTM. Would you want to throw in some tests? You can use
> cl-letf to bind `url-retrieve-synchronously' to a function returning
> a buffer containing some dumb (valid or invalid) output.
>
Done! I, though used invalid URLs in the test. (If they become valid at
some point in distant future (I doubt), then we can update the test :))
--
Kaushal Modi
[-- Attachment #1.2: Type: text/html, Size: 3843 bytes --]
[-- Attachment #2: 0001-Allow-org-file-contents-to-fetch-file-contents-from-.patch --]
[-- Type: application/octet-stream, Size: 16867 bytes --]
From 711d71895cc3fec23ee4665299df665842fe9065 Mon Sep 17 00:00:00 2001
From: Kaushal Modi <kaushal.modi@gmail.com>
Date: Fri, 9 Jun 2017 12:56:11 -0400
Subject: [PATCH] Allow org-file-contents to fetch file contents from a URL
* lisp/org.el (org--file-cache): New internal variable to store
downloaded files' cache.
* lisp/org.el (org-reset-file-cache): New function to clear the above
file cache.
* lisp/org.el (org-mode-restart): Use org-reset-file-cache to clear
the file cache.
* lisp/org.el (org-file-url-p): New function to test if the input
argument is a URL.
* lisp/org.el (org-file-contents): Allow the FILE argument to be a
URL. If the URL contents are already cached, return the cache
contents, else download the file and return contents of that. The
file is automatically cached each time it is downloaded. Add a new
optional argument NOCACHE. If this is non-nil, the URL is always
downloaded afresh. Use `org--file-cache' and `org-file-url-p'.
* lisp/org.el (org-edit-special): Do not allow editing the "file" if a
URL is specified for the "#+SETUPFILE".
* lisp/ox.el (org-export--list-bound-variables)
(org-export--prepare-file-contents):
* lisp/org-macro.el (org-macro--collect-macros) : Adapt to the
possibility that the input to `org-file-contents' can be a URL too.
* doc/org.texi (Export settings, In-buffer settings)
(The very busy C-c C-c key): Mention that #+SETUPFILE keyword can now
take a URL as a value, and that C-c C-c on the #+SETUPFILE line will
clear the org file cache.
* testing/lisp/test-org.el (test-org/org-file-contents-url)
(test-org/org-file-contents-file): Add tests for org-file-contents.
---
doc/org.texi | 37 +++++++++++++-----------
etc/ORG-NEWS | 12 +++++++-
lisp/org-macro.el | 22 ++++++++------
lisp/org.el | 74 ++++++++++++++++++++++++++++++++++++++++--------
lisp/ox.el | 38 +++++++++++++++----------
testing/lisp/test-org.el | 41 +++++++++++++++++++++++++++
6 files changed, 172 insertions(+), 52 deletions(-)
diff --git a/doc/org.texi b/doc/org.texi
index dee46d1054..195e37ad9f 100644
--- a/doc/org.texi
+++ b/doc/org.texi
@@ -10406,14 +10406,14 @@ override options set at a more general level.
@cindex #+SETUPFILE
In-buffer settings may appear anywhere in the file, either directly or
-indirectly through a file included using @samp{#+SETUPFILE: filename} syntax.
-Option keyword sets tailored to a particular back-end can be inserted from
-the export dispatcher (@pxref{The export dispatcher}) using the @code{Insert
-template} command by pressing @key{#}. To insert keywords individually,
-a good way to make sure the keyword is correct is to type @code{#+} and then
-to use @kbd{M-@key{TAB}}@footnote{Many desktops intercept @kbd{M-TAB} to
-switch windows. Use @kbd{C-M-i} or @kbd{@key{ESC} @key{TAB}} instead.} for
-completion.
+indirectly through a file included using @samp{#+SETUPFILE: filename or URL}
+syntax. Option keyword sets tailored to a particular back-end can be
+inserted from the export dispatcher (@pxref{The export dispatcher}) using the
+@code{Insert template} command by pressing @key{#}. To insert keywords
+individually, a good way to make sure the keyword is correct is to type
+@code{#+} and then to use @kbd{M-@key{TAB}}@footnote{Many desktops intercept
+@kbd{M-TAB} to switch windows. Use @kbd{C-M-i} or @kbd{@key{ESC} @key{TAB}}
+instead.} for completion.
The export keywords available for every back-end, and their equivalent global
variables, include:
@@ -17174,13 +17174,16 @@ have a lower ASCII number than the lowest priority.
This line sets a default inheritance value for entries in the current
buffer, most useful for specifying the allowed values of a property.
@cindex #+SETUPFILE
-@item #+SETUPFILE: file
-The setup file is for additional in-buffer settings. Org loads this file and
-parses it for any settings in it only when Org opens the main file. @kbd{C-c
-C-c} on the settings line will also parse and load. Org also parses and
-loads the file during normal exporting process. Org parses the contents of
-this file as if it was included in the buffer. It can be another Org file.
-To visit the file, @kbd{C-c '} while the cursor is on the line with the file
+@item #+SETUPFILE: file or URL
+The setup file or a URL pointing to such file is for additional in-buffer
+settings. Org loads this file and parses it for any settings in it only when
+Org opens the main file. If URL is specified, the contents are downloaded
+and stored in a temporary cache. @kbd{C-c C-c} on the settings line will
+also parse and load. @kbd{C-c C-c} on the @code{#+SETUPFILE:} line will also
+reset the temporary file cache. Org also parses and loads the document
+during normal exporting process. Org parses the contents of this document as
+if it was included in the buffer. It can be another Org file. To visit the
+file (not a URL), @kbd{C-c '} while the cursor is on the line with the file
name.
@item #+STARTUP:
@cindex #+STARTUP
@@ -17422,7 +17425,9 @@ If any highlights shown in the buffer from the creation of a sparse tree, or
from clock display, remove such highlights.
@item
If the cursor is in one of the special @code{#+KEYWORD} lines, scan the
-buffer for these lines and update the information.
+buffer for these lines and update the information. Also reset the Org file
+cache used to temporary store the contents of URLs used as values for
+keywords like @code{#+SETUPFILE}.
@item
If the cursor is inside a table, realign the table. The table realigns even
if automatic table editor is turned off.
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index b984103e04..7a1c68ca2e 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -203,7 +203,7 @@ manual for details.
**** Add global macros through ~org-export-global-macros~
With this variable, one can define macros available for all documents.
**** New keyword ~#+EXPORT_FILE_NAME~
-Simiralry to ~:EXPORT_FILE_NAME:~ property, this keyword allows the
+Similarly to ~:EXPORT_FILE_NAME:~ property, this keyword allows the
user to specify the name of the output file upon exporting the
document. This also has an effect on publishing.
**** Horizontal rules are no longer ignored in LaTeX table math mode
@@ -240,6 +240,16 @@ which causes refile targets to be prefixed with the buffer’s
name. This is particularly useful when used in conjunction with
~uniquify.el~.
+*** ~org-file-contents~ now allows the FILE argument to be a URL.
+This allows ~#+SETUPFILE:~ to accept a URL instead of a local file
+path. The URL contents are auto-downloaded and saved to a temporary
+cache ~org--file-cache~. A new optional argument ~NOCACHE~ is added
+to ~org-file-contents~.
+
+*** ~org-mode-restart~ now resets the newly added ~org--file-cache~.
+Using ~C-c C-c~ on any keyword (like ~#+SETUPFILE~) will reset the
+that file cache.
+
** Removed functions
*** Org Timeline
diff --git a/lisp/org-macro.el b/lisp/org-macro.el
index 96461f9a95..ffbfa91add 100644
--- a/lisp/org-macro.el
+++ b/lisp/org-macro.el
@@ -55,7 +55,8 @@
(declare-function org-element-macro-parser "org-element" ())
(declare-function org-element-property "org-element" (property element))
(declare-function org-element-type "org-element" (element))
-(declare-function org-file-contents "org" (file &optional noerror))
+(declare-function org-file-url-p "org" (file))
+(declare-function org-file-contents "org" (file &optional noerror nocache))
(declare-function org-mode "org" ())
(declare-function vc-backend "vc-hooks" (f))
(declare-function vc-call "vc-hooks" (fun file &rest args) t)
@@ -101,16 +102,21 @@ Return an alist containing all macro templates found."
(if old-cell (setcdr old-cell template)
(push (cons name template) templates))))
;; Enter setup file.
- (let ((file (expand-file-name
- (org-unbracket-string "\"" "\"" val))))
- (unless (member file files)
+ (let* ((uri (org-unbracket-string "\"" "\"" (org-trim val)))
+ (uri-is-url (org-file-url-p uri))
+ (uri (if uri-is-url
+ uri
+ (expand-file-name uri))))
+ ;; Avoid circular dependencies.
+ (unless (member uri files)
(with-temp-buffer
- (setq default-directory
- (file-name-directory file))
+ (unless uri-is-url
+ (setq default-directory
+ (file-name-directory uri)))
(org-mode)
- (insert (org-file-contents file 'noerror))
+ (insert (org-file-contents uri 'noerror))
(setq templates
- (funcall collect-macros (cons file files)
+ (funcall collect-macros (cons uri files)
templates)))))))))))
templates))))
(funcall collect-macros nil nil)))
diff --git a/lisp/org.el b/lisp/org.el
index 2101ec7d13..191990db75 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -181,6 +181,8 @@ Stars are put in group 1 and the trimmed body in group 2.")
(declare-function org-export-get-environment "ox" (&optional backend subtreep ext-plist))
(declare-function org-latex-make-preamble "ox-latex" (info &optional template snippet?))
+(defvar ffap-url-regexp) ;Silence byte-compiler
+
(defsubst org-uniquify (list)
"Non-destructively remove duplicate elements from LIST."
(let ((res (copy-sequence list))) (delete-dups res)))
@@ -5280,17 +5282,62 @@ a string, summarizing TAGS, as a list of strings."
(setq current-group (list tag))))
(_ nil)))))
-(defun org-file-contents (file &optional noerror)
- "Return the contents of FILE, as a string."
- (if (and file (file-readable-p file))
+(defvar org--file-cache (make-hash-table :test #'equal)
+ "Hash table to store contents of files referenced via a URL.
+This is the cache of file URLs read using `org-file-contents'.")
+
+(defun org-reset-file-cache ()
+ "Reset the cache of files downloaded by `org-file-contents'."
+ (clrhash org--file-cache))
+
+(defun org-file-url-p (file)
+ "Non-nil if FILE is a URL."
+ (require 'ffap)
+ (string-match-p ffap-url-regexp file))
+
+(defun org-file-contents (file &optional noerror nocache)
+ "Return the contents of FILE, as a string.
+
+FILE can be a file name or URL.
+
+If FILE is a URL, download the contents. If the URL contents are
+already cached in the `org--file-cache' hash table, the download step
+is skipped.
+
+If NOERROR is non-nil, ignore the error when unable to read the FILE
+from file or URL.
+
+If NOCACHE is non-nil, do a fresh fetch of FILE even if cached version
+is available. This option applies only if FILE is a URL."
+ (let* ((is-url (org-file-url-p file))
+ (cache (and is-url
+ (not nocache)
+ (gethash file org--file-cache))))
+ (cond
+ (cache)
+ (is-url
+ (with-current-buffer (url-retrieve-synchronously file)
+ (goto-char (point-min))
+ ;; Move point to after the url-retrieve header.
+ (search-forward "\n\n" nil :move)
+ ;; Search for the success code only in the url-retrieve header.
+ (if (save-excursion (re-search-backward "HTTP.*\\s-+200\\s-OK" nil :noerror))
+ ;; Update the cache `org--file-cache' and return contents.
+ (puthash file
+ (buffer-substring-no-properties (point) (point-max))
+ org--file-cache)
+ (funcall (if noerror #'message #'user-error)
+ "Unable to fetch file from %S"
+ file))))
+ (t
(with-temp-buffer
- (insert-file-contents file)
- (buffer-string))
- (funcall (if noerror 'message 'error)
- "Cannot read file \"%s\"%s"
- file
- (let ((from (buffer-file-name (buffer-base-buffer))))
- (if from (concat " (referenced in file \"" from "\")") "")))))
+ (condition-case err
+ (progn
+ (insert-file-contents file)
+ (buffer-string))
+ (file-error
+ (funcall (if noerror #'message #'user-error)
+ (error-message-string err)))))))))
(defun org-extract-log-state-settings (x)
"Extract the log state setting from a TODO keyword string.
@@ -20687,7 +20734,9 @@ Otherwise, return a user error."
(format "[[%s]]"
(expand-file-name
(let ((value (org-element-property :value element)))
- (cond ((not (org-string-nw-p value))
+ (cond ((org-file-url-p value)
+ (user-error "The file is specified as a URL, cannot be edited"))
+ ((not (org-string-nw-p value))
(user-error "No file to edit"))
((string-match "\\`\"\\(.*?\\)\"" value)
(match-string 1 value))
@@ -20951,7 +21000,8 @@ Use `\\[org-edit-special]' to edit table.el tables"))
(funcall major-mode)
(hack-local-variables)
(when (and indent-status (not (bound-and-true-p org-indent-mode)))
- (org-indent-mode -1)))
+ (org-indent-mode -1))
+ (org-reset-file-cache))
(message "%s restarted" major-mode))
(defun org-kill-note-or-show-branches ()
diff --git a/lisp/ox.el b/lisp/ox.el
index 53d35bba8f..3b793a00f2 100644
--- a/lisp/ox.el
+++ b/lisp/ox.el
@@ -1499,17 +1499,20 @@ Assume buffer is in Org mode. Narrowing, if any, is ignored."
(cond
;; Options in `org-export-special-keywords'.
((equal key "SETUPFILE")
- (let ((file
- (expand-file-name
- (org-unbracket-string "\"" "\"" (org-trim val)))))
+ (let* ((uri (org-unbracket-string "\"" "\"" (org-trim val)))
+ (uri-is-url (org-file-url-p uri))
+ (uri (if uri-is-url
+ uri
+ (expand-file-name uri))))
;; Avoid circular dependencies.
- (unless (member file files)
+ (unless (member uri files)
(with-temp-buffer
- (setq default-directory
- (file-name-directory file))
- (insert (org-file-contents file 'noerror))
+ (unless uri-is-url
+ (setq default-directory
+ (file-name-directory uri)))
+ (insert (org-file-contents uri 'noerror))
(let ((org-inhibit-startup t)) (org-mode))
- (funcall get-options (cons file files))))))
+ (funcall get-options (cons uri files))))))
((equal key "OPTIONS")
(setq plist
(org-combine-plists
@@ -1647,17 +1650,22 @@ an alist where associations are (VARIABLE-NAME VALUE)."
"BIND")
(push (read (format "(%s)" val)) alist)
;; Enter setup file.
- (let ((file (expand-file-name
- (org-unbracket-string "\"" "\"" val))))
- (unless (member file files)
+ (let* ((uri (org-unbracket-string "\"" "\"" val))
+ (uri-is-url (org-file-url-p uri))
+ (uri (if uri-is-url
+ uri
+ (expand-file-name uri))))
+ ;; Avoid circular dependencies.
+ (unless (member uri files)
(with-temp-buffer
- (setq default-directory
- (file-name-directory file))
+ (unless uri-is-url
+ (setq default-directory
+ (file-name-directory uri)))
(let ((org-inhibit-startup t)) (org-mode))
- (insert (org-file-contents file 'noerror))
+ (insert (org-file-contents uri 'noerror))
(setq alist
(funcall collect-bind
- (cons file files)
+ (cons uri files)
alist))))))))))
alist)))))
;; Return value in appropriate order of appearance.
diff --git a/testing/lisp/test-org.el b/testing/lisp/test-org.el
index e55ee077b8..31bda86073 100644
--- a/testing/lisp/test-org.el
+++ b/testing/lisp/test-org.el
@@ -6498,6 +6498,47 @@ Paragraph<point>"
(org-show-set-visibility 'minimal)
(org-invisible-p2))))
+(ert-deftest test-org/org-file-contents-file ()
+ "Test `org-file-contents' with a file as input."
+ (should
+ (string= "#+BIND: variable value
+#+DESCRIPTION: l2
+#+LANGUAGE: en
+#+SELECT_TAGS: b
+#+TITLE: b
+#+PROPERTY: a 1
+" (org-file-contents (expand-file-name "setupfile3.org"
+ (concat org-test-dir "examples/")))))
+
+ (let ((invalid-file "this-file-must-not-exist"))
+ ;; Throw error when trying to access an invalid file.
+ (should-error
+ (org-file-contents invalid-file))
+ ;; Try to access an invalid file, but do not throw an error.
+ (should
+ (string-match-p "\\`Opening input file: No such file or directory"
+ (org-file-contents invalid-file :noerror)))))
+
+(ert-deftest test-org/org-file-contents-url ()
+ "Test `org-file-contents' with a URL as input."
+ (should
+ (string= "#+BIND: variable value
+#+DESCRIPTION: l2
+#+LANGUAGE: en
+#+SELECT_TAGS: b
+#+TITLE: b
+#+PROPERTY: a 1
+" (org-file-contents "http://orgmode.org/cgit.cgi/org-mode.git/plain/testing/examples/setupfile3.org")))
+
+ (let ((invalid-url "http://this-url-must-not-exist"))
+ ;; Throw error when trying to access an invalid URL.
+ (should-error
+ (org-file-contents invalid-url))
+ ;; Try to access an invalid URL, but do not throw an error.
+ (should
+ (string= (format "Unable to fetch file from \"%s\"" invalid-url)
+ (org-file-contents invalid-url :noerror)))))
+
(provide 'test-org)
--
2.13.0
next prev parent reply other threads:[~2017-06-09 17:00 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-12-03 17:23 Allow #+SETUPFILE to point to an URL for the org file Kaushal Modi
2016-12-08 11:51 ` Kaushal Modi
2016-12-08 14:22 ` John Kitchin
2016-12-08 14:31 ` Nicolas Goaziou
2016-12-08 14:44 ` Kaushal Modi
2016-12-08 21:48 ` Nicolas Goaziou
2016-12-08 22:07 ` Kaushal Modi
2016-12-08 22:40 ` Nicolas Goaziou
2017-03-13 17:37 ` Kaushal Modi
2017-03-30 7:43 ` Nicolas Goaziou
2017-05-23 19:07 ` Kaushal Modi
2017-05-25 10:13 ` Nicolas Goaziou
2017-05-25 10:18 ` Nicolas Goaziou
2017-05-25 11:43 ` Kaushal Modi
2017-05-25 15:15 ` Kaushal Modi
2017-05-26 7:47 ` Nicolas Goaziou
2017-05-26 20:24 ` Kaushal Modi
2017-05-28 7:35 ` Nicolas Goaziou
2017-05-28 10:04 ` Kaushal Modi
2017-06-09 16:59 ` Kaushal Modi [this message]
2017-06-12 19:32 ` Kaushal Modi
2017-06-13 12:43 ` Nicolas Goaziou
2017-06-13 15:45 ` Kaushal Modi
2017-06-13 21:32 ` Nicolas Goaziou
2017-06-13 21:42 ` Kaushal Modi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAFyQvY00s0kGq2BymT6vd7we7sKC=KSM_=ogCN=HeaNT+EHOYw@mail.gmail.com' \
--to=kaushal.modi@gmail.com \
--cc=emacs-orgmode@gnu.org \
--cc=mail@nicolasgoaziou.fr \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).