From: Kaushal Modi <kaushal.modi@gmail.com>
To: Nicolas Goaziou <mail@nicolasgoaziou.fr>
Cc: emacs-org list <emacs-orgmode@gnu.org>
Subject: Re: Allow #+SETUPFILE to point to an URL for the org file
Date: Fri, 26 May 2017 20:24:00 +0000 [thread overview]
Message-ID: <CAFyQvY3-rOGXQmmAa-L++Z8tYiP3xfU2HeZrTn8-ox57rdALng@mail.gmail.com> (raw)
In-Reply-To: <87o9ug83fb.fsf@nicolasgoaziou.fr>
[-- Attachment #1.1: Type: text/plain, Size: 1653 bytes --]
Thanks for another round of review.
I have attached a patch, rebased to master and with all suggestions
implemented.
There are some additional changes in the patch this time:
- Prevent org-edit-special from attempting to open the "file" for editing
if the value is a URL.
- Silence byte-compiler for ffap-url-regexp. We do a require, but the
byte-compiler still complains of a free variable.
- Update org.texi at few places (first time editing a texi)
On Fri, May 26, 2017 at 3:47 AM Nicolas Goaziou <mail@nicolasgoaziou.fr>
wrote:
> I understand the use case for `org-file-clear-cache'. My suggestion is
> to remove that need by clearing cache automatically. Of course, the
> drawback is the cache is cleared more often than necessary. Users could
> get very surprising results if they forget to call this function. So it
> might be a good idea to call it from time to time, on user's behalf.
>
I liked your C-c C-c idea below! :)
> `org-file-contents' is a developer-facing function. I don't really see
> a good place in the manual for it. That's the problem of
> `org-file-clear-cache', which is really an implementation detail users
> shouldn't care about.
>
Understood.
> Here's another idea: call it from `org-mode-restart'. So cache is reset
> whenever `C-c C-c' is pressed on a keyword. So we don't need to document
> the function anymore. Instead, we could drop a note about the cache in
> (info "(org) The very busy C-c C-c key").
>
Done!
> OK. Then the following at least doesn't have the overhead of creating
> a string:
>
Integrated. Thanks!
Also, mind the full stop at the end of the comments.
>
Will do.
--
Kaushal Modi
[-- Attachment #1.2: Type: text/html, Size: 2809 bytes --]
[-- Attachment #2: 0001-Allow-org-file-contents-to-fetch-file-contents-from-.patch --]
[-- Type: application/octet-stream, Size: 14334 bytes --]
From 80bb0f81fc54e2bdd7f00c03f0d0cae789204877 Mon Sep 17 00:00:00 2001
From: Kaushal Modi <kaushal.modi@gmail.com>
Date: Thu, 25 May 2017 11:08:20 -0400
Subject: [PATCH] Allow org-file-contents to fetch file contents from a URL
* lisp/org.el (org--file-cache): New internal variable to store
downloaded files' cache.
* lisp/org.el (org-mode-restart): Now also clears the above file
cache.
* lisp/org.el (org-file-url-p): New function to test if the input
argument is a URL.
* lisp/org.el (org-file-contents): Allow the FILE argument to be a
URL. If the URL contents are already cached, return the cache
contents, else download the file and return contents of that. The
file is automatically cached each time it is downloaded. Add a new
optional argument NOCACHE. If this is non-nil, the URL is always
downloaded afresh. Use `org--file-cache' and `org-file-url-p'.
* lisp/org.el (org-edit-special): Do not allow editing the "file" if a
URL is specified for the "#+SETUPFILE".
* lisp/ox.el (org-export--list-bound-variables)
(org-export--prepare-file-contents):
* lisp/org-macro.el (org-macro--collect-macros) : Adapt to the
possibility that the input to `org-file-contents' can be a URL too.
* doc/org.texi (Export settings, In-buffer settings)
(The very busy C-c C-c key): Mention that #+SETUPFILE keyword can now
take a URL as a value, and that C-c C-c on the #+SETUPFILE line will
clear the org file cache.
---
doc/org.texi | 37 ++++++++++++++++-------------
etc/ORG-NEWS | 12 +++++++++-
lisp/org-macro.el | 22 ++++++++++-------
lisp/org.el | 70 +++++++++++++++++++++++++++++++++++++++++++++----------
lisp/ox.el | 38 ++++++++++++++++++------------
5 files changed, 127 insertions(+), 52 deletions(-)
diff --git a/doc/org.texi b/doc/org.texi
index 142fa9627..3ebc01c0c 100644
--- a/doc/org.texi
+++ b/doc/org.texi
@@ -10406,14 +10406,14 @@ override options set at a more general level.
@cindex #+SETUPFILE
In-buffer settings may appear anywhere in the file, either directly or
-indirectly through a file included using @samp{#+SETUPFILE: filename} syntax.
-Option keyword sets tailored to a particular back-end can be inserted from
-the export dispatcher (@pxref{The export dispatcher}) using the @code{Insert
-template} command by pressing @key{#}. To insert keywords individually,
-a good way to make sure the keyword is correct is to type @code{#+} and then
-to use @kbd{M-@key{TAB}}@footnote{Many desktops intercept @kbd{M-TAB} to
-switch windows. Use @kbd{C-M-i} or @kbd{@key{ESC} @key{TAB}} instead.} for
-completion.
+indirectly through a file included using @samp{#+SETUPFILE: filename/URL}
+syntax. Option keyword sets tailored to a particular back-end can be
+inserted from the export dispatcher (@pxref{The export dispatcher}) using the
+@code{Insert template} command by pressing @key{#}. To insert keywords
+individually, a good way to make sure the keyword is correct is to type
+@code{#+} and then to use @kbd{M-@key{TAB}}@footnote{Many desktops intercept
+@kbd{M-TAB} to switch windows. Use @kbd{C-M-i} or @kbd{@key{ESC} @key{TAB}}
+instead.} for completion.
The export keywords available for every back-end, and their equivalent global
variables, include:
@@ -17147,13 +17147,16 @@ have a lower ASCII number than the lowest priority.
This line sets a default inheritance value for entries in the current
buffer, most useful for specifying the allowed values of a property.
@cindex #+SETUPFILE
-@item #+SETUPFILE: file
-The setup file is for additional in-buffer settings. Org loads this file and
-parses it for any settings in it only when Org opens the main file. @kbd{C-c
-C-c} on the settings line will also parse and load. Org also parses and
-loads the file during normal exporting process. Org parses the contents of
-this file as if it was included in the buffer. It can be another Org file.
-To visit the file, @kbd{C-c '} while the cursor is on the line with the file
+@item #+SETUPFILE: file/URL
+The setup file or a URL pointing to such file is for additional in-buffer
+settings. Org loads this file and parses it for any settings in it only when
+Org opens the main file. If URL is specified, the contents are downloaded
+and stored in a temporary cache. @kbd{C-c C-c} on the settings line will
+also parse and load. @kbd{C-c C-c} on the @code{#+SETUPFILE:} line will also
+reset the temporary file cache. Org also parses and loads the file/URL
+during normal exporting process. Org parses the contents of this file/URL as
+if it was included in the buffer. It can be another Org file. To visit the
+file (not a URL), @kbd{C-c '} while the cursor is on the line with the file
name.
@item #+STARTUP:
@cindex #+STARTUP
@@ -17395,7 +17398,9 @@ If any highlights shown in the buffer from the creation of a sparse tree, or
from clock display, remove such highlights.
@item
If the cursor is in one of the special @code{#+KEYWORD} lines, scan the
-buffer for these lines and update the information.
+buffer for these lines and update the information. Also reset the org file
+cache used to temporary store the contents of URLs used as values for
+keywords like @code{#+SETUPFILE}.
@item
If the cursor is inside a table, realign the table. The table realigns even
if automatic table editor is turned off.
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 044f167ce..a84eb98c1 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -203,7 +203,7 @@ manual for details.
**** Add global macros through ~org-export-global-macros~
With this variable, one can define macros available for all documents.
**** New keyword ~#+EXPORT_FILE_NAME~
-Simiralry to ~:EXPORT_FILE_NAME:~ property, this keyword allows the
+Similarly to ~:EXPORT_FILE_NAME:~ property, this keyword allows the
user to specify the name of the output file upon exporting the
document. This also has an effect on publishing.
**** Horizontal rules are no longer ignored in LaTeX table math mode
@@ -234,6 +234,16 @@ which causes refile targets to be prefixed with the buffer’s
name. This is particularly useful when used in conjunction with
~uniquify.el~.
+*** ~org-file-contents~ now allows the FILE argument to be a URL.
+This allows ~#+SETUPFILE:~ to accept a URL instead of a local file
+path. The URL contents are auto-downloaded and saved to a temporary
+cache ~org--file-cache~. A new optional argument ~NOCACHE~ is added
+to ~org-file-contents~.
+
+*** ~org-mode-restart~ now resets the newly added ~org--file-cache~.
+Using ~C-c C-c~ on any keyword (like ~#+SETUPFILE~) will reset the
+that file cache.
+
** Removed functions
*** Org Timeline
diff --git a/lisp/org-macro.el b/lisp/org-macro.el
index f5ddb92e4..9f6e0ebaf 100644
--- a/lisp/org-macro.el
+++ b/lisp/org-macro.el
@@ -59,7 +59,8 @@
(&optional granularity visible-only))
(declare-function org-element-property "org-element" (property element))
(declare-function org-element-type "org-element" (element))
-(declare-function org-file-contents "org" (file &optional noerror))
+(declare-function org-file-url-p "org" (file))
+(declare-function org-file-contents "org" (file &optional noerror nocache))
(declare-function org-mode "org" ())
(declare-function vc-backend "vc-hooks" (f))
(declare-function vc-call "vc-hooks" (fun file &rest args) t)
@@ -105,16 +106,21 @@ Return an alist containing all macro templates found."
(if old-cell (setcdr old-cell template)
(push (cons name template) templates))))
;; Enter setup file.
- (let ((file (expand-file-name
- (org-unbracket-string "\"" "\"" val))))
- (unless (member file files)
+ (let* ((uri (org-unbracket-string "\"" "\"" (org-trim val)))
+ (uri-is-url (org-file-url-p uri))
+ (uri (if uri-is-url
+ uri
+ (expand-file-name uri))))
+ ;; Avoid circular dependencies.
+ (unless (member uri files)
(with-temp-buffer
- (setq default-directory
- (file-name-directory file))
+ (unless uri-is-url
+ (setq default-directory
+ (file-name-directory uri)))
(org-mode)
- (insert (org-file-contents file 'noerror))
+ (insert (org-file-contents uri 'noerror))
(setq templates
- (funcall collect-macros (cons file files)
+ (funcall collect-macros (cons uri files)
templates)))))))))))
templates))))
(funcall collect-macros nil nil)))
diff --git a/lisp/org.el b/lisp/org.el
index 102a9b265..171b10cd7 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -5273,17 +5273,59 @@ a string, summarizing TAGS, as a list of strings."
(setq current-group (list tag))))
(_ nil)))))
-(defun org-file-contents (file &optional noerror)
- "Return the contents of FILE, as a string."
- (if (and file (file-readable-p file))
+(defvar org--file-cache (make-hash-table :test #'equal)
+ "Hash table to store contents of files referenced via a URL.
+This is the cache of file URLs read using `org-file-contents'.")
+
+(defvar ffap-url-regexp) ;Silence byte-compiler
+(defun org-file-url-p (file)
+ "Non-nil if FILE is a URL."
+ (require 'ffap)
+ (string-match-p ffap-url-regexp file))
+
+(defun org-file-contents (file &optional noerror nocache)
+ "Return the contents of FILE, as a string.
+
+FILE can be a file name or URL.
+
+If FILE is a URL, download the contents. If the URL contents are
+already cached in the `org--file-cache' hash table, the download step
+is skipped.
+
+If NOERROR is non-nil, ignore the error when unable to read the FILE
+from file or URL.
+
+If NOCACHE is non-nil, do a fresh fetch of FILE even if cached version
+is available. This option applies only if FILE is a URL."
+ (let* ((is-url (org-file-url-p file))
+ (cache (and is-url
+ (not nocache)
+ (gethash file org--file-cache))))
+ (cond
+ (cache)
+ (is-url
+ (with-current-buffer (url-retrieve-synchronously file)
+ (goto-char (point-min))
+ ;; Move point to after the url-retrieve header.
+ (search-forward "\n\n" nil :move)
+ ;; Search for the success code only in the url-retrieve header.
+ (if (save-excursion (re-search-backward "HTTP.*\\s-+200\\s-OK" nil :noerror))
+ ;; Update the cache `org--file-cache' and return contents.
+ (puthash file
+ (buffer-substring-no-properties (point) (point-max))
+ org--file-cache)
+ (funcall (if noerror #'message #'user-error)
+ "Unable to fetch file from %S"
+ file))))
+ (t
(with-temp-buffer
- (insert-file-contents file)
- (buffer-string))
- (funcall (if noerror 'message 'error)
- "Cannot read file \"%s\"%s"
- file
- (let ((from (buffer-file-name (buffer-base-buffer))))
- (if from (concat " (referenced in file \"" from "\")") "")))))
+ (condition-case err
+ (progn
+ (insert-file-contents file)
+ (buffer-string))
+ (file-error
+ (funcall (if noerror #'message #'user-error)
+ (error-message-string err)))))))))
(defun org-extract-log-state-settings (x)
"Extract the log state setting from a TODO keyword string.
@@ -20685,7 +20727,9 @@ Otherwise, return a user error."
(format "[[%s]]"
(expand-file-name
(let ((value (org-element-property :value element)))
- (cond ((not (org-string-nw-p value))
+ (cond ((org-file-url-p value)
+ (user-error "The file is specified as a URL, cannot be edited"))
+ ((not (org-string-nw-p value))
(user-error "No file to edit"))
((string-match "\\`\"\\(.*?\\)\"" value)
(match-string 1 value))
@@ -20949,7 +20993,9 @@ Use `\\[org-edit-special]' to edit table.el tables"))
(funcall major-mode)
(hack-local-variables)
(when (and indent-status (not (bound-and-true-p org-indent-mode)))
- (org-indent-mode -1)))
+ (org-indent-mode -1))
+ ;; Reset the cache of files downloaded by `org-file-contents'.
+ (clrhash org--file-cache))
(message "%s restarted" major-mode))
(defun org-kill-note-or-show-branches ()
diff --git a/lisp/ox.el b/lisp/ox.el
index ac8d8ce68..7d1012974 100644
--- a/lisp/ox.el
+++ b/lisp/ox.el
@@ -1499,17 +1499,20 @@ Assume buffer is in Org mode. Narrowing, if any, is ignored."
(cond
;; Options in `org-export-special-keywords'.
((equal key "SETUPFILE")
- (let ((file
- (expand-file-name
- (org-unbracket-string "\"" "\"" (org-trim val)))))
+ (let* ((uri (org-unbracket-string "\"" "\"" (org-trim val)))
+ (uri-is-url (org-file-url-p uri))
+ (uri (if uri-is-url
+ uri
+ (expand-file-name uri))))
;; Avoid circular dependencies.
- (unless (member file files)
+ (unless (member uri files)
(with-temp-buffer
- (setq default-directory
- (file-name-directory file))
- (insert (org-file-contents file 'noerror))
+ (unless uri-is-url
+ (setq default-directory
+ (file-name-directory uri)))
+ (insert (org-file-contents uri 'noerror))
(let ((org-inhibit-startup t)) (org-mode))
- (funcall get-options (cons file files))))))
+ (funcall get-options (cons uri files))))))
((equal key "OPTIONS")
(setq plist
(org-combine-plists
@@ -1647,17 +1650,22 @@ an alist where associations are (VARIABLE-NAME VALUE)."
"BIND")
(push (read (format "(%s)" val)) alist)
;; Enter setup file.
- (let ((file (expand-file-name
- (org-unbracket-string "\"" "\"" val))))
- (unless (member file files)
+ (let* ((uri (org-unbracket-string "\"" "\"" val))
+ (uri-is-url (org-file-url-p uri))
+ (uri (if uri-is-url
+ uri
+ (expand-file-name uri))))
+ ;; Avoid circular dependencies.
+ (unless (member uri files)
(with-temp-buffer
- (setq default-directory
- (file-name-directory file))
+ (unless uri-is-url
+ (setq default-directory
+ (file-name-directory uri)))
(let ((org-inhibit-startup t)) (org-mode))
- (insert (org-file-contents file 'noerror))
+ (insert (org-file-contents uri 'noerror))
(setq alist
(funcall collect-bind
- (cons file files)
+ (cons uri files)
alist))))))))))
alist)))))
;; Return value in appropriate order of appearance.
--
2.13.0
next prev parent reply other threads:[~2017-05-26 20:24 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-12-03 17:23 Allow #+SETUPFILE to point to an URL for the org file Kaushal Modi
2016-12-08 11:51 ` Kaushal Modi
2016-12-08 14:22 ` John Kitchin
2016-12-08 14:31 ` Nicolas Goaziou
2016-12-08 14:44 ` Kaushal Modi
2016-12-08 21:48 ` Nicolas Goaziou
2016-12-08 22:07 ` Kaushal Modi
2016-12-08 22:40 ` Nicolas Goaziou
2017-03-13 17:37 ` Kaushal Modi
2017-03-30 7:43 ` Nicolas Goaziou
2017-05-23 19:07 ` Kaushal Modi
2017-05-25 10:13 ` Nicolas Goaziou
2017-05-25 10:18 ` Nicolas Goaziou
2017-05-25 11:43 ` Kaushal Modi
2017-05-25 15:15 ` Kaushal Modi
2017-05-26 7:47 ` Nicolas Goaziou
2017-05-26 20:24 ` Kaushal Modi [this message]
2017-05-28 7:35 ` Nicolas Goaziou
2017-05-28 10:04 ` Kaushal Modi
2017-06-09 16:59 ` Kaushal Modi
2017-06-12 19:32 ` Kaushal Modi
2017-06-13 12:43 ` Nicolas Goaziou
2017-06-13 15:45 ` Kaushal Modi
2017-06-13 21:32 ` Nicolas Goaziou
2017-06-13 21:42 ` Kaushal Modi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAFyQvY3-rOGXQmmAa-L++Z8tYiP3xfU2HeZrTn8-ox57rdALng@mail.gmail.com \
--to=kaushal.modi@gmail.com \
--cc=emacs-orgmode@gnu.org \
--cc=mail@nicolasgoaziou.fr \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).