Re: Allow #+SETUPFILE to point to an URL for the org file

[Kaushal Modi <kaushal.modi@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 5135 bytes --]

On Thu, Dec 8, 2016 at 5:40 PM Nicolas Goaziou <mail@nicolasgoaziou.fr>
wrote:

> Kaushal Modi <kaushal.modi@gmail.com> writes:
>
> You can use something like `url-insert' and `url-retrieve' or
> `url-retrieve-synchronously'.
>

Thanks. I am using url-retrieve-synchronously.

>
> Sure, but SETUPFILE still accepts local file names. So I was pointing
> out that you need to check if the URL is a local file name before
> proceeding. In particular, this check needs to happen when using "C-c '"
> (which may display URL contents in a read-only buffer, BTW).
>

Understood. I might need some help when baking this into org.el,
org-macros.el, etc.


>
> Is it necessary? File contents could be stored in, e.g., a hash table,
> url being the key. The file is downloaded only if the entry doesn't
> exist in the table and the user didn't force download.-
>

Correct. Thanks for the idea. I am now using hash table for this.

Here is my implementation.. it is still not baked into org.el, etc and
provided as a complete patch; I have some questions..

=====
(defvar org-setupfile-ht (make-hash-table :test 'equal)
  "Hash table to store the org SETUPFILE.
This acts as a cache of setup files read using
`org-insert-setupfile-contents'.")

(defun org-setupfile-clear-cache ()
  "Clear the SETUPFILE cache stored in `org-setupfile-ht'."
  (interactive)
  (clrhash org-setupfile-ht))

(defun org-insert-setupfile-contents (setupfile &optional nocache noerror)
  "Insert the contents of SETUPFILE.
SETUPFILE can be a file path or URL.

If SETUPFILE is a file path, use `org-file-contents' to get the file
contents.
If it is a URL instead, download the contents. If the URL contents are
already
cached in the `org-setupfile-ht' hash table, the download step is skipped.

If NOCACHE is non-nil, do a fresh fetch of SETUPFILE even if cached version
is
available. This option applies only if SETUPFILE is a URL.

If NOERROR is non-nil, ignore the error when unable to read the SETUPFILE
from
file or URL."
  (require 'ffap)                       ;for `ffap-url-regexp'
  (let* ((is-url (string-match-p ffap-url-regexp setupfile))
         (cache (when (and is-url (not nocache))
                  (gethash setupfile org-setupfile-ht)))
         (contents (when (and is-url cache) cache)))
    (if is-url
        (unless cache
          (let (url-retrieve-header)
            (with-current-buffer (url-retrieve-synchronously setupfile)
              (goto-char (point-min))
              ;; Take point to after the url-retrieve header
              (re-search-forward "\n\n") ; 2 consecutive new-line chars
              (setq url-retrieve-header (buffer-substring-no-properties
                                         (point-min) (point)))
              (message url-retrieve-header) ;Dump the URL retrieve header
to *Messages*
              (if (string-match-p "HTTP.*\\s-+200\\s-OK"
url-retrieve-header) ;URL retrieved correctly
                  (progn
                    (setq contents (buffer-substring-no-properties (point)
(point-max)))
                    ;; Update the cache `org-setupfile-ht'
                    (puthash setupfile contents org-setupfile-ht))
                (funcall (if noerror 'message 'error)
                         "Unable to fetch SETUPFILE from `%s'"
setupfile)))))
      (setq contents (org-file-contents setupfile noerror)))
    (when contents
      (save-excursion
        (insert contents)))))
=====

Question:

- All the places where the content of SETUPFILE is inserted in a temp
buffer, it is assumed that the file is retrieved from disk and not from URL.

Example in ox.el:

 ((equal key "SETUPFILE")
  (let ((file
 (expand-file-name
  (org-unbracket-string "\"" "\"" (org-trim val)))))
    ;; Avoid circular dependencies.
    (unless (member file files)
      (with-temp-buffer
(setq default-directory
  (file-name-directory file))
(insert (org-file-contents file 'noerror))
(let ((org-inhibit-startup t)) (org-mode))
(funcall get-options (cons file files))))))


Note the use of expand-file-name, (member file files), default-directory,
(funcall get-options (cons file files)).

How do we deal with those parts of the code when the 'file' is a URL.

Using my implementation above,

    (org-insert-setupfile-contents "/file/path/or/url" nil :noerror)

works the same way as

    (insert (org-file-contents "/file/path" 'noerror))

So org-insert-setupfile-contents can replace (insert (org-file-contents
..)) easily. The unknown is how to deal with the surrounding code that
deals with

> expand-file-name, (member file files), default-directory, (funcall
get-options (cons file files)).

Here's a similar code snippet around setupfile insertion in ox.el again, in
org-export--list-bound-variables:

=====
    ;; Enter setup file.
    (let ((file (expand-file-name
 (org-unbracket-string "\"" "\"" val))))
      (unless (member file files)
(with-temp-buffer
  (setq default-directory
(file-name-directory file))
  (let ((org-inhibit-startup t)) (org-mode))
  (insert (org-file-contents file 'noerror))
  (setq alist
(funcall collect-bind
 (cons file files)
 alist))))))))))
=====




-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 9864 bytes --]