From mboxrd@z Thu Jan 1 00:00:00 1970 From: Kaushal Modi Subject: Re: Allow #+SETUPFILE to point to an URL for the org file Date: Mon, 13 Mar 2017 17:37:03 +0000 Message-ID: References: <87h96eh4qb.fsf@nicolasgoaziou.fr> <871sxigkhk.fsf@nicolasgoaziou.fr> <87twaef3iq.fsf@nicolasgoaziou.fr> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=001a114dbbba82a520054aa0292e Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:33941) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cnTuJ-0004lT-NL for emacs-orgmode@gnu.org; Mon, 13 Mar 2017 13:37:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cnTuI-0004dl-12 for emacs-orgmode@gnu.org; Mon, 13 Mar 2017 13:37:19 -0400 Received: from mail-vk0-x22c.google.com ([2607:f8b0:400c:c05::22c]:36190) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cnTuH-0004cm-Po for emacs-orgmode@gnu.org; Mon, 13 Mar 2017 13:37:17 -0400 Received: by mail-vk0-x22c.google.com with SMTP id t8so39180445vke.3 for ; Mon, 13 Mar 2017 10:37:16 -0700 (PDT) In-Reply-To: <87twaef3iq.fsf@nicolasgoaziou.fr> List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: "Emacs-orgmode" To: emacs-org list , Nicolas Goaziou --001a114dbbba82a520054aa0292e Content-Type: text/plain; charset=UTF-8 On Thu, Dec 8, 2016 at 5:40 PM Nicolas Goaziou wrote: > Kaushal Modi writes: > > You can use something like `url-insert' and `url-retrieve' or > `url-retrieve-synchronously'. > Thanks. I am using url-retrieve-synchronously. > > Sure, but SETUPFILE still accepts local file names. So I was pointing > out that you need to check if the URL is a local file name before > proceeding. In particular, this check needs to happen when using "C-c '" > (which may display URL contents in a read-only buffer, BTW). > Understood. I might need some help when baking this into org.el, org-macros.el, etc. > > Is it necessary? File contents could be stored in, e.g., a hash table, > url being the key. The file is downloaded only if the entry doesn't > exist in the table and the user didn't force download.- > Correct. Thanks for the idea. I am now using hash table for this. Here is my implementation.. it is still not baked into org.el, etc and provided as a complete patch; I have some questions.. ===== (defvar org-setupfile-ht (make-hash-table :test 'equal) "Hash table to store the org SETUPFILE. This acts as a cache of setup files read using `org-insert-setupfile-contents'.") (defun org-setupfile-clear-cache () "Clear the SETUPFILE cache stored in `org-setupfile-ht'." (interactive) (clrhash org-setupfile-ht)) (defun org-insert-setupfile-contents (setupfile &optional nocache noerror) "Insert the contents of SETUPFILE. SETUPFILE can be a file path or URL. If SETUPFILE is a file path, use `org-file-contents' to get the file contents. If it is a URL instead, download the contents. If the URL contents are already cached in the `org-setupfile-ht' hash table, the download step is skipped. If NOCACHE is non-nil, do a fresh fetch of SETUPFILE even if cached version is available. This option applies only if SETUPFILE is a URL. If NOERROR is non-nil, ignore the error when unable to read the SETUPFILE from file or URL." (require 'ffap) ;for `ffap-url-regexp' (let* ((is-url (string-match-p ffap-url-regexp setupfile)) (cache (when (and is-url (not nocache)) (gethash setupfile org-setupfile-ht))) (contents (when (and is-url cache) cache))) (if is-url (unless cache (let (url-retrieve-header) (with-current-buffer (url-retrieve-synchronously setupfile) (goto-char (point-min)) ;; Take point to after the url-retrieve header (re-search-forward "\n\n") ; 2 consecutive new-line chars (setq url-retrieve-header (buffer-substring-no-properties (point-min) (point))) (message url-retrieve-header) ;Dump the URL retrieve header to *Messages* (if (string-match-p "HTTP.*\\s-+200\\s-OK" url-retrieve-header) ;URL retrieved correctly (progn (setq contents (buffer-substring-no-properties (point) (point-max))) ;; Update the cache `org-setupfile-ht' (puthash setupfile contents org-setupfile-ht)) (funcall (if noerror 'message 'error) "Unable to fetch SETUPFILE from `%s'" setupfile))))) (setq contents (org-file-contents setupfile noerror))) (when contents (save-excursion (insert contents))))) ===== Question: - All the places where the content of SETUPFILE is inserted in a temp buffer, it is assumed that the file is retrieved from disk and not from URL. Example in ox.el: ((equal key "SETUPFILE") (let ((file (expand-file-name (org-unbracket-string "\"" "\"" (org-trim val))))) ;; Avoid circular dependencies. (unless (member file files) (with-temp-buffer (setq default-directory (file-name-directory file)) (insert (org-file-contents file 'noerror)) (let ((org-inhibit-startup t)) (org-mode)) (funcall get-options (cons file files)))))) Note the use of expand-file-name, (member file files), default-directory, (funcall get-options (cons file files)). How do we deal with those parts of the code when the 'file' is a URL. Using my implementation above, (org-insert-setupfile-contents "/file/path/or/url" nil :noerror) works the same way as (insert (org-file-contents "/file/path" 'noerror)) So org-insert-setupfile-contents can replace (insert (org-file-contents ..)) easily. The unknown is how to deal with the surrounding code that deals with > expand-file-name, (member file files), default-directory, (funcall get-options (cons file files)). Here's a similar code snippet around setupfile insertion in ox.el again, in org-export--list-bound-variables: ===== ;; Enter setup file. (let ((file (expand-file-name (org-unbracket-string "\"" "\"" val)))) (unless (member file files) (with-temp-buffer (setq default-directory (file-name-directory file)) (let ((org-inhibit-startup t)) (org-mode)) (insert (org-file-contents file 'noerror)) (setq alist (funcall collect-bind (cons file files) alist)))))))))) ===== -- Kaushal Modi --001a114dbbba82a520054aa0292e Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable


On Thu= , Dec 8, 2016 at 5:40 PM Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:
Kaushal Modi <kaushal.modi@gmail.com> writes:=

You can use something like `url-insert' and `ur= l-retrieve' or
`url-retrieve-synchronously'.
=
Thanks. I am using url-retrieve-synchronously.=C2=A0

Sure, but SETUPFILE still accepts local file names. So I was pointing
out that you need to check if the URL is a local file name before
proceeding. In particular, this check needs to happen when using "C-c = '"
(which may display URL contents in a read-only buffer, BTW).

Understood. I might need some hel= p when baking this into org.el, org-macros.el, etc.
=C2=A0
<= blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px= #ccc solid;padding-left:1ex">
Is it necessary? File contents could be stored in, e.g., a hash table,
url being the key. The file is downloaded only if the entry doesn't
exist in the table and the user didn't force download.-

Correct. Thanks for the idea. I am= now using hash table for this.=C2=A0

Here is my i= mplementation.. it is still not baked into org.el, etc and provided as a co= mplete patch; I have some questions..

=3D=3D=3D=3D= =3D
(defvar org-setupfile-ht (make-hash-table :test 'equ= al)
=C2=A0 "Hash table to store the org SETUPFILE.
This acts as a cache of setup files read using `org-insert-setupfile-conte= nts'.")

(defun org-setupfile-clear-cache = ()
=C2=A0 "Clear the SETUPFILE cache stored in `org-setupfil= e-ht'."
=C2=A0 (interactive)
=C2=A0 (clrhash o= rg-setupfile-ht))

(defun org-insert-setupfile-cont= ents (setupfile &optional nocache noerror)
=C2=A0 "Inser= t the contents of SETUPFILE.
SETUPFILE can be a file path or URL.=

If SETUPFILE is a file path, use `org-file-conten= ts' to get the file contents.
If it is a URL instead, downloa= d the contents. If the URL contents are already
cached in the `or= g-setupfile-ht' hash table, the download step is skipped.
If NOCACHE is non-nil, do a fresh fetch of SETUPFILE even if ca= ched version is
available. This option applies only if SETUPFILE = is a URL.

If NOERROR is non-nil, ignore the error = when unable to read the SETUPFILE from
file or URL."
=C2=A0 (require 'ffap) =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ;for `ffap-url-regexp'
=C2=A0= (let* ((is-url (string-match-p ffap-url-regexp setupfile))
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(cache (when (and is-url (not nocache))
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (getha= sh setupfile org-setupfile-ht)))
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0(contents (when (and is-url cache) cache)))
=C2=A0 =C2=A0 (if = is-url
=C2=A0 =C2=A0 =C2=A0 =C2=A0 (unless cache
=C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 (let (url-retrieve-header)
=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (with-current-buffer (url-retrieve-synch= ronously setupfile)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 (goto-char (point-min))
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 ;; Take point to after the url-retrieve header
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (re-search-forward "\n\n= ") ; 2 consecutive new-line chars
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 (setq url-retrieve-header (buffer-substring-no-pro= perties
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0(point-min) (point)))
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 (message url-retrieve-header) ;Dump the URL retrie= ve header to *Messages*
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 (if (string-match-p "HTTP.*\\s-+200\\s-OK" url-retrieve-h= eader) ;URL retrieved correctly
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (progn
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (setq contents (buffer-substr= ing-no-properties (point) (point-max)))
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ;; Update the cache `org-setu= pfile-ht'
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 (puthash setupfile contents org-setupfile-ht))
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 (funcall (if noe= rror 'message 'error)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"Unable to fetc= h SETUPFILE from `%s'" setupfile)))))
=C2=A0 =C2=A0 =C2= =A0 (setq contents (org-file-contents setupfile noerror)))
=C2=A0= =C2=A0 (when contents
=C2=A0 =C2=A0 =C2=A0 (save-excursion
=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 (insert contents)))))
=3D= =3D=3D=3D=3D

Question:

- = All the places where the content of SETUPFILE is inserted in a temp buffer,= it is assumed that the file is retrieved from disk and not from URL.
=

Example in ox.el:

=C2=A0((equal = key "SETUPFILE")
=C2=A0 (let ((file
=C2=A0(expand-file-n= ame
= =C2=A0 (org-unbracket-string "\"" "\""= ; (org-trim val)))))
=C2=A0 =C2=A0 ;; Avoid circular dependencies.
=
= =C2=A0 =C2=A0 (unless (member file files)
=C2=A0 =C2=A0 =C2=A0 (with-tem= p-buffer
(setq default-directory
=C2=A0 (file-name-directory file))=
(insert (org-file-contents file 'noerror))
(let ((org-inhibi= t-startup t)) (org-mode))
(funcall get-options (cons file files))))))


Note the use of expand-file-na= me, (member file files), default-directory, (funcall get-options (cons file= files)).

How do we deal with those parts of the c= ode when the 'file' is a URL.

Using my imp= lementation above,=C2=A0

=C2=A0 =C2=A0 (org-insert= -setupfile-contents "/file/path/or/url" nil :noerror)

works the same way as=C2=A0

=C2= =A0 =C2=A0 (insert (org-file-contents "/file/path" 'noerror))=

So org-insert-setupfile-contents can replace (ins= ert (org-file-contents ..)) easily. The unknown is how to deal with the sur= rounding code that deals with

> expand-file-nam= e, (member file files), default-directory, (funcall get-options (cons file = files)).

Here's a similar code snippet around setupfile = insertion in ox.el again, in org-export--list-bound-variables:

=3D=3D=3D=3D=3D
=C2=A0 =C2=A0 ;; Enter se= tup file.
=C2=A0 =C2=A0 (let ((file (expand-file-n= ame
=C2=A0(org-unbracket-string "\"&qu= ot; "\"" val))))
=C2=A0 =C2=A0 =C2= =A0 (unless (member file files)
(with-temp-buffer=
=C2=A0 (setq default-directory
= (file-name-directory file))
=C2=A0 (l= et ((org-inhibit-startup t)) (org-mode))
=C2=A0 (= insert (org-file-contents file 'noerror))
=C2= =A0 (setq alist
(funcall collect-bind
=C2=A0(cons file files)
<= span class=3D"Apple-tab-span" style=3D"white-space:pre"> =C2= =A0alist))))))))))
=3D=3D=3D=3D=3D


<= /div>


--

Kaushal Modi

--001a114dbbba82a520054aa0292e--