* [babel] exports, caching, remote execution
@ 2010-06-17 6:55 Austin Frank
2010-06-17 14:42 ` Eric Schulte
0 siblings, 1 reply; 7+ messages in thread
From: Austin Frank @ 2010-06-17 6:55 UTC (permalink / raw)
To: emacs-orgmode
[-- Attachment #1.1: Type: text/plain, Size: 1081 bytes --]
Hey all--
Two (hopefully quick) questions:
1) Does the exporter respect the :cache argument? When I evaluate a
buffer, I can tell that cached blocks are not re-run, as expected.
When I export to \LaTeX or PDF, it seems that all blocks in the file
are re-run. Is there a way to force the exporter to respect caching?
2) Is there a way to generate graphical output on a remote host and
still have it included in export? A block like
#+BEGIN_SRC R :exports results :dir /user@example.com: :file test.png
plot(my-object)
#+END_SRC
Gives me
#+results:
[[file:/scpc:user@example.com:/users/home/user/test.png]]
When I export to latex or pdf, the graphic isn't included because the
file for export is local but the graphic is remote. Is there a way
to include remotely generated graphics in a locally exported file?
Something like downloading via scp before exporting?
Thanks for any help,
/au
--
Austin Frank
http://aufrank.net
GPG Public Key (D7398C2F): http://aufrank.net/personal.asc
[-- Attachment #1.2: Type: application/pgp-signature, Size: 194 bytes --]
[-- Attachment #2: Type: text/plain, Size: 201 bytes --]
_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [babel] exports, caching, remote execution
2010-06-17 6:55 [babel] exports, caching, remote execution Austin Frank
@ 2010-06-17 14:42 ` Eric Schulte
2010-06-17 19:29 ` Austin Frank
` (2 more replies)
0 siblings, 3 replies; 7+ messages in thread
From: Eric Schulte @ 2010-06-17 14:42 UTC (permalink / raw)
To: Austin Frank; +Cc: emacs-orgmode
Hi Austin,
Austin Frank <austin.frank@gmail.com> writes:
> Hey all--
>
> Two (hopefully quick) questions:
>
> 1) Does the exporter respect the :cache argument? When I evaluate a
> buffer, I can tell that cached blocks are not re-run, as expected.
> When I export to \LaTeX or PDF, it seems that all blocks in the file
> are re-run. Is there a way to force the exporter to respect caching?
>
I believe the exporter does respect caching, the following minimal
example worked (i.e. was not re-run) for me on export to html. Could
you provide an example that demonstrates the problem?
--8<---------------cut here---------------start------------->8---
** cache on export
do we export cached blocks
#+begin_src sh :cache yes :exports results
date
#+end_src
#+results[06ed73c6d8d022cf9c323d92af885952865add17]:
: Thu Jun 17 07:35:19 PDT 2010
--8<---------------cut here---------------end--------------->8---
>
> 2) Is there a way to generate graphical output on a remote host and
> still have it included in export?
Not that I'm currently aware of. Dan has handled most of the remote
execution work so he may know more than me in this regard.
> A block like
>
> #+source: rplot
> #+BEGIN_SRC R :exports results :dir /user@example.com: :file test.png
> plot(my-object)
> #+END_SRC
>
> Gives me
>
> #+results:
> [[file:/scpc:user@example.com:/users/home/user/test.png]]
>
Is "scpc" in the line above a transport protocol? Maybe this should be
an org-mode wide features, i.e. the ability to resolve remote file
references with C-c C-o and on export. Does that sound reasonable, and
would it take care of the need in this particular case?
>
> When I export to latex or pdf, the graphic isn't included because
> the file for export is local but the graphic is remote. Is there a
> way to include remotely generated graphics in a locally exported
> file? Something like downloading via scp before exporting?
>
A block like the following may be an acceptable workaround in the near
term (notice I named your block above).
#+begin_src sh :file test.png :var remote=rplot :exports results
scp $remote ./test.png
#+end_src
>
>
> Thanks for any help,
> /au
Thanks -- Eric
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [babel] exports, caching, remote execution
2010-06-17 14:42 ` Eric Schulte
@ 2010-06-17 19:29 ` Austin Frank
2010-06-18 15:35 ` David Maus
2010-08-19 20:59 ` Tom Short
2 siblings, 0 replies; 7+ messages in thread
From: Austin Frank @ 2010-06-17 19:29 UTC (permalink / raw)
To: emacs-orgmode
[-- Attachment #1.1: Type: text/plain, Size: 1673 bytes --]
Eric--
Thanks for the reply.
On Thu, Jun 17 2010, Eric Schulte wrote:
> I believe the exporter does respect caching, the following minimal
> example worked (i.e. was not re-run) for me on export to html. Could
> you provide an example that demonstrates the problem?
Yes, I'll keep an eye on this and try to narrow it down to an example if
I can.
> Not that I'm currently aware of. Dan has handled most of the remote
> execution work so he may know more than me in this regard.
And thanks to him for that! I tipped him off to using TRAMP with ESS,
and he took it and ran further than I ever hoped!
> Is "scpc" in the line above a transport protocol?
Yes, it is scp over an existing ssh connection. The following org-style
link goes to the relevant info page.
[[info:tramp:External%20methods][info:tramp:External methods]]
> Maybe this should be an org-mode wide features, i.e. the ability to
> resolve remote file references with C-c C-o and on export. Does that
> sound reasonable, and would it take care of the need in this
> particular case?
C-c C-o actually works! I was extremely encouraged to see this, and it
got my hopes up for export =)
If remote file references could be resolved during export, that would be
ideal.
> A block like the following may be an acceptable workaround in the near
> term (notice I named your block above).
>
> #+begin_src sh :file test.png :var remote=rplot :exports results
> scp $remote ./test.png
> #+end_src
Nice suggestions, I'll try it later today.
Thanks,
/au
--
Austin Frank
http://aufrank.net
GPG Public Key (D7398C2F): http://aufrank.net/personal.asc
[-- Attachment #1.2: Type: application/pgp-signature, Size: 194 bytes --]
[-- Attachment #2: Type: text/plain, Size: 201 bytes --]
_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [babel] exports, caching, remote execution
2010-06-17 14:42 ` Eric Schulte
2010-06-17 19:29 ` Austin Frank
@ 2010-06-18 15:35 ` David Maus
2010-08-19 20:59 ` Tom Short
2 siblings, 0 replies; 7+ messages in thread
From: David Maus @ 2010-06-18 15:35 UTC (permalink / raw)
To: Eric Schulte; +Cc: emacs-orgmode
[-- Attachment #1.1: Type: text/plain, Size: 4235 bytes --]
Eric Schulte wrote:
>Is "scpc" in the line above a transport protocol? Maybe this should be
>an org-mode wide features, i.e. the ability to resolve remote file
>references with C-c C-o and on export. Does that sound reasonable, and
>would it take care of the need in this particular case?
I've created two IDEA issue in the issue file for this and another
operation on links especially during export: Link resolving and link
dereferencing.
Just out of my head without many thoughs about how to implement:
1.
Following the URI specs (RFC 3986) resolving and dereferencing means:
"the process of determining an access mechanism and the appropriate
parameters necessary to dereference a URI (sc. resolving, D.M.); this
resolution may require several iterations. To use that access
mechanism to perform an action on the URI's resource is to
"dereference" the URI." (1.2.2)
The basic idea is, that an Org mode link is a URI identifying a
resource; upon export it might be necessary or desired to provide
either an access mechanism suitable for the output format to access
the entity referenced by the URI or another URI (see below).
E.g.: An info reference to the Org mode manual [[info:org]] could be
resolved to a web link [[http://orgmode.org/manual/]] when exporting
to html.
Resolving is already done for files when exporting to html:
[[file:foo.baz]] is resolved to a relative web link, pointing to
foo.baz.
Dereferencing a link like
[[file:/scpc:user@example.com:/users/home/user/test.png]]
could or would hence be part of resolving a file link. It fetches the
resource (test.png), puts it into current directory and returns a link
suitable for the export context.
Another example would be a hypothetical Org mode link type "rfc" that
opens and stores links to RFCs using rfcview-mode.
When exporting an Org mode document to html, the function
`org-rfc-resolve-link' is called and returns a web link pointing to a
web resource of the RFC. When exporting to latex, the function
`org-rfc-resolve-link' returns a link of type "bib" with the BibTeX
key of the RFC (e.g. rfc:4287 -> bib:RFC4287), assuming that a
reference to a RFC in a LaTeX document is a bibliographic reference.
2.
To complete this way of link handling we require a third function:
Link markup. An Org mode link handler may provide a markup-link
function that takes the exporting context and the link and returns
suitable markup of LINK in CONTEXT.
E.g. the handler for BibTeX links provides the function
`org-bib-markup-link' that returns proper markup of a bibliographic
reference. In context of LaTeX export his would be a \cite command
with the BibTeX key of the bib link (bib:RFC4287 -> \cite{4287}).
3.
For the sake of Lispness and to avoid repeated parsing, we define a
Lisp representation of an Org link to be:
((TYPE AI RESOURCE) . PROPS)
Where
- TYPE :: Is a symbol of the type ("scheme") of the link
- RESOURCE :: Is the path or name of the referenced resource
- AI :: Is t, if RESOURCE contains an authority, otherwise it is nil.
Cf. RFC3986, 3.2; in the written representation of a URI the
presence of an authority is indicated by the two slashes
after the colon.
- PROPS :: Is a property list of additional link properties
(e.g. description)
Examples:
- [[http://orgmode.org/manual/][Org mode manual]] is represented as:
((http t "orgmode.org/manual/") . (:description "Org mode manual"))
- [[info:org][Org mode manual]] is represented as
((info nil "org") . (:description "Org mode manual"))
- [[file:foo/baz.org]] is represented as
((file nil "foo/baz.org"))
Having this, resolving a link is modifying TYPE, RESOURCE and AI.
E.g. ((file nil "foo/baz.org")) -> ((http nil "foo/baz.html"))
((file nil "/scp:user@example.com:/users/home/user/test.png"))
-> ((file nil "test.png")) [dereferenced]
-> ((http nil "test.png")) [finally resolved]
4.
The question of how to actually resolve a link is up to the respective
link handler and I am not sure how to provide easy customization of
such a system.
Comments are welcome,
-- David
--
OpenPGP... 0x99ADB83B5A4478E6
Jabber.... dmjena@jabber.org
Email..... dmaus@ictsoc.de
[-- Attachment #1.2: Type: application/pgp-signature, Size: 230 bytes --]
[-- Attachment #2: Type: text/plain, Size: 201 bytes --]
_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [babel] exports, caching, remote execution
2010-06-17 14:42 ` Eric Schulte
2010-06-17 19:29 ` Austin Frank
2010-06-18 15:35 ` David Maus
@ 2010-08-19 20:59 ` Tom Short
2010-08-25 18:04 ` Eric Schulte
2010-09-19 2:11 ` Eric Schulte
2 siblings, 2 replies; 7+ messages in thread
From: Tom Short @ 2010-08-19 20:59 UTC (permalink / raw)
To: emacs-orgmode
Eric Schulte <schulte.eric <at> gmail.com> writes:
>
> Hi Austin,
>
> Austin Frank <austin.frank <at> gmail.com> writes:
>
> > Hey all--
> >
> > Two (hopefully quick) questions:
> >
> > 1) Does the exporter respect the :cache argument? When I evaluate a
> > buffer, I can tell that cached blocks are not re-run, as expected.
> > When I export to \LaTeX or PDF, it seems that all blocks in the file
> > are re-run. Is there a way to force the exporter to respect caching?
> >
>
> I believe the exporter does respect caching, the following minimal
> example worked (i.e. was not re-run) for me on export to html. Could
> you provide an example that demonstrates the problem?
Eric, here's an example where the exporter does not respect caching. - Tom
#+BABEL: :session *R* :results output :exports both :cache yes
* A test of caching
#+begin_src R
cat("random result:", runif(1), "\n")
Sys.sleep(2)
alarm()
#+end_src
#+results[b2549fac8a1ec2923ae289d47ce55fb2853dd1de]:
: random result: 0.2799064
#+begin_src R
cat("random result:", runif(1), "\n")
Sys.sleep(2)
alarm()
#+end_src
#+results[b2549fac8a1ec2923ae289d47ce55fb2853dd1de]:
: random result: 0.1625634
** cache on export
do we export cached blocks
#+begin_src emacs-lisp :cache yes :exports results
(random)
#+end_src
#+results[46632b4fe2e3a23e847953c95adcba58c270b381]:
: 490528137
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Re: [babel] exports, caching, remote execution
2010-08-19 20:59 ` Tom Short
@ 2010-08-25 18:04 ` Eric Schulte
2010-09-19 2:11 ` Eric Schulte
1 sibling, 0 replies; 7+ messages in thread
From: Eric Schulte @ 2010-08-25 18:04 UTC (permalink / raw)
To: Tom Short; +Cc: emacs-orgmode
Hi Tom,
Thanks for pointing this out. It appears that during export the code
block is evaluated in an altered version of the original buffer in which
(for some reason) Babel is sometimes not able to find properties which
are tucked away in either a #+Babel: line at the top of the file, or as
properties of an enclosing headline level.
This certainly merits a closer investigation, and has been recorded as a
bug. Most likely it will require picking apart the subtleties of the
actions taken by the exporter.
Thanks -- Eric
Tom Short <tshort@epri.com> writes:
> Eric Schulte <schulte.eric <at> gmail.com> writes:
>
>>
>> Hi Austin,
>>
>> Austin Frank <austin.frank <at> gmail.com> writes:
>>
>> > Hey all--
>> >
>> > Two (hopefully quick) questions:
>> >
>> > 1) Does the exporter respect the :cache argument? When I evaluate a
>> > buffer, I can tell that cached blocks are not re-run, as expected.
>> > When I export to \LaTeX or PDF, it seems that all blocks in the file
>> > are re-run. Is there a way to force the exporter to respect caching?
>> >
>>
>> I believe the exporter does respect caching, the following minimal
>> example worked (i.e. was not re-run) for me on export to html. Could
>> you provide an example that demonstrates the problem?
>
> Eric, here's an example where the exporter does not respect caching. - Tom
>
> #+BABEL: :session *R* :results output :exports both :cache yes
>
> * A test of caching
>
> #+begin_src R
> cat("random result:", runif(1), "\n")
> Sys.sleep(2)
> alarm()
> #+end_src
>
> #+results[b2549fac8a1ec2923ae289d47ce55fb2853dd1de]:
> : random result: 0.2799064
>
> #+begin_src R
> cat("random result:", runif(1), "\n")
> Sys.sleep(2)
> alarm()
> #+end_src
>
> #+results[b2549fac8a1ec2923ae289d47ce55fb2853dd1de]:
> : random result: 0.1625634
>
> ** cache on export
> do we export cached blocks
>
> #+begin_src emacs-lisp :cache yes :exports results
> (random)
> #+end_src
>
> #+results[46632b4fe2e3a23e847953c95adcba58c270b381]:
> : 490528137
>
>
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Re: [babel] exports, caching, remote execution
2010-08-19 20:59 ` Tom Short
2010-08-25 18:04 ` Eric Schulte
@ 2010-09-19 2:11 ` Eric Schulte
1 sibling, 0 replies; 7+ messages in thread
From: Eric Schulte @ 2010-09-19 2:11 UTC (permalink / raw)
To: Tom Short; +Cc: emacs-orgmode
Hi Tom,
This turned out to be a fairly complicated issue.
During export of a narrowed region the narrowed region will be coppied
to a temporary buffer, when exporting from the buffer all buffer-wide,
and headline-wide parameters which are not included in the region are
ignored. This results in a different set of header arguments being used
for interactive evaluation than were used for export evaluation. These
differences in header arguments then invalidate the cache saved during
the interactive evaluation.
I've just put together and pushed a fix which involves jumping back to
the original org file to collect header arguments in context during
export. This appears to resolve the issue (at least for me). Please
let me know if it works for you.
Best -- Eric
Tom Short <tshort@epri.com> writes:
> Eric Schulte <schulte.eric <at> gmail.com> writes:
>
>>
>> Hi Austin,
>>
>> Austin Frank <austin.frank <at> gmail.com> writes:
>>
>> > Hey all--
>> >
>> > Two (hopefully quick) questions:
>> >
>> > 1) Does the exporter respect the :cache argument? When I evaluate a
>> > buffer, I can tell that cached blocks are not re-run, as expected.
>> > When I export to \LaTeX or PDF, it seems that all blocks in the file
>> > are re-run. Is there a way to force the exporter to respect caching?
>> >
>>
>> I believe the exporter does respect caching, the following minimal
>> example worked (i.e. was not re-run) for me on export to html. Could
>> you provide an example that demonstrates the problem?
>
> Eric, here's an example where the exporter does not respect caching. - Tom
>
> #+BABEL: :session *R* :results output :exports both :cache yes
>
> * A test of caching
>
> #+begin_src R
> cat("random result:", runif(1), "\n")
> Sys.sleep(2)
> alarm()
> #+end_src
>
> #+results[b2549fac8a1ec2923ae289d47ce55fb2853dd1de]:
> : random result: 0.2799064
>
> #+begin_src R
> cat("random result:", runif(1), "\n")
> Sys.sleep(2)
> alarm()
> #+end_src
>
> #+results[b2549fac8a1ec2923ae289d47ce55fb2853dd1de]:
> : random result: 0.1625634
>
> ** cache on export
> do we export cached blocks
>
> #+begin_src emacs-lisp :cache yes :exports results
> (random)
> #+end_src
>
> #+results[46632b4fe2e3a23e847953c95adcba58c270b381]:
> : 490528137
>
>
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2010-09-19 17:17 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-06-17 6:55 [babel] exports, caching, remote execution Austin Frank
2010-06-17 14:42 ` Eric Schulte
2010-06-17 19:29 ` Austin Frank
2010-06-18 15:35 ` David Maus
2010-08-19 20:59 ` Tom Short
2010-08-25 18:04 ` Eric Schulte
2010-09-19 2:11 ` Eric Schulte
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).