emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed

* [babel] exports, caching, remote execution
@ 2010-06-17  6:55 Austin Frank
  2010-06-17 14:42 ` Eric Schulte
  0 siblings, 1 reply; 7+ messages in thread
From: Austin Frank @ 2010-06-17  6:55 UTC (permalink / raw)
  To: emacs-orgmode


[-- Attachment #1.1: Type: text/plain, Size: 1081 bytes --]

Hey all--

Two (hopefully quick) questions:

1) Does the exporter respect the :cache argument?  When I evaluate a
   buffer, I can tell that cached blocks are not re-run, as expected.
   When I export to \LaTeX or PDF, it seems that all blocks in the file
   are re-run.  Is there a way to force the exporter to respect caching?

2) Is there a way to generate graphical output on a remote host and
   still have it included in export?  A block like

   #+BEGIN_SRC R :exports results :dir /user@example.com: :file test.png
   plot(my-object)
   #+END_SRC
   
   Gives me
   
   #+results:
   [[file:/scpc:user@example.com:/users/home/user/test.png]]

   When I export to latex or pdf, the graphic isn't included because the
   file for export is local but the graphic is remote.  Is there a way
   to include remotely generated graphics in a locally exported file?
   Something like downloading via scp before exporting?


Thanks for any help,
/au
   
-- 
Austin Frank
http://aufrank.net
GPG Public Key (D7398C2F): http://aufrank.net/personal.asc

[-- Attachment #1.2: Type: application/pgp-signature, Size: 194 bytes --]

[-- Attachment #2: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

^ permalink raw reply	[flat|nested] 7+ messages in thread

• * Re: [babel] exports, caching, remote execution
    2010-06-17  6:55 [babel] exports, caching, remote execution Austin Frank
  @ 2010-06-17 14:42 ` Eric Schulte
    2010-06-17 19:29   ` Austin Frank
                       ` (2 more replies)
    0 siblings, 3 replies; 7+ messages in thread
  From: Eric Schulte @ 2010-06-17 14:42 UTC (permalink / raw)
    To: Austin Frank; +Cc: emacs-orgmode
  
  Hi Austin,
  
  Austin Frank <austin.frank@gmail.com> writes:
  
  > Hey all--
  >
  > Two (hopefully quick) questions:
  >
  > 1) Does the exporter respect the :cache argument?  When I evaluate a
  >    buffer, I can tell that cached blocks are not re-run, as expected.
  >    When I export to \LaTeX or PDF, it seems that all blocks in the file
  >    are re-run.  Is there a way to force the exporter to respect caching?
  >
  
  I believe the exporter does respect caching, the following minimal
  example worked (i.e. was not re-run) for me on export to html.  Could
  you provide an example that demonstrates the problem?
  
  --8<---------------cut here---------------start------------->8---
  ** cache on export
  do we export cached blocks
  
  #+begin_src sh :cache yes :exports results
    date
  #+end_src
  
  #+results[06ed73c6d8d022cf9c323d92af885952865add17]:
  : Thu Jun 17 07:35:19 PDT 2010
  --8<---------------cut here---------------end--------------->8---
  
  >
  > 2) Is there a way to generate graphical output on a remote host and
  >    still have it included in export?
  
  Not that I'm currently aware of.  Dan has handled most of the remote
  execution work so he may know more than me in this regard.
  
  >    A block like
  >
  >    #+source: rplot
  >    #+BEGIN_SRC R :exports results :dir /user@example.com: :file test.png
  >    plot(my-object)
  >    #+END_SRC
  >    
  >    Gives me
  >    
  >    #+results:
  >    [[file:/scpc:user@example.com:/users/home/user/test.png]]
  >
  
  Is "scpc" in the line above a transport protocol?  Maybe this should be
  an org-mode wide features, i.e. the ability to resolve remote file
  references with C-c C-o and on export.  Does that sound reasonable, and
  would it take care of the need in this particular case?
  
  >
  >    When I export to latex or pdf, the graphic isn't included because
  >    the file for export is local but the graphic is remote.  Is there a
  >    way to include remotely generated graphics in a locally exported
  >    file?  Something like downloading via scp before exporting?
  >
  
  A block like the following may be an acceptable workaround in the near
  term (notice I named your block above).
  
  #+begin_src sh :file test.png :var remote=rplot :exports results
    scp $remote ./test.png
  #+end_src
  
  >
  >
  > Thanks for any help,
  > /au
  
  Thanks -- Eric
  
  ^ permalink raw reply	[flat|nested] 7+ messages in thread

• • * Re: [babel] exports, caching, remote execution
      2010-06-17 14:42 ` Eric Schulte
    @ 2010-06-17 19:29   ` Austin Frank
      2010-06-18 15:35   ` David Maus
      2010-08-19 20:59   ` Tom Short
      2 siblings, 0 replies; 7+ messages in thread
    From: Austin Frank @ 2010-06-17 19:29 UTC (permalink / raw)
      To: emacs-orgmode
    
    
    [-- Attachment #1.1: Type: text/plain, Size: 1673 bytes --]
    
    Eric--
    
    Thanks for the reply.
    
    On Thu, Jun 17 2010, Eric Schulte wrote:
    
    > I believe the exporter does respect caching, the following minimal
    > example worked (i.e. was not re-run) for me on export to html.  Could
    > you provide an example that demonstrates the problem?
    
    Yes, I'll keep an eye on this and try to narrow it down to an example if
    I can.
    
    > Not that I'm currently aware of.  Dan has handled most of the remote
    > execution work so he may know more than me in this regard.
    
    And thanks to him for that!  I tipped him off to using TRAMP with ESS,
    and he took it and ran further than I ever hoped!
    
    > Is "scpc" in the line above a transport protocol?
    
    Yes, it is scp over an existing ssh connection.  The following org-style
    link goes to the relevant info page.
    
    [[info:tramp:External%20methods][info:tramp:External methods]]
    
    > Maybe this should be an org-mode wide features, i.e. the ability to
    > resolve remote file references with C-c C-o and on export.  Does that
    > sound reasonable, and would it take care of the need in this
    > particular case?
    
    C-c C-o actually works!  I was extremely encouraged to see this, and it
    got my hopes up for export =)
    
    If remote file references could be resolved during export, that would be
    ideal.
    
    > A block like the following may be an acceptable workaround in the near
    > term (notice I named your block above).
    >
    > #+begin_src sh :file test.png :var remote=rplot :exports results
    >   scp $remote ./test.png
    > #+end_src
    
    Nice suggestions, I'll try it later today.
    
    Thanks,
    /au
    
    -- 
    Austin Frank
    http://aufrank.net
    GPG Public Key (D7398C2F): http://aufrank.net/personal.asc
    
    [-- Attachment #1.2: Type: application/pgp-signature, Size: 194 bytes --]
    
    [-- Attachment #2: Type: text/plain, Size: 201 bytes --]
    
    _______________________________________________
    Emacs-orgmode mailing list
    Please use `Reply All' to send replies to the list.
    Emacs-orgmode@gnu.org
    http://lists.gnu.org/mailman/listinfo/emacs-orgmode
    
    ^ permalink raw reply	[flat|nested] 7+ messages in thread

  • * Re: [babel] exports, caching, remote execution
      2010-06-17 14:42 ` Eric Schulte
      2010-06-17 19:29   ` Austin Frank
    @ 2010-06-18 15:35   ` David Maus
      2010-08-19 20:59   ` Tom Short
      2 siblings, 0 replies; 7+ messages in thread
    From: David Maus @ 2010-06-18 15:35 UTC (permalink / raw)
      To: Eric Schulte; +Cc: emacs-orgmode
    
    
    [-- Attachment #1.1: Type: text/plain, Size: 4235 bytes --]
    
    Eric Schulte wrote:
    
    >Is "scpc" in the line above a transport protocol?  Maybe this should be
    >an org-mode wide features, i.e. the ability to resolve remote file
    >references with C-c C-o and on export.  Does that sound reasonable, and
    >would it take care of the need in this particular case?
    
    I've created two IDEA issue in the issue file for this and another
    operation on links especially during export: Link resolving and link
    dereferencing.
    
    Just out of my head without many thoughs about how to implement:
    
    1.
    
    Following the URI specs (RFC 3986) resolving and dereferencing means:
    
    "the process of determining an access mechanism and the appropriate
    parameters necessary to dereference a URI (sc. resolving, D.M.); this
    resolution may require several iterations.  To use that access
    mechanism to perform an action on the URI's resource is to
    "dereference" the URI." (1.2.2)
    
    The basic idea is, that an Org mode link is a URI identifying a
    resource; upon export it might be necessary or desired to provide
    either an access mechanism suitable for the output format to access
    the entity referenced by the URI or another URI (see below).
    
    E.g.: An info reference to the Org mode manual [[info:org]] could be
    resolved to a web link [[http://orgmode.org/manual/]] when exporting
    to html.
    
    Resolving is already done for files when exporting to html:
    [[file:foo.baz]] is resolved to a relative web link, pointing to
    foo.baz.
    
    Dereferencing a link like
    
    [[file:/scpc:user@example.com:/users/home/user/test.png]]
    
    could or would hence be part of resolving a file link.  It fetches the
    resource (test.png), puts it into current directory and returns a link
    suitable for the export context.
    
    Another example would be a hypothetical Org mode link type "rfc" that
    opens and stores links to RFCs using rfcview-mode.
    
    When exporting an Org mode document to html, the function
    `org-rfc-resolve-link' is called and returns a web link pointing to a
    web resource of the RFC.  When exporting to latex, the function
    `org-rfc-resolve-link' returns a link of type "bib" with the BibTeX
    key of the RFC (e.g. rfc:4287 -> bib:RFC4287), assuming that a
    reference to a RFC in a LaTeX document is a bibliographic reference.
    
    2.
    
    To complete this way of link handling we require a third function:
    Link markup.  An Org mode link handler may provide a markup-link
    function that takes the exporting context and the link and returns
    suitable markup of LINK in CONTEXT.
    
    E.g. the handler for BibTeX links provides the function
    `org-bib-markup-link' that returns proper markup of a bibliographic
    reference.  In context of LaTeX export his would be a \cite command
    with the BibTeX key of the bib link (bib:RFC4287 -> \cite{4287}).
    
    3.
    
    For the sake of Lispness and to avoid repeated parsing, we define a
    Lisp representation of an Org link to be:
    
    ((TYPE AI RESOURCE) . PROPS)
    
    Where
    
      - TYPE :: Is a symbol of the type ("scheme") of the link
    
      - RESOURCE :: Is the path or name of the referenced resource
    
      - AI :: Is t, if RESOURCE contains an authority, otherwise it is nil.
    
              Cf. RFC3986, 3.2; in the written representation of a URI the
              presence of an authority is indicated by the two slashes
              after the colon.
    
      - PROPS :: Is a property list of additional link properties
                 (e.g. description)
    
    Examples:
    
      - [[http://orgmode.org/manual/][Org mode manual]] is represented as:
    
        ((http t "orgmode.org/manual/") . (:description "Org mode manual"))
    
      - [[info:org][Org mode manual]] is represented as
    
        ((info nil "org") . (:description "Org mode manual"))
    
      - [[file:foo/baz.org]] is represented as
    
        ((file nil "foo/baz.org"))
    
    Having this, resolving a link is modifying TYPE, RESOURCE and AI.
    
    E.g. ((file nil "foo/baz.org")) -> ((http nil "foo/baz.html"))
    
         ((file nil "/scp:user@example.com:/users/home/user/test.png"))
    
         -> ((file nil "test.png")) [dereferenced]
    
         -> ((http nil "test.png")) [finally resolved]
    
    4.
    
    The question of how to actually resolve a link is up to the respective
    link handler and I am not sure how to provide easy customization of
    such a system.
    
    Comments are welcome,
    
     -- David
    
    
    --
    OpenPGP... 0x99ADB83B5A4478E6
    Jabber.... dmjena@jabber.org
    Email..... dmaus@ictsoc.de
    
    [-- Attachment #1.2: Type: application/pgp-signature, Size: 230 bytes --]
    
    [-- Attachment #2: Type: text/plain, Size: 201 bytes --]
    
    _______________________________________________
    Emacs-orgmode mailing list
    Please use `Reply All' to send replies to the list.
    Emacs-orgmode@gnu.org
    http://lists.gnu.org/mailman/listinfo/emacs-orgmode
    
    ^ permalink raw reply	[flat|nested] 7+ messages in thread

  • * Re: [babel] exports, caching, remote execution
      2010-06-17 14:42 ` Eric Schulte
      2010-06-17 19:29   ` Austin Frank
      2010-06-18 15:35   ` David Maus
    @ 2010-08-19 20:59   ` Tom Short
      2010-08-25 18:04     ` Eric Schulte
      2010-09-19  2:11     ` Eric Schulte
      2 siblings, 2 replies; 7+ messages in thread
    From: Tom Short @ 2010-08-19 20:59 UTC (permalink / raw)
      To: emacs-orgmode
    
    Eric Schulte <schulte.eric <at> gmail.com> writes:
    
    > 
    > Hi Austin,
    > 
    > Austin Frank <austin.frank <at> gmail.com> writes:
    > 
    > > Hey all--
    > >
    > > Two (hopefully quick) questions:
    > >
    > > 1) Does the exporter respect the :cache argument?  When I evaluate a
    > >    buffer, I can tell that cached blocks are not re-run, as expected.
    > >    When I export to \LaTeX or PDF, it seems that all blocks in the file
    > >    are re-run.  Is there a way to force the exporter to respect caching?
    > >
    > 
    > I believe the exporter does respect caching, the following minimal
    > example worked (i.e. was not re-run) for me on export to html.  Could
    > you provide an example that demonstrates the problem?
    
    Eric, here's an example where the exporter does not respect caching. - Tom
    
    #+BABEL: :session *R* :results output :exports both :cache yes
    
    * A test of caching
    
    #+begin_src R 
      cat("random result:", runif(1), "\n")
      Sys.sleep(2)
      alarm()
    #+end_src 
    
    #+results[b2549fac8a1ec2923ae289d47ce55fb2853dd1de]:
    : random result: 0.2799064
    
    #+begin_src R 
      cat("random result:", runif(1), "\n")
      Sys.sleep(2)
      alarm()
    #+end_src 
    
    #+results[b2549fac8a1ec2923ae289d47ce55fb2853dd1de]:
    : random result: 0.1625634
    
    ** cache on export
    do we export cached blocks
    
    #+begin_src emacs-lisp :cache yes :exports results
      (random)
    #+end_src
    
    #+results[46632b4fe2e3a23e847953c95adcba58c270b381]:
    : 490528137
    
    ^ permalink raw reply	[flat|nested] 7+ messages in thread

  • • * Re: Re: [babel] exports, caching, remote execution
        2010-08-19 20:59   ` Tom Short
      @ 2010-08-25 18:04     ` Eric Schulte
        2010-09-19  2:11     ` Eric Schulte
        1 sibling, 0 replies; 7+ messages in thread
      From: Eric Schulte @ 2010-08-25 18:04 UTC (permalink / raw)
        To: Tom Short; +Cc: emacs-orgmode
      
      Hi Tom,
      
      Thanks for pointing this out.  It appears that during export the code
      block is evaluated in an altered version of the original buffer in which
      (for some reason) Babel is sometimes not able to find properties which
      are tucked away in either a #+Babel: line at the top of the file, or as
      properties of an enclosing headline level.
      
      This certainly merits a closer investigation, and has been recorded as a
      bug.  Most likely it will require picking apart the subtleties of the
      actions taken by the exporter.
      
      Thanks -- Eric
      
      Tom Short <tshort@epri.com> writes:
      
      > Eric Schulte <schulte.eric <at> gmail.com> writes:
      >
      >> 
      >> Hi Austin,
      >> 
      >> Austin Frank <austin.frank <at> gmail.com> writes:
      >> 
      >> > Hey all--
      >> >
      >> > Two (hopefully quick) questions:
      >> >
      >> > 1) Does the exporter respect the :cache argument?  When I evaluate a
      >> >    buffer, I can tell that cached blocks are not re-run, as expected.
      >> >    When I export to \LaTeX or PDF, it seems that all blocks in the file
      >> >    are re-run.  Is there a way to force the exporter to respect caching?
      >> >
      >> 
      >> I believe the exporter does respect caching, the following minimal
      >> example worked (i.e. was not re-run) for me on export to html.  Could
      >> you provide an example that demonstrates the problem?
      >
      > Eric, here's an example where the exporter does not respect caching. - Tom
      >
      > #+BABEL: :session *R* :results output :exports both :cache yes
      >
      > * A test of caching
      >
      > #+begin_src R 
      >   cat("random result:", runif(1), "\n")
      >   Sys.sleep(2)
      >   alarm()
      > #+end_src 
      >
      > #+results[b2549fac8a1ec2923ae289d47ce55fb2853dd1de]:
      > : random result: 0.2799064
      >
      > #+begin_src R 
      >   cat("random result:", runif(1), "\n")
      >   Sys.sleep(2)
      >   alarm()
      > #+end_src 
      >
      > #+results[b2549fac8a1ec2923ae289d47ce55fb2853dd1de]:
      > : random result: 0.1625634
      >
      > ** cache on export
      > do we export cached blocks
      >
      > #+begin_src emacs-lisp :cache yes :exports results
      >   (random)
      > #+end_src
      >
      > #+results[46632b4fe2e3a23e847953c95adcba58c270b381]:
      > : 490528137
      >
      >
      >
      >
      > _______________________________________________
      > Emacs-orgmode mailing list
      > Please use `Reply All' to send replies to the list.
      > Emacs-orgmode@gnu.org
      > http://lists.gnu.org/mailman/listinfo/emacs-orgmode
      
      ^ permalink raw reply	[flat|nested] 7+ messages in thread

    • * Re: Re: [babel] exports, caching, remote execution
        2010-08-19 20:59   ` Tom Short
        2010-08-25 18:04     ` Eric Schulte
      @ 2010-09-19  2:11     ` Eric Schulte
        1 sibling, 0 replies; 7+ messages in thread
      From: Eric Schulte @ 2010-09-19  2:11 UTC (permalink / raw)
        To: Tom Short; +Cc: emacs-orgmode
      
      Hi Tom,
      
      This turned out to be a fairly complicated issue.
      
      During export of a narrowed region the narrowed region will be coppied
      to a temporary buffer, when exporting from the buffer all buffer-wide,
      and headline-wide parameters which are not included in the region are
      ignored.  This results in a different set of header arguments being used
      for interactive evaluation than were used for export evaluation.  These
      differences in header arguments then invalidate the cache saved during
      the interactive evaluation.
      
      I've just put together and pushed a fix which involves jumping back to
      the original org file to collect header arguments in context during
      export.  This appears to resolve the issue (at least for me).  Please
      let me know if it works for you.
      
      Best -- Eric
      
      Tom Short <tshort@epri.com> writes:
      
      > Eric Schulte <schulte.eric <at> gmail.com> writes:
      >
      >> 
      >> Hi Austin,
      >> 
      >> Austin Frank <austin.frank <at> gmail.com> writes:
      >> 
      >> > Hey all--
      >> >
      >> > Two (hopefully quick) questions:
      >> >
      >> > 1) Does the exporter respect the :cache argument?  When I evaluate a
      >> >    buffer, I can tell that cached blocks are not re-run, as expected.
      >> >    When I export to \LaTeX or PDF, it seems that all blocks in the file
      >> >    are re-run.  Is there a way to force the exporter to respect caching?
      >> >
      >> 
      >> I believe the exporter does respect caching, the following minimal
      >> example worked (i.e. was not re-run) for me on export to html.  Could
      >> you provide an example that demonstrates the problem?
      >
      > Eric, here's an example where the exporter does not respect caching. - Tom
      >
      > #+BABEL: :session *R* :results output :exports both :cache yes
      >
      > * A test of caching
      >
      > #+begin_src R 
      >   cat("random result:", runif(1), "\n")
      >   Sys.sleep(2)
      >   alarm()
      > #+end_src 
      >
      > #+results[b2549fac8a1ec2923ae289d47ce55fb2853dd1de]:
      > : random result: 0.2799064
      >
      > #+begin_src R 
      >   cat("random result:", runif(1), "\n")
      >   Sys.sleep(2)
      >   alarm()
      > #+end_src 
      >
      > #+results[b2549fac8a1ec2923ae289d47ce55fb2853dd1de]:
      > : random result: 0.1625634
      >
      > ** cache on export
      > do we export cached blocks
      >
      > #+begin_src emacs-lisp :cache yes :exports results
      >   (random)
      > #+end_src
      >
      > #+results[46632b4fe2e3a23e847953c95adcba58c270b381]:
      > : 490528137
      >
      >
      >
      >
      > _______________________________________________
      > Emacs-orgmode mailing list
      > Please use `Reply All' to send replies to the list.
      > Emacs-orgmode@gnu.org
      > http://lists.gnu.org/mailman/listinfo/emacs-orgmode
      
      ^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2010-09-19 17:17 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-06-17  6:55 [babel] exports, caching, remote execution Austin Frank
2010-06-17 14:42 ` Eric Schulte
2010-06-17 19:29   ` Austin Frank
2010-06-18 15:35   ` David Maus
2010-08-19 20:59   ` Tom Short
2010-08-25 18:04     ` Eric Schulte
2010-09-19  2:11     ` Eric Schulte

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).