emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed

* org-babel order of evaluation
@ 2012-01-12  0:25 Rick Frankel
  2012-01-12 14:43 ` Eric Schulte
  0 siblings, 1 reply; 8+ messages in thread
From: Rick Frankel @ 2012-01-12  0:25 UTC (permalink / raw)
  To: emacs-orgmode

There is a problem with the order of execution of interspersed source
and call blocks will not be executed in order because of the way
org-babel-execute-buffer is written (first all the source blocks, then
all the call blocks). 

Therefore, when executing an entire buffer, there is no way to have
the execution of a call block dependent on the prior execution of a
source block.

Given the following:

	#+name: one(v="one")
	#+begin_src elisp
	v
	#+end_src

	#+call: one("two")

	#+name: three(v="three")
	#+begin_src elisp
	v
	#+end_src

The message buffer shows:

	executing Elisp code block (one)...

	(v (quote "one"))

	Code block evaluation complete.
	executing Elisp code block (three)...

	(v (quote "three"))

	Code block evaluation complete.
	executing Elisp code block (one)...

	(v (quote "two"))

	"two"
	executing Emacs-Lisp code block...

	(results (quote "two"))

^ permalink raw reply	[flat|nested] 8+ messages in thread

• * Re: org-babel order of evaluation
    2012-01-12  0:25 org-babel order of evaluation Rick Frankel
  @ 2012-01-12 14:43 ` Eric Schulte
    2012-01-12 22:35   ` Leo Alekseyev
    0 siblings, 1 reply; 8+ messages in thread
  From: Eric Schulte @ 2012-01-12 14:43 UTC (permalink / raw)
    To: emacs-orgmode
  
  Rick Frankel <rick@rickster.com> writes:
  
  > There is a problem with the order of execution of interspersed source
  > and call blocks will not be executed in order because of the way
  > org-babel-execute-buffer is written (first all the source blocks, then
  > all the call blocks). 
  >
  > Therefore, when executing an entire buffer, there is no way to have
  > the execution of a call block dependent on the prior execution of a
  > source block.
  >
  
  It would be better to make the dependency explicit by passing the
  results of the call line as a (potentially unused) variable to the code
  block.  For example;
  
    #+name: three(v="three")
    #+begin_src elisp :var foo=one("two")
    v
    #+end_src
  
  There is (at least currently) no guarantee that evaluation order will be
  buffer order.
  
  -- 
  Eric Schulte
  http://cs.unm.edu/~eschulte/
  
  ^ permalink raw reply	[flat|nested] 8+ messages in thread

• • * Re: org-babel order of evaluation
      2012-01-12 14:43 ` Eric Schulte
    @ 2012-01-12 22:35   ` Leo Alekseyev
      2012-01-12 22:51     ` Rick Frankel
      0 siblings, 1 reply; 8+ messages in thread
    From: Leo Alekseyev @ 2012-01-12 22:35 UTC (permalink / raw)
      To: Emacs orgmode
    
    >> Therefore, when executing an entire buffer, there is no way to have
    >> the execution of a call block dependent on the prior execution of a
    >> source block.
    >>
    >
    > It would be better to make the dependency explicit by passing the
    > results of the call line as a (potentially unused) variable to the code
    > block.  For example;
    [snip]
    >
    > There is (at least currently) no guarantee that evaluation order will be
    > buffer order.
    
    I've been extremely confused by this in the past; this should be
    prominently documented.  In the long run, I would like to see this
    behavior changed.  One would intuitively expect all the source code in
    the file to be evaluated in order.  This is how it works in pretty
    much any other interpreter, why should org-babel be different?
    
    (I'm a big fan of the principle of least surprise, and this behavior
    violates it with vengeance :)  )
    
    This is particularly nasty because many users start by treating an
    org-babel file as a fancier version of the original source code with
    nice annotations and outline levels; typically in a single language.
    Thus, operationally, there isn't a distinction between tangling the
    blocks into a single source file and feeding that to the interpreter
    and running execute on the whole buffer.  But then, of course, one
    might start using named blocks, variables, and #+call directives.  It
    achieves the same effect as writing wrapper functions (or issuing
    statements like source("somefile")) in the original language.  So,
    when it results in a completely different execution order, it's a huge
    surprise.
    
    Even if this can be fixed by putting dummy dependencies in by hand,
    this fix  seems inelegant and hacky.
    
    Is there some deep rationale for the current behavior that I'm not
    seeing?  Are there big obstacles to enforcing ligeral execution order?
    
    --Leo
    
    ^ permalink raw reply	[flat|nested] 8+ messages in thread

  • • * Re: org-babel order of evaluation
        2012-01-12 22:35   ` Leo Alekseyev
      @ 2012-01-12 22:51     ` Rick Frankel
        2012-01-13  1:07       ` Eric Schulte
        0 siblings, 1 reply; 8+ messages in thread
      From: Rick Frankel @ 2012-01-12 22:51 UTC (permalink / raw)
        To: emacs-orgmode
      
      On Thu, Jan 12, 2012 at 04:35:31PM -0600, Leo Alekseyev wrote:
      > >> Therefore, when executing an entire buffer, there is no way to have
      > >> the execution of a call block dependent on the prior execution of a
      > >> source block.
      > >>
      > >
      > > It would be better to make the dependency explicit by passing the
      > > results of the call line as a (potentially unused) variable to the code
      > > block.  For example;
      > [snip]
      
      The problem w/ this is that the (potentially time consuming) dependent
      will be executed twice when doing a buffer eval.
      
      > > There is (at least currently) no guarantee that evaluation order will be
      > > buffer order.
      > 
      > Is there some deep rationale for the current behavior that I'm not
      > seeing?  Are there big obstacles to enforcing ligeral execution order?
      
      It's because prior to 7.8, call blocks were not executed during a
      buffer execute. The solution was to execute all the call blocks after
      executing the src block. (Eric would have to comment on how hard it
      would be to merge the two functions :).
      
      rick
      
      ^ permalink raw reply	[flat|nested] 8+ messages in thread

    • • * Re: org-babel order of evaluation
          2012-01-12 22:51     ` Rick Frankel
        @ 2012-01-13  1:07       ` Eric Schulte
          2012-01-13  1:52         ` Rick Frankel
          0 siblings, 1 reply; 8+ messages in thread
        From: Eric Schulte @ 2012-01-13  1:07 UTC (permalink / raw)
          To: emacs-orgmode
        
        [-- Attachment #1: Type: text/plain, Size: 1379 bytes --]
        
        Rick Frankel <rick@rickster.com> writes:
        
        > On Thu, Jan 12, 2012 at 04:35:31PM -0600, Leo Alekseyev wrote:
        >> >> Therefore, when executing an entire buffer, there is no way to have
        >> >> the execution of a call block dependent on the prior execution of a
        >> >> source block.
        >> >>
        >> >
        >> > It would be better to make the dependency explicit by passing the
        >> > results of the call line as a (potentially unused) variable to the code
        >> > block.  For example;
        >> [snip]
        >
        > The problem w/ this is that the (potentially time consuming) dependent
        > will be executed twice when doing a buffer eval.
        >
        >> > There is (at least currently) no guarantee that evaluation order will be
        >> > buffer order.
        >> 
        >> Is there some deep rationale for the current behavior that I'm not
        >> seeing?  Are there big obstacles to enforcing ligeral execution order?
        >
        > It's because prior to 7.8, call blocks were not executed during a
        > buffer execute. The solution was to execute all the call blocks after
        > executing the src block. (Eric would have to comment on how hard it
        > would be to merge the two functions :).
        >
        
        Turns out it was not that difficult to change this behavior.  You and
        Leo are both correct that in-buffer-order evaluation is more natural and
        expected than the previous behavior.  I've just pushed up a fix after
        which evaluating the following
        
        
        [-- Warning: decoded text below may be mangled, UTF-8 assumed --]
        [-- Attachment #2: execute-in-order.org --]
        [-- Type: text/x-org, Size: 211 bytes --]
        
        #+Title: Execute all executables in Order
        #+Property: results silent
        
        #+name: foo
        #+BEGIN_SRC sh :var it="one"
          echo $it >> debug
        #+END_SRC
        
        #+call: foo("two")
        
        #+BEGIN_SRC sh
          echo "three" >> debug
        #+END_SRC
        
        [-- Attachment #3: Type: text/plain, Size: 196 bytes --]
        
        
        results in the creation of a "debug" file in the same directory reading;
        
        ,----
        | one
        | two
        | three
        `----
        
        Thanks for bringing this up,
        
        >
        > rick
        >
        
        -- 
        Eric Schulte
        http://cs.unm.edu/~eschulte/
        
        ^ permalink raw reply	[flat|nested] 8+ messages in thread

      • • * Re: org-babel order of evaluation
            2012-01-13  1:07       ` Eric Schulte
          @ 2012-01-13  1:52         ` Rick Frankel
            2012-01-20 17:58           ` Leo Alekseyev
            0 siblings, 1 reply; 8+ messages in thread
          From: Rick Frankel @ 2012-01-13  1:52 UTC (permalink / raw)
            To: emacs-orgmode
          
          On Thu, Jan 12, 2012 at 06:07:41PM -0700, Eric Schulte wrote:
          > Rick Frankel <rick@rickster.com> writes:
          > 
          > Turns out it was not that difficult to change this behavior.  You and
          > Leo are both correct that in-buffer-order evaluation is more natural and
          > expected than the previous behavior.  I've just pushed up a fix after
          > which evaluating the following
          > 
          
          Brillant! thank you Eric. This saves me much pain.
          
          rick
          
          ^ permalink raw reply	[flat|nested] 8+ messages in thread

        • • * Re: org-babel order of evaluation
              2012-01-13  1:52         ` Rick Frankel
            @ 2012-01-20 17:58           ` Leo Alekseyev
              2012-01-23 17:56             ` Eric Schulte
              0 siblings, 1 reply; 8+ messages in thread
            From: Leo Alekseyev @ 2012-01-20 17:58 UTC (permalink / raw)
              To: emacs-orgmode
            
            [-- Attachment #1: Type: text/plain, Size: 1125 bytes --]
            
            On Thu, Jan 12, 2012 at 7:52 PM, Rick Frankel <rick@rickster.com> wrote:
            > On Thu, Jan 12, 2012 at 06:07:41PM -0700, Eric Schulte wrote:
            >> Rick Frankel <rick@rickster.com> writes:
            >>
            >> Turns out it was not that difficult to change this behavior.  You and
            >> Leo are both correct that in-buffer-order evaluation is more natural and
            >> expected than the previous behavior.  I've just pushed up a fix after
            >> which evaluating the following
            
            Eric,
            The fix doesn't seem to be working for me when I export the buffer to
            HTML.  The ordering of call and source blocks once again becomes
            randomized, and in general, exported file is missing a bunch of stuff
            unless I run org-babel-execute-buffer prior to export.  Since the
            export engine does its own evaluation, it doesn't seem like
            org-babel-execute-buffer should be a necessity.  But I can't run
            org-babel-execute-buffer on anything with a src_<language> inline
            block as it gives me an error.
            
            I'm attaching two files which do not export correctly, at least when
            one doesn't run org-babel-execute-buffer; just do C-c C-e h and look
            at the output.
            
            --Leo
            
            [-- Attachment #2: test-export4.org --]
            [-- Type: application/octet-stream, Size: 1101 bytes --]
            
            #+property: session *R-babel*
            
            * export fail
            
            #+NAME: elo_sim_analysis
            #+HEADER: :var idx=1 :var print="TRUE" :var plot.filename="conv1.png" :var plot.disp="FALSE"
            #+BEGIN_SRC R :results output replace 
              if (print == TRUE) {cat("some output\n")}
              
              img.dir = "images"
              cat.fname.link <- function() { cat(img.dir,"/",plot.filename,"\n",sep="") } 
              
              cat(img.dir,"/",plot.filename,"\n",sep="")
              
            #+END_SRC
            
            
            Should have all conv1 stuff
            
            #+call: elo_sim_analysis(idx=1,print="TRUE",plot.filename="conv1.png",plot.disp="FALSE")
            
            src_R[:results output replace]{ cat.fname.link() }
            
            Should have all conv2 stuff 
            
            #+call: elo_sim_analysis(idx=8,print="TRUE",plot.filename="conv2.png",plot.disp="FALSE")
            
            src_R[:results output replace]{ cat.fname.link() } 
            
            Should have all conv3 stuff
            #+call: elo_sim_analysis(idx=1,print="TRUE",plot.filename="conv3.png",plot.disp="FALSE")
            
            src_R[:results output replace]{ cat.fname.link() } 
            
            Should have all conv4 stuff
            
            #+call: elo_sim_analysis(idx=1,print="TRUE",plot.filename="conv4.png",plot.disp="FALSE")
            
            src_R[:results output replace]{ cat.fname.link() } 
            
            
            
            [-- Attachment #3: test-export6.org --]
            [-- Type: application/octet-stream, Size: 1573 bytes --]
            
            
            #+startup: noindent
            #+property: session *R-babel*
            #+property: tangle yes
            
            * code
            
            #+NAME: elo_sim_analysis
            #+HEADER: :var idx=1 :var print="TRUE" :var plot.filename="conv1.png" :var plot.disp="FALSE"
            #+BEGIN_SRC R :results output replace 
              gen.sample.plots <- function(idx) {
                hist(rnorm(100))
                title(paste("Plot for index ",idx, sep=""))
              }
              
              if (print == TRUE) {cat("some output\n")}
              
              img.dir = "images"
              cat.fname.link <- function() { cat("\nfile:",img.dir,"/",plot.filename,"\n",sep="") } 
              
              cat("file:",img.dir,"/",plot.filename,"\n",sep="")
              
            #+END_SRC
            
            #+RESULTS: elo_sim_analysis
            :  some output
            :  file:images/conv1.png
            
            #+NAME: gen_link
            #+HEADER: :var plot.filename="conv1.png" 
            #+BEGIN_SRC R :results output replace 
              img.dir = "images"
              cat.fname.link <- function() { cat("\nfile:",img.dir,"/",plot.filename,"\n",sep="") } 
              cat.fname.link()
            #+END_SRC
            
            * ordered execution fail
            
            Should have all conv1 stuff
            
            #+call: elo_sim_analysis(idx=1,print="TRUE",plot.filename="conv1.png",plot.disp="FALSE")
            
            #+call: gen_link(plot.filename="conv1.png")
            
            Should have all conv2 stuff 
            
            #+call: elo_sim_analysis(idx=8,print="TRUE",plot.filename="conv2.png",plot.disp="FALSE")
            
            #+call: gen_link(plot.filename="conv2.png")
            
            Should have all conv3 stuff
            
            #+call: elo_sim_analysis(idx=1,print="TRUE",plot.filename="conv3.png",plot.disp="FALSE")
            
            #+call: gen_link(plot.filename="conv3.png")
            
            Should have all conv4 stuff
            
            #+call: elo_sim_analysis(idx=1,print="TRUE",plot.filename="conv4.png",plot.disp="FALSE")
            
            #+call: gen_link(plot.filename="conv4.png")
            
            
            
            
            ^ permalink raw reply	[flat|nested] 8+ messages in thread

          • • * Re: org-babel order of evaluation
                2012-01-20 17:58           ` Leo Alekseyev
              @ 2012-01-23 17:56             ` Eric Schulte
                0 siblings, 0 replies; 8+ messages in thread
              From: Eric Schulte @ 2012-01-23 17:56 UTC (permalink / raw)
                To: Leo Alekseyev; +Cc: emacs-orgmode
              
              Leo Alekseyev <dnquark@gmail.com> writes:
              
              > On Thu, Jan 12, 2012 at 7:52 PM, Rick Frankel <rick@rickster.com> wrote:
              >> On Thu, Jan 12, 2012 at 06:07:41PM -0700, Eric Schulte wrote:
              >>> Rick Frankel <rick@rickster.com> writes:
              >>>
              >>> Turns out it was not that difficult to change this behavior.  You and
              >>> Leo are both correct that in-buffer-order evaluation is more natural and
              >>> expected than the previous behavior.  I've just pushed up a fix after
              >>> which evaluating the following
              >
              > Eric,
              > The fix doesn't seem to be working for me when I export the buffer to
              > HTML.
              
              My previous fix only set the order of evaluation for interactive buffer
              evaluation.  I've just pushed up another fix which sets the order of
              evaluation during export.
              
              Best,
              
              > The ordering of call and source blocks once again becomes randomized,
              > and in general, exported file is missing a bunch of stuff unless I run
              > org-babel-execute-buffer prior to export.  Since the export engine
              > does its own evaluation, it doesn't seem like org-babel-execute-buffer
              > should be a necessity.  But I can't run org-babel-execute-buffer on
              > anything with a src_<language> inline block as it gives me an error.
              >
              > I'm attaching two files which do not export correctly, at least when
              > one doesn't run org-babel-execute-buffer; just do C-c C-e h and look
              > at the output.
              >
              > --Leo
              >
              >
              
              -- 
              Eric Schulte
              http://cs.unm.edu/~eschulte/
              
              ^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2012-01-23 18:09 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-01-12  0:25 org-babel order of evaluation Rick Frankel
2012-01-12 14:43 ` Eric Schulte
2012-01-12 22:35   ` Leo Alekseyev
2012-01-12 22:51     ` Rick Frankel
2012-01-13  1:07       ` Eric Schulte
2012-01-13  1:52         ` Rick Frankel
2012-01-20 17:58           ` Leo Alekseyev
2012-01-23 17:56             ` Eric Schulte

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).