Re: Bug? R: Org babel block execution *drastically* slower than in ESS session directly

[tsd@tsdye.com (Thomas S. Dye)]

Hi John,

Have you tried wrapping your R read in system.time()?  If you are right
about :results silent eating up lots of time, then this should fix the
problem.  system.time yields just a bit of output, so shouldn't slow
things down if writing out the data is indeed the problem as you
suspect. 

hth,
Tom


John Hendy <jw.hendy@gmail.com> writes:

> On Wed, Oct 31, 2012 at 3:12 PM, <cberry@tajo.ucsd.edu> wrote:
>
>> John Hendy <jw.hendy@gmail.com> writes:
>>
>> > On Wed, Oct 31, 2012 at 11:41 AM,  <span dir="ltr"><mailto:
>> cberry@tajo.ucsd.edu></span> wrote:
>> > John Hendy <mailto:jw.hendy@gmail.com> writes:
>> >
>> >> I edited the subject to be more concise/clear.I let orgmode chug away
>> >> on reading in some ~10-30mb csv files for nearly 30min.
>> >
>> > [rest deleted]
>> >
>> > You need an ECM.I did my best to provide one, other than the file, which
>> I offered to provide if others requested that I upload it somewhere. Since
>> you have done so, so have I:
>> > - https://docs.google.com/open?id=0BzQupOSnvw08WHdabHh5VVczRGM
>>
>> > Let me know if that doesn&#39;t work. I put it on Google docs and
>> sometimes have issues with the sharing settings...
>>
>> Not an ECM in my book, but ...
>>
>>
> What else would you like? I provided:
> - the config
> - the data
> - how to [attempt to] reproduce
> - the org-mode text
>
>
>
>> On my 4 year old MacBook:
>>
>> ,----
>> |
>> | #+PROPERTY: session *R*
>> |
>> | #+name: bigcsv
>> | #+begin_src R
>> | bigcsv <- Sys.glob("~/Downloads/*.csv")
>> | #+end_src
>> |
>> | #+RESULTS: bigcsv
>> | : /Users/cberry/Downloads/test-file.csv
>> |
>> | #+name: readbig
>> | #+begin_src R :results output
>> |   system.time(
>> |     tmp <- read.csv(bigcsv)
>> |     )
>> |
>> | #+end_src
>> |
>> | #+RESULTS: readbig
>> | :    user  system elapsed
>> | :   5.679   0.306   6.002
>> |
>> `----
>>
>> About the same as running from ESS.
>>
>>
> Not sure what to say. Looking for ways to troubleshoot or confirm. Since
> you can't confirm, any suggestions on where I should look for my issue? I
> can't explain it! All I know is that org chugs and chugs and the direct
> execution in ESS session is lightning fast.
>
>
>
> FYI,
>>
>> > dim(tmp)
>> [1] 964805      9
>>
>>
>>
> Was that just to show the size, or is something wrong with my file?
>
>
>> Chuck
>>
>>
>
> Thanks for attempting to replicate!
> John
>
>
>> [ rest deleted ]
>>
>>
>>
>>
> On Wed, Oct 31, 2012 at 3:12 PM, <cberry@tajo.ucsd.edu> wrote:
>
>     John Hendy <jw.hendy@gmail.com> writes:
>     
>     > On Wed, Oct 31, 2012 at 11:41 AM,  <span
>     dir="ltr"><mailto:cberry@tajo.ucsd.edu></span> wrote:
>     > John Hendy <mailto:jw.hendy@gmail.com> writes:
>     >
>     >> I edited the subject to be more concise/clear.I let orgmode
>     chug away
>     >> on reading in some ~10-30mb csv files for nearly 30min.
>     >
>     > [rest deleted]
>     >
>     > You need an ECM.I did my best to provide one, other than the
>     file, which I offered to provide if others requested that I upload
>     it somewhere. Since you have done so, so have I:
>     > - https://docs.google.com/open?id=0BzQupOSnvw08WHdabHh5VVczRGM
>     
>     > Let me know if that doesn&#39;t work. I put it on Google docs
>     and sometimes have issues with the sharing settings...
>     
>     Not an ECM in my book, but ...
>     
>     
>
> What else would you like? I provided:
> - the config
> - the data
> - how to [attempt to] reproduce
> - the org-mode text
>
>  
>
>     On my 4 year old MacBook:
>     
>     ,----
>     |
>     | #+PROPERTY: session *R*
>     |
>     | #+name: bigcsv
>     | #+begin_src R
>     | bigcsv <- Sys.glob("~/Downloads/*.csv")
>     | #+end_src
>     |
>     | #+RESULTS: bigcsv
>     | : /Users/cberry/Downloads/test-file.csv
>     |
>     | #+name: readbig
>     | #+begin_src R :results output
>     |   system.time(
>     |     tmp <- read.csv(bigcsv)
>     |     )
>     |
>     | #+end_src
>     |
>     | #+RESULTS: readbig
>     | :    user  system elapsed
>     | :   5.679   0.306   6.002
>     |
>     `----
>     
>     About the same as running from ESS.
>     
>     
>
> Not sure what to say. Looking for ways to troubleshoot or confirm.
> Since you can't confirm, any suggestions on where I should look for my
> issue? I can't explain it! All I know is that org chugs and chugs and
> the direct execution in ESS session is lightning fast.
>
>
>
>     FYI,
>     
>     > dim(tmp)
>     [1] 964805      9
>     
>     
>     
>
> Was that just to show the size, or is something wrong with my file?
>  
>
>     Chuck
>     
>     
>
>
> Thanks for attempting to replicate!
> John
>  
>
>     [ rest deleted ]
>     
>     
>     
>     
>

-- 
Thomas S. Dye
http://www.tsdye.com