emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
From: Nick Dokos <nicholas.dokos@hp.com>
To: "Thomas S. Dye" <tsd@tsdye.com>
Cc: cberry@tajo.ucsd.edu, emacs-orgmode@gnu.org
Subject: Re: Bug? R: Org babel block execution *drastically* slower than in ESS session directly
Date: Wed, 31 Oct 2012 19:00:20 -0400	[thread overview]
Message-ID: <3596.1351724420@alphaville> (raw)
In-Reply-To: Message from tsd@tsdye.com (Thomas S. Dye) of "Wed\, 31 Oct 2012 10\:56\:16 -1000." <m1txta743z.fsf@tsdye.com>

Thomas S. Dye <tsd@tsdye.com> wrote:

> Hi John,
> 
> Have you tried wrapping your R read in system.time()?  If you are right
> about :results silent eating up lots of time, then this should fix the
> problem.  system.time yields just a bit of output, so shouldn't slow
> things down if writing out the data is indeed the problem as you
> suspect. 
> 

I missed this in the thread so I thought I'd add one more thing to try:

o run iostat (or equivalent) to check how your disks are doing.
  on linux, iostat is part of a package usually called sysstat.

Nick

> 
> 
> John Hendy <jw.hendy@gmail.com> writes:
> 
> > On Wed, Oct 31, 2012 at 3:12 PM, <cberry@tajo.ucsd.edu> wrote:
> >
> >> John Hendy <jw.hendy@gmail.com> writes:
> >>
> >> > On Wed, Oct 31, 2012 at 11:41 AM,  <span dir="ltr"><mailto:
> >> cberry@tajo.ucsd.edu></span> wrote:
> >> > John Hendy <mailto:jw.hendy@gmail.com> writes:
> >> >
> >> >> I edited the subject to be more concise/clear.I let orgmode chug away
> >> >> on reading in some ~10-30mb csv files for nearly 30min.
> >> >
> >> > [rest deleted]
> >> >
> >> > You need an ECM.I did my best to provide one, other than the file, which
> >> I offered to provide if others requested that I upload it somewhere. Since
> >> you have done so, so have I:
> >> > - https://docs.google.com/open?id=0BzQupOSnvw08WHdabHh5VVczRGM
> >>
> >> > Let me know if that doesn&#39;t work. I put it on Google docs and
> >> sometimes have issues with the sharing settings...
> >>
> >> Not an ECM in my book, but ...
> >>
> >>
> > What else would you like? I provided:
> > - the config
> > - the data
> > - how to [attempt to] reproduce
> > - the org-mode text
> >
> >
> >
> >> On my 4 year old MacBook:
> >>
> >> ,----
> >> |
> >> | #+PROPERTY: session *R*
> >> |
> >> | #+name: bigcsv
> >> | #+begin_src R
> >> | bigcsv <- Sys.glob("~/Downloads/*.csv")
> >> | #+end_src
> >> |
> >> | #+RESULTS: bigcsv
> >> | : /Users/cberry/Downloads/test-file.csv
> >> |
> >> | #+name: readbig
> >> | #+begin_src R :results output
> >> |   system.time(
> >> |     tmp <- read.csv(bigcsv)
> >> |     )
> >> |
> >> | #+end_src
> >> |
> >> | #+RESULTS: readbig
> >> | :    user  system elapsed
> >> | :   5.679   0.306   6.002
> >> |
> >> `----
> >>
> >> About the same as running from ESS.
> >>
> >>
> > Not sure what to say. Looking for ways to troubleshoot or confirm. Since
> > you can't confirm, any suggestions on where I should look for my issue? I
> > can't explain it! All I know is that org chugs and chugs and the direct
> > execution in ESS session is lightning fast.
> >
> >
> >
> > FYI,
> >>
> >> > dim(tmp)
> >> [1] 964805      9
> >>
> >>
> >>
> > Was that just to show the size, or is something wrong with my file?
> >
> >
> >> Chuck
> >>
> >>
> >
> > Thanks for attempting to replicate!
> > John
> >
> >
> >> [ rest deleted ]
> >>
> >>
> >>
> >>
> > On Wed, Oct 31, 2012 at 3:12 PM, <cberry@tajo.ucsd.edu> wrote:
> >
> >     John Hendy <jw.hendy@gmail.com> writes:
> >     
> >     > On Wed, Oct 31, 2012 at 11:41 AM,  <span
> >     dir="ltr"><mailto:cberry@tajo.ucsd.edu></span> wrote:
> >     > John Hendy <mailto:jw.hendy@gmail.com> writes:
> >     >
> >     >> I edited the subject to be more concise/clear.I let orgmode
> >     chug away
> >     >> on reading in some ~10-30mb csv files for nearly 30min.
> >     >
> >     > [rest deleted]
> >     >
> >     > You need an ECM.I did my best to provide one, other than the
> >     file, which I offered to provide if others requested that I upload
> >     it somewhere. Since you have done so, so have I:
> >     > - https://docs.google.com/open?id=0BzQupOSnvw08WHdabHh5VVczRGM
> >     
> >     > Let me know if that doesn&#39;t work. I put it on Google docs
> >     and sometimes have issues with the sharing settings...
> >     
> >     Not an ECM in my book, but ...
> >     
> >     
> >
> > What else would you like? I provided:
> > - the config
> > - the data
> > - how to [attempt to] reproduce
> > - the org-mode text
> >
> >  
> >
> >     On my 4 year old MacBook:
> >     
> >     ,----
> >     |
> >     | #+PROPERTY: session *R*
> >     |
> >     | #+name: bigcsv
> >     | #+begin_src R
> >     | bigcsv <- Sys.glob("~/Downloads/*.csv")
> >     | #+end_src
> >     |
> >     | #+RESULTS: bigcsv
> >     | : /Users/cberry/Downloads/test-file.csv
> >     |
> >     | #+name: readbig
> >     | #+begin_src R :results output
> >     |   system.time(
> >     |     tmp <- read.csv(bigcsv)
> >     |     )
> >     |
> >     | #+end_src
> >     |
> >     | #+RESULTS: readbig
> >     | :    user  system elapsed
> >     | :   5.679   0.306   6.002
> >     |
> >     `----
> >     
> >     About the same as running from ESS.
> >     
> >     
> >
> > Not sure what to say. Looking for ways to troubleshoot or confirm.
> > Since you can't confirm, any suggestions on where I should look for my
> > issue? I can't explain it! All I know is that org chugs and chugs and
> > the direct execution in ESS session is lightning fast.
> >
> >
> >
> >     FYI,
> >     
> >     > dim(tmp)
> >     [1] 964805      9
> >     
> >     
> >     
> >
> > Was that just to show the size, or is something wrong with my file?
> >  
> >
> >     Chuck
> >     
> >     
> >
> >
> > Thanks for attempting to replicate!
> > John
> >  
> >
> >     [ rest deleted ]
> >     
> >     
> >     
> >     
> >
> 
> -- 
> Thomas S. Dye
> http://www.tsdye.com
> 

  reply	other threads:[~2012-10-31 23:00 UTC|newest]

Thread overview: 217+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-10-31  6:12 Bug? R: Org babel block execution *drastically* slower than in ESS session directly John Hendy
2012-10-31 16:41 ` cberry
2012-10-31 17:18   ` John Hendy
2012-10-31 20:12     ` cberry
2012-10-31 20:23       ` John Hendy
2012-10-31 20:56         ` Thomas S. Dye
2012-10-31 23:00           ` Nick Dokos [this message]
2012-10-31 22:53         ` Nick Dokos
2012-11-01 14:53           ` John Hendy
2012-11-01 15:38             ` Nick Dokos
2012-11-01 18:17               ` John Hendy
2012-11-01 18:18               ` John Hendy
2012-11-01 18:48                 ` Nick Dokos
2012-11-14  3:27                   ` Aaron Ecay
2012-11-14  4:52                     ` John Hendy
2012-11-14  6:32                       ` Aaron Ecay
2012-11-14  8:28                         ` Thomas S. Dye
2012-11-14  8:35                           ` Andreas Leha
2012-11-16 15:45                             ` Eric Schulte
2012-11-16 17:47                               ` Andreas Leha
2012-11-16 19:00                                 ` John Hendy
2012-11-16 20:25                                 ` Achim Gratz
2012-11-17 17:12                                   ` Eric Schulte
2012-11-18  0:41                               ` Aaron Ecay
2012-11-18  0:57                                 ` Eric Schulte
2012-11-18  1:41                                   ` Aaron Ecay
2012-11-19  1:18                                     ` Eric Schulte
2012-11-19  5:11                                       ` Eric Schulte
  -- strict thread matches above, loose matches on Subject: below --
2013-03-19 21:40 python sessions Gary Oberbrunner
2013-03-20  0:04 ` John Hendy
2013-03-20  3:07   ` Nick Dokos
2013-03-20  6:44     ` Andreas Röhler
2013-03-20 12:54       ` Ista Zahn
2013-03-20 14:08         ` Nick Dokos
2013-03-20 14:28           ` Gary Oberbrunner
2013-03-20 17:21         ` Andreas Röhler
2013-03-20  7:08   ` Andreas Röhler
2013-03-20 15:38     ` John Hendy
2013-03-20 15:42       ` John Hendy
2013-03-20 16:06       ` Nick Dokos
     [not found]         ` <CAFChFygsUeBejEY6m2bSU0C5cjLYz0nUADH=s+BYxajfZ2sKKQ@mail.gmail.com>
2013-03-20 16:12           ` Fwd: " Gary Oberbrunner
2013-03-20 17:08         ` Andreas Röhler
2013-03-20 17:25           ` Nick Dokos
2013-03-20 17:54             ` John Hendy
2013-03-20 18:15               ` Nick Dokos
2013-03-20 18:32                 ` Ista Zahn
2013-03-20 18:39                   ` Nick Dokos
2013-03-20 18:43                   ` Andreas Röhler
2013-03-20 19:42                     ` Ista Zahn
2013-03-20 19:53                       ` Andreas Röhler
2013-03-20 20:12                         ` Ista Zahn
2013-03-20 20:25                           ` Gary Oberbrunner
2013-03-20 22:06                             ` Ista Zahn
2013-03-20 18:42                 ` Andreas Röhler
2013-03-20 19:01             ` Andreas Röhler
     [not found]       ` <CAFChFygPrhuHHyMo2Vk2_OdksWVB=oACA7jnOXN+itUNqh-nqw@mail.gmail.com>
     [not found]         ` <CA+M2ft9bO7DU61vVeYrhduVvkg1V6PMuiyvsOy5tNWuXcSUEpw@mail.gmail.com>
     [not found]           ` <CAFChFyhbeXE4HNLKA6Ubii+MtQwD7=4khAypa+--coZrQiJekA@mail.gmail.com>
2013-03-21  1:22             ` Gary Oberbrunner
2013-03-21  1:54               ` John Hendy
2013-03-21  1:56                 ` Gary Oberbrunner
2013-03-21  7:42                   ` Andreas Röhler
2013-03-21  7:43                     ` Bastien
2013-03-21  8:13                       ` Andreas Röhler
2013-03-23 22:07                         ` Eric Schulte
2013-03-24  1:29                           ` John Hendy
2013-03-24 22:59                             ` Eric Schulte
2013-03-25  2:38                               ` Nick Dokos
2013-03-25  2:59                                 ` John Hendy
2013-03-25  6:34                                   ` Andreas Röhler
2013-03-25 15:40                                   ` Eric Schulte
2013-03-25 15:55                                     ` John Hendy
2013-03-25 16:07                                       ` Eric Schulte
2013-03-25 16:41                                         ` Nick Dokos
2013-03-25 16:01                                     ` Ista Zahn
2013-03-25 16:23                                       ` John Hendy
2013-03-25 16:43                                         ` Eric Schulte
2013-03-25 17:27                                           ` Andreas Röhler
2013-03-25 17:41                                             ` John Hendy
2013-03-25 18:16                                               ` Ista Zahn
2013-03-25 19:30                                             ` Ivan Andrus
2013-03-24  7:47                           ` Andreas Röhler
2013-03-24 14:47                             ` John Hendy
2013-03-24 15:30                               ` Andreas Röhler
2013-03-24 16:58                               ` Nick Dokos
2013-03-24 16:59                                 ` John Hendy
2013-03-24 18:41                                   ` Nick Dokos
2013-03-25 20:46                                     ` Andreas Röhler
2013-03-25 21:37                                       ` Eric Schulte
2013-03-26  6:29                                         ` Andreas Röhler
2013-03-26 12:32                                           ` Eric Schulte
2013-03-26 21:41                                             ` Gary Oberbrunner
2013-03-26 21:47                                               ` John Hendy
2013-03-27  6:19                                               ` Andreas Röhler
2013-03-30  7:19                                                 ` Andreas Röhler
2012-08-26 20:11 Alternate format for datetree c b
2012-08-27  1:16 ` John Hendy
2012-08-27  4:04   ` Nick Dokos
2012-08-27  6:08     ` John Hendy
2012-08-28  8:49       ` Ian Barton
2012-08-28 12:49         ` Nick Dokos
2012-08-28 12:50         ` Nick Dokos
2012-08-29  7:49           ` Ian Barton
     [not found]             ` <lists@manor-farm.org>
2012-08-29 14:17               ` Nick Dokos
2012-08-29 14:25             ` John Hendy
2012-08-29 20:01               ` Ian Barton
2012-09-06  7:46                 ` Ian Barton
2012-09-06  5:42       ` c b
2012-09-06  6:26         ` Nick Dokos
2012-09-06  7:00         ` Jambunathan K
2012-09-06 15:33         ` John Hendy
2012-09-06 16:08           ` Jonathan Leech-Pepin
2011-11-02  2:22 Pass LaTeX exporter option prior to \documentclass John Hendy
2011-11-02  2:43 ` suvayu ali
2011-11-02 10:30   ` suvayu ali
2011-11-03 17:48     ` John Hendy
2011-11-03 17:55       ` suvayu ali
2011-11-03 18:07         ` John Hendy
2011-11-04 17:29           ` Nick Dokos
2011-11-04 19:29             ` suvayu ali
2011-10-19 14:54 Prompt for time when clocking in? Nathan Neff
2011-10-19 15:12 ` John Hendy
2011-10-19 15:35   ` Nick Dokos
2011-10-20 16:15     ` Nathan Neff
2011-10-20 17:09       ` Nick Dokos
2011-10-20 17:44         ` Nathan Neff
2011-10-19 16:37 ` Bernt Hansen
2011-08-05 22:15 Handling errors in command line exporting of agenda? John Hendy
2011-08-05 23:34 ` suvayu ali
2011-08-05 23:35   ` suvayu ali
2011-08-08 18:12   ` John Hendy
2011-08-08 18:28     ` suvayu ali
2011-08-08 18:53       ` John Hendy
2011-08-08 18:59         ` suvayu ali
2011-08-08 19:48           ` John Hendy
2011-08-08 20:22             ` suvayu ali
2011-08-08 20:31               ` John Hendy
2011-08-08 20:33               ` Nick Dokos
2011-08-08 20:37             ` Nick Dokos
2011-08-08 20:54               ` John Hendy
2011-08-08 21:09                 ` Nick Dokos
2011-08-08 21:27                   ` John Hendy
2011-08-08 21:30               ` suvayu ali
2011-08-08 22:17                 ` Nick Dokos
2011-08-08 21:46             ` suvayu ali
2011-06-15 10:29 BUG: Newest org-mode interferes with desktop save Rainer M Krug
2011-06-15 16:49 ` Eric Schulte
2011-06-15 17:35   ` Nick Dokos
2011-06-15 18:23     ` Rainer M Krug
2011-06-15 18:21   ` Rainer M Krug
2011-06-15 18:31     ` Eric Schulte
2011-06-15 18:59       ` Rainer M Krug
2011-06-15 22:40         ` Nick Dokos
2011-06-16  7:15           ` Rainer M Krug
2011-06-16  7:29             ` Jambunathan K
2011-06-16  8:22               ` Rainer M Krug
2011-05-24 15:54 Passing font size to exported LaTeX table John Hendy
2011-05-24 16:44 ` Sebastien Vauban
2011-05-24 18:57   ` John Hendy
2011-05-24 19:25     ` Nick Dokos
2011-05-24 19:31       ` John Hendy
2011-05-24 19:58         ` Thomas S. Dye
2011-05-24 20:42           ` Nick Dokos
2011-05-24 21:12             ` Nick Dokos
2011-05-24 21:19               ` Nick Dokos
2011-05-25  7:22                 ` Thomas S. Dye
2011-05-27  0:58                   ` suvayu ali
2011-05-27  6:46                     ` Thomas S. Dye
2011-05-27  7:07                       ` suvayu ali
2011-05-27 16:17                         ` Thomas S. Dye
2011-05-27 16:50                           ` Suvayu Ali
2011-05-27 17:37                             ` Thomas S. Dye
2011-05-27 18:30                               ` Suvayu Ali
2011-05-27 21:25                                 ` Thomas S. Dye
2011-05-28  7:17                                   ` Sebastien Vauban
2011-05-28 10:14                                     ` Suvayu Ali
2011-05-28 15:21                                       ` Nick Dokos
2011-05-28 18:10                                         ` Suvayu Ali
2011-06-01 18:50                                   ` Nick Dokos
2011-06-01 19:08                                     ` Suvayu Ali
2011-06-10 22:31                                       ` Suvayu Ali
2011-06-13  1:38                                         ` suvayu ali
2011-06-13  1:52                                         ` Nick Dokos
2011-06-13  2:40                                           ` Suvayu Ali
2011-06-13  3:29                                             ` Nick Dokos
2011-06-13  3:45                                           ` Thomas S. Dye
2011-06-13  4:27                                             ` Nick Dokos
2011-06-13  4:56                                               ` Suvayu Ali
2011-06-13  5:29                                                 ` Nick Dokos
2011-06-13  6:42                                                   ` Suvayu Ali
2011-05-24 20:06         ` Nick Dokos
2011-05-24 19:42       ` Sebastien Vauban
2010-08-21 19:30 Gnuplot unevenly spaced non-numeric data plot? John Hendy
2010-08-21 20:35 ` Eric S Fraga
2010-08-21 20:41 ` Nick Dokos
2010-08-21 21:11   ` John Hendy
2010-08-21 22:41     ` John Hendy
2010-08-21 23:17       ` Nick Dokos
2010-08-23 14:22         ` John Hendy
2010-08-23 15:16           ` Nick Dokos
     [not found]             ` <AANLkTi=WTLBL1Giq0GTrjCo-A1s=iP4u1Qxn57cH-xnB@mail.gmail.com>
     [not found]               ` <AANLkTimSVm=-2o39CQ9wSMT276Dee4Tuj9jLKJH-c+cd@mail.gmail.com>
2010-08-24  2:44                 ` [For Worg?] " John Hendy
2010-08-24  5:25                   ` suvayu ali
2010-08-24  5:56               ` Nick Dokos
2010-08-25  0:32                 ` Eric Schulte
2010-08-25  1:06                   ` John Hendy
2010-08-25  1:35                     ` Eric Schulte
2010-08-25 18:38                       ` John Hendy
2010-08-25 18:52                         ` Eric Schulte
     [not found]         ` <jw.hendy@gmail.com>
2012-03-12 21:13           ` Beamer specific setupfile? John Hendy
2012-03-12 21:48             ` Nick Dokos
2012-03-12 22:09               ` John Hendy
2012-03-12 22:30                 ` Nick Dokos
2012-03-12 22:44                   ` Nick Dokos
2012-03-13  5:02                     ` John Hendy
2012-03-13  5:46                       ` Nick Dokos
2012-04-25  8:16                         ` Eric Fraga
2012-04-25  8:59                           ` Eric Fraga
2012-03-12 23:01                 ` Nick Dokos
2012-03-12 21:51             ` Nick Dokos
2012-03-12 22:05             ` Nick Dokos

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://www.orgmode.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3596.1351724420@alphaville \
    --to=nicholas.dokos@hp.com \
    --cc=cberry@tajo.ucsd.edu \
    --cc=emacs-orgmode@gnu.org \
    --cc=tsd@tsdye.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).