From mboxrd@z Thu Jan 1 00:00:00 1970 From: Nick Dokos Subject: Re: Bug? R: Org babel block execution *drastically* slower than in ESS session directly Date: Wed, 31 Oct 2012 19:00:20 -0400 Message-ID: <3596.1351724420@alphaville> References: <874nlappb1.fsf@tajo.ucsd.edu> <878vam1jvh.fsf@tajo.ucsd.edu> Reply-To: nicholas.dokos@hp.com Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([208.118.235.92]:51536) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TThGx-0003Yr-Cw for emacs-orgmode@gnu.org; Wed, 31 Oct 2012 19:00:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TThGr-000495-22 for emacs-orgmode@gnu.org; Wed, 31 Oct 2012 19:00:31 -0400 Received: from g4t0014.houston.hp.com ([15.201.24.17]:4583) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TThGq-00048i-QR for emacs-orgmode@gnu.org; Wed, 31 Oct 2012 19:00:24 -0400 In-Reply-To: Message from tsd@tsdye.com (Thomas S. Dye) of "Wed\, 31 Oct 2012 10\:56\:16 -1000." List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: "Thomas S. Dye" Cc: cberry@tajo.ucsd.edu, emacs-orgmode@gnu.org Thomas S. Dye wrote: > Hi John, >=20 > Have you tried wrapping your R read in system.time()? If you are right > about :results silent eating up lots of time, then this should fix the > problem. system.time yields just a bit of output, so shouldn't slow > things down if writing out the data is indeed the problem as you > suspect.=20 >=20 I missed this in the thread so I thought I'd add one more thing to try: o run iostat (or equivalent) to check how your disks are doing. on linux, iostat is part of a package usually called sysstat. Nick >=20 >=20 > John Hendy writes: >=20 > > On Wed, Oct 31, 2012 at 3:12 PM, wrote: > > > >> John Hendy writes: > >> > >> > On Wed, Oct 31, 2012 at 11:41 AM, >> cberry@tajo.ucsd.edu> wrote: > >> > John Hendy writes: > >> > > >> >> I edited the subject to be more concise/clear.I let orgmode chug aw= ay > >> >> on reading in some ~10-30mb csv files for nearly 30min. > >> > > >> > [rest deleted] > >> > > >> > You need an ECM.I did my best to provide one, other than the file, w= hich > >> I offered to provide if others requested that I upload it somewhere. S= ince > >> you have done so, so have I: > >> > - https://docs.google.com/open?id=3D0BzQupOSnvw08WHdabHh5VVczRGM > >> > >> > Let me know if that doesn't work. I put it on Google docs and > >> sometimes have issues with the sharing settings... > >> > >> Not an ECM in my book, but ... > >> > >> > > What else would you like? I provided: > > - the config > > - the data > > - how to [attempt to] reproduce > > - the org-mode text > > > > > > > >> On my 4 year old MacBook: > >> > >> ,---- > >> | > >> | #+PROPERTY: session *R* > >> | > >> | #+name: bigcsv > >> | #+begin_src R > >> | bigcsv <- Sys.glob("~/Downloads/*.csv") > >> | #+end_src > >> | > >> | #+RESULTS: bigcsv > >> | : /Users/cberry/Downloads/test-file.csv > >> | > >> | #+name: readbig > >> | #+begin_src R :results output > >> | system.time( > >> | tmp <- read.csv(bigcsv) > >> | ) > >> | > >> | #+end_src > >> | > >> | #+RESULTS: readbig > >> | : user system elapsed > >> | : 5.679 0.306 6.002 > >> | > >> `---- > >> > >> About the same as running from ESS. > >> > >> > > Not sure what to say. Looking for ways to troubleshoot or confirm. Since > > you can't confirm, any suggestions on where I should look for my issue?= I > > can't explain it! All I know is that org chugs and chugs and the direct > > execution in ESS session is lightning fast. > > > > > > > > FYI, > >> > >> > dim(tmp) > >> [1] 964805 9 > >> > >> > >> > > Was that just to show the size, or is something wrong with my file? > > > > > >> Chuck > >> > >> > > > > Thanks for attempting to replicate! > > John > > > > > >> [ rest deleted ] > >> > >> > >> > >> > > On Wed, Oct 31, 2012 at 3:12 PM, wrote: > > > > John Hendy writes: > >=20=20=20=20=20 > > > On Wed, Oct 31, 2012 at 11:41 AM, =C2=A0 > dir=3D"ltr"> wrote: > > > John Hendy writes: > > > > > >> I edited the subject to be more concise/clear.I let orgmode > > chug away > > >> on reading in some ~10-30mb csv files for nearly 30min. > > > > > > [rest deleted] > > > > > > You need an ECM.I did my best to provide one, other than the > > file, which I offered to provide if others requested that I upload > > it somewhere. Since you have done so, so have I: > > > -=C2=A0https://docs.google.com/open?id=3D0BzQupOSnvw08WHdabHh5VVc= zRGM > >=20=20=20=20=20 > > > Let me know if that doesn't work. I put it on Google docs > > and sometimes have issues with the sharing settings... > >=20=20=20=20=20 > > Not an ECM in my book, but ... > >=20=20=20=20=20 > >=20=20=20=20=20 > > > > What else would you like? I provided: > > - the config > > - the data > > - how to [attempt to] reproduce > > - the org-mode text > > > > =C2=A0 > > > > On my 4 year old MacBook: > >=20=20=20=20=20 > > ,---- > > | > > | #+PROPERTY: session *R* > > | > > | #+name: bigcsv > > | #+begin_src R > > | bigcsv <- Sys.glob("~/Downloads/*.csv") > > | #+end_src > > | > > | #+RESULTS: bigcsv > > | : /Users/cberry/Downloads/test-file.csv > > | > > | #+name: readbig > > | #+begin_src R :results output > > | =C2=A0 system.time( > > | =C2=A0 =C2=A0 tmp <- read.csv(bigcsv) > > | =C2=A0 =C2=A0 ) > > | > > | #+end_src > > | > > | #+RESULTS: readbig > > | : =C2=A0 =C2=A0user =C2=A0system elapsed > > | : =C2=A0 5.679 =C2=A0 0.306 =C2=A0 6.002 > > | > > `---- > >=20=20=20=20=20 > > About the same as running from ESS. > >=20=20=20=20=20 > >=20=20=20=20=20 > > > > Not sure what to say. Looking for ways to troubleshoot or confirm. > > Since you can't confirm, any suggestions on where I should look for my > > issue? I can't explain it! All I know is that org chugs and chugs and > > the direct execution in ESS session is lightning fast. > > > > > > > > FYI, > >=20=20=20=20=20 > > > dim(tmp) > > [1] 964805 =C2=A0 =C2=A0 =C2=A09 > >=20=20=20=20=20 > >=20=20=20=20=20 > >=20=20=20=20=20 > > > > Was that just to show the size, or is something wrong with my file? > > =C2=A0 > > > > Chuck > >=20=20=20=20=20 > >=20=20=20=20=20 > > > > > > Thanks for attempting to replicate! > > John > > =C2=A0 > > > > [ rest deleted ] > >=20=20=20=20=20 > >=20=20=20=20=20 > >=20=20=20=20=20 > >=20=20=20=20=20 > > >=20 > --=20 > Thomas S. Dye > http://www.tsdye.com >=20