From mboxrd@z Thu Jan 1 00:00:00 1970 From: tsd@tsdye.com (Thomas S. Dye) Subject: Re: Bug? R: Org babel block execution *drastically* slower than in ESS session directly Date: Wed, 31 Oct 2012 10:56:16 -1000 Message-ID: References: <874nlappb1.fsf@tajo.ucsd.edu> <878vam1jvh.fsf@tajo.ucsd.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([208.118.235.92]:53874) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TTfL9-0004cZ-Dk for emacs-orgmode@gnu.org; Wed, 31 Oct 2012 16:56:44 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TTfL7-0003df-PM for emacs-orgmode@gnu.org; Wed, 31 Oct 2012 16:56:43 -0400 Received: from oproxy9.bluehost.com ([69.89.24.6]:42593) by eggs.gnu.org with smtp (Exim 4.71) (envelope-from ) id 1TTfL7-0003d9-Et for emacs-orgmode@gnu.org; Wed, 31 Oct 2012 16:56:41 -0400 In-Reply-To: (John Hendy's message of "Wed, 31 Oct 2012 15:23:53 -0500") List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: John Hendy Cc: emacs-orgmode@gnu.org, cberry@tajo.ucsd.edu Hi John, Have you tried wrapping your R read in system.time()? If you are right about :results silent eating up lots of time, then this should fix the problem. system.time yields just a bit of output, so shouldn't slow things down if writing out the data is indeed the problem as you suspect.=20 hth, Tom John Hendy writes: > On Wed, Oct 31, 2012 at 3:12 PM, wrote: > >> John Hendy writes: >> >> > On Wed, Oct 31, 2012 at 11:41 AM, > cberry@tajo.ucsd.edu> wrote: >> > John Hendy writes: >> > >> >> I edited the subject to be more concise/clear.I let orgmode chug away >> >> on reading in some ~10-30mb csv files for nearly 30min. >> > >> > [rest deleted] >> > >> > You need an ECM.I did my best to provide one, other than the file, whi= ch >> I offered to provide if others requested that I upload it somewhere. Sin= ce >> you have done so, so have I: >> > - https://docs.google.com/open?id=3D0BzQupOSnvw08WHdabHh5VVczRGM >> >> > Let me know if that doesn't work. I put it on Google docs and >> sometimes have issues with the sharing settings... >> >> Not an ECM in my book, but ... >> >> > What else would you like? I provided: > - the config > - the data > - how to [attempt to] reproduce > - the org-mode text > > > >> On my 4 year old MacBook: >> >> ,---- >> | >> | #+PROPERTY: session *R* >> | >> | #+name: bigcsv >> | #+begin_src R >> | bigcsv <- Sys.glob("~/Downloads/*.csv") >> | #+end_src >> | >> | #+RESULTS: bigcsv >> | : /Users/cberry/Downloads/test-file.csv >> | >> | #+name: readbig >> | #+begin_src R :results output >> | system.time( >> | tmp <- read.csv(bigcsv) >> | ) >> | >> | #+end_src >> | >> | #+RESULTS: readbig >> | : user system elapsed >> | : 5.679 0.306 6.002 >> | >> `---- >> >> About the same as running from ESS. >> >> > Not sure what to say. Looking for ways to troubleshoot or confirm. Since > you can't confirm, any suggestions on where I should look for my issue? I > can't explain it! All I know is that org chugs and chugs and the direct > execution in ESS session is lightning fast. > > > > FYI, >> >> > dim(tmp) >> [1] 964805 9 >> >> >> > Was that just to show the size, or is something wrong with my file? > > >> Chuck >> >> > > Thanks for attempting to replicate! > John > > >> [ rest deleted ] >> >> >> >> > On Wed, Oct 31, 2012 at 3:12 PM, wrote: > > John Hendy writes: >=20=20=20=20=20 > > On Wed, Oct 31, 2012 at 11:41 AM, =C2=A0 dir=3D"ltr"> wrote: > > John Hendy writes: > > > >> I edited the subject to be more concise/clear.I let orgmode > chug away > >> on reading in some ~10-30mb csv files for nearly 30min. > > > > [rest deleted] > > > > You need an ECM.I did my best to provide one, other than the > file, which I offered to provide if others requested that I upload > it somewhere. Since you have done so, so have I: > > -=C2=A0https://docs.google.com/open?id=3D0BzQupOSnvw08WHdabHh5VVczR= GM >=20=20=20=20=20 > > Let me know if that doesn't work. I put it on Google docs > and sometimes have issues with the sharing settings... >=20=20=20=20=20 > Not an ECM in my book, but ... >=20=20=20=20=20 >=20=20=20=20=20 > > What else would you like? I provided: > - the config > - the data > - how to [attempt to] reproduce > - the org-mode text > > =C2=A0 > > On my 4 year old MacBook: >=20=20=20=20=20 > ,---- > | > | #+PROPERTY: session *R* > | > | #+name: bigcsv > | #+begin_src R > | bigcsv <- Sys.glob("~/Downloads/*.csv") > | #+end_src > | > | #+RESULTS: bigcsv > | : /Users/cberry/Downloads/test-file.csv > | > | #+name: readbig > | #+begin_src R :results output > | =C2=A0 system.time( > | =C2=A0 =C2=A0 tmp <- read.csv(bigcsv) > | =C2=A0 =C2=A0 ) > | > | #+end_src > | > | #+RESULTS: readbig > | : =C2=A0 =C2=A0user =C2=A0system elapsed > | : =C2=A0 5.679 =C2=A0 0.306 =C2=A0 6.002 > | > `---- >=20=20=20=20=20 > About the same as running from ESS. >=20=20=20=20=20 >=20=20=20=20=20 > > Not sure what to say. Looking for ways to troubleshoot or confirm. > Since you can't confirm, any suggestions on where I should look for my > issue? I can't explain it! All I know is that org chugs and chugs and > the direct execution in ESS session is lightning fast. > > > > FYI, >=20=20=20=20=20 > > dim(tmp) > [1] 964805 =C2=A0 =C2=A0 =C2=A09 >=20=20=20=20=20 >=20=20=20=20=20 >=20=20=20=20=20 > > Was that just to show the size, or is something wrong with my file? > =C2=A0 > > Chuck >=20=20=20=20=20 >=20=20=20=20=20 > > > Thanks for attempting to replicate! > John > =C2=A0 > > [ rest deleted ] >=20=20=20=20=20 >=20=20=20=20=20 >=20=20=20=20=20 >=20=20=20=20=20 > --=20 Thomas S. Dye http://www.tsdye.com