From mboxrd@z Thu Jan 1 00:00:00 1970 From: John Hendy Subject: Re: Bug? R: Org babel block execution *drastically* slower than in ESS session directly Date: Wed, 31 Oct 2012 12:18:55 -0500 Message-ID: References: <874nlappb1.fsf@tajo.ucsd.edu> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=e89a8fb1ff2cf19bfd04cd5e18e2 Return-path: Received: from eggs.gnu.org ([208.118.235.92]:49987) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TTbwU-0008Ek-8w for emacs-orgmode@gnu.org; Wed, 31 Oct 2012 13:19:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TTbwP-0007mH-7I for emacs-orgmode@gnu.org; Wed, 31 Oct 2012 13:19:02 -0400 Received: from mail-oa0-f41.google.com ([209.85.219.41]:54018) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TTbwP-0007l4-1W for emacs-orgmode@gnu.org; Wed, 31 Oct 2012 13:18:57 -0400 Received: by mail-oa0-f41.google.com with SMTP id k14so1897178oag.0 for ; Wed, 31 Oct 2012 10:18:56 -0700 (PDT) In-Reply-To: <874nlappb1.fsf@tajo.ucsd.edu> List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: cberry@tajo.ucsd.edu Cc: emacs-orgmode@gnu.org --e89a8fb1ff2cf19bfd04cd5e18e2 Content-Type: text/plain; charset=UTF-8 On Wed, Oct 31, 2012 at 11:41 AM, wrote: > John Hendy writes: > > > I edited the subject to be more concise/clear.I let orgmode chug away > > on reading in some ~10-30mb csv files for nearly 30min. > > [rest deleted] > > You need an ECM. > I did my best to provide one, other than the file, which I offered to provide if others requested that I upload it somewhere. Since you have done so, so have I: - https://docs.google.com/open?id=0BzQupOSnvw08WHdabHh5VVczRGM Let me know if that doesn't work. I put it on Google docs and sometimes have issues with the sharing settings... You will find the rest needed for a complete ECM of my setup in the original email: - minimal config I used with `emacs -Q --load=file` - org file with minimal babel R block Let me know if you can reproduce with the linked file. I'm open to suggestions on how to bisect a file, though it's a column of strings and several columns of numbers, I'm not exactly sure what to look for. Best regards, John > > I cannot reproduce your issue. > > This runs in the same amount of time, whether I execute the src block or > run the code from ESS: > > > #+begin_src R :session > write.csv(diag(10)[rep(1:10,100000),],file="abc.csv") > #+end_src > > #+begin_src R :session :results output > system.time(read.csv("abc.csv")) > #+end_src > > > abc.csv is >25MB > > If the behavior you describe only happens with some files, I suggest you > try to bisect them to find the issue. > > HTH, > > Chuck > > > --e89a8fb1ff2cf19bfd04cd5e18e2 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

On Wed, Oct 31, 2012 at 11:41 AM, <= cberry@tajo.ucsd.edu> wrote:
jw.hendy@gmail.com= > writes:

> I edited the subject to be more concise/clear.I let orgmode chug away<= br>
> on reading in some ~10-30mb csv files for nearly 30m= in.

[rest deleted]

You need an ECM.

I did my best to provi= de one, other than the file, which I offered to provide if others requested= that I upload it somewhere. Since you have done so, so have I:
-=C2=A0https://docs.google.com/open?id=3D0BzQupOSnvw08WHdabHh5VVczRGM

Let me know if that doesn't work. I put it on G= oogle docs and sometimes have issues with the sharing settings...

You will find the rest needed for a complete ECM of my = setup in the original email:
- minimal config I used with `emacs = -Q --load=3Dfile`
- org file with minimal babel R block

Let me know if you can reproduce with the linked file. I'= ;m open to suggestions on how to bisect a file, though it's a column of= strings and several columns of numbers, I'm not exactly sure what to l= ook for.


Best regards,
John
= =C2=A0

I cannot reproduce your issue.

This runs in the same amount of time, whether I execute the src block or run the code from ESS:


#+begin_src R :session
write.csv(diag(10)[rep(1:10,100000),],file=3D"abc.csv")
#+end_src

#+begin_src R :session :results output
=C2=A0 system.time(read.csv("abc.csv"))
#+end_src


abc.csv is >25MB

If the behavior you describe only happens with some files, I suggest you try to bisect them to find the issue.

HTH,

Chuck



--e89a8fb1ff2cf19bfd04cd5e18e2--