From mboxrd@z Thu Jan 1 00:00:00 1970 From: Nick Dokos Subject: Re: Bug? R: Org babel block execution *drastically* slower than in ESS session directly Date: Wed, 31 Oct 2012 18:53:08 -0400 Message-ID: <3477.1351723988@alphaville> References: <874nlappb1.fsf@tajo.ucsd.edu> <878vam1jvh.fsf@tajo.ucsd.edu> Reply-To: nicholas.dokos@hp.com Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([208.118.235.92]:40677) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TTh9y-0001BG-Gy for emacs-orgmode@gnu.org; Wed, 31 Oct 2012 18:53:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TTh9w-00024D-OL for emacs-orgmode@gnu.org; Wed, 31 Oct 2012 18:53:18 -0400 Received: from g6t0185.atlanta.hp.com ([15.193.32.62]:13277) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TTh9w-00022B-J4 for emacs-orgmode@gnu.org; Wed, 31 Oct 2012 18:53:16 -0400 In-Reply-To: Message from John Hendy of "Wed\, 31 Oct 2012 15\:23\:53 CDT." List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: John Hendy Cc: emacs-orgmode@gnu.org, cberry@tajo.ucsd.edu John Hendy wrote: > On Wed, Oct 31, 2012 at 3:12 PM, wrote: >=20 > John Hendy writes: >=20=20=20=20 > > On Wed, Oct 31, 2012 at 11:41 AM, =C2=A0 wrote: > > John Hendy writes: > > > >> I edited the subject to be more concise/clear.I let orgmode chug a= way > >> on reading in some ~10-30mb csv files for nearly 30min. > > > > [rest deleted] > > > > You need an ECM.I did my best to provide one, other than the file, = which I offered to provide > if others requested that I upload it somewhere. Since you have done s= o, so have I: > > -=C2=A0https://docs.google.com/open?id=3D0BzQupOSnvw08WHdabHh5VVczR= GM >=20=20=20=20 > > Let me know if that doesn't work. I put it on Google docs and s= ometimes have issues with > the sharing settings... >=20=20=20=20 > Not an ECM in my book, but ... >=20 > What else would you like? I provided: > - the config > - the data > - how to [attempt to] reproduce > - the org-mode text >=20 Smaller set of data I'd guess :-) But it does not seem to be the size of the data that matters. > =C2=A0 >=20 > On my 4 year old MacBook: >=20=20=20=20 > ,---- > | > | #+PROPERTY: session *R* > | > | #+name: bigcsv > | #+begin_src R > | bigcsv <- Sys.glob("~/Downloads/*.csv") > | #+end_src > | > | #+RESULTS: bigcsv > | : /Users/cberry/Downloads/test-file.csv > | > | #+name: readbig > | #+begin_src R :results output > | =C2=A0 system.time( > | =C2=A0 =C2=A0 tmp <- read.csv(bigcsv) > | =C2=A0 =C2=A0 ) > | > | #+end_src > | > | #+RESULTS: readbig > | : =C2=A0 =C2=A0user =C2=A0system elapsed > | : =C2=A0 5.679 =C2=A0 0.306 =C2=A0 6.002 > | > `---- >=20=20=20=20 > About the same as running from ESS. >=20 > Not sure what to say. Looking for ways to troubleshoot or confirm. Since = you can't confirm, any > suggestions on where I should look for my issue? I can't explain it! All = I know is that org chugs > and chugs and the direct execution in ESS session is lightning fast. >=20 A few things to try in no particular order: o run top (or whatever equivalent is available on your OS) and see whether the CPU (or one of the CPUs) gets pegged at 100% utilization and stays there. If yes, that's an indication of an infinite loop somewhere. o run vmstat (or equivalent) and see if any of the counters are out of whac= k. That requires some experience though. o use elp-instrument-package to instrument org and run the test, getting a profile. I'm not sure whether the results will be useful, since you are going to interrupt the test when you run out of patience, but it cannot hurt and it might tell you something useful. o run your ECM on a different computer/OS/emacs installation. Being able to compare things side by side is often very useful. o Halve your file and run the test on each half (but that's probably not the problem given Chuck's results). o Reinstall org from scratch - you might have some corruption in one of the compiled files that's causing it to go into an infinite loop. o Turn on debug-on-quit, start your test, wait a bit and then interrupt it. Check the backtrace. Do it again and check whether the backtrace looks the same. That's often an indication of an infinite loop (inferring an infinite loop from a two element sample is statistically suspect of course, but surprisingly effective nevertheless). The point here is that the infinite loop is in emacs and the backtrace tells you something about the parties involved. These are obviously not independent and the results of one experiment will have to guide you in what you try next. Good luck, Nick