From mboxrd@z Thu Jan 1 00:00:00 1970 From: John Hendy Subject: Bug? R: Org babel block execution *drastically* slower than in ESS session directly Date: Wed, 31 Oct 2012 01:12:38 -0500 Message-ID: Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=e89a8f3b9ec71d9ed404cd54cad1 Return-path: Received: from eggs.gnu.org ([208.118.235.92]:49347) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TTRXe-0007pr-3i for emacs-orgmode@gnu.org; Wed, 31 Oct 2012 02:12:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TTRXc-0002IW-1h for emacs-orgmode@gnu.org; Wed, 31 Oct 2012 02:12:42 -0400 Received: from mail-oa0-f41.google.com ([209.85.219.41]:35546) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TTRXb-0002Hp-PL for emacs-orgmode@gnu.org; Wed, 31 Oct 2012 02:12:39 -0400 Received: by mail-oa0-f41.google.com with SMTP id k14so1209568oag.0 for ; Tue, 30 Oct 2012 23:12:38 -0700 (PDT) List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: emacs-orgmode --e89a8f3b9ec71d9ed404cd54cad1 Content-Type: text/plain; charset=UTF-8 I edited the subject to be more concise/clear. I let orgmode chug away on reading in some ~10-30mb csv files for nearly 30min. I finally gave up with Ctrl + g. Out of curiosity, I copied my babel block code (four simple lines in the form of `var <- read.csv("./file.csv", header = T, sep = ",")` into the ESS session already started by my first block execution, and it finished in less than 10 seconds. Something seems seriously wrong with however org + babel is executing. I'm suspicious of it printing the output to the minibuffer. I have about 200^2 - 400^2 lines in the csv files. Was shocked that I'd been waiting for org this whole time and could have just run the code in the ESS R session directly in a minuscule fraction of the time. I could post a reproducible csv file if others would like to try this? I tried the following to make sure it wasn't my setup: 1) from terminal: `emacs -Q --load=~/Desktop/basic-config`, where basic-config contains: ;; set load paths ;; set load dirs and global config options (add-to-list 'load-path "~/.elisp/site-lisp/") (add-to-list 'load-path "~/.elisp/org.git/contrib/lisp/") (add-to-list 'load-path "~/.elisp/org.git/lisp/") (add-to-list 'load-path "~/.elisp/site-lisp/ess/lisp/") ;; setup org (require 'org-install) ;; setup babel languages (require 'ess-site) (org-babel-do-load-languages 'org-babel-load-languages '((R . t))) 2) Created a simple org file: ---------- * headline #+begin_src R :session r :results silent data <- read.csv("/media/vault/misc/benefit-plan-analysis/2012/contour_table.csv") #+end_src ---------- 3) Run =C-c C-c YES RET= on block 4) Change to R session and enter the same line manually. The results are unbelievably different. I didn't let the org session finish as once it took longer than 30sec I figured I'd get the same results as with my actual file in this example. Changing to the R session and it was done in less than 5sec with a ~70mb csv. Any suggestions on troubleshooting this? Best regards, John On Tue, Oct 30, 2012 at 2:46 AM, John Hendy wrote: > I'm working with a large dataset. I try to minimize overhead by doing > some bigger data rearranging and calculations in a single block that I > execute once per session and then switch :eval to "no." > > This block is set with :results silent, but I know with this option > the results are still printed to the minibuffer. Usually this isn't a > problem, but I've had some hangs with emacs and started wondering if > this could be due to Orgmode regurgitating the results. Could this be > the case? If so, could Org be made to stop this behavior? > > If I accidentally type the name of a data frame in the R session > buffer, even when I use C-c C-c to stop the print out, it still takes > quite a while to catch up and stop itself. It made me wonder if trying > to print all of those results is increasing my compilation times > unnecessarily. > > > Thanks for any thoughts, > John > --e89a8f3b9ec71d9ed404cd54cad1 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I edited the subject to be more concise/clear.

I let org= mode chug away on reading in some ~10-30mb csv files for nearly 30min. I fi= nally gave up with Ctrl + g.=C2=A0Out of curiosity, I copied my babel block= code (four simple lines in the form of `var <- read.csv("./file.cs= v", header =3D T, sep =3D ",")` into the ESS session already= started by my first block execution, and it finished in less than 10 secon= ds.

Something seems seriously wrong with however org + babe= l is executing. I'm suspicious of it printing the output to the minibuf= fer. I have about 200^2 - 400^2 lines in the csv files. Was shocked that I&= #39;d been waiting for org this whole time and could have just run the code= in the ESS R session directly in a=C2=A0minuscule fraction of the time.

I could post a reproducible csv file if others would li= ke to try this?

I tried the following to make sure= it wasn't my setup:

1) from terminal: `emacs = -Q --load=3D~/Desktop/basic-config`, where basic-config contains:

;; set load paths
;; set load dirs and g= lobal config options
(add-to-list 'load-path "~/.elisp/s= ite-lisp/")
(add-to-list 'load-path "~/.elisp/org.g= it/contrib/lisp/")
(add-to-list 'load-path "~/.elisp/org.git/lisp/")
<= div>(add-to-list 'load-path "~/.elisp/site-lisp/ess/lisp/")

;; setup org
(require 'org-install)

;; setup babel languages
(require 'ess-si= te)
(org-babel-do-load-languages
=C2=A0'org-babel-l= oad-languages
=C2=A0'((R . t)))

2) Created a simple org file:

----------
* headline

#+begin_src R :session r :results silent
data <- read.c= sv("/media/vault/misc/benefit-plan-analysis/2012/contour_table.csv&quo= t;)
#+end_src
----------

3) Run = =3DC-c C-c YES RET=3D on block

4) Change to R sess= ion and enter the same line manually.

The results = are unbelievably different. I didn't let the org session finish as once= it took longer than 30sec I figured I'd get the same results as with m= y actual file in this example. Changing to the R session and it was done in= less than 5sec with a ~70mb csv.

Any suggestions on troubleshooting this?

=

Best regards,
John=C2=A0

On Tue, Oct 30, 2012 at 2:46 AM, John Hendy <jw.hen= dy@gmail.com> wrote:
I'm working with a large dataset. I try = to minimize overhead by doing
some bigger data rearranging and calculations in a single block that I
execute once per session and then switch :eval to "no."

This block is set with :results silent, but I know with this option
the results are still printed to the minibuffer. Usually this isn't a problem, but I've had some hangs with emacs and started wondering if this could be due to Orgmode regurgitating the results. Could this be
the case? If so, could Org be made to stop this behavior?

If I accidentally type the name of a data frame in the R session
buffer, even when I use C-c C-c =C2=A0to stop the print out, it still takes=
quite a while to catch up and stop itself. It made me wonder if trying
to print all of those results is increasing my compilation times
unnecessarily.


Thanks for any thoughts,
John

--e89a8f3b9ec71d9ed404cd54cad1--