From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Charles C. Berry" Subject: Re: Emacs/ESS/org freezes/hangs on big data/ RAM(~256GB) processes when run in org/babel Date: Wed, 17 Jun 2015 20:17:42 -0700 Message-ID: References: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:50752) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z5QKv-0001Dj-LO for emacs-orgmode@gnu.org; Wed, 17 Jun 2015 23:17:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Z5QKp-0000FK-Pi for emacs-orgmode@gnu.org; Wed, 17 Jun 2015 23:17:53 -0400 Received: from iport-acv6-out.ucsd.edu ([132.239.0.13]:9810) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Z5QKp-0000Ec-Gn for emacs-orgmode@gnu.org; Wed, 17 Jun 2015 23:17:47 -0400 In-Reply-To: List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: William Denton Cc: Xebar Saram , org mode On Wed, 17 Jun 2015, William Denton wrote: > On 17 June 2015, Xebar Saram wrote: > >> I do alot of modeling work that involves using huge datasets and run >> process intensive R processes (such as complex mixed models, Gamms etc). in >> R studio all works well yet when i use the orgmode eval on R code blocks it >> works well for small simple process but 90% of the time when dealing with >> complex models and bug data (up to 256GB) it will just freeze emacs/ess. >> sometimes i can C-c or C-g it and other times i need to physically kill >> emacs. > > I've been having the same problem for a while, but wasn't able to isolate it > any more than large data sets, lack of memory, and heavy CPU usage. > Sometimes everything hangs and I need to power cycle the computer. :( > And you (both) have `ess-eval-visibly' set to nil, right? I do statistical genomics, which can be compute intensive. Sometimes processes need to run for a while, and I get impatient having to wait. I wrote (and use) ox-ravel[1] to speed up my write-run-revise cycle in org-mode. Basically, ravel will export Org mode to a format that knitr (and the like) can run - turning src blocks into `code chunks'. That allows me to set the cache=TRUE chunk option, etc. I run knitr on the exported document to initialize objects for long running computations or to produce a finished report. When I start a session, I run knitr in the R session, then all the cached objects are loaded in and ready to use. If I write a src block I know will take a long time to export, I export from org mode to update the knitr document and re-knit it to refresh the cache. Mostly, I work in org-mode adding src blocks, revising existing ones, or editing text and graphics. If you decide to try ravel I recommend the `ravel-lang' branch[2] as that will soon replace master. HTH, Chuck [1] https://github.com/chasberry/orgmode-accessories [2] https://github.com/chasberry/orgmode-accessories/tree/ravel-lang