From: Xebar Saram <zeltakc@gmail.com>
To: "Charles C. Berry" <ccberry@ucsd.edu>, org mode <emacs-orgmode@gnu.org>
Subject: Re: Emacs/ESS/org freezes/hangs on big data/ RAM(~256GB) processes when run in org/babel
Date: Thu, 18 Jun 2015 14:39:18 +0300 [thread overview]
Message-ID: <CAOQHXPqZQis0bYn3G2R=HHEaWs1eH-_fccNTGtX5NPjR+vfkvg@mail.gmail.com> (raw)
In-Reply-To: <alpine.OSX.2.11.1506171920250.537@charles-berrys-macbook.local>
[-- Attachment #1: Type: text/plain, Size: 2361 bytes --]
Thx Chuck
this sounds great. could you perhaps point us to some documentation on
this, or perhpas consider sharing a detailed overview of your workflow?
this seems it could really fit my needs.
thx so much in advance
Z
On Thu, Jun 18, 2015 at 6:17 AM, Charles C. Berry <ccberry@ucsd.edu> wrote:
> On Wed, 17 Jun 2015, William Denton wrote:
>
> On 17 June 2015, Xebar Saram wrote:
>>
>> I do alot of modeling work that involves using huge datasets and run
>>> process intensive R processes (such as complex mixed models, Gamms etc).
>>> in
>>> R studio all works well yet when i use the orgmode eval on R code blocks
>>> it
>>> works well for small simple process but 90% of the time when dealing with
>>> complex models and bug data (up to 256GB) it will just freeze emacs/ess.
>>> sometimes i can C-c or C-g it and other times i need to physically kill
>>> emacs.
>>>
>>
>> I've been having the same problem for a while, but wasn't able to isolate
>> it any more than large data sets, lack of memory, and heavy CPU usage.
>> Sometimes everything hangs and I need to power cycle the computer. :(
>>
>>
> And you (both) have `ess-eval-visibly' set to nil, right?
>
> I do statistical genomics, which can be compute intensive. Sometimes
> processes need to run for a while, and I get impatient having to wait.
>
> I wrote (and use) ox-ravel[1] to speed up my write-run-revise cycle in
> org-mode.
>
> Basically, ravel will export Org mode to a format that knitr (and the
> like) can run - turning src blocks into `code chunks'. That allows me to
> set the cache=TRUE chunk option, etc. I run knitr on the exported document
> to initialize objects for long running computations or to produce a
> finished report.
>
> When I start a session, I run knitr in the R session, then all the cached
> objects are loaded in and ready to use.
>
> If I write a src block I know will take a long time to export, I export
> from org mode to update the knitr document and re-knit it to refresh the
> cache.
>
> Mostly, I work in org-mode adding src blocks, revising existing ones, or
> editing text and graphics.
>
> If you decide to try ravel I recommend the `ravel-lang' branch[2] as that
> will soon replace master.
>
> HTH,
>
> Chuck
>
>
> [1] https://github.com/chasberry/orgmode-accessories
> [2] https://github.com/chasberry/orgmode-accessories/tree/ravel-lang
>
[-- Attachment #2: Type: text/html, Size: 3320 bytes --]
next prev parent reply other threads:[~2015-06-18 11:39 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-06-17 19:09 Emacs/ESS/org freezes/hangs on big data/ RAM(~256GB) processes when run in org/babel Xebar Saram
2015-06-17 19:19 ` William Denton
2015-06-18 3:17 ` Charles C. Berry
2015-06-18 11:39 ` Xebar Saram [this message]
2015-06-18 18:45 ` Charles C. Berry
2015-06-18 12:34 ` Rainer M Krug
2015-06-18 20:20 ` Charles C. Berry
2015-06-18 21:20 ` Andreas Leha
2015-06-19 22:13 ` Charles C. Berry
2015-06-19 22:25 ` Andreas Leha
2015-06-19 22:31 ` Andreas Leha
2015-06-20 15:05 ` Rainer M Krug
2015-06-20 21:20 ` Andreas Leha
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAOQHXPqZQis0bYn3G2R=HHEaWs1eH-_fccNTGtX5NPjR+vfkvg@mail.gmail.com' \
--to=zeltakc@gmail.com \
--cc=ccberry@ucsd.edu \
--cc=emacs-orgmode@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).