Emacs/ESS/org freezes/hangs on big data/ RAM(~256GB) processes when run in org/babel

[Xebar Saram <zeltakc@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1633 bytes --]

Hi all

im not sure if this is absolutely the correct forum to raise this and if
not would be also happy to get input on where to persue this issue.

i recently dived into using orgmode, ESS, Babel etc to run code and im
really love it. The problem i have (and im not sure if its a org,Emacs or
ess issue) is that emacs sometimes (and mostly when dealing with R
processes involving HUGE databases) will just hang/freeze.

I do alot of modeling work that involves using huge datasets and run
process intensive R processes (such as complex mixed models, Gamms etc). in
R studio all works well yet when i use the orgmode eval on R code blocks it
works well for small simple process but 90% of the time when dealing with
complex models and bug data (up to 256GB) it will just freeze emacs/ess.
sometimes i can C-c or C-g it and other times i need to physically kill
emacs.

here is an example of such process that hangs


**** lmer
run the lmer part regressing stage 2 pred Vs mean pm

#+BEGIN_SRC R  :session Rorg  :results none
m2.smooth = lme(pred.m2 ~ meanPM25,random = list(aodid= ~1 +
meanPM25),control=lmeControl(opt = "optim"), data= mod2 )
#correlate to see everything from mod2 and the mpm works
mod2[, pred.t31 := predict(m2.smooth)]
mod2[, resid  := residuals(m2.smooth)]
print(summary(lm(pred.m2~pred.t31,data=mod2))$r.squared)
#+END_SRC

i usually issue org-babel-execute-subtree to eval several subsections under
a main header.

again i dont know if its an org mode isse perse but would love to hear from
people that have experience using R/org with big data/RAM and maybe point
me to where to raise these issues

best

Z

[-- Attachment #2: Type: text/html, Size: 2005 bytes --]