emacs-orgmode
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [O] Emacs/ESS/org freezes/hangs on big data/ RAM(~256GB) processes w


From: Charles C. Berry
Subject: Re: [O] Emacs/ESS/org freezes/hangs on big data/ RAM(~256GB) processes when run in org/babel
Date: Wed, 17 Jun 2015 20:17:42 -0700
User-agent: Alpine 2.11 (OSX 23 2013-08-11)

On Wed, 17 Jun 2015, William Denton wrote:

On 17 June 2015, Xebar Saram wrote:

I do alot of modeling work that involves using huge datasets and run
process intensive R processes (such as complex mixed models, Gamms etc). in
R studio all works well yet when i use the orgmode eval on R code blocks it
works well for small simple process but 90% of the time when dealing with
complex models and bug data (up to 256GB) it will just freeze emacs/ess.
sometimes i can C-c or C-g it and other times i need to physically kill
emacs.
I've been having the same problem for a while, but wasn't able to isolate it 
any more than large data sets, lack of memory, and heavy CPU usage. 
Sometimes everything hangs and I need to power cycle the computer. :(
And you (both) have `ess-eval-visibly' set to nil, right?

I do statistical genomics, which can be compute intensive. Sometimes processes need to run for a while, and I get impatient having to wait.
I wrote (and use) ox-ravel[1] to speed up my write-run-revise cycle in 
org-mode.
Basically, ravel will export Org mode to a format that knitr (and the 
like) can run - turning src blocks into `code chunks'. That allows me to 
set the cache=TRUE chunk option, etc. I run knitr on the exported document 
to initialize objects for long running computations or to produce a 
finished report.
When I start a session, I run knitr in the R session, then all the cached 
objects are loaded in and ready to use.
If I write a src block I know will take a long time to export, I export 
from org mode to update the knitr document and re-knit it to refresh the 
cache.
Mostly, I work in org-mode adding src blocks, revising existing ones, or 
editing text and graphics.
If you decide to try ravel I recommend the `ravel-lang' branch[2] as that 
will soon replace master.
HTH,

Chuck


[1] https://github.com/chasberry/orgmode-accessories
[2] https://github.com/chasberry/orgmode-accessories/tree/ravel-lang



reply via email to

[Prev in Thread] Current Thread [Next in Thread]