From mboxrd@z Thu Jan  1 00:00:00 1970
From: Christoph Groth <christoph@grothesque.org>
Subject: Re: babel and long-running computations
Date: Thu, 17 Apr 2014 12:52:36 +0200
Message-ID: <87lhv4ut1n.fsf@grothesque.org>
References: <87lhv4qr7l.fsf@grothesque.org> <534FA332.1080304@gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Return-path: <emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org>
Received: from eggs.gnu.org ([2001:4830:134:3::10]:55826)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <geo-emacs-orgmode@m.gmane.org>) id 1WajwB-0002wh-2a
	for emacs-orgmode@gnu.org; Thu, 17 Apr 2014 06:53:06 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <geo-emacs-orgmode@m.gmane.org>) id 1Wajw3-0000tQ-8R
	for emacs-orgmode@gnu.org; Thu, 17 Apr 2014 06:52:59 -0400
Received: from plane.gmane.org ([80.91.229.3]:39665)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <geo-emacs-orgmode@m.gmane.org>) id 1Wajw3-0000tM-2O
	for emacs-orgmode@gnu.org; Thu, 17 Apr 2014 06:52:51 -0400
Received: from list by plane.gmane.org with local (Exim 4.69)
	(envelope-from <geo-emacs-orgmode@m.gmane.org>) id 1Wajw1-0005y7-Ny
	for emacs-orgmode@gnu.org; Thu, 17 Apr 2014 12:52:49 +0200
Received: from dra38-5-82-246-248-175.fbx.proxad.net ([82.246.248.175])
	by main.gmane.org with esmtp (Gmexim 0.1 (Debian))
	id 1AlnuQ-0007hv-00
	for <emacs-orgmode@gnu.org>; Thu, 17 Apr 2014 12:52:49 +0200
Received: from christoph by dra38-5-82-246-248-175.fbx.proxad.net with local
	(Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00
	for <emacs-orgmode@gnu.org>; Thu, 17 Apr 2014 12:52:49 +0200
List-Id: "General discussions about Org-mode." <emacs-orgmode.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/emacs-orgmode>,
	<mailto:emacs-orgmode-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/emacs-orgmode>
List-Post: <mailto:emacs-orgmode@gnu.org>
List-Help: <mailto:emacs-orgmode-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/emacs-orgmode>,
	<mailto:emacs-orgmode-request@gnu.org?subject=subscribe>
Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org
Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org
To: emacs-orgmode@gnu.org

Hi Julian,

> Generally, org-babel is suited for long running computations.  Its
> caching can help you avoid rerunning code chunks.  And long runtime does
> not conflict with the idea of reproducible research, it just may be not
> very comfortable for the user.

I agree of course that it’s generally a good idea to structure the
analysis in small logical steps and to save intermediate results.  Let
me narrow down my question then: does org-babel support working with
tasks that take several minutes to execute?  By this I mean (most
important first):

- Not freezing the editor during the execution of a task
- Being able to execute multiple independent tasks in parallel
- Being able to interrupt a running task
- Being able to inspect the incomplete output of a running task

I’d love to hear about any frameworks or workflows that fulfill these
requirements.

Cheers

Christoph


PS.

I’m using ipython notebooks but I’m not happy with them because of the
freezing problem and the complete lack of isolation of tasks within a
single notebook (they live in a common mutable namespace).  I think a
useful framework must be “functional” at the highest level for caching
and dependencies to be useful.