From mboxrd@z Thu Jan 1 00:00:00 1970 From: Christoph Groth Subject: Re: babel and long-running computations Date: Sat, 19 Apr 2014 23:49:30 +0200 Message-ID: <874n1pouqd.fsf@grothesque.org> References: <87lhv4qr7l.fsf@grothesque.org> Mime-Version: 1.0 Content-Type: text/plain Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:52051) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Wbd91-0000sq-Gh for emacs-orgmode@gnu.org; Sat, 19 Apr 2014 17:50:03 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Wbd8u-0008Pz-19 for emacs-orgmode@gnu.org; Sat, 19 Apr 2014 17:49:55 -0400 Received: from plane.gmane.org ([80.91.229.3]:33523) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Wbd8t-0008Pr-Ql for emacs-orgmode@gnu.org; Sat, 19 Apr 2014 17:49:47 -0400 Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1Wbd8r-0005NZ-O5 for emacs-orgmode@gnu.org; Sat, 19 Apr 2014 23:49:45 +0200 Received: from dra38-5-82-246-248-175.fbx.proxad.net ([82.246.248.175]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 19 Apr 2014 23:49:45 +0200 Received: from christoph by dra38-5-82-246-248-175.fbx.proxad.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 19 Apr 2014 23:49:45 +0200 List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: emacs-orgmode@gnu.org Cc: John Kitchin Thank you, John, for your detailed reply. > we routinely do this, in the following way. We run jobs that may take > up to a week to finish, and they are usually run on a cluster. Our > setup relies on the following behavior for a script. > > 1. you can run the script anytime you want, and it can tell the state > of the calculation by some means. If the script has never been run > before, it submits the job to a queue and exits. If the job is still > in the queue, it exits, and if the job is done, it gives you the > result. Returning immediately with whatever state the long-running computation is in currently seems indeed to be a good solution. I think I will setup something similar. Would you share your experience on the following issues? - How do you interface such jobs from orgmode? With org-babel, do you execute Python code, or do you run shell commands? - Do you run your Emacs on the master node of the cluster? Or does your setup involve running emacs on the machine you are working on and talking to the cluster over the network? Cheers, Christoph