From: John Kitchin <jkitchin@andrew.cmu.edu>
To: Christoph Groth <christoph@grothesque.org>,
"emacs-orgmode@gnu.org" <emacs-orgmode@gnu.org>
Subject: Re: babel and long-running computations
Date: Sun, 20 Apr 2014 09:21:09 -0400 [thread overview]
Message-ID: <CAJ51EToheB2YgZff2=nU8U_gXXQ2meoZG=B+eAqi2czYhwxo=Q@mail.gmail.com> (raw)
In-Reply-To: <874n1pouqd.fsf@grothesque.org>
[-- Attachment #1: Type: text/plain, Size: 1861 bytes --]
On Sat, Apr 19, 2014 at 5:49 PM, Christoph Groth
<christoph@grothesque.org>wrote:
> The following message is a courtesy copy of an article
> that has been posted to gmane.emacs.orgmode as well.
>
> Thank you, John, for your detailed reply.
>
> > we routinely do this, in the following way. We run jobs that may take
> > up to a week to finish, and they are usually run on a cluster. Our
> > setup relies on the following behavior for a script.
> >
> > 1. you can run the script anytime you want, and it can tell the state
> > of the calculation by some means. If the script has never been run
> > before, it submits the job to a queue and exits. If the job is still
> > in the queue, it exits, and if the job is done, it gives you the
> > result.
>
> Returning immediately with whatever state the long-running computation
> is in currently seems indeed to be a good solution. I think I will
> setup something similar. Would you share your experience on the
> following issues?
>
> - How do you interface such jobs from orgmode? With org-babel, do
> you execute Python code, or do you run shell commands?
>
We just have code blocks in org-mode. They are usually python blocks, but
we can also do shell, emacs-lisp, etc... Anything that can run a system
command, and get the output will do.
>
> - Do you run your Emacs on the master node of the cluster? Or does your
> setup involve running emacs on the machine you are working on and
> talking to the cluster over the network?
>
Currently, we run emacs on the master node. Once upon a time I had a
sophisticated ssh setup that would allow me to do this on a local machine,
rsync the necessary files to the cluster, ssh some commands to run the
jobs, and then when they were done to rsync the files back. It was pretty
sweet, but I have stopped used and maintaining it.
>
> Cheers,
> Christoph
>
[-- Attachment #2: Type: text/html, Size: 2614 bytes --]
prev parent reply other threads:[~2014-04-20 13:21 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-04-17 8:45 babel and long-running computations Christoph Groth
2014-04-17 9:47 ` Julian Gehring
2014-04-17 10:52 ` Christoph Groth
2014-04-17 12:06 ` Thorsten Jolitz
2014-04-17 12:45 ` Christoph Groth
2014-04-17 14:58 ` Waldemar Quevedo
2014-04-17 15:29 ` Eric Schulte
2014-04-17 16:29 ` Christoph Groth
2014-04-17 16:44 ` Nick Dokos
2014-04-17 15:33 ` Eric S Fraga
2014-04-20 13:29 ` John Kitchin
2014-04-18 13:20 ` John Kitchin
2014-04-19 21:49 ` Christoph Groth
2014-04-20 13:21 ` John Kitchin [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAJ51EToheB2YgZff2=nU8U_gXXQ2meoZG=B+eAqi2czYhwxo=Q@mail.gmail.com' \
--to=jkitchin@andrew.cmu.edu \
--cc=christoph@grothesque.org \
--cc=emacs-orgmode@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).