From mboxrd@z Thu Jan 1 00:00:00 1970 From: Eric Lubeck Subject: Re: Using org-mode for laboratory notes. Date: Wed, 19 Sep 2012 20:18:13 -0700 Message-ID: References: <3EBF4F09D06D482C9F7BD50A74669FA8@gmail.com> <87a9wlalvs.fsf@gmx.com> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="505a8af5_4c2a7166_f5" Return-path: Received: from eggs.gnu.org ([208.118.235.92]:33570) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TEXHR-0007VI-Of for emacs-orgmode@gnu.org; Wed, 19 Sep 2012 23:18:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TEXHP-0005TX-9v for emacs-orgmode@gnu.org; Wed, 19 Sep 2012 23:18:21 -0400 Received: from mail-pb0-f41.google.com ([209.85.160.41]:37325) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TEXHO-0005TG-S7 for emacs-orgmode@gnu.org; Wed, 19 Sep 2012 23:18:19 -0400 Received: by pbbro12 with SMTP id ro12so4461707pbb.0 for ; Wed, 19 Sep 2012 20:18:18 -0700 (PDT) In-Reply-To: List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Torsten Wagner Cc: emacs-orgmode@gnu.org, Eric Schulte --505a8af5_4c2a7166_f5 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Hey, Thanks for the good idea. I'll have to look into figuring that system out. On the broader point of organizing the notebook, I am still having a bit of a dilemma coming up with an effective system. My first thought was to just place all my work in a dated hierarchy, such as with org-datetree. This would be simple and mirror a conventional notebook, but would loose a lot of the logical hierarchy possible with digital tools. On the other hand, organizing all my experiments as a non-linear outline is getting a bit messy. I found myself navigating around headlines everyday searching for items I need to schedule for the next day. As my notebook gets bigger, this system will probably get very inefficient. If i properly tag and schedule my tasks for the day this should be less of a problem, but i still foresee potential chaos if I get lazy. In addition to tasks I intend to record other observations in the notebook that may not be associated with a recent task, yet are important for me planning future experiments. Without proper timestamps I could loose these observations over time. This is where I came up with the idea of tagging all of my headlines with their entry date and timestamp. Potentially such a system would enable me to view the logical hierarchy of an experiment, but also view my work in the conventional linear order. Anyone have any other ideas for reasonable systems? I'm also a bit confused about the proper way to implement such a system. I imagine I could hack together some auto-timestamp property, but than it would only apply to headlines, not to my nested observations in list form. For this reason I have little used lists at all in org-mode, as it seems that any data that could potentially be nested, such as with different tasks or properties, must be converted back to headline form before it can be annotated. Am I missing something? Also, I have a general question about nesting headings demonstrated by the below example. * Today's Experiment :EXPERIMENT: ** Do today's Experiment :RATIONALE: ** Data link :DATA: ** Experiments are sad. :DISCUSSION: ** Repeat, but change X :FOLLOW-UP: * Tomorrow's Experiment :EXPERIMENT: ** Failed :RATIONALE: *** Determine if X was the cause In the above case I have two options, to either continually nest all follow-up experiments, or rely on a network of links to get me back to the data that led to the follow-up experiments. Anybody have any advice on pursing either option? Thanks for the help, Eric Lubeck On Wednesday, September 19, 2012 at 6:44 PM, Torsten Wagner wrote: > Hi, > > one way which works rather differently is the combination of git and org-mode. > You could write your protocols in separate org-files and link to them > in your records. > org allows (at least it did a while ago) to link not only to a file > but also to a specific version of a file. > You could do small modification in the protocol-files as you need them > and check them into the git system. > Link to them and you will see the version you used for exactly this experiments. > > Actually using something like git and a git sensitive link is > important if you might plan link to a lot of external files. Imaging > you overwrite a file by accident or because you can't remember you > referred to the original file already. A normal link would quietly > point to the new file and would not be in-sync with anything you > mentioned in your org-file. > > Other benefits are gits diff, merge and change-recording capabilities. > If you set-up the git repro with entire lab-book on a server (a PC > reachable from all your other devices) you could easily add data from > within the lab, go to your office to add more data and at a certain > point merge all this together. Both PCs could work offline and only > need to be online for check-in and check-out new data. > Another benefit of combining org-mode and git... you can tag certain > versions of your lab book. E.g. tag them whenever you write a paper > and make a notice in org-mode. This enables you to get back to all the > measurement and reps. data evaluation results as you found them during > writing your paper, even years and many many changes later (e.g. you > might improved your data analysis method over time but for the paper > you still want to see the old stupid way how you dealt with the data). > > Recently we got an org-file sensitive git-module, which makes merging > org-files much more nice. > Check here: > > http://orgmode.org/worg/org-contrib/org-git-link.html > http://orgmode.org/worg/org-contrib/gsoc2012/student-projects/git-merge-tool/index.html > > Albeit I have to say I like to do all kind of stuff in org-mode, I > faced problems using only org-mode as lab-notebook. Sometimes things > in a lab are to numerous and to verbose to type them all in as they > happen. Sometimes a little sketch, some quick scribbled note, etc. > contains the real important data.... sentences like "Damn, Joe slammed > the door AGAIN", during an AFM measurement. Or in your case "Uhh.. > what are the funny little flakes in my buffer solutions".... I think > sometimes a keyboard still filters to much and hence org-mode might > not really contain all the necessary info. > As a summary: > org-mode as a lab-book will work fine if you are strict in using it > and force yourself to be verbose enough. > > Hope thats helps > > Totti > > CC. There might be some legal issue with real lab books and electronic > once. Back in the good old time where scientists didn't publish each > and every result and where it could take many many months until some > discovery reached the other side of the planet, the laboratory books > where the legal evidence of the original work. If someone made a wrong > claim or someone accused someone else of falsify reporting, the > laboratory books where used to proof those claims. That is the reason, > you find e.g. nicely archived laboratory books of all the great > scientists of the Bell laboratories. Not sure how much this is still > relevant today. > > > On 20 September 2012 03:49, Eric Schulte wrote: > > Eric Lubeck writes: > > > > > Hello Everybody, > > > > > > I had a look around the web for awhile, but couldn't find any > > > information pertaining to my particular needs. I hope somebody here > > > will be able to help me out. > > > > > > Anyway, I've been looking around for quite a while for the proper > > > system to set-up an electronic laboratory notebook in. I will be > > > using org-mode to document wet-lab experiments in addition to > > > computational work. > > > > > > One of my particular concerns is this: I'm accustomed to using a > > > chronological laboratory notebook for recording all of my data. The > > > agenda views in org-mode seem to provide a means to retrieve > > > chronological information out of my outlines, but I would than need to > > > timestamp every single entry in my outline. Is there a means for > > > doing this? Currently I am manually typing C-u C-c ! , but it would > > > be helpful to have something automatically configured to timestamp and > > > place the time in a drawer for any entry in a particular file. > > > > > > > > > I believe such automated functionality may exist (although I don't use > > it personally). Take a look at this portion of the manual [1]. > > > > > > > > My other question pertains to efficiently representing linked or > > > nested data. I'd like to record my detailed laboratory protocols in > > > another outline. As most of my day-to-day work is using these > > > protocols with minimal modifications, I'd like to record in my primary > > > outline a property or hyperlink that points to the primary protocol > > > and suggests that this days experiment "inherits" from the main > > > protocol with given modifications. It would be really awesome if the > > > protocol tree could than pick up on these distant inheriting protocols > > > and transclude in the dates I have performed this protocol and > > > subsequent modifications from the "lab notebook" section. Is such a > > > task possible with org-mode, or must I look towards a more traditional > > > database? > > > > > > > > > If I read this paragraph correctly, you want to be able to collect the > > dates from every entry in which a particular protocol was used. If you > > assign each protocol a short tag name, then you could tag each daily > > experiment with the date and the protocol used. Armed with your tagged > > headlines Org-mode should provide everything required to search for and > > collect your experiments based on tags, and to do things like collect > > the dates on which each tag was used. Look at the tag and searching > > sections of the Org-mode manual [2], and maybe look at the org-collector > > in the contrib portion of Worg [3]. > > > > > > > > Thanks, > > > Eric Lubeck > > > > > > > > > > > Footnotes: > > [1] http://orgmode.org/manual/Clocking-work-time.html > > > > [2] http://orgmode.org/manual/Tags.html > > > > [3] http://orgmode.org/worg/org-contrib/org-collector.html > > > > -- > > Eric Schulte > > http://cs.unm.edu/~eschulte > > > > > --505a8af5_4c2a7166_f5 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline
Hey,

Thanks for the good idea.  = ;I'll have to look into figuring that system out.

On the broader point of organizing the notebook, I am still having a bi= t of a dilemma coming up with an effective system.  My first thought= was to just place all my work in a dated hierarchy, such as with org-dat= etree.  This would be simple and mirror a conventional notebook, but= would loose a lot of the logical hierarchy possible with digital tools. =  

On the other hand, organizing all my expe= riments as a non-linear outline is getting a bit messy.  I found mys= elf navigating around headlines everyday searching for items I need to sc= hedule for the next day.  As my notebook gets bigger, this system wi= ll probably get very inefficient.  If i properly tag and schedule my= tasks for the day this should be less of a problem, but i still foresee = potential chaos if I get lazy.  In addition to tasks I intend to rec= ord other observations in the notebook that may not be associated with a = recent task, yet are important for me planning future experiments.  = Without proper timestamps I could loose these observations over time.

This is where I came up with the idea of tagging al= l of my headlines with their entry date and timestamp.  Potentially = such a system would enable me to view the logical hierarchy of an experim= ent, but also view my work in the conventional linear order. 
<= div>
Anyone have any other ideas for reasonable systems=3F<= /div>

I'm also a bit confused about the proper way to = implement such a system.  I imagine I could hack together some auto-= timestamp property, but than it would only apply to headlines, not to my = nested observations in list form.  =46or this reason I have little u= sed lists at all in org-mode, as it seems that any data that could potent= ially be nested, such as with different tasks or properties, must be conv= erted back to headline form before it can be annotated.  Am I missin= g something=3F

Also, I have a general question a= bout nesting headings demonstrated by the below example.

* Today's Experiment       :EXPERIMENT:
** Do today's Experiment :RATIONALE:
** Data link :DATA:
=
** Experiments are sad.  :DISCUSSION:
** Repeat, but = change X <Tomorrow>  :=46OLLOW-UP:
* Tomorrow's= Experiment  :EXPERIMENT:
** <Yesterday's Experiment>= ; =46ailed :RATIONALE:
*** Determine if X was the cause

In the above case I have two options, to either continu= ally nest all follow-up experiments, or rely on a network of links to get= me back to the data that led to the follow-up experiments.  Anybody= have any advice on pursing either option=3F

Tha= nks for the help,
Eric Lubeck

=20

On Wednesday, Septembe= r 19, 2012 at 6:44 PM, Torsten Wagner wrote:

Hi,

one= way which works rather differently is the combination of git and org-mod= e.
You could write your protocols in separate org-files and lin= k to them
in your records.
org allows (at least it di= d a while ago) to link not only to a file
but also to a specifi= c version of a file.
You could do small modification in the pro= tocol-files as you need them
and check them into the git system= .
Link to them and you will see the version you used for exactl= y this experiments.

Actually using something lik= e git and a git sensitive link is
important if you might plan l= ink to a lot of external files. Imaging
you overwrite a file by= accident or because you can't remember you
referred to the ori= ginal file already. A normal link would quietly
point to the ne= w file and would not be in-sync with anything you
mentioned in = your org-file.

Other benefits are gits diff, mer= ge and change-recording capabilities.
If you set-up the git rep= ro with entire lab-book on a server (a PC
reachable from all yo= ur other devices) you could easily add data from
within the lab= , go to your office to add more data and at a certain
point mer= ge all this together. Both PCs could work offline and only
need= to be online for check-in and check-out new data.
Another bene= fit of combining org-mode and git... you can tag certain
versio= ns of your lab book. E.g. tag them whenever you write a paper
a= nd make a notice in org-mode. This enables you to get back to all the
measurement and reps. data evaluation results as you found them du= ring
writing your paper, even years and many many changes later= (e.g. you
might improved your data analysis method over time b= ut for the paper
you still want to see the old stupid way how y= ou dealt with the data).

Recently we got an org-= file sensitive git-module, which makes merging
org-files much m= ore nice.
Check here:


Albeit I have to say I= like to do all kind of stuff in org-mode, I
faced problems usi= ng only org-mode as lab-notebook. Sometimes things
in a lab are= to numerous and to verbose to type them all in as they
happen.= Sometimes a little sketch, some quick scribbled note, etc.
con= tains the real important data.... sentences like =22Damn, Joe slammed
the door AGAIN=22, during an A=46M measurement. Or in your case =22= Uhh..
what are the funny little flakes in my buffer solutions=22= .... I think
sometimes a keyboard still filters to much and hen= ce org-mode might
not really contain all the necessary info.
As a summary:
org-mode as a lab-book will work fine if = you are strict in using it
and force yourself to be verbose eno= ugh.

Hope thats helps

T= otti

CC. There might be some legal issue with re= al lab books and electronic
once. Back in the good old time whe= re scientists didn't publish each
and every result and where it= could take many many months until some
discovery reached the o= ther side of the planet, the laboratory books
where the legal e= vidence of the original work. If someone made a wrong
claim or = someone accused someone else of falsify reporting, the
laborato= ry books where used to proof those claims. That is the reason,
= you find e.g. nicely archived laboratory books of all the great
scientists of the Bell laboratories. Not sure how much this is still
relevant today.


On 20 Sept= ember 2012 03:49, Eric Schulte <eric.schulte=40gmx.com> wrote:
Eric Lubeck <eric.lubeck=40gmail.com> writes:

Hello Everybody,

<= div>I had a look around the web for awhile, but couldn't find any
information pertaining to my particular needs. I hope somebody here
will be able to help me out.

Anyway, I'= ve been looking around for quite a while for the proper
system = to set-up an electronic laboratory notebook in. I will be
usin= g org-mode to document wet-lab experiments in addition to
compu= tational work.

One of my particular concerns is = this: I'm accustomed to using a
chronological laboratory notebo= ok for recording all of my data. The
agenda views in org-mode = seem to provide a means to retrieve
chronological information o= ut of my outlines, but I would than need to
timestamp every sin= gle entry in my outline. Is there a means for
doing this=3F C= urrently I am manually typing C-u C-c =21 , but it would
be hel= pful to have something automatically configured to timestamp and
place the time in a drawer for any entry in a particular file.

I believe such automated functionalit= y may exist (although I don't use
it personally). Take a look = at this portion of the manual =5B1=5D.


My other question pertains to ef= ficiently representing linked or
nested data. I'd like to reco= rd my detailed laboratory protocols in
another outline. As mos= t of my day-to-day work is using these
protocols with minimal m= odifications, I'd like to record in my primary
outline a proper= ty or hyperlink that points to the primary protocol
and suggest= s that this days experiment =22inherits=22 from the main
protoc= ol with given modifications. It would be really awesome if the
protocol tree could than pick up on these distant inheriting protocols
and transclude in the dates I have performed this protocol and
subsequent modifications from the =22lab notebook=22 section. I= s such a
task possible with org-mode, or must I look towards a = more traditional
database=3F

<= /div>
If I read this paragraph correctly, you want to be able to coll= ect the
dates from every entry in which a particular protocol w= as used. If you
assign each protocol a short tag name, then yo= u could tag each daily
experiment with the date and the protoco= l used. Armed with your tagged
headlines Org-mode should provi= de everything required to search for and
collect your experimen= ts based on tags, and to do things like collect
the dates on wh= ich each tag was used. Look at the tag and searching
sections = of the Org-mode manual =5B2=5D, and maybe look at the org-collector
=
in the contrib portion of Worg =5B3=5D.


Thanks,
Eric Lub= eck


=46ootnote= s:
<= div>
--
Eric Schulte
= =20 =20 =20 =20 =20

--505a8af5_4c2a7166_f5--