Re: Gather properties for use by babel source block?

[Tim Burt <tcburt@rochester.rr.com>]

[-- Attachment #1: Type: text/plain, Size: 4335 bytes --]

Colin Hall <colinghall@gmail.com> writes:

> Tim Burt <tcburt <at> rochester.rr.com> writes:
>>
>> I want to gather data from properties into something that can be used by
>> a babel source block (e.g. plot the data).  Searches in the manual,
>> worg, and gmane have not yielded the method, but my best guess is that
>> I've missed it.  If so, this is simply a request for a pointer to the
>> documentation I should read.
>
> Any luck with this, Tim? I'm trying to do something very similar.

After the hints of Darlan and Suvayu last August I cobbled a workflow
described below.  Errors and instability in the early weeks have been
resolved into the current set of workable blemishes described below the
workflow.  The attachment is an org file that should be a working example
if gnuplot is installed and configured for org-babel.

Thank you Colin for asking the question about progress, because I should
have shared it with the mailing list long ago warts and all.



This is an example file for gathering and plotting health data (some of
which has been fudged to protect the guilty ;-).  The workflow is this:
 1. DAILY: Use capture templates to query for data into a datetree under
    the heading '2011' which has an ID property.  Each data headline is
    marked with tags for the data class (e.g. weight, blood sugar) that is
    used as a hook for collection.
 2. ONE-TIME: Define the plots, in this case using gnuplot (thanks to
    Suvayu for the noweb example), under the heading 'Calculation and
    Visualization'.  Another one-time setup is to define propview blocks that
    org-collector will populate for each data class (e.g. weight).  Both
    the ID and tags are used to extract the information.
 3. ONE-TIME: Another one-time setup is to define propview blocks that
    org-collector will populate for each data class (e.g. weight).  Both
    the ID and tags are used to extract the information and this is under
    the 'Summaries' heading.
    - A #+tblname is placed after the #+BEGIN to provide a hook for the
      babel blocks for plotting.
 4. ON-DEMAND: Refresh each collector block (in this case 4 separate ones),
    then org-babel-execute-buffer to generate the plots.

This method has been mostly working but it has a few weaknesses, borne mostly from
my ignorance. I haven't thought about the problem in several months, so
maybe the act of writing the questions will spur me to the act of making it
better.  Here are my observations and questions: 
 - At various times in the past year the ON-DEMAND step has not worked
   consistently, especially in regards to the #+tblname within the
   collector block.  Sometimes the refresh would fail, often by wiping out
   the existing static view and replacing with an empty line under
   the #+tblname.  No matter what happened in the past, it is stable enough
   for this post.
   - The problem could have been one of configuration (e.g. org-collector
     not loaded) or data (e.g. error in the date tree) or something else
     entirely.  I have not done sufficiently thorough troubleshooting to
     really identify the different problems.
 - Collection is done by matching the tags in the data headlines, but the
   same type of information can be gleaned from the properties themselves.
   How can a match invocation be crafted to use the properties?
   - This weakness comes straight from my ignorance of lisp.
 - Collection is done for each separate table which does not scale well as
   the number of tables increases.  Scaling itself is not the issue with
   just 4 tables, but forgetting to update each table is a slight problem.
   - Is there a way to automate the org-collector step so that one action
     updates all tables?
   - Of course, for the current story another solution would be to create
     one single data table with org-collector then modify the plotting
     routines to extract only the necessary columns.  I'd like the ability
     to do monolithic or separate then decide which to use for which
     problems. 
 - The capture templates yield blank properties that simply clutter the
   drawer and make data extraction logic slightly more complicated since
   existence alone is not a complete test.
   - Can a capture template be written to ask all the questions, but not
     yield a property if the answer is 'null'?



[-- Attachment #2: TCBHealth-Example.org --]
[-- Type: application/vnd.lotus-organizer, Size: 21674 bytes --]

[-- Attachment #3: Type: text/plain, Size: 147 bytes --]


>
> Cheers,
> Colin.
>
>
>
>
>
>

-- 
Tim Burt
www.rketburt.org
"It is healthful to every sane man to utter the art within him;" -- GK Chesterton