Re: Org-mode as a Quantified Self platform

Re: Org-mode as a Quantified Self platform

[Eric Schulte]

John Hendy <jw.hendy@gmail.com> writes:

> I've really been getting into the Quantified Self movement recently,
> and by "getting into," I mean reading and watching videos and
> /thinking/ about things I'd like to track about myself. [1] I've been
> pondering what kind of "platform" to use for my data collection...
> Excel? Some time tracking application? Those were after-thoughts. I
> thought of Org-mode right away and it seemed like an obvious choice.
> Why?
>
> 1) I have access to it at home and work pretty easily
>
> 2) The ability to quickly insert time stamps/clock data
>
> 3) The habit family of features -- set up some initial goals
> (recurring todo headlines) and then just got to the headline and mark
> done (possibly with a note) to record the event.
>
> 4) Exportability (though I have some questions on this one)
>
> 5) I recently got an Android tablet and have mobile-org on it now,
> which I'm hoping will allow me to have even better access to data
> entering.
>
> 6) Should some application "go under," I'll still always have my data
> in plain text.
>
> On to my questions:
> -- Has anyone done something like this? I see it as very similar to
> habit tracking. To give an example, I've had a bit of a persistent
> cough. I developed it when I was sick at one point, went on
> antibiotics, and got better... but the cough kind of stuck around but
> is on and off. Now I'm wondering if it's coffee/dehydration related so
> I'd like to create headlines for Coffee, glass of water, and cough.
> Then I can just mark them done whenever I finish a cup of coffee,
> drink a glass of water, or cough. At some point I'd like to plot them
> to visualize any potential relationships.
>
> -- On that note, what's the best way to get data out of
> org-habit/headlines and into something a bit more usable like csv? I'm
> looking for something more than just looking through headlines for
> missed days. I'm going to need full access to the time stamps in a
> usable format (say, R) and everything agglomerated into one single
> data file.
>
> -- I see I can insert [inactive] timestamps in mobile-org. Making
> something a TODO seems to require manual input? I find the
> documentation fairly lacking with respect to simple basic terminology
> (Outline view vs. Document view?) and have not used it much. Any
> suggestions on using this in conjunction with the above would be
> great.
>
> I think org-mode could make a very spectacular data tracker due to its
> time stamp and clocking abilities along with what feels natural with
> respect to adding in narrative text/notes and being able to structure
> things how I'd like. In addition, I'll have git ability to sync with
> my home computer.
>
> Thoughts? Thanks for any input!
>
> [1] For those unfamiliar: http://quantifiedself.com/
>
>
> John
>

Hi John,

If you end up storing measurements in properties you could use
org-collector [1] to collect these values into tables.  From tables they
can easily be analyzed with spreadsheet formulas [2] or passed to code
blocks [3] for more sophisticated analysis and plotting.

Best,

Footnotes: 
[1]  http://orgmode.org/worg/org-contrib/org-collector.html

[2]  http://orgmode.org/manual/The-spreadsheet.html

[3]  http://orgmode.org/manual/Working-With-Source-Code.html

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Org-mode as a Quantified Self platform

[John Hendy]

I've really been getting into the Quantified Self movement recently,
and by "getting into," I mean reading and watching videos and
/thinking/ about things I'd like to track about myself. [1] I've been
pondering what kind of "platform" to use for my data collection...
Excel? Some time tracking application? Those were after-thoughts. I
thought of Org-mode right away and it seemed like an obvious choice.
Why?

1) I have access to it at home and work pretty easily

2) The ability to quickly insert time stamps/clock data

3) The habit family of features -- set up some initial goals
(recurring todo headlines) and then just got to the headline and mark
done (possibly with a note) to record the event.

4) Exportability (though I have some questions on this one)

5) I recently got an Android tablet and have mobile-org on it now,
which I'm hoping will allow me to have even better access to data
entering.

6) Should some application "go under," I'll still always have my data
in plain text.

On to my questions:
-- Has anyone done something like this? I see it as very similar to
habit tracking. To give an example, I've had a bit of a persistent
cough. I developed it when I was sick at one point, went on
antibiotics, and got better... but the cough kind of stuck around but
is on and off. Now I'm wondering if it's coffee/dehydration related so
I'd like to create headlines for Coffee, glass of water, and cough.
Then I can just mark them done whenever I finish a cup of coffee,
drink a glass of water, or cough. At some point I'd like to plot them
to visualize any potential relationships.

-- On that note, what's the best way to get data out of
org-habit/headlines and into something a bit more usable like csv? I'm
looking for something more than just looking through headlines for
missed days. I'm going to need full access to the time stamps in a
usable format (say, R) and everything agglomerated into one single
data file.

-- I see I can insert [inactive] timestamps in mobile-org. Making
something a TODO seems to require manual input? I find the
documentation fairly lacking with respect to simple basic terminology
(Outline view vs. Document view?) and have not used it much. Any
suggestions on using this in conjunction with the above would be
great.

I think org-mode could make a very spectacular data tracker due to its
time stamp and clocking abilities along with what feels natural with
respect to adding in narrative text/notes and being able to structure
things how I'd like. In addition, I'll have git ability to sync with
my home computer.

Thoughts? Thanks for any input!

[1] For those unfamiliar: http://quantifiedself.com/


John

Re: Org-mode as a Quantified Self platform

[Zack Mayson]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 30/04/2012 19:03, John Hendy wrote:
> On to my questions:
> -- Has anyone done something like this? I see it as very similar to
> habit tracking. To give an example, I've had a bit of a persistent
> cough. I developed it when I was sick at one point, went on
> antibiotics, and got better... but the cough kind of stuck around but
> is on and off. Now I'm wondering if it's coffee/dehydration related so
> I'd like to create headlines for Coffee, glass of water, and cough.
> Then I can just mark them done whenever I finish a cup of coffee,
> drink a glass of water, or cough. At some point I'd like to plot them
> to visualize any potential relationships.

You might like to checkout worg; it has a few examples of GTD setups (is
Quantified Self like GTD? I assume it is).

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJPntobAAoJEAQWago++pDWRDgH/0nGGyeDCETncn+KTav5fx8Q
Mf31NjyXqmfVV1O4ZwKBJQPEtAL7FdfKJOOQDSAWyV/98wdGXNTLmimPouI9+XOH
PJqIvTi0RcARn3T+3zyGhQzdlo3/srzabG1BWie5cyYM1hWgBxqSnQXgkL/aFG90
vf+kufSfQIHB+BGTkJ0bx7wPabncfJT96vPAZJeMz2SEdfPnfDlIOKmFpcG3KOl6
8lV7dI3+snwI//aRpFKdfSt3wU99rUN5pCSaFkEiVjXPA2JVY9CmMIs0ggPX853b
fmeSk0NoCm36K3dak/teog2kXy1+7gqmWfaiMxrNecEQu45OssE9u2/XGm6z69w=
=2tMf
-----END PGP SIGNATURE-----

Re: Org-mode as a Quantified Self platform

[Zack Mayson]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 30/04/2012 19:03, John Hendy wrote:


You might like to checkout worg; it has a few examples of GTD setups (is
Quantified Self like GTD? I assume it is).

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJPntoLAAoJEAQWago++pDWGf4IAJ2WqXII51HwRJpa26dmPxMA
6ag/yV1w4QsoZJW0NIB6C5tTnkfIRi8p5lnj0hyhLb8BwBehzRBzzI4MKbOHhPVr
/lH/HA+eNHyxX1+8XBiUVGJ1FK+ZMhCccM22SkcZ54t5qOTZAnM7VfZE2rvwl92T
aOcOY4C+G/fmslq/AQtwvQL+TYFYUVUv2agTQpbiLuI1vFI2TnFApe8YqjUwHy6W
DDe3MA1Cg76wbzYy/5LwjlivkrQGQvPwUb1AUJe8DhIf2ckZQVZcoNL4OuGmfRe6
wl2003bA0l2xghpW5daKwmxL4C/8ovD91nuM6Wq75nmTtFIHARG0QyhvUu8b6Ac=
=fLJX
-----END PGP SIGNATURE-----

Re: Org-mode as a Quantified Self platform

[John Hendy]

On Mon, Apr 30, 2012 at 1:29 PM, Zack Mayson <zmayson46@googlemail.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 30/04/2012 19:03, John Hendy wrote:
>
>
> You might like to checkout worg; it has a few examples of GTD setups (is
> Quantified Self like GTD? I assume it is).

Not really. Quantified Self (check the link in my original email) is
the practice of... well... quantifying yourself. What do you want to
know about yourself?
-- Hrs sleep/night?
-- How much time you waste?
-- What you eat?
-- Your mood or ailments (then perhaps correlated to diet records)?

Stuff like that. I'm interested in using org-mode to record things I
do and time I spend on things for later analysis.

It's quite well set up to do that already with respect to recurring
todos/habit features. My primary missing piece is getting the data
into csv easily somehow.


Best regards,
John

>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iQEcBAEBAgAGBQJPntoLAAoJEAQWago++pDWGf4IAJ2WqXII51HwRJpa26dmPxMA
> 6ag/yV1w4QsoZJW0NIB6C5tTnkfIRi8p5lnj0hyhLb8BwBehzRBzzI4MKbOHhPVr
> /lH/HA+eNHyxX1+8XBiUVGJ1FK+ZMhCccM22SkcZ54t5qOTZAnM7VfZE2rvwl92T
> aOcOY4C+G/fmslq/AQtwvQL+TYFYUVUv2agTQpbiLuI1vFI2TnFApe8YqjUwHy6W
> DDe3MA1Cg76wbzYy/5LwjlivkrQGQvPwUb1AUJe8DhIf2ckZQVZcoNL4OuGmfRe6
> wl2003bA0l2xghpW5daKwmxL4C/8ovD91nuM6Wq75nmTtFIHARG0QyhvUu8b6Ac=
> =fLJX
> -----END PGP SIGNATURE-----

Re: Org-mode as a Quantified Self platform

[Achim Gratz]

Zack Mayson writes:
> (is Quantified Self like GTD? I assume it is).

No, Quantified Self is more like how much you weighed before and after
GTD.  :-)


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

DIY Stuff:
http://Synth.Stromeko.net/DIY.html

Re: Org-mode as a Quantified Self platform

[Kyle Machulis]

My "Quantified Coder" talk at Google IO last year was going to focus
toward emacs and org-mode, actually:

http://www.youtube.com/watch?v=52Ml_zax4A0#t=12m47s

Unfortunately I haven't gotten time to actually start on it yet, but I
know there's community interest in org-mode and tracking as much as
possible.

On Mon, Apr 30, 2012 at 12:21 PM, Achim Gratz <Stromeko@nexgo.de> wrote:
> Zack Mayson writes:
>> (is Quantified Self like GTD? I assume it is).
>
> No, Quantified Self is more like how much you weighed before and after
> GTD.  :-)
>
>
> Regards,
> Achim.
> --
> +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+
>
> DIY Stuff:
> http://Synth.Stromeko.net/DIY.html
>
>

Re: Org-mode as a Quantified Self platform

[Sacha Chua]

John Hendy <jw.hendy@gmail.com> writes:

> 3) The habit family of features -- set up some initial goals
> (recurring todo headlines) and then just got to the headline and mark
> done (possibly with a note) to record the event.

org-agenda is a handy way of marking tasks as complete, too. I have an
Org subtree with my daily, weekly, monthly, and yearly routines, and
another section for "In case of..." lists that cover event-driven
routines.

> -- Has anyone done something like this? I see it as very similar to
> habit tracking. To give an example, I've had a bit of a persistent

I've been using Emacs and Ledger to keep track of my finances since
2005. I've also been tracking miscellaneous things (clothes, time, etc.)
on quantifiedawesome.com . I share my notes at
http://sachachua.com/blog/category/quantified . 

I often go back and use data from notes that I've taken using
org-capture. For example, if you set it to clock in and out
automatically, you can tell how long it takes to write a blog post. It's
easy to write a function that counts the words in a subtree and
calculates your WPM.

Effort / time elapsed information might also be very interesting for
self-tracking. You can use org-set-effort to estimate the time it takes
to complete a task, clock into a task when you start working on it, have
it clock out when you're done, and then check how accurate your
estimates are. Bonus: you get a modeline reminder of time elapsed vs
time estimated, and it turns a different colour when you go overtime.

I'm working on using org-contacts to quantify my social interactions
like the way that I was using BBDB to do so before.

I tracked e-mail interactions (# of days this message waited for a
reply, # of messages I've sent to people) when I used Gnus to do my
mail, but mail setup is a bit more complicated with Gmail and a Windows
system, so I haven't done this for a while.

I have some basic Emacs integration with QuantifiedAwesome - I can post
some records to my system from Emacs using the REST API. I've been
thinking about having it update my time tracking system when I clock in
and out of tasks, as that would be cool.

> -- I see I can insert [inactive] timestamps in mobile-org. Making
> something a TODO seems to require manual input? I find the

One option might be to use a different app like Tap Log Records to
capture timestamped records and generate a CSV, then write some code to
parse the CSV and update your Org file. It's totally a hack, though.

Sacha Chua

Re: Org-mode as a Quantified Self platform

[Sacha Chua]

John Hendy <jw.hendy@gmail.com> writes:

> -- On that note, what's the best way to get data out of
> org-habit/headlines and into something a bit more usable like csv? I'm
> looking for something more than just looking through headlines for
> missed days. I'm going to need full access to the time stamps in a
> usable format (say, R) and everything agglomerated into one single
> data file.

A different approach might be to use org-capture with type table-line,
which inserts a new line in the first table at the target location. You
can then use org-capture to automatically capture the timestamp, prompt
you for the type of drink (you can define a completion table) and any
other notes you want to add, and file it automatically. "Drink coffee"
and "Drink tea" aren't really TODOs or habits, so this might be better
for you.

Sacha Chua

Re: Org-mode as a Quantified Self platform

[Karl Voit]

Hi!

* John Hendy <jw.hendy@gmail.com> wrote:
> I've really been getting into the Quantified Self movement recently,

Same holds to me. I just wanted to post almost the same things you
did. :-)

> and by "getting into," I mean reading and watching videos and
> /thinking/ about things I'd like to track about myself. [1] I've been
> pondering what kind of "platform" to use for my data collection...
> Excel? Some time tracking application? Those were after-thoughts. I
> thought of Org-mode right away and it seemed like an obvious choice.

I chose Org-mode before I got into QS. And it seems to me an almost
perfect match.

> -- 

Sidemark: double-dash followed by a space will be interpreted as the
start of your signature. At least in my newsreader (ML->Gmane) your
posts consists of /awful/ long signatures :-)

> Has anyone done something like this? 

I did not /plan/ my QS/Org-mode setup thoroughly (yet):

I am using habits to track some things that re-appear within five
weeks or less. 

For tracking events, I am using todos that repeat every 20 years or
so. Those todos I mark as done whenever an event occurs. This
results in drawers like:

:LOGBOOK:
- State "DONE"       from "TODO"       [2011-10-05 Wed 00:16]
- State "DONE"       from "TODO"       [2011-09-24 Sat 15:27]
- State "DONE"       from "TODO"       [2011-09-08 Thu 08:49]
- State "DONE"       from "TODO"       [2011-08-05 Fri 08:35]
- State "DONE"       from "TODO"       <2011-07-03 Sun 14:28>
:END:

I admit that I did not know about org-collector.el before this
thread. I will definitely look into that as well.

I do not have anything useful for analyzing or visualizing Org-mode
data yet. If org-collector.el does not turn out to be the solution,
I plan to write my own Org-mode parser in Python. My brain is not
«compatible» with LISP, I'm afraid.

When I've got the time stamps in Python, I can do whatever I like.

I'd love to see an Android app for quick input. Mobile Org for
Android has to much overhead for this purpose. I want to have an app
that I can start and touch one single button when I want to log a
timestamp of an event. Optionally it should ask me for a
text/number/amount. It'd be perfect when this QS-app is providing
widgets so that I can put such «event XY happened»-icons on the main
screen to be able to quickly log any event. One button type for each
event I want to capture.

I played around with Android apps that try to log my position using
WiFi network, cell triangulation, or GPS. They all had their issues.
Bad luck so far. (LifeLogger, GPSLogger, ...)

> -- On that note, what's the best way to get data out of
> org-habit/headlines and into something a bit more usable like csv? 

Sorry, nothing I know of so far.

But something very nice I already did: Memacs[2]

With Memacs, you can get many different data sources back into your
possession, back into your agenda: emails, tweets, bookmarks,
git-commits, text messages, phone calls, photographs, usenet
postings, and so forth will appear in your agenda. I do like Memacs
a *lot*. 

Once you have set up Memacs, it runs by its own and provides a diary
that writes itself :-) Every Memacs-user reported very positive
aspects she/he did not think of this before so far.

I plan to write that Python-Parser also to fetch Memacs-data and
generate (general) visualizations like:
* events per day/week/month/year
* average time between events
* ...

  2. https://github.com/novoid/Memacs
-- 
Karl Voit