Tracking time from one state to another?

Tracking time from one state to another?

[Malcolm Matalka]

Hello,

Is it possible in org-mode to track, and report, the duration of certain
state transitions in org-mode?  In particular, I'm interested in
tracking how long it takes me to go from a state that means I'm actively
working on an item to it being in a done state?

In my case, an item might go from working, to waiting, to working, to
done.  And in this case I'm only really interested in the last working
to done time.

For my case, I'm looking to do a weekly report on how long it takes me
to complete tasks that I have said I will work on.

Thanks,
/Malcolm

Re: Tracking time from one state to another?

[Marcin Borkowski]

On 2019-04-17, at 14:20, Malcolm Matalka <mmatalka@gmail.com> wrote:

> Hello,
>
> Is it possible in org-mode to track, and report, the duration of certain
> state transitions in org-mode?  In particular, I'm interested in
> tracking how long it takes me to go from a state that means I'm actively
> working on an item to it being in a done state?
>
> In my case, an item might go from working, to waiting, to working, to
> done.  And in this case I'm only really interested in the last working
> to done time.
>
> For my case, I'm looking to do a weekly report on how long it takes me
> to complete tasks that I have said I will work on.

Hi Malcolm,

that sounds interesting.  Since you can turn on logging of state changes
(as you probably know), this is in principle possible, though I don't
think it is built in.

I guess writing a bit of Elisp to accomplish this should not be very
difficult, though it seems that currently the problem is a bit
underspecified.  If you could elaborate, e.g., provide an example of
your state change log and describe the result you would like to get,
I could be tempted to coding this.

Best,

-- 
Marcin Borkowski
http://mbork.pl

Re: Tracking time from one state to another?

[Malcolm Matalka]

[-- Attachment #1: Type: text/plain, Size: 2728 bytes --]


Marcin Borkowski <mbork@mbork.pl> writes:

> On 2019-04-17, at 14:20, Malcolm Matalka <mmatalka@gmail.com> wrote:
>
>> Hello,
>>
>> Is it possible in org-mode to track, and report, the duration of certain
>> state transitions in org-mode?  In particular, I'm interested in
>> tracking how long it takes me to go from a state that means I'm actively
>> working on an item to it being in a done state?
>>
>> In my case, an item might go from working, to waiting, to working, to
>> done.  And in this case I'm only really interested in the last working
>> to done time.
>>
>> For my case, I'm looking to do a weekly report on how long it takes me
>> to complete tasks that I have said I will work on.
>
> Hi Malcolm,
>
> that sounds interesting.  Since you can turn on logging of state changes
> (as you probably know), this is in principle possible, though I don't
> think it is built in.
>
> I guess writing a bit of Elisp to accomplish this should not be very
> difficult, though it seems that currently the problem is a bit
> underspecified.  If you could elaborate, e.g., provide an example of
> your state change log and describe the result you would like to get,
> I could be tempted to coding this.
>
> Best,

Sure!

So for my use case, I'm mostly interested in the last transition to a
finished state, but for simplicity I'll specify the two states I'm
interested in:

Given states NEXT and DONE, I want to know the time between going into
NEXT and over to DONE.

For example, given the following logbook:

   :LOGBOOK:
   - State "DONE"       from "NEXT"       [2019-04-01 Mon 11:07]
   - State "NEXT"       from "TODO"       [2019-04-01 Mon 10:35]
   - State "NEXT"       from "TODO"       [2018-07-02 Mon 11:03]
   :END:


This item would be 32 minutes.

This one:

   :LOGBOOK:
   - State "DONE"       from "NEXT"       [2019-04-10 Wed 09:56]
   - State "NEXT"       from "WAITING"    [2019-04-10 Wed 09:40]
   - State "WAITING"    from "NEXT"       [2019-04-09 Tue 10:44]
   - State "NEXT"       from "WAITING"    [2019-04-09 Tue 10:10]
   - State "WAITING"    from "NEXT"       [2019-04-08 Mon 16:39]
   - State "NEXT"       from "TODO"       [2019-04-08 Mon 11:14]
   :END:


Would be 16 minutes.

I'd like to specify what time range to do this for, and be able to sort
by duration.

I think a dynamic block, like clocktable, would probably be a fine.

Any idea what the level of work involved is to accomplish this?  I'm
guessing a very hacky version might be to modify clock table and look at
the logbook instead of clock and filter out all but the last transition?

Also, this is just the usecase I'm interested in, so if you have
thoughts on what a more general form would look like, that would be interested.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Tracking time from one state to another?

[Marcin Borkowski]

On 2019-04-18, at 17:34, Malcolm Matalka <mmatalka@gmail.com> wrote:

> Marcin Borkowski <mbork@mbork.pl> writes:
>
>> On 2019-04-17, at 14:20, Malcolm Matalka <mmatalka@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> Is it possible in org-mode to track, and report, the duration of certain
>>> state transitions in org-mode?  In particular, I'm interested in
>>> tracking how long it takes me to go from a state that means I'm actively
>>> working on an item to it being in a done state?
>>>
>>> In my case, an item might go from working, to waiting, to working, to
>>> done.  And in this case I'm only really interested in the last working
>>> to done time.
>>>
>>> For my case, I'm looking to do a weekly report on how long it takes me
>>> to complete tasks that I have said I will work on.
>>
>> Hi Malcolm,
>>
>> that sounds interesting.  Since you can turn on logging of state changes
>> (as you probably know), this is in principle possible, though I don't
>> think it is built in.
>>
>> I guess writing a bit of Elisp to accomplish this should not be very
>> difficult, though it seems that currently the problem is a bit
>> underspecified.  If you could elaborate, e.g., provide an example of
>> your state change log and describe the result you would like to get,
>> I could be tempted to coding this.
>>
>> Best,
>
> Sure!
>
> So for my use case, I'm mostly interested in the last transition to a
> finished state, but for simplicity I'll specify the two states I'm
> interested in:
>
> Given states NEXT and DONE, I want to know the time between going into
> NEXT and over to DONE.
>
> For example, given the following logbook:
>
>    :LOGBOOK:
>    - State "DONE"       from "NEXT"       [2019-04-01 Mon 11:07]
>    - State "NEXT"       from "TODO"       [2019-04-01 Mon 10:35]
>    - State "NEXT"       from "TODO"       [2018-07-02 Mon 11:03]
>    :END:
>
>
> This item would be 32 minutes.
>
> This one:
>
>    :LOGBOOK:
>    - State "DONE"       from "NEXT"       [2019-04-10 Wed 09:56]
>    - State "NEXT"       from "WAITING"    [2019-04-10 Wed 09:40]
>    - State "WAITING"    from "NEXT"       [2019-04-09 Tue 10:44]
>    - State "NEXT"       from "WAITING"    [2019-04-09 Tue 10:10]
>    - State "WAITING"    from "NEXT"       [2019-04-08 Mon 16:39]
>    - State "NEXT"       from "TODO"       [2019-04-08 Mon 11:14]
>    :END:
>
>
> Would be 16 minutes.

Are those :LOGBOOK: drawers real?  They are not sorted chronologically,
as they probably should be (though I'm not sure, I don't se them much).

> I'd like to specify what time range to do this for, and be able to sort
> by duration.
>
> I think a dynamic block, like clocktable, would probably be a fine.
>
> Any idea what the level of work involved is to accomplish this?  I'm
> guessing a very hacky version might be to modify clock table and look at
> the logbook instead of clock and filter out all but the last transition?

I don't think that's the best way - Org-mode code is famously
complicated, it might be easier to do it from scratch (it'd be perhaps
less general then, though).

Anyway, it doesn't look like a lot of work - an hour or two for a rough
prototype might be enough.

> Also, this is just the usecase I'm interested in, so if you have
> thoughts on what a more general form would look like, that would be interested.

No idea yet.

Best,

-- 
Marcin Borkowski
http://mbork.pl

Re: Tracking time from one state to another?

[Julius Dittmar]

Hi Marcin,

Am 19.04.19 um 06:37 schrieb Marcin Borkowski:
> Are those :LOGBOOK: drawers real?  They are not sorted chronologically,
> as they probably should be (though I'm not sure, I don't se them much).

They are. With the newest entry at the top.

Yours,
Julius

Re: Tracking time from one state to another?

[Malcolm Matalka]

Marcin Borkowski <mbork@mbork.pl> writes:

> On 2019-04-18, at 17:34, Malcolm Matalka <mmatalka@gmail.com> wrote:
>
>> Marcin Borkowski <mbork@mbork.pl> writes:
>>
>>> On 2019-04-17, at 14:20, Malcolm Matalka <mmatalka@gmail.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> Is it possible in org-mode to track, and report, the duration of certain
>>>> state transitions in org-mode?  In particular, I'm interested in
>>>> tracking how long it takes me to go from a state that means I'm actively
>>>> working on an item to it being in a done state?
>>>>
>>>> In my case, an item might go from working, to waiting, to working, to
>>>> done.  And in this case I'm only really interested in the last working
>>>> to done time.
>>>>
>>>> For my case, I'm looking to do a weekly report on how long it takes me
>>>> to complete tasks that I have said I will work on.
>>>
>>> Hi Malcolm,
>>>
>>> that sounds interesting.  Since you can turn on logging of state changes
>>> (as you probably know), this is in principle possible, though I don't
>>> think it is built in.
>>>
>>> I guess writing a bit of Elisp to accomplish this should not be very
>>> difficult, though it seems that currently the problem is a bit
>>> underspecified.  If you could elaborate, e.g., provide an example of
>>> your state change log and describe the result you would like to get,
>>> I could be tempted to coding this.
>>>
>>> Best,
>>
>> Sure!
>>
>> So for my use case, I'm mostly interested in the last transition to a
>> finished state, but for simplicity I'll specify the two states I'm
>> interested in:
>>
>> Given states NEXT and DONE, I want to know the time between going into
>> NEXT and over to DONE.
>>
>> For example, given the following logbook:
>>
>>    :LOGBOOK:
>>    - State "DONE"       from "NEXT"       [2019-04-01 Mon 11:07]
>>    - State "NEXT"       from "TODO"       [2019-04-01 Mon 10:35]
>>    - State "NEXT"       from "TODO"       [2018-07-02 Mon 11:03]
>>    :END:
>>
>>
>> This item would be 32 minutes.
>>
>> This one:
>>
>>    :LOGBOOK:
>>    - State "DONE"       from "NEXT"       [2019-04-10 Wed 09:56]
>>    - State "NEXT"       from "WAITING"    [2019-04-10 Wed 09:40]
>>    - State "WAITING"    from "NEXT"       [2019-04-09 Tue 10:44]
>>    - State "NEXT"       from "WAITING"    [2019-04-09 Tue 10:10]
>>    - State "WAITING"    from "NEXT"       [2019-04-08 Mon 16:39]
>>    - State "NEXT"       from "TODO"       [2019-04-08 Mon 11:14]
>>    :END:
>>
>>
>> Would be 16 minutes.
>
> Are those :LOGBOOK: drawers real?  They are not sorted chronologically,
> as they probably should be (though I'm not sure, I don't se them
> much).

Yes they are real.  I'm not tracking going into a TODO state which is
hwy the first one looks funny.  And they are sorted chronologically, but
in reverse order, so top is most recent.

>
>> I'd like to specify what time range to do this for, and be able to sort
>> by duration.
>>
>> I think a dynamic block, like clocktable, would probably be a fine.
>>
>> Any idea what the level of work involved is to accomplish this?  I'm
>> guessing a very hacky version might be to modify clock table and look at
>> the logbook instead of clock and filter out all but the last transition?
>
> I don't think that's the best way - Org-mode code is famously
> complicated, it might be easier to do it from scratch (it'd be perhaps
> less general then, though).
>
> Anyway, it doesn't look like a lot of work - an hour or two for a rough
> prototype might be enough.

Is this something you're interested in doing?  If not if you have a
pointers I can try to take a poke at it.  My elisp skills are very-much
lacking but some progress is better than no progress.

>
>> Also, this is just the usecase I'm interested in, so if you have
>> thoughts on what a more general form would look like, that would be interested.
>
> No idea yet.
>
> Best,

Re: Tracking time from one state to another?

[Ken Mankoff]

Hi Malcom,

On 2019-04-17 at 14:20 +0200, Malcolm Matalka <mmatalka@gmail.com> wrote...
> Is it possible in org-mode to track, and report, the duration of
> certain state transitions in org-mode? In particular, I'm interested
> in tracking how long it takes me to go from a state that means I'm
> actively working on an item to it being in a done state?
>
> In my case, an item might go from working, to waiting, to working, to
> done.  And in this case I'm only really interested in the last working
> to done time.

Org doesn't have a built-in feature to turn on/off the clock when you change state, as far as I know. But it has the opposite - you can change state when you clock in or out of a task.

See section 7 here https://writequit.org/denver-emacs/presentations/2017-04-11-time-clocking-with-org.html#helpful-things

And/or maybe thread here w/ some code https://lists.gnu.org/archive/html/emacs-orgmode//2009-04/msg00315.html

So perhaps you can achieve this by changing your workflow a bit. Instead of changing state, just clock in and have the state set to INPROGRESS. Clock otu and have it set to WAITING. When it is done, it may be a two-step process of setting to DONE and making sure the clock is stopped.

Would this achieve what you're looking for?

  -k.

Re: Tracking time from one state to another?

[Malcolm Matalka]

Ken Mankoff <mankoff@gmail.com> writes:

> Hi Malcom,
>
> On 2019-04-17 at 14:20 +0200, Malcolm Matalka <mmatalka@gmail.com> wrote...
>> Is it possible in org-mode to track, and report, the duration of
>> certain state transitions in org-mode? In particular, I'm interested
>> in tracking how long it takes me to go from a state that means I'm
>> actively working on an item to it being in a done state?
>>
>> In my case, an item might go from working, to waiting, to working, to
>> done.  And in this case I'm only really interested in the last working
>> to done time.
>
> Org doesn't have a built-in feature to turn on/off the clock when you
> change state, as far as I know. But it has the opposite - you can
> change state when you clock in or out of a task.

I don't want to turn on/off the clock when I change state, I want to get
a report on the time between certain state transitions.  That
information is stored in the LOGBOOK (assuming one has that turned on).

>
> See section 7 here https://writequit.org/denver-emacs/presentations/2017-04-11-time-clocking-with-org.html#helpful-things
>
> And/or maybe thread here w/ some code https://lists.gnu.org/archive/html/emacs-orgmode//2009-04/msg00315.html
>
> So perhaps you can achieve this by changing your workflow a bit. Instead of changing state, just clock in and have the state set to INPROGRESS. Clock otu and have it set to WAITING. When it is done, it may be a two-step process of setting to DONE and making sure the clock is stopped.
>
> Would this achieve what you're looking for?

Unfortunately no.  I need to track the total time I spend in a task for
billing reasons, but what I'm interested in here is how quickly I
complete a task when there are no blockers/dependencies.  And the way
that presents itself in my LOGBOOK is the time between the final
transition from NEXT to DONE.  Before that, a TODO can go from NEXT, to
WAITING, and back and forth.

>
>   -k.

Re: Tracking time from one state to another?

[Ken Mankoff]

Hi Malcom,

I think we're talking about two different things here. When you change TODO state, the time of the change is recorded. This is what you're doing.

Org also has a time-tracking feature that is totally separate. You can "clock in" and "clock out" of a task which starts and stops a timer. You can then generate reports about that timer. The timer can be activated regardless of the TODO state, but you could also have the timer start (clock in) every time you set the state to "WORKING_ON_IT" or "INPROGRESS", and the timer stop (clock out) whenever the task is in NEXT or WAITING state.

Then you can generate a CLOCK REPORT for each task and sub-task, which is the amount of time you actually worked on it (i.e. the amount of time it spent in the WORKING_ON_IT state.

This is a different workflow than your current one - it uses the Org clocking feature, not just the TODO status feature.

  -k.

On 2019-04-23 at 08:07 +0200, Malcolm Matalka <mmatalka@gmail.com> wrote...
> Ken Mankoff <mankoff@gmail.com> writes:
>
>> Hi Malcom,
>>
>> On 2019-04-17 at 14:20 +0200, Malcolm Matalka <mmatalka@gmail.com> wrote...
>>> Is it possible in org-mode to track, and report, the duration of
>>> certain state transitions in org-mode? In particular, I'm interested
>>> in tracking how long it takes me to go from a state that means I'm
>>> actively working on an item to it being in a done state?
>>>
>>> In my case, an item might go from working, to waiting, to working, to
>>> done.  And in this case I'm only really interested in the last working
>>> to done time.
>>
>> Org doesn't have a built-in feature to turn on/off the clock when you
>> change state, as far as I know. But it has the opposite - you can
>> change state when you clock in or out of a task.
>
> I don't want to turn on/off the clock when I change state, I want to get
> a report on the time between certain state transitions.  That
> information is stored in the LOGBOOK (assuming one has that turned on).
>
>>
>> See section 7 here https://writequit.org/denver-emacs/presentations/2017-04-11-time-clocking-with-org.html#helpful-things
>>
>> And/or maybe thread here w/ some code https://lists.gnu.org/archive/html/emacs-orgmode//2009-04/msg00315.html
>>
>> So perhaps you can achieve this by changing your workflow a bit. Instead of changing state, just clock in and have the state set to INPROGRESS. Clock otu and have it set to WAITING. When it is done, it may be a two-step process of setting to DONE and making sure the clock is stopped.
>>
>> Would this achieve what you're looking for?
>
> Unfortunately no.  I need to track the total time I spend in a task for
> billing reasons, but what I'm interested in here is how quickly I
> complete a task when there are no blockers/dependencies.  And the way
> that presents itself in my LOGBOOK is the time between the final
> transition from NEXT to DONE.  Before that, a TODO can go from NEXT, to
> WAITING, and back and forth.
>
>>
>>   -k.

Re: Tracking time from one state to another?

[Malcolm Matalka]

Ken Mankoff <mankoff@gmail.com> writes:

> Hi Malcom,
>
> I think we're talking about two different things here. When you change TODO state, the time of the change is recorded. This is what you're doing.
>
> Org also has a time-tracking feature that is totally separate. You can "clock in" and "clock out" of a task which starts and stops a timer. You can then generate reports about that timer. The timer can be activated regardless of the TODO state, but you could also have the timer start (clock in) every time you set the state to "WORKING_ON_IT" or "INPROGRESS", and the timer stop (clock out) whenever the task is in NEXT or WAITING state.
>
> Then you can generate a CLOCK REPORT for each task and sub-task, which
> is the amount of time you actually worked on it (i.e. the amount of
> time it spent in the WORKING_ON_IT state.

Yes we're talking about different things.  I do not want the total
amount of time I worked on an item, in this case, I'm already tracking
that by clocking in and out.  I'm after the time it took to go from NEXT
to DONE.  In this case I'm trying to track the time an item is in queue,
and not the time the item is worked on.

>
> This is a different workflow than your current one - it uses the Org clocking feature, not just the TODO status feature.
>
>   -k.
>
> On 2019-04-23 at 08:07 +0200, Malcolm Matalka <mmatalka@gmail.com> wrote...
>> Ken Mankoff <mankoff@gmail.com> writes:
>>
>>> Hi Malcom,
>>>
>>> On 2019-04-17 at 14:20 +0200, Malcolm Matalka <mmatalka@gmail.com> wrote...
>>>> Is it possible in org-mode to track, and report, the duration of
>>>> certain state transitions in org-mode? In particular, I'm interested
>>>> in tracking how long it takes me to go from a state that means I'm
>>>> actively working on an item to it being in a done state?
>>>>
>>>> In my case, an item might go from working, to waiting, to working, to
>>>> done.  And in this case I'm only really interested in the last working
>>>> to done time.
>>>
>>> Org doesn't have a built-in feature to turn on/off the clock when you
>>> change state, as far as I know. But it has the opposite - you can
>>> change state when you clock in or out of a task.
>>
>> I don't want to turn on/off the clock when I change state, I want to get
>> a report on the time between certain state transitions.  That
>> information is stored in the LOGBOOK (assuming one has that turned on).
>>
>>>
>>> See section 7 here https://writequit.org/denver-emacs/presentations/2017-04-11-time-clocking-with-org.html#helpful-things
>>>
>>> And/or maybe thread here w/ some code https://lists.gnu.org/archive/html/emacs-orgmode//2009-04/msg00315.html
>>>
>>> So perhaps you can achieve this by changing your workflow a bit. Instead of changing state, just clock in and have the state set to INPROGRESS. Clock otu and have it set to WAITING. When it is done, it may be a two-step process of setting to DONE and making sure the clock is stopped.
>>>
>>> Would this achieve what you're looking for?
>>
>> Unfortunately no.  I need to track the total time I spend in a task for
>> billing reasons, but what I'm interested in here is how quickly I
>> complete a task when there are no blockers/dependencies.  And the way
>> that presents itself in my LOGBOOK is the time between the final
>> transition from NEXT to DONE.  Before that, a TODO can go from NEXT, to
>> WAITING, and back and forth.
>>
>>>
>>>   -k.