Bug: No property change from ‘org-clock-sum’

Bug: No property change from ‘org-clock-sum’

[Ben Finney]

Howdy all,

When I use ‘org-clock-sum’ in Org-mode version 7.4, nothing appears to
happen: the items in the subtree are not updated and no ‘CLOCKSUM’
property appears.

Org-mode version 7.4
GNU Emacs 23.2.1 (powerpc-unknown-linux-gnu, GTK+ Version 2.20.1) of
2010-12-12 on porpora, modified by Debian


I'm trying to generate an invoice report using ‘org-invoice.el’.

The items on which I want to report have clock entries, generated
correctly with ‘org-clock-in’ and ‘org-clock-out’. An example:

=====
* foo
** bar
*** [2011-03-24 Thu]
    CLOCK: [2011-03-24 Thu 14:15]--[2011-03-24 Thu 18:03] =>  3:48
    CLOCK: [2011-03-24 Thu 09:10]--[2011-03-24 Thu 12:27] =>  3:17
*** [2011-03-23 Wed]
    CLOCK: [2011-03-23 Wed 14:10]--[2011-03-23 Wed 18:16] =>  4:06
    CLOCK: [2011-03-23 Wed 09:30]--[2011-03-23 Wed 13:10] =>  3:40
*** [2011-03-22 Tue]
    CLOCK: [2011-03-22 Tue 14:00]--[2011-03-22 Tue 17:30] =>  3:30
    CLOCK: [2011-03-22 Tue 08:58]--[2011-03-22 Tue 13:31] =>  4:33
=====

According to the doc string for ‘org-invoice-report’, the ‘CLOCKSUM’
property on each item in the subtree will be used for the report.

So apparently I need to ‘org-clock-sum’ on the tree before updating the
report, in order to automatically generate the ‘CLOCKSUM’ property. But
that function doesn't change anything.

Why wouldn't the ‘org-clock-sum’ function do what its doc string says?
How should I be updating the ‘CLOCKSUM’ property?

-- 
 \        “The industrial system is profoundly dependent on commercial |
  `\   television and could not exist in its present form without it.” |
_o__)        —John Kenneth Galbraith, _The New Industrial State_, 1967 |
Ben Finney

Re: Bug: No property change from ‘org-clock-sum’

[Ben Finney]

Ben Finney <ben+emacs@benfinney.id.au> writes:

> When I use ‘org-clock-sum’ in Org-mode version 7.4, nothing appears to
> happen: the items in the subtree are not updated and no ‘CLOCKSUM’
> property appears.
>
> Org-mode version 7.4
> GNU Emacs 23.2.1 (powerpc-unknown-linux-gnu, GTK+ Version 2.20.1) of
> 2010-12-12 on porpora, modified by Debian

This is still the case after an upgrade:

Org-mode version 7.5
GNU Emacs 23.3.1 (powerpc-unknown-linux-gnu, GTK+ Version 2.24.4) of
2011-04-11 on praetorius, modified by Debian

> Why wouldn't the ‘org-clock-sum’ function do what its doc string says?
> How should I be updating the ‘CLOCKSUM’ property?

Can anyone shed light on this? I'd like to use the function as
specified.

-- 
 \       “To have the choice between proprietary software packages, is |
  `\      being able to choose your master. Freedom means not having a |
_o__)                        master.” —Richard M. Stallman, 2007-05-16 |
Ben Finney

Re: Bug: No property change from ‘org-clock-sum’

[Bernt Hansen]

Ben Finney <ben+emacs@benfinney.id.au> writes:

> Ben Finney <ben+emacs@benfinney.id.au> writes:
>
>> When I use ‘org-clock-sum’ in Org-mode version 7.4, nothing appears to
>> happen: the items in the subtree are not updated and no ‘CLOCKSUM’
>> property appears.
>>
>> Org-mode version 7.4
>> GNU Emacs 23.2.1 (powerpc-unknown-linux-gnu, GTK+ Version 2.20.1) of
>> 2010-12-12 on porpora, modified by Debian
>
> This is still the case after an upgrade:
>
> Org-mode version 7.5
> GNU Emacs 23.3.1 (powerpc-unknown-linux-gnu, GTK+ Version 2.24.4) of
> 2011-04-11 on praetorius, modified by Debian
>
>> Why wouldn't the ‘org-clock-sum’ function do what its doc string says?
>> How should I be updating the ‘CLOCKSUM’ property?
>
> Can anyone shed light on this? I'd like to use the function as
> specified.

Hi Ben,

I don't think the org-clock-sum function does what you think/want it to
do.  The docstring says it sets text properties on the headline - so I
don't think it is intended to update a CLOCKSUM property in the
headings.

These text properties are temporary (ie. not saved in the org file) and
I think these are used by the column view overlays for display-only.

The org-invoice.el contributed file created by Peter Jones uses a
CLOCKSUM property but I'm not sure if this is intended to be updated
manually or automatically from the clocking lines in org.  I've CC-ed
Peter in case he can shed any light on this.

Regards,
Bernt

Re: Bug: No property change from ‘org-clock-sum’

[Ben Finney]

Bernt Hansen <bernt@norang.ca> writes:

> I don't think the org-clock-sum function does what you think/want it
> to do. The docstring says it sets text properties on the headline - so
> I don't think it is intended to update a CLOCKSUM property in the
> headings.

I don't underastand the distinction. Isn't the CLOCKSUM property a text
property? What do I need to know here?

> These text properties are temporary (ie. not saved in the org file) and
> I think these are used by the column view overlays for display-only.
>
> The org-invoice.el contributed file created by Peter Jones uses a
> CLOCKSUM property but I'm not sure if this is intended to be updated
> manually or automatically from the clocking lines in org.  I've CC-ed
> Peter in case he can shed any light on this.

Thanks. It does seem that they're much the same purpose, so I hope they
can be unified.

-- 
 \         “To succeed in the world it is not enough to be stupid, you |
  `\                            must also be well-mannered.” —Voltaire |
_o__)                                                                  |
Ben Finney

Re: Bug: No property change from ‘org-clock-sum’

[Bernt Hansen]

Ben Finney <ben+emacs@benfinney.id.au> writes:

> Bernt Hansen <bernt@norang.ca> writes:
>
>> I don't think the org-clock-sum function does what you think/want it
>> to do. The docstring says it sets text properties on the headline - so
>> I don't think it is intended to update a CLOCKSUM property in the
>> headings.
>
> I don't underastand the distinction. Isn't the CLOCKSUM property a text
> property? What do I need to know here?

Sorry I didn't explain that very well.  Hopefully this will shed a
little more light on this.

Text properties is an emacs feature where you can add extra (invisible,
and temporary) data to text in an emacs buffer.

An Org-mode properties are part of the org file and normally lives in
the :PROPERTY: drawer.

The column view overlay is displayed using emacs text properties - and
this doesn't alter the data in the org file.  Thus the org-invoice.el
code that queries the _org_ property CLOCKSUM never gets a value.

>
>> These text properties are temporary (ie. not saved in the org file) and
>> I think these are used by the column view overlays for display-only.
>>
>> The org-invoice.el contributed file created by Peter Jones uses a
>> CLOCKSUM property but I'm not sure if this is intended to be updated
>> manually or automatically from the clocking lines in org.  I've CC-ed
>> Peter in case he can shed any light on this.
>
> Thanks. It does seem that they're much the same purpose, so I hope they
> can be unified.

I think the fix is probably to make org-invoice.el look for the CLOCKSUM
text property instead of a CLOCKSUM org-mode property value.
org-invoice.el should probably also automatically call org-clock-sum so
this doesn't need to be a manual step.

Regards,
Bernt

Re: Bug: No property change from ‘org-clock-sum’

[Peter Jones]

Bernt Hansen <bernt@norang.ca> writes:
> The org-invoice.el contributed file created by Peter Jones uses a
> CLOCKSUM property but I'm not sure if this is intended to be updated
> manually or automatically from the clocking lines in org.  I've CC-ed
> Peter in case he can shed any light on this.

IIRC, when I wrote org-invoice.el the function (org-clock-sum) updated
the CLOCKSUM entry in the property drawer.  I don't remember anything
about text properties being updated, and that doesn't seem very useful
to me.

-- 
Peter Jones - pmade inc.
303-219-0226 http://pmade.com

Re: Bug: No property change from ‘org-clock-sum’

[Carsten Dominik]

On 6.5.2011, at 15:56, Bernt Hansen wrote:

> Ben Finney <ben+emacs@benfinney.id.au> writes:
> 
>> Bernt Hansen <bernt@norang.ca> writes:
>> 
>>> I don't think the org-clock-sum function does what you think/want it
>>> to do. The docstring says it sets text properties on the headline - so
>>> I don't think it is intended to update a CLOCKSUM property in the
>>> headings.
>> 
>> I don't underastand the distinction. Isn't the CLOCKSUM property a text
>> property? What do I need to know here?
> 
> Sorry I didn't explain that very well.  Hopefully this will shed a
> little more light on this.
> 
> Text properties is an emacs feature where you can add extra (invisible,
> and temporary) data to text in an emacs buffer.
> 
> An Org-mode properties are part of the org file and normally lives in
> the :PROPERTY: drawer.
> 
> The column view overlay is displayed using emacs text properties - and
> this doesn't alter the data in the org file.  Thus the org-invoice.el
> code that queries the _org_ property CLOCKSUM never gets a value.
> 
>> 
>>> These text properties are temporary (ie. not saved in the org file) and
>>> I think these are used by the column view overlays for display-only.
>>> 
>>> The org-invoice.el contributed file created by Peter Jones uses a
>>> CLOCKSUM property but I'm not sure if this is intended to be updated
>>> manually or automatically from the clocking lines in org.  I've CC-ed
>>> Peter in case he can shed any light on this.
>> 
>> Thanks. It does seem that they're much the same purpose, so I hope they
>> can be unified.
> 
> I think the fix is probably to make org-invoice.el look for the CLOCKSUM
> text property instead of a CLOCKSUM org-mode property value.
> org-invoice.el should probably also automatically call org-clock-sum so
> this doesn't need to be a manual step.

No.  CLOCKSUM is a special property, and it can be used to
access the clocking sum of an entry after (org-clock-sum) has
been called.  This is what org-invoice does.  It calls
org-clock-sum to do the computation, and then used the property 
API to get to the value.

However, I seen now that there is a bug here, cause by a
rewrite (for speed) of property access some time ago.
After that rewrite, the special CLOCKSUM property used
by org-invoice.el was no longer accessible.

This problem has just been fixed, so with the current git master,
org-invoice.el should work again.

- Carsten

Re: Bug: No property change from ‘org-clock-sum’

[Carsten Dominik]

On 7.4.2011, at 23:51, Ben Finney wrote:

> Howdy all,
> 
> When I use ‘org-clock-sum’ in Org-mode version 7.4, nothing appears to
> happen: the items in the subtree are not updated and no ‘CLOCKSUM’
> property appears.

CLOCKSUM is a special property which is only a way to access clocking
information, but the property is never set literally.
org-invoice was broken due to changes in the property code in org.el,
fixed now.

HTH

- Carsten

> 
> Org-mode version 7.4
> GNU Emacs 23.2.1 (powerpc-unknown-linux-gnu, GTK+ Version 2.20.1) of
> 2010-12-12 on porpora, modified by Debian
> 
> 
> I'm trying to generate an invoice report using ‘org-invoice.el’.
> 
> The items on which I want to report have clock entries, generated
> correctly with ‘org-clock-in’ and ‘org-clock-out’. An example:
> 
> =====
> * foo
> ** bar
> *** [2011-03-24 Thu]
>    CLOCK: [2011-03-24 Thu 14:15]--[2011-03-24 Thu 18:03] =>  3:48
>    CLOCK: [2011-03-24 Thu 09:10]--[2011-03-24 Thu 12:27] =>  3:17
> *** [2011-03-23 Wed]
>    CLOCK: [2011-03-23 Wed 14:10]--[2011-03-23 Wed 18:16] =>  4:06
>    CLOCK: [2011-03-23 Wed 09:30]--[2011-03-23 Wed 13:10] =>  3:40
> *** [2011-03-22 Tue]
>    CLOCK: [2011-03-22 Tue 14:00]--[2011-03-22 Tue 17:30] =>  3:30
>    CLOCK: [2011-03-22 Tue 08:58]--[2011-03-22 Tue 13:31] =>  4:33
> =====
> 
> According to the doc string for ‘org-invoice-report’, the ‘CLOCKSUM’
> property on each item in the subtree will be used for the report.
> 
> So apparently I need to ‘org-clock-sum’ on the tree before updating the
> report, in order to automatically generate the ‘CLOCKSUM’ property. But
> that function doesn't change anything.
> 
> Why wouldn't the ‘org-clock-sum’ function do what its doc string says?
> How should I be updating the ‘CLOCKSUM’ property?
> 
> -- 
> \        “The industrial system is profoundly dependent on commercial |
>  `\   television and could not exist in its present form without it.” |
> _o__)        —John Kenneth Galbraith, _The New Industrial State_, 1967 |
> Ben Finney
> 
> 

Re: Bug: No property change from ‘org-clock-sum’

[Bernt Hansen]

Carsten Dominik <carsten.dominik@gmail.com> writes:

> No.  CLOCKSUM is a special property, and it can be used to
> access the clocking sum of an entry after (org-clock-sum) has
> been called.  This is what org-invoice does.  It calls
> org-clock-sum to do the computation, and then used the property 
> API to get to the value.
>
> However, I seen now that there is a bug here, cause by a
> rewrite (for speed) of property access some time ago.
> After that rewrite, the special CLOCKSUM property used
> by org-invoice.el was no longer accessible.
>
> This problem has just been fixed, so with the current git master,
> org-invoice.el should work again.

Ah! Thanks Carsten.  It does indeed return values after this fix.

However there are numerous problems with the resulting invoice for a
sample tree I have with clocked data.

I'll send these details to the list shortly.

Regards,
Bernt

Re: Bug: No property change from ‘org-clock-sum’

[Bernt Hansen]

Peter Jones <mlists@pmade.com> writes:

> Bernt Hansen <bernt@norang.ca> writes:
>> The org-invoice.el contributed file created by Peter Jones uses a
>> CLOCKSUM property but I'm not sure if this is intended to be updated
>> manually or automatically from the clocking lines in org.  I've CC-ed
>> Peter in case he can shed any light on this.
>
> IIRC, when I wrote org-invoice.el the function (org-clock-sum) updated
> the CLOCKSUM entry in the property drawer.  I don't remember anything
> about text properties being updated, and that doesn't seem very useful
> to me.

Hi Peter,

Carsten just pushed a fix so org-invoice.el can again access the
CLOCKSUM special property.

I've mocked up a sample task tree with clock times and dates and tried
to generate an invoice using a rate of $60 - which makes each clock
minute $1.  The total time in the tree is 55 minutes so this invoice
should be $55 but the total is $115.

Please let me know if I'm doing this completely wrong.

org-invoice.el has a number of problems on a sample tree I ran this
against.

The clocked time in this file is as follows:

#+BEGIN: columnview :hlines 1 :id: local
| Task                       | Estimated Effort | CLOCKSUM |
|----------------------------+------------------+----------|
| ** DONE Project Foo        |                  |     0:55 |
| *** DONE Task 1 for foo    |                  |     0:05 |
| *** DONE Task 2 for foo    |                  |     0:20 |
| **** DONE Task 2.1 for foo |                  |     0:05 |
| *** DONE Task 3 for foo    |                  |     0:20 |
| **** DONE Task 3.1 for foo |                  |     0:05 |
| *** DONE Task 4 for foo    |                  |     0:05 |
#+END:

There are 5 minutes clocked in the Project Foo task and all other
clocked times are in the subtasks.  There are 55 minutes total clocked
on this project.

The clock detail per task is

| Task                       | Clocked Minutes |
|----------------------------+-----------------|
| ** DONE Project Foo        |            0:05 |
| *** DONE Task 1 for foo    |            0:05 |
| *** DONE Task 2 for foo    |            0:15 |
| **** DONE Task 2.1 for foo |            0:05 |
| *** DONE Task 3 for foo    |            0:15 |
| **** DONE Task 3.1 for foo |            0:05 |
| *** DONE Task 4 for foo    |            0:05 |


and the resulting invoice looks like this:

| Task / Date               | Time |  Price |
|---------------------------+------+--------|
| Tuesday, April 19, 2011   | 0:55 |  55.00 |
| DONE Project Foo          | 0:55 |  55.00 |
|---------------------------+------+--------|
| Monday, April 18, 2011    | 0:05 |   5.00 |
| DONE Task 1 for foo       | 0:05 |   5.00 |
|---------------------------+------+--------|
| Tuesday, April 19, 2011   | 0:25 |  25.00 |
| DONE Task 2 for foo       | 0:20 |  20.00 |
| DONE Task 2.1 for foo     | 0:05 |   5.00 |
|---------------------------+------+--------|
| Wednesday, April 20, 2011 | 0:25 |  25.00 |
| DONE Task 3 for foo       | 0:20 |  20.00 |
| DONE Task 3.1 for foo     | 0:05 |   5.00 |
|---------------------------+------+--------|
| Tuesday, April 19, 2011   | 0:05 |   5.00 |
| DONE Task 3 for foo       | 0:05 |   5.00 |
|---------------------------+------+--------|
| Total:                    | 1:55 | 115.00 |


There is double accounting of clock times in the invoice.

   1) Trees with subtasks are double accounted for time.
      This inflates the invoice and is incorrect.
   2) The total time at the bottom of the invoice also includes double
      accounted for clock times.

The 3rd entry

|-------------------------+------+-------|
| Tuesday, April 19, 2011 | 0:25 | 25.00 |
| DONE Task 2 for foo     | 0:20 | 20.00 |
| DONE Task 2.1 for foo   | 0:05 |  5.00 |
|-------------------------+------+-------|

is really only 20 minutes long with 15 minutes clocked in the Task 2
parent task and 5 minutes clocked in the Task 2.1 subtask.  The total
for the day should be 0:20 not 0:25 which also affects the invoice total
at the bottom.

Ditto for the 4th entry.

The Project Foo entry includes all 55 minutes for the project and is
added to all of the subtask entries which inflates the total for the
invoice.

Regards,
Bernt


--8<---------------cut here---------------start------------->8---
* Invoices
** DONE Project Foo
CLOSED: [2011-04-19 Tue 13:57]
:LOGBOOK:
- State "DONE"       from "STARTED"    [2011-04-19 Tue 13:57]
CLOCK: [2011-04-18 Mon 16:13]--[2011-04-18 Mon 16:18] =>  0:05
:END:
:PROPERTIES:
:RATE: 60
:END:
[2011-04-18 Mon 15:36]

#+BEGIN: columnview :hlines 1 :id: local
| Task                       | Estimated Effort | CLOCKSUM |
|----------------------------+------------------+----------|
| ** DONE Project Foo        |                  |     0:55 |
| *** DONE Task 1 for foo    |                  |     0:05 |
| *** DONE Task 2 for foo    |                  |     0:20 |
| **** DONE Task 2.1 for foo |                  |     0:05 |
| *** DONE Task 3 for foo    |                  |     0:20 |
| **** DONE Task 3.1 for foo |                  |     0:05 |
| *** DONE Task 4 for foo    |                  |     0:05 |
#+END:


#+BEGIN: invoice
| Task / Date               | Time |  Price |
|---------------------------+------+--------|
| Tuesday, April 19, 2011   | 0:55 |  55.00 |
| DONE Project Foo          | 0:55 |  55.00 |
|---------------------------+------+--------|
| Monday, April 18, 2011    | 0:05 |   5.00 |
| DONE Task 1 for foo       | 0:05 |   5.00 |
|---------------------------+------+--------|
| Tuesday, April 19, 2011   | 0:25 |  25.00 |
| DONE Task 2 for foo       | 0:20 |  20.00 |
| DONE Task 2.1 for foo     | 0:05 |   5.00 |
|---------------------------+------+--------|
| Wednesday, April 20, 2011 | 0:25 |  25.00 |
| DONE Task 3 for foo       | 0:20 |  20.00 |
| DONE Task 3.1 for foo     | 0:05 |   5.00 |
|---------------------------+------+--------|
| Tuesday, April 19, 2011   | 0:05 |   5.00 |
| DONE Task 4 for foo       | 0:05 |   5.00 |
|---------------------------+------+--------|
| Total:                    | 1:55 | 115.00 |

#+END:

*** DONE Task 1 for foo
SCHEDULED: <2011-04-18 Mon> CLOSED: [2011-04-18 Mon 12:09]
:LOGBOOK:
- State "DONE"       from "TODO"       [2011-04-18 Mon 12:09]
CLOCK: [2011-04-18 Mon 10:17]--[2011-04-18 Mon 10:22] =>  0:05
:END:
[2011-04-17 Sun 06:33]

*** DONE Task 2 for foo
CLOSED: [2011-04-19 Tue 13:56]
:LOGBOOK:
- State "DONE"       from "TODO"       [2011-04-19 Tue 13:56]
CLOCK: [2011-04-18 Mon 18:20]--[2011-04-18 Mon 18:35] =>  0:15
:END:
[2011-04-18 Mon 18:12]
**** DONE Task 2.1 for foo
CLOSED: [2011-04-19 Tue 13:56]
:LOGBOOK:
- State "DONE"       from "STARTED"    [2011-04-19 Tue 13:56]
CLOCK: [2011-04-19 Tue 13:56]--[2011-04-19 Tue 14:01] =>  0:05
:END:
[2011-04-19 Tue 12:05]

*** DONE Task 3 for foo
CLOSED: [2011-04-20 Wed 13:56]
:LOGBOOK:
- State "DONE"       from "TODO"       [2011-04-20 Wed 13:56]
CLOCK: [2011-04-19 Tue 18:20]--[2011-04-19 Tue 18:35] =>  0:15
:END:
[2011-04-19 Tue 18:12]
**** DONE Task 3.1 for foo
CLOSED: [2011-04-20 Wed 13:56]
:LOGBOOK:
- State "DONE"       from "STARTED"    [2011-04-20 Wed 13:56]
CLOCK: [2011-04-20 Wed 13:56]--[2011-04-20 Wed 14:01] =>  0:05
:END:
[2011-04-20 Wed 12:05]

*** DONE Task 4 for foo
CLOSED: [2011-04-19 Tue 17:01]
:LOGBOOK:
- State "DONE"       from "STARTED"    [2011-04-19 Tue 17:01]
CLOCK: [2011-04-19 Tue 16:58]--[2011-04-19 Tue 17:03] =>  0:05
:END:
[2011-04-19 Tue 16:58]
--8<---------------cut here---------------end--------------->8---

Re: Bug: No property change from ‘org-clock-sum’

[Ben Finney]

Carsten Dominik <carsten.dominik@gmail.com> writes:

> However, I seen now that there is a bug here, cause by a rewrite (for
> speed) of property access some time ago. After that rewrite, the
> special CLOCKSUM property used by org-invoice.el was no longer
> accessible.

Thanks! I am glad to finally have attracted enough attention to this bug
:-)


Bernt Hansen <bernt@norang.ca> writes:

> Carsten Dominik <carsten.dominik@gmail.com> writes:
> > This problem has just been fixed, so with the current git master,
> > org-invoice.el should work again.
>
> Ah! Thanks Carsten.  It does indeed return values after this fix.

Great. I look forward to a new release making its way into Debian.

Thanks again for the diagnosis and bug fix, guys.

-- 
 \         “Pinky, are you pondering what I'm pondering?” “I think so, |
  `\    Brain, but if we give peas a chance, won't the lima beans feel |
_o__)                                left out?” —_Pinky and The Brain_ |
Ben Finney