Bug: Appointments duration and effort sums in agenda column view [9.3.7 (release_9.3.7-700-ga1e5be @ ~/.emacs.d/straight/build/org/)]

Bug: Appointments duration and effort sums in agenda column view [9.3.7 (release_9.3.7-700-ga1e5be @ ~/.emacs.d/straight/build/org/)]

[Stanislav Vlasov]

Dear org community,

It seems that adding appointment duration to efforts sum is not working for Agenda Column View. Setting `agenda-columns-add-appointments-to-effort-sum' does not affect that.

Looking at the org-colview.el code I think that the reason why column view in org agenda can not add entry duration to efforts (even when org-agenda-columns-add-appointments-to-effort-sum is set to t) is because the function org-columns--collect-values which is called on each org entry by org-agenda-columns cannot actually get the text property 'duration at the point where it tries to do that.

I proposed a temporary workaround as an answer to the related question at  Emacs StackExchange - https://emacs.stackexchange.com/questions/58875/how-do-i-add-appointments-to-effort-sum

Kind regards,
Stas

------------------------------------------------------------------------

Emacs  : GNU Emacs 26.3 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.20)
 of 2020-05-19
Package: Org mode version 9.3.7 (release_9.3.7-700-ga1e5be @ ~/.emacs.d/straight/build/org/)

Re: Bug: Appointments duration and effort sums in agenda column view [9.3.7 (release_9.3.7-700-ga1e5be @ ~/.emacs.d/straight/build/org/)]

[Bastien]

Dear Stanislav,

thanks for reporting this, I confirm the bug.

I shared a note on the SO entry asking for a patch: since the
solution is mostly written, it would help if a patch could be
sent to this mailing list directly.

See https://orgmode.org/worg/org-contribute.html on how to
contribute.

Thanks!

-- 
 Bastien

Re: Bug: Appointments duration and effort sums in agenda column view [9.3.7 (release_9.3.7-700-ga1e5be @ ~/.emacs.d/straight/build/org/)]

[Bastien]

Dear Stanislav,

I'd like to revive this thread: did you have time to ask the person
on reddit to share the solution as a patch?  Or could you make this
patch yourself, by any chance?

Thanks a lot,

-- 
 Bastien

Re: Bug: Appointments duration and effort sums in agenda column view [9.3.7 (release_9.3.7-700-ga1e5be @ ~/.emacs.d/straight/build/org/)]

[Mamoru Miura]

[-- Attachment #1: Type: text/plain, Size: 327 bytes --]

Dear Bastien and org community,

I have also faced the problem and have made a patch to fix it. This 
patch would not be the best solution since I'm unfamiliar with Emacs 
Lisp and the detailed implementation of org-mode. I'd appreciate that if 
you would review and merge the patch attached to this mail.

Kind regards,
Mamoru

[-- Attachment #2: 0001-lisp-org-colview.el-Fix-missing-addition-of-appointm.patch --]
[-- Type: text/plain, Size: 1441 bytes --]

From 314877d3e4ec994a025f267a12ebc4d16b5a2aa5 Mon Sep 17 00:00:00 2001
From: Mamoru Miura <mamo3gr@gmail.com>
Date: Mon, 18 Jul 2022 18:52:40 +0900
Subject: [PATCH] lisp/org-colview.el: Fix missing addition of appointments to
 effort

* lisp/org-colview.el (org-columns--collect-values): re-compute
appointments' duration because this funtion can not get text property
`'duration'.

TINYCHANGE
---
 lisp/org-colview.el | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/lisp/org-colview.el b/lisp/org-colview.el
index 13643101b..35acc9deb 100644
--- a/lisp/org-colview.el
+++ b/lisp/org-colview.el
@@ -288,10 +288,9 @@ possible to override it with optional argument COMPILED-FMT."
 			     ;; to use appointment duration.
 			     org-agenda-columns-add-appointments-to-effort-sum
 			     (string= p (upcase org-effort-property))
-			     (get-text-property (point) 'duration)
-			     (propertize (org-duration-from-minutes
-					  (get-text-property (point) 'duration))
-					 'face 'org-warning))
+			     (when-let* ((formatted-item (org-agenda-format-item nil (org-get-entry) nil nil nil t))
+			                 (duration-string (get-text-property 0 'duration formatted-item)))
+			       (propertize (org-duration-from-minutes duration-string 'face 'org-warning)))
 			"")))
 	    ;; A non-nil COMPILED-FMT means we're calling from Org
 	    ;; Agenda mode, where we do not want leading stars for
-- 
2.37.0

Re: Bug: Appointments duration and effort sums in agenda column view [9.3.7 (release_9.3.7-700-ga1e5be @ ~/.emacs.d/straight/build/org/)]

[Mamoru Miura]

[-- Attachment #1: Type: text/plain, Size: 322 bytes --]

Dear org community,

I have also faced the problem and have made a patch to fix it. This patch would
not be the best solution since I'm unfamiliar with Emacs Lisp and the detailed
implementation of org-mode. I'd appreciate that if you would review and merge
the patch attached to this mail.

Kind regards,
Mamoru

[-- Attachment #2: 0001-lisp-org-colview.el-Fix-missing-addition-of-appointm.patch --]
[-- Type: application/octet-stream, Size: 1477 bytes --]

From 314877d3e4ec994a025f267a12ebc4d16b5a2aa5 Mon Sep 17 00:00:00 2001
From: Mamoru Miura <mamo3gr@gmail.com>
Date: Mon, 18 Jul 2022 18:52:40 +0900
Subject: [PATCH] lisp/org-colview.el: Fix missing addition of appointments to
 effort

* lisp/org-colview.el (org-columns--collect-values): re-compute
appointments' duration because this funtion can not get text property
`'duration'.

TINYCHANGE
---
 lisp/org-colview.el | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/lisp/org-colview.el b/lisp/org-colview.el
index 13643101b..35acc9deb 100644
--- a/lisp/org-colview.el
+++ b/lisp/org-colview.el
@@ -288,10 +288,9 @@ possible to override it with optional argument COMPILED-FMT."
 			     ;; to use appointment duration.
 			     org-agenda-columns-add-appointments-to-effort-sum
 			     (string= p (upcase org-effort-property))
-			     (get-text-property (point) 'duration)
-			     (propertize (org-duration-from-minutes
-					  (get-text-property (point) 'duration))
-					 'face 'org-warning))
+			     (when-let* ((formatted-item (org-agenda-format-item nil (org-get-entry) nil nil nil t))
+			                 (duration-string (get-text-property 0 'duration formatted-item)))
+			       (propertize (org-duration-from-minutes duration-string 'face 'org-warning)))
 			"")))
 	    ;; A non-nil COMPILED-FMT means we're calling from Org
 	    ;; Agenda mode, where we do not want leading stars for
-- 
2.37.0

Re: Bug: Appointments duration and effort sums in agenda column view [9.3.7 (release_9.3.7-700-ga1e5be @ ~/.emacs.d/straight/build/org/)]

[Ihor Radchenko]

Mamoru Miura <mamo0110@gmail.com> writes:

> Dear org community,
>
> I have also faced the problem and have made a patch to fix it. This patch would
> not be the best solution since I'm unfamiliar with Emacs Lisp and the detailed
> implementation of org-mode. I'd appreciate that if you would review and merge
> the patch attached to this mail.
>
> Kind regards,
> Mamoru
> From 314877d3e4ec994a025f267a12ebc4d16b5a2aa5 Mon Sep 17 00:00:00 2001
> From: Mamoru Miura <mamo3gr@gmail.com>
> Date: Mon, 18 Jul 2022 18:52:40 +0900
> Subject: [PATCH] lisp/org-colview.el: Fix missing addition of appointments to
>  effort
>
> * lisp/org-colview.el (org-columns--collect-values): re-compute
> appointments' duration because this funtion can not get text property
> `'duration'.

Could you please describe what kind of issue the patch fixes?
Ideally, can you provide steps required to see the issue. 

Best,
Ihor

Re: Bug: Appointments duration and effort sums in agenda column view [9.3.7 (release_9.3.7-700-ga1e5be @ ~/.emacs.d/straight/build/org/)]

[Mamoru Miura]

[-- Attachment #1: Type: text/plain, Size: 1991 bytes --]

Dear Ihor,

Thank you for responding.
The issue is failing to collect (and sum) appointments' duration as
effort even when `org-agenda-columns-add-appointments-to-effort-sum'
is t.

My environment:

* Emacs 28.1
* Org 9.5.4

Here's the setting (~/.emacs.d/init.el):

(setq org-directory "~/org/")
(setq org-agenda-files (list org-directory))
(setq org-columns-default-format "%Item %Effort{:}")
(setq org-agenda-columns-add-appointments-to-effort-sum t)

With putting org-file (~/org/test.org):

* My appointment
  SCHEDULED: <2022-07-20 Wed 10:00-11:00>
* My appointment with effort
  SCHEDULED: <2022-07-20 Wed 11:00-12:00>
  :PROPERTIES:
  :Effort:   0:30
  :END:

Steps to see this issue:

1. run M-x org-agenda
2. type "a" to show "agenda for current week or day"
3. run M-x org-agenda-columns (C-c C-x C-c)

Then, you see that the column "Effort" for the item "My appointment"
is empty and not added to the daily effort summation.

Bug investigation and solving approach:

* The function `org-columns--collect-values' collects appointment
duration: https://github.com/bzg/org-mode/blob/e0b05b07528dea684f3439c017370436b8d37b50/lisp/org-colview.el#L291
* With running M-x describe-text-properties on the item "My
appointment" in the *Org Agenda* buffer, it has a text property
"duration" and its value is correct (60.0).
Given these facts, the property "duration" seems to disappear
somewhere or the function fails to collect it. (Unfortunately, I
didn't have the room to deep dive into further details.)
* To solve this, instead of collecting value, get the corresponding
item (entry) and recompute the duration from it. I found a computing
function `org-agenda-format-item' and re-use it.

I re-attach a patch because the previous patch contains a bug.

See also (insightful reports by Stanislav):

* https://emacs.stackexchange.com/questions/58875/how-do-i-add-appointments-to-effort-sum
* https://lists.gnu.org/archive/html/emacs-orgmode/2020-08/msg00090.html

Kind regards,
Mamoru

[-- Attachment #2: 0001-lisp-org-colview.el-Fix-missing-addition-of-appointm.patch --]
[-- Type: application/octet-stream, Size: 1442 bytes --]

From bfdb96b0646eac2174d3ac632986e97e45711225 Mon Sep 17 00:00:00 2001
From: Mamoru Miura <mamo3gr@gmail.com>
Date: Wed, 20 Jul 2022 07:06:32 +0900
Subject: [PATCH] lisp/org-colview.el: Fix missing addition of appointments to
 effort

* lisp/org-colview.el (org-columns--collect-values): re-compute
appointments' duration because this funtion can not get text property
`'duration'.

TINYCHANGE
---
 lisp/org-colview.el | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/lisp/org-colview.el b/lisp/org-colview.el
index 13643101b..eae09e096 100644
--- a/lisp/org-colview.el
+++ b/lisp/org-colview.el
@@ -288,10 +288,9 @@ possible to override it with optional argument COMPILED-FMT."
 			     ;; to use appointment duration.
 			     org-agenda-columns-add-appointments-to-effort-sum
 			     (string= p (upcase org-effort-property))
-			     (get-text-property (point) 'duration)
-			     (propertize (org-duration-from-minutes
-					  (get-text-property (point) 'duration))
-					 'face 'org-warning))
+			     (when-let* ((formatted-item (org-agenda-format-item nil (org-get-entry) nil nil nil t))
+			                 (duration-string (get-text-property 0 'duration formatted-item)))
+			       (propertize (org-duration-from-minutes duration-string) 'face 'org-warning)))
 			"")))
 	    ;; A non-nil COMPILED-FMT means we're calling from Org
 	    ;; Agenda mode, where we do not want leading stars for
-- 
2.37.0

Re: Bug: Appointments duration and effort sums in agenda column view [9.3.7 (release_9.3.7-700-ga1e5be @ ~/.emacs.d/straight/build/org/)]

[Ihor Radchenko]

Mamoru Miura <mamo3gr@gmail.com> writes:

> ...
> Steps to see this issue:
>
> 1. run M-x org-agenda
> 2. type "a" to show "agenda for current week or day"
> 3. run M-x org-agenda-columns (C-c C-x C-c)

Thanks for providing the details! I have also found the relevant thread.
It was not immediately visible because of my MUA settings.

> -			     (get-text-property (point) 'duration)
> -			     (propertize (org-duration-from-minutes
> -					  (get-text-property (point) 'duration))
> -					 'face 'org-warning))
> +			     (when-let* ((formatted-item (org-agenda-format-item nil (org-get-entry) nil nil nil t))
> +			                 (duration-string (get-text-property 0 'duration formatted-item)))
> +			       (propertize (org-duration-from-minutes duration-string) 'face 'org-warning)))

I cannot say that I like calling org-agenda functions, but I cannot
propose anything better either (agenda code sucks...).

Ideally, we should factor out the duration calculation from the depths
of the agenda code into a separate function and then use it. But I am
not even sure if this calculation is always same in different agenda
types...

For the code, it will err when the time range is located inside the
headline and org-agenda-search-headline-for-time is set to non-nil
(which it is, by default)

Best,
Ihor

Re: Bug: Appointments duration and effort sums in agenda column view [9.3.7 (release_9.3.7-700-ga1e5be @ ~/.emacs.d/straight/build/org/)]

[Mamoru Miura]

Thank you for reviewing my patch.

>Ideally, we should factor out the duration calculation from the depths of the agenda code into a separate function and then use it.

I agree with it.

>But I am not even sure if this calculation is always same in different agenda types...

Where can I get the specification of agenda types? I'm interested in
the refactoring (but cannot work on it immediately...)

Kind regards,
Mamoru

2022年7月21日(木) 20:47 Ihor Radchenko <yantar92@gmail.com>:
>
> Mamoru Miura <mamo3gr@gmail.com> writes:
>
> > ...
> > Steps to see this issue:
> >
> > 1. run M-x org-agenda
> > 2. type "a" to show "agenda for current week or day"
> > 3. run M-x org-agenda-columns (C-c C-x C-c)
>
> Thanks for providing the details! I have also found the relevant thread.
> It was not immediately visible because of my MUA settings.
>
> > -                          (get-text-property (point) 'duration)
> > -                          (propertize (org-duration-from-minutes
> > -                                       (get-text-property (point) 'duration))
> > -                                      'face 'org-warning))
> > +                          (when-let* ((formatted-item (org-agenda-format-item nil (org-get-entry) nil nil nil t))
> > +                                      (duration-string (get-text-property 0 'duration formatted-item)))
> > +                            (propertize (org-duration-from-minutes duration-string) 'face 'org-warning)))
>
> I cannot say that I like calling org-agenda functions, but I cannot
> propose anything better either (agenda code sucks...).
>
> Ideally, we should factor out the duration calculation from the depths
> of the agenda code into a separate function and then use it. But I am
> not even sure if this calculation is always same in different agenda
> types...
>
> For the code, it will err when the time range is located inside the
> headline and org-agenda-search-headline-for-time is set to non-nil
> (which it is, by default)
>
> Best,
> Ihor

Re: Bug: Appointments duration and effort sums in agenda column view [9.3.7 (release_9.3.7-700-ga1e5be @ ~/.emacs.d/straight/build/org/)]

[Ihor Radchenko]

Mamoru Miura <mamo3gr@gmail.com> writes:

>>But I am not even sure if this calculation is always same in different agenda types...
>
> Where can I get the specification of agenda types? I'm interested in
> the refactoring (but cannot work on it immediately...)

1. 11.2 The Agenda Dispatcher section of the manual
2. org-agenda-custom-commands docstring
3. org-agenda function inside org-agenda.el
   Look for the lines
   (pcase type
    (`agenda

Hope it helps.

Best,
Ihor

P.S. Take your time if you need some and feel free to ask questions if
     you have any. We are not in hurry here. It is unlikely that anyone
     else will make significant changes in the agenda code any time
     soon.

Re: Bug: Appointments duration and effort sums in agenda column view [9.3.7 (release_9.3.7-700-ga1e5be @ ~/.emacs.d/straight/build/org/)]

[Mamoru Miura]

Thank you for your helpful information!
I'll try it.

Best,
Mamoru

2022年7月27日(水) 13:05 Ihor Radchenko <yantar92@gmail.com>:
>
> Mamoru Miura <mamo3gr@gmail.com> writes:
>
> >>But I am not even sure if this calculation is always same in different agenda types...
> >
> > Where can I get the specification of agenda types? I'm interested in
> > the refactoring (but cannot work on it immediately...)
>
> 1. 11.2 The Agenda Dispatcher section of the manual
> 2. org-agenda-custom-commands docstring
> 3. org-agenda function inside org-agenda.el
>    Look for the lines
>    (pcase type
>     (`agenda
>
> Hope it helps.
>
> Best,
> Ihor
>
> P.S. Take your time if you need some and feel free to ask questions if
>      you have any. We are not in hurry here. It is unlikely that anyone
>      else will make significant changes in the agenda code any time
>      soon.