clock-table and hooking that into org-capture file+olp+datetree

clock-table and hooking that into org-capture file+olp+datetree

[Christopher Causer]

Hello everyone! Here's a reasonably easy (I think) question because I'm quite new to Emacs and org-mode.

I have an org-capture template using file+olp+datetree[1], which works great at filing my thoughts for the day. Separately I know I can generate clock tables[2] based on dynamic blocks to show me what I've been doing with my time for any given period. What I'm struggling with is to glue parts of these together to achieve the following:

1. I org-capture to the datetree. When it does so it either creates or updates an org-clock-report at the top of the datetree header  (the bit that says "2020-11-12 Thursday", for example.) I guess this would be the parent of what I'm capturing.

2. For all my historical journal entries, if I could move point to a headline with a date such as the example below and it would pull the date out and add a clocktable below via an interactive function that would be my ideal. This is less of a problem for me as I don't have much in the way of history in my diary yet or my other org files.

An example tree would be

#+BEGIN_QUOTE
* Work
** 2021
*** 2021-01 January
**** 2021-01-07 Thursday
***** Ate some chips
***** Drank some soda
#+END_QUOTE

I tried looking at the org-mode source, and it is too advanced for me to follow. The closest I can get (or at least here's a scrap of code to prove I did actually try) is as follows:

#+BEGIN_SRC emacs-lisp
(defun cc/create-or-update-effort-table ()
  (save-excursion
    (find-file (concat org-directory "/diary.org"))
    (goto-char (org-find-olp (list (concat org-directory "/diary.org" ) "Work" "2021" "2021-01 January" "2021-01-07 Thursday")))
    (beginning-of-line)
    (next-line)
    (if (looking-at-p "[[:space:]]*#\\+BEGIN: clocktable") (forward-word 3) (progn (insert "\n")(previous-line)))
    (org-clock-report)))
#+END_SRC

A lot of beginner missteps there that I'm sure people can correct, but hopefully you can see the intent. If you see anything amiss please let me know, but my main problems are:

1. I hardcoded the datetree part whereas I'd like this to be today's date or even the date picker built into org-mode. I am very far off using the datepicker to generate ("%Y" "%Y-%m %B" "%Y-%m-%d %A") [3] or even from today's date.

2. I obviously don't understand how markers work well enough because I had to add the ~find-file~ whereas I would imagine you could just do it all in the  goto-char line, if I knew how to use markers better .

3. The function would fail if the tree doesn't exist. I'd like it created like org-capture would. Not all clocked items are in my diary, so I may want to run the function before any diary items exist for today.

4. I would like to customize the variables org-clock-report uses to generate the report. It looks to be assigned by the defaults ~org-clocktable-defaults~. The  ~:block~ option is the obvious option I'd like to change to a set date. In other words, I want to change the options in the function above but not use org-clocktable-defaults unless it reverts after the function is finished.

5. Even if the function above worked, I have no idea how to hook that into file+olp+datetree. Would I need to switch to file+function and add the clock table as a side-effect?

The function cc/create-or-update-effort-table has to use olp, not headline, as the headline "2021-01-07 Thursday" is not unique in the file.

Thank you for taking the time to read this far, and thank you to all the people who've contributed to Emacs and Org-Mode. I wish I'd discovered them sooner.

Christopher.


[1] https://orgmode.org/manual/Template-elements.html
[2] https://orgmode.org/manual/The-clock-table.html
[3] https://man7.org/linux/man-pages/man1/date.1.html

The links are for my benefit when I come back to this email, rather than anyone else's, but I guess it doesn't hurt to include them for other people new to org-mode like myself.

clock-table and hooking that into org-capture file+olp+datetree

[Christopher Causer]

Hello everyone! Here's a reasonably easy (I think) question because I'm quite new to Emacs and org-mode.

I have an org-capture template using file+olp+datetree[1], which works great at filing my thoughts for the day. Separately I know I can generate clock tables[2] based on dynamic blocks to show me what I've been doing with my time for any given period. What I'm struggling with is to glue parts of these together to achieve the following:

1. I org-capture to a subheading of datetree. When it does so it either creates or updates an org-clock-report just below the datetree header  (the bit that says "2020-11-12 Thursday", for example.) I guess this would be the parent of what I'm capturing.

2. For all my historical journal entries, if I could move point to a headline with a date such as the example below and it would pull the date out and add a clocktable below via an interactive function that would be my ideal. This is less of a problem for me as I don't have much in the way of history in my diary yet or my other org files.

An example tree would be

#+BEGIN_QUOTE
* Work
** 2021
*** 2021-01 January
**** 2021-01-07 Thursday
***** Ate some chips
***** Drank some soda
#+END_QUOTE

I tried looking at the org-mode source, and it is too advanced for me to follow. The closest I can get (or at least here's a scrap of code to prove I did actually try) is as follows:

#+BEGIN_SRC emacs-lisp
(defun cc/create-or-update-effort-table ()
  (save-excursion
    (find-file (concat org-directory "/diary.org"))
    (goto-char (org-find-olp (list (concat org-directory "/diary.org" ) "Work" "2021" "2021-01 January" "2021-01-07 Thursday")))
    (beginning-of-line)
    (next-line)
    (if (looking-at-p "[[:space:]]*#\\+BEGIN: clocktable") (forward-word 3) (progn (insert "\n")(previous-line)))
    (org-clock-report)))
#+END_SRC

A lot of beginner missteps there that I'm sure people can correct, but hopefully you can see the intent. If you see anything amiss please let me know, but my main problems are:

1. I hardcoded the datetree part whereas I'd like this to be today's date or even the date picker built into org-mode. I am very far off using the datepicker to generate ("%Y" "%Y-%m %B" "%Y-%m-%d %A") [3]. This would also be needed for point 4.

2. I obviously don't understand how markers work well enough because I had to add the ~find-file~ whereas I would imagine you could just do it all in the  goto-char line, if I knew how to use markers better.

3. The function would fail if the tree doesn't exist. I'd like it created like org-capture would. Not all clocked items are in my diary, so I may want to run the function before any diary items exist for today.

4. I would like to customize the variables org-clock-report uses to generate the report. It looks to be assigned by the defaults ~org-clocktable-defaults~. The  ~:block~ option is the obvious option I'd like to change to a set date. In other words, I want to change the options in the function above but not use org-clocktable-defaults unless it reverts after the function is finished.

5. Even if the function above worked, I have no idea how to hook that into file+olp+datetree. Would I need to switch to file+function and add the clock table as a side-effect?

The function cc/create-or-update-effort-table has to use olp, not headline, as the headline "2021-01-07 Thursday" is not unique in the file.

Thank you for taking the time to read this far, and thank you to all the people who've contributed to Emacs and Org-Mode. I wish I'd discovered them sooner.

Christopher.


[1] https://orgmode.org/manual/Template-elements.html
[2] https://orgmode.org/manual/The-clock-table.html
[3] https://man7.org/linux/man-pages/man1/date.1.html

The links are for my benefit when I come back to this email, rather than anyone else's, but I guess it doesn't hurt to include them for other people new to org-mode like myself.

Re: clock-table and hooking that into org-capture file+olp+datetree

[Richard Lawrence]

Hi Christopher,

"Christopher Causer" <ml-emacs-orgmode@chyc.co.uk> writes:

> Hello everyone! Here's a reasonably easy (I think) question because I'm quite new to Emacs and org-mode.
>
> I have an org-capture template using file+olp+datetree[1], which works great at filing my thoughts for the day. Separately I know I can generate clock tables[2] based on dynamic blocks to show me what I've been doing with my time for any given period. What I'm struggling with is to glue parts of these together to achieve the following:
>
> 1. I org-capture to a subheading of datetree. When it does so it either creates or updates an org-clock-report just below the datetree header  (the bit that says "2020-11-12 Thursday", for example.) I guess this would be the parent of what I'm capturing.
>
> 2. For all my historical journal entries, if I could move point to a headline with a date such as the example below and it would pull the date out and add a clocktable below via an interactive function that would be my ideal. This is less of a problem for me as I don't have much in the way of history in my diary yet or my other org files.
>

If I understand right, what you need for both of these things is a
function to jump to a date in your diary datetree and update the
clocktable there. Right?

Some functions that will help with this:
- org-datetree-find-date-create
- org-narrow-to-subtree

So, something like this should get you started:

#+begin_src emacs-lisp
(defun org-update-clocktable-on-date (date)
  (save-excursion
    ;; open the file containing the datetree:
    (find-file (concat org-directory "/diary.org"))
    ;; jump to the subtree for the given date:
    ;; note: date must look like (m d y) where all three values are integers
    (org-datetree-find-date-create date)
    ;; narrow to the subtree for this date, so we don't update
    ;; any other clocktables
    (org-narrow-to-subtree)
    ;; update the clock report, or create it if it doesn't exist
    ;; note: we pass a prefix argument to tell org-clock-report to
    ;; update the first clocktable it finds in the (narrowed) buffer
    (org-clock-report t)
    ;; widen to the whole buffer again
    (widen)))
#+end_src

Then you can call this function, providing the date, in different
contexts where you want to create or update the clocktable.

Note that org-datetree-find-date has a slightly annoying interface, in
that you need to provide a list of three integers representing a
calendar date. One easy way to do that interactively is with
calendar-read-date, which prompts you for the year, month and day, so
you could say

(org-update-clocktable-on-date (calendar-read-date))

calendar-read-date is not as nice to use interactively as org-read-date,
but as far as I know, there is no easy way to get the calendar (m d y) format
out of its return value, which is either a string like "2021-01-30" or a
value in Emacs' internal time representation format.  But you can do
something like

(let*
    ;; prompt for the date and decode the resulting internal time as a list:
    ((decoded (decode-time (org-read-date nil t nil "Update on date:")))
    ;; unpack the date as a list (m d y) from the decoded time:
     (date (list (nth 4 decoded) ; month
                 (nth 3 decoded) ; day
                 (nth 5 decoded)))) ; year

  (org-update-clocktable-on-date date))

Hope that helps get you to your next step!

-- 
Best,
Richard

Re: clock-table and hooking that into org-capture file+olp+datetree

[Christopher Causer]

On Sat, 30 Jan 2021, at 12:53, Richard Lawrence wrote:
> #+begin_src emacs-lisp
> (defun org-update-clocktable-on-date (date)
>   (save-excursion
>     ;; open the file containing the datetree:
>     (find-file (concat org-directory "/diary.org"))
>     ;; jump to the subtree for the given date:
>     ;; note: date must look like (m d y) where all three values are integers
>     (org-datetree-find-date-create date)
>     ;; narrow to the subtree for this date, so we don't update
>     ;; any other clocktables
>     (org-narrow-to-subtree)
>     ;; update the clock report, or create it if it doesn't exist
>     ;; note: we pass a prefix argument to tell org-clock-report to
>     ;; update the first clocktable it finds in the (narrowed) buffer
>     (org-clock-report t)
>     ;; widen to the whole buffer again
>     (widen)))
> #+end_src

This is wonderful Richard, and a great help to me. I had no idea of the org-datree-find-date-create, and the argument to org-clock-report cuts out a lot of my code. 

> Note that org-datetree-find-date has a slightly annoying interface, in
> that you need to provide a list of three integers representing a
> calendar date. 

Yes, that is a little awkward. What I did think of using was substrings to extract the date from the picker interface.

#+BEGIN_SRC emacs-lisp
(defun org-date-picker-to-list ()
  (let* ((date-string (org-read-date))
	 (year (substring date-string 0 4))
	 (month (substring date-string 5 7))
	 (day (substring date-string 8 10)))
    (mapcar 'string-to-number (list month day year))))
#+END_SRC

Does that look sensible? 

My two next things to tackle are 

1. A hook to run the function when I run org-capture.
2. Changing the org-clock-report options in your function above, but not the defaults.

For the second point, is there some trick to swap a global variable for the run of a function? The variable in this case would be org-clocktable-defaults.

Thank you so much Richard. You've probably saved me days of going through the org-mode documentation and source. 

Re: clock-table and hooking that into org-capture file+olp+datetree

[Richard Lawrence]

Hi Christopher and all,

"Christopher Causer" <ml-emacs-orgmode@chyc.co.uk> writes:

>> Note that org-datetree-find-date has a slightly annoying interface, in
>> that you need to provide a list of three integers representing a
>> calendar date. 
>
> Yes, that is a little awkward. What I did think of using was substrings to extract the date from the picker interface.
>
> #+BEGIN_SRC emacs-lisp
> (defun org-date-picker-to-list ()
>   (let* ((date-string (org-read-date))
> 	 (year (substring date-string 0 4))
> 	 (month (substring date-string 5 7))
> 	 (day (substring date-string 8 10)))
>     (mapcar 'string-to-number (list month day year))))
> #+END_SRC
>
> Does that look sensible? 

As sensible as anything :) 

I recently added my version, using decode-time and nth, as a helper
function to my .emacs. I feel sure that it must already exist
*somewhere* in Emacs but I searched quite a while for it and didn't find
it. I will use this again below:

#+begin_src emacs-lisp 
  (defun time-as-calendar-date (time)
    "Convert time in Emacs' time format to a calendar date list (MONTH DAY YEAR)"
    (let ((parsed-time (decode-time time)))
      (list
       (nth 4 parsed-time)
       (nth 3 parsed-time)
       (nth 5 parsed-time))))
#+end_src

> My two next things to tackle are 
>
> 1. A hook to run the function when I run org-capture.

Here you might find it useful to grab the value of the :default-time key
from org-capture-plist (which should contain either the time you entered
at the date prompt during capture, or the current time, in Emacs' time
format). Then, using the functions above, you can say something like
this in your hook:

#+being_src emacs-lisp
(let* ((default-time (plist-get org-capture-plist :default-time))
       (date (time-as-calendar-date default-time)))
  (org-update-clocktable-on-date date))
#+end_src

> 2. Changing the org-clock-report options in your function above, but not the defaults.
>
> For the second point, is there some trick to swap a global variable for the run of a function? The variable in this case would be org-clocktable-defaults.

Emacs Lisp has dynamic scope by default, which makes this is pretty easy
in general: just set the value you want to use in a let form around the
code that uses this variable, like:

(let ((org-clocktable-defaults your-custom-value-here))
  ...)

> Thank you so much Richard. You've probably saved me days of going through the org-mode documentation and source. 

No problem! That's what this list is for. I recently spent a fair amount
of time digging through the datetree stuff myself, so I was glad to have
a chance to share what I learned.

-- 
Best,
Richard

Re: clock-table and hooking that into org-capture file+olp+datetree

[Christopher Causer]

Thanks again Richard. I now have a working solution which I share below, warts and all.

One snag I hit that I feel I should mention is what I believe to be a mistake in the documentation[1]. I was scratching my head as why org-clocktable-defaults wasn't working, but it was only when I brought up the inline documentation that I see that this probably is some vestigial variableand it has since moved to org-clock-clocktable-default-properties. Is mentioning this here on the list enough or should I report it somewhere else?

Christopher 

#+BEGIN_SRC emacs-lisp
(defun org-date-to-list ()
  "Run `org-read-date' and return it in the form '(mm dd YYYY).

This form is more useful for functions that require it in
this format, such as `org-datetree-find-date-create'."

  (let* ((date-string (org-read-date))
	 (year (substring date-string 0 4))
	 (month (substring date-string 5 7))
	 (day (substring date-string 8 10)))
    (mapcar 'string-to-number (list month day year))))

(defun org-update-clocktable-on-date (date)
  (let* ((year (number-to-string (nth 2 date)))
	 (month (number-to-string (nth 0 date)))
	 (day (number-to-string (nth 1 date)))
	 (org-clock-clocktable-default-properties
	  (list :scope 'agenda :maxlevel 6 :block (concat year "-" month "-" day))))
	 (save-excursion
	   ;; open the file containing the datetree:
	   (find-file (concat org-directory "/diary.org"))
	   ;; jump to the subtree for the given date:
	   ;; note: date must look like (m d y) where all three values are integers
	   (org-datetree-find-date-create date)
	   ;; narrow to the subtree for this date, so we don't update
	   ;; any other clocktables
	   (org-narrow-to-subtree)
	   ;; update the clock report, or create it if it doesn't exist
	   ;; note: we pass a prefix argument to tell org-clock-report to
	   ;; update the first clocktable it finds in the (narrowed) buffer
	   (org-clock-report t)
	   ;; widen to the whole buffer again
	   (widen))))


;; Usage
; (org-update-clocktable-on-date (org-date-to-list))
#+END_SRC

[1] Variable org-clocktable-defaults is mentioned here https://orgmode.org/manual/The-clock-table.html

Re: clock-table and hooking that into org-capture file+olp+datetree

[Kyle Meyer]

Christopher Causer writes:

> One snag I hit that I feel I should mention is what I believe to be a
> mistake in the documentation[1]. I was scratching my head as why
> org-clocktable-defaults wasn't working, but it was only when I brought
> up the inline documentation that I see that this probably is some
> vestigial variableand it has since moved to
> org-clock-clocktable-default-properties.

Despite their confusingly similar names as well as
org-clock-clocktable-default-properties not being documented in the
manual, they serve distinct purposes as far as I understand.
org-clock-clocktable-default-properties is about which properties are
inserted as part of the BEGIN line, while org-clocktable-defaults is the
set of defaults that are used when the value is not taken from the BEGIN
line.

For example, here's the result of calling
org-dynamic-block-insert-dblock and selecting "clocktable" with the
built-in values for both these options:

  * one
  :LOGBOOK:
  CLOCK: [2021-02-01 Mon 17:48]--[2021-02-01 Mon 19:48] =>  2:00
  
  #+BEGIN: clocktable :scope subtree :maxlevel 2
  #+CAPTION: Clock summary at [2021-02-02 Tue 00:08]
  | Headline     | Time   |
  |--------------+--------|
  | *Total time* | *2:00* |
  |--------------+--------|
  | one          | 2:00   |
  #+END:

Redoing that after

  (setq org-clock-clocktable-default-properties '(:fileskip0 nil))

it looks like this

  * one
  :LOGBOOK:
  CLOCK: [2021-02-01 Mon 17:48]--[2021-02-01 Mon 19:48] =>  2:00
  
  #+BEGIN: clocktable :scope subtree :fileskip0 nil
  #+CAPTION: Clock summary at [2021-02-02 Tue 00:10]
  | Headline     | Time   |
  |--------------+--------|
  | *Total time* | *2:00* |
  |--------------+--------|
  | one          | 2:00   |
  #+END:

Notice the ":fileskip0 nil" in the BEGIN line.

And redoing it once more with

  (setq org-clocktable-defaults
        (plist-put org-clocktable-defaults :link t))

on top

  * one
  :LOGBOOK:
  CLOCK: [2021-02-01 Mon 17:48]--[2021-02-01 Mon 19:48] =>  2:00
  
  #+BEGIN: clocktable :scope subtree :fileskip0 nil
  #+CAPTION: Clock summary at [2021-02-02 Tue 00:11]
  | Headline     | Time   |
  |--------------+--------|
  | *Total time* | *2:00* |
  |--------------+--------|
  | [[file:/tmp/scratch.org::*one][one]]          | 2:00   |
  #+END:

Notice that `:link t' has an effect, but it's not inserted in the BEGIN
line.