Help with dblocks and table formulas (and an RFC on quantitative logging)

Help with dblocks and table formulas (and an RFC on quantitative logging)

[Eric Abrahamsen]

Hi all,

I'm finally trying to scratch a very old itch of mine: quantitative
logging in Org. Basically it's the same idea as org-clock, except
generalized to log random values. Rather than explain at length, below is
what a sample todo would look like, plus the created dblock. You specify
a LOG_VALUES property (multiple labels are possible), and then anytime
you take a note, you're prompted to enter values for those labels. Then
there's a command for making the dblock.

*** CHECK Test blood glucose
SCHEDULED: <2019-06-19 Wed +1d>
:PROPERTIES:
:ID:       dummy
:LOG_VALUES: MG
:LAST_REPEAT: [2019-06-18 Tue 19:55]
:END:
:LOGBOOK:
- State "DONE"      from "CHECK"      [2019-06-18 Tue 19:56] \\
  MG: 98;
- State "DONE"      from "CHECK"      [2019-06-17 Mon 19:55] \\
  MG: 110;
- State "DONE"      from "CHECK"      [2019-06-16 Sun 19:55] \\
  MG: 103;
:END:

#+BEGIN: log :id "dummy"
| Timestamp              |  Mg |
|------------------------+-----|
| [2019-06-18 Tue 19:56] |  98 |
| [2019-06-17 Mon 19:55] | 110 |
| [2019-06-16 Sun 19:55] | 103 |
#+END:

The code for this is below, I'm sure it's a mess and doesn't take into
account hardly any of the variety of Org configs.

But it sort of works! I have a few questions:

1. I can't reliably trigger the note prompting using TODO keyword setup.
   The above example is actually manually adjusted because I couldn't
   get it to prompt for a note when going into DONE state, even with
   DONE(@). But presumably that's my own confusion.
2. I don't know how to handle defining table formulas. Are users meant
   to insert them manually into the dblock? As a property on the
   heading? What's the usual thing to do?
3. What I'd really like is to have this integrated with clocking, so
   that clocking out also prompts for log values, and then you can make
   tables that show not only how many hours you worked today, but also
   how many... widgets you frobbed in that time. I guess this isn't
   really a question, just an indication of where I hope it might go.
   
Anyway, this is a total work in progress, but I would be curious if
anyone would find this useful, and for what.

Eric

#+src elisp
;;; Code:

(require 'org)

(defun org-log-prompt ()
  "Prompt the user for log values.
Insert values into the current log note."
  ;; This is called in the log buffer.  Only fire for state and note;
  ;; later add clock-out.
  (when (memq org-log-note-purpose '(state note))
    (let ((values
	   (with-current-buffer (marker-buffer org-log-note-marker)
	     (save-excursion
	       (goto-char org-log-note-marker)
	       (org-entry-get (point) "LOG_VALUES" 'selective)))))
      (when (and (stringp values) (null (string-empty-p values)))
	(unless (bolp)		     ; This might follow a clock line.
	  (insert "; "))
	(dolist (val (split-string values))
	  ;; Maybe strip off units.
	  (setq val (substring val 0 (string-match-p "\\[" val)))
	  (insert val ": ")
	  (insert (read-string (format "%s: " val)) "; "))))))

(defun org-log--collect-data (id)
  "Collect log data from heading with id ID.
When valid data is found, it is returned as a list of lists.
Each sublist starts with the timestamp of the log entry, followed
by data keys and values, in the order they were found in the log
entry.

If no valid data is found, return nil."
  (save-excursion
    (org-id-goto id)
    (goto-char (org-log-beginning))
    (let* ((struct (org-list-struct))
	   (labels (org-entry-get (point) "LOG_VALUES" 'selective))
	   (entries (when (and (stringp labels)
			       (null (string-empty-p labels)))
		      ;; First element is a list of value labels.
		      (list (cons "TIMESTAMP"
				  (mapcar (lambda (str)
					    (substring
					     str 0 (string-match-p "\\[" str)))
					  (split-string labels))))))
	   elt data)
      (when (and entries struct)
	;; Get onto the list item.
	(forward-char)
	(while (equal 'item (org-element-type
			     (setq elt (org-element-at-point))))
	  ;; Move past state/timestamp line.
	  (forward-line)
	  (while (re-search-forward "[[:upper:]]+: \\([^;]+\\)"
				    (point-at-eol) t)
	    (push (match-string-no-properties 1) data))
	  (when data
	    (save-excursion
	      (forward-line -1)
	      ;; Get the log entry timestamp.
	      (setq data
		    (cons (if (re-search-forward org-ts-regexp-inactive
						 (point-at-eol) t)
			      (match-string-no-properties 0)
			    "none")
			  (nreverse data))))
	    (push data entries)
	    (setq data nil))
	  (goto-char (org-element-property :end elt)))
	(nreverse entries)))))

(defun org-log-create-dblock ()
  "Create a dblock with a table for this heading's log data."
  (interactive)
  (let ((id (org-id-get-create)))
    ;; Anyway, this call is the heart of it.
    (org-create-dblock
     `(:name "log" :id ,id))
    (org-update-dblock)))

(defun org-dblock-write:log (params)
  "Write the log dblock table."
  (let ((data (org-log--collect-data (plist-get params :id))))
    (when data
      (save-excursion
	(insert
	 (format "|%s|\n" (mapconcat #'capitalize (pop data) "|"))
	 "|-|\n"
	 (mapconcat (lambda (row)
		      (format "|%s|" (mapconcat #'identity row "|")))
		    data "\n")))
      (org-table-align))))

(add-hook 'org-log-buffer-setup-hook #'org-log-prompt)

(provide 'org-log)
;;; org-log.el ends here
#+end

Re: Help with dblocks and table formulas (and an RFC on quantitative logging)

[Fraga, Eric]

On Tuesday, 18 Jun 2019 at 20:18, Eric Abrahamsen wrote:
> I'm finally trying to scratch a very old itch of mine: quantitative
> logging in Org. 
[...]

> 1. I can't reliably trigger the note prompting using TODO keyword setup.

What about using org-after-todo-stage-change-hook?
    
> Anyway, this is a total work in progress, but I would be curious if
> anyone would find this useful, and for what.

It's interesting and I can see at least one use: logging expenses.  I
normally use ledger but this is overkill for some simple tasks.

-- 
Eric S Fraga via Emacs 27.0.50, Org release_9.2.3-379-gff2bf2

Re: Help with dblocks and table formulas (and an RFC on quantitative logging)

[Eric Abrahamsen]

"Fraga, Eric" <e.fraga@ucl.ac.uk> writes:

> On Tuesday, 18 Jun 2019 at 20:18, Eric Abrahamsen wrote:
>> I'm finally trying to scratch a very old itch of mine: quantitative
>> logging in Org. 
> [...]
>
>> 1. I can't reliably trigger the note prompting using TODO keyword setup.
>
> What about using org-after-todo-stage-change-hook?

Hmm, that's a possibility, thanks. There are so many fiddly bits with
getting the right information out of the right dynamic variables, it
might take a while to pin this down...

>> Anyway, this is a total work in progress, but I would be curious if
>> anyone would find this useful, and for what.
>
> It's interesting and I can see at least one use: logging expenses.  I
> normally use ledger but this is overkill for some simple tasks.

Cool, well, I'll report back when I have something a little more
mature...

eric