Recording Time - org-clock-sum

Recording Time - org-clock-sum

[Fredrik Unger]

Hi,

I have been using Emacs Orgmode for 3-4 years recording time.
It started out with Sacha Chua:s article "Clocking Time with Emacs Org" 
[1] where the org-dblock-write:rangereport did what I needed.

As time went by I have made some adaptations to it and simplified it
but kept the core idea [2].
One addition is: that it is not based no increments of 24*60*60 but 1 
day, which removed problems by Daylight savings time (in October).
Maybe more type safe functions exists like in Perl, [3] but I did not 
find in org-mode. With my change I rely on that emacs date calculations
are in the same shape as [3]. It worked for this case and I stopped at that.

I am not a full out user of Orgmode and I do not keep project files.
I use one simple file per month to record my work, and it has been a 
good choice for me. Maybe someday I can switch to some other structure 
but as the CLOCK lines takes some space, I'd rather "forget" CLOCK 
history on a monthly basis. Projects do not last more than 2-3 month 
anyway and the catch-all internal project that is "never ending" would 
become too crowded with CLOCK lines.
It is also easier to spot clocking mistakes if the file is limited to a 
month.

Projects can not be summarized across month, but for my reporting needs
that is not needed.

The data is clocked as follows [4] per month. More project,
items and clocked times are in a normal org file, but this
simplified case should do.

I want to expand the report to show start/end time + breaks.

Eg from :
Time report November 2014
<2014-11-05 Wed> -- 8:00 (8.00)

to :
Time report November 2014
<2014-11-05 Wed> -- 8:00 (8.00)  s 07:00 e 16:00 b 11:00-12:00,

Format is not set, but I need these time for my reporting.
In the normal case it is start-end and lunch.

I have tried to work with org-clock-sum to get the data from the CLOCK 
entries. My elisp is rusty, and I missed that the regexp was */level 
dependent. I managed to get a list of dates but just from the CLOCK 
under the Item.
(((2014-11-05 12:00) (2014-11-05 14:00))
  ((2014-11-05 14:00) (2014-11-05 16:00)))

There were some strange duplicates as well so I guess I just did things 
wrong.

To get the wanted result I would need a regexp that scans all CLOCK:
and reports the two timestamps and generate a list of
((time1 time2) (time2 time3) (time4 time5) (time5 time6)) filtered
on the call with tstart tend like org-clock-sum.
For me the actual heading is not needed.

With this list I can then flaten it, and if needed sort it :
(time1 time2 time2 time3 time4 time5 time5 time6)
If I then remove duplicates all together I will get :
(time1 time3 time4 time6)
where min is start and max is end, and pairs in between are breaks.

s time1 e time6 b time3-time4,

I did not manage to create this list of all CLOCK values filtered on 
tstart/tend (which in my case would be one day at the time just like
I call org-clock-sum for the total sum).

I would be grateful for advice or pointers to a better way to find
all CLOCK values. Feel free to publish [2] on the wiki if you find
it valuable.
If/when I have a full working solution I might publish it on my site.

Thank you,

Fredrik Unger

[1] http://sachachua.com/blog/2007/12/clocking-time-with-emacs-org/
[2] https://tree.se/rangereport.el
[3] https://metacpan.org/pod/DateTime#How-DateTime-Math-Works
[4] https://tree.se/t.org

Re: Recording Time - org-clock-sum

[Sebastien Vauban]

Just a side note...

Fredrik Unger wrote:
> I am not a full out user of Orgmode and I do not keep project files.
> I use one simple file per month to record my work, and it has been a good choice
> for me. Maybe someday I can switch to some other structure but as the CLOCK
> lines takes some space, I'd rather "forget" CLOCK history on a monthly
> basis. Projects do not last more than 2-3 month anyway and the catch-all
> internal project that is "never ending" would become too crowded with CLOCK
> lines.
> It is also easier to spot clocking mistakes if the file is limited to a month.

I use CLOCKING for several years now (around 4) and commit to the same
entries for repeating tasks such as "read emails". One information for
you : there is no problem to spot clocking mistakes if you use `v c'
("clockcheck"); you can easily trap clocking gaps and overlaps. Without
that, yes, that'd be pretty unusable...

Best regards,
  Seb

-- 
Sebastien Vauban

Re: Recording Time - org-clock-sum

[Fredrik Unger]

[-- Attachment #1: Type: text/plain, Size: 4540 bytes --]

Hi,

I implemented a couple of functions that provide me with the
needed timestamps for my timereport [1].

ex.
<2014-11-14 Fri>    [s 08:00 e 14:00   (11:00 12:00)]   [ 5:00 (5.00)]

Summarized from :

* Test
** first
     CLOCK: [2014-11-14 Fri 08:00]--[2014-11-14 Fri 09:00] =>  1:00
     CLOCK: [2014-11-14 Fri 12:00]--[2014-11-14 Fri 13:00] =>  1:00
** second
     CLOCK: [2014-11-14 Fri 09:00]--[2014-11-14 Fri 11:00] =>  2:00
     CLOCK: [2014-11-14 Fri 13:00]--[2014-11-14 Fri 14:00] =>  1:00


Started work at 8, ended at 14 lunch break 11-12. worked 5 h.
(h:m and decimal representation)

One can adapt the format with "timestr".

It is based on the org-clock-sum function. Tested only with simple
files with complete Time stamps (no dangling clocks etc).

timeuniq takes a list of timestamps and returns only timestamps that 
were unique in the initial list.

Suggestion for improvements are welcome, as I am an elisp/org-mode beginner.

Fredrik Unger

[1] attached range-report.emacs


On 11/05/2014 06:00 PM, Fredrik Unger wrote:
> Hi,
>
> I have been using Emacs Orgmode for 3-4 years recording time.
> It started out with Sacha Chua:s article "Clocking Time with Emacs Org"
> [1] where the org-dblock-write:rangereport did what I needed.
>
> As time went by I have made some adaptations to it and simplified it
> but kept the core idea [2].
> One addition is: that it is not based no increments of 24*60*60 but 1
> day, which removed problems by Daylight savings time (in October).
> Maybe more type safe functions exists like in Perl, [3] but I did not
> find in org-mode. With my change I rely on that emacs date calculations
> are in the same shape as [3]. It worked for this case and I stopped at
> that.
>
> I am not a full out user of Orgmode and I do not keep project files.
> I use one simple file per month to record my work, and it has been a
> good choice for me. Maybe someday I can switch to some other structure
> but as the CLOCK lines takes some space, I'd rather "forget" CLOCK
> history on a monthly basis. Projects do not last more than 2-3 month
> anyway and the catch-all internal project that is "never ending" would
> become too crowded with CLOCK lines.
> It is also easier to spot clocking mistakes if the file is limited to a
> month.
>
> Projects can not be summarized across month, but for my reporting needs
> that is not needed.
>
> The data is clocked as follows [4] per month. More project,
> items and clocked times are in a normal org file, but this
> simplified case should do.
>
> I want to expand the report to show start/end time + breaks.
>
> Eg from :
> Time report November 2014
> <2014-11-05 Wed> -- 8:00 (8.00)
>
> to :
> Time report November 2014
> <2014-11-05 Wed> -- 8:00 (8.00)  s 07:00 e 16:00 b 11:00-12:00,
>
> Format is not set, but I need these time for my reporting.
> In the normal case it is start-end and lunch.
>
> I have tried to work with org-clock-sum to get the data from the CLOCK
> entries. My elisp is rusty, and I missed that the regexp was */level
> dependent. I managed to get a list of dates but just from the CLOCK
> under the Item.
> (((2014-11-05 12:00) (2014-11-05 14:00))
>   ((2014-11-05 14:00) (2014-11-05 16:00)))
>
> There were some strange duplicates as well so I guess I just did things
> wrong.
>
> To get the wanted result I would need a regexp that scans all CLOCK:
> and reports the two timestamps and generate a list of
> ((time1 time2) (time2 time3) (time4 time5) (time5 time6)) filtered
> on the call with tstart tend like org-clock-sum.
> For me the actual heading is not needed.
>
> With this list I can then flaten it, and if needed sort it :
> (time1 time2 time2 time3 time4 time5 time5 time6)
> If I then remove duplicates all together I will get :
> (time1 time3 time4 time6)
> where min is start and max is end, and pairs in between are breaks.
>
> s time1 e time6 b time3-time4,
>
> I did not manage to create this list of all CLOCK values filtered on
> tstart/tend (which in my case would be one day at the time just like
> I call org-clock-sum for the total sum).
>
> I would be grateful for advice or pointers to a better way to find
> all CLOCK values. Feel free to publish [2] on the wiki if you find
> it valuable.
> If/when I have a full working solution I might publish it on my site.
>
> Thank you,
>
> Fredrik Unger
>
> [1] http://sachachua.com/blog/2007/12/clocking-time-with-emacs-org/
> [2] https://tree.se/rangereport.el
> [3] https://metacpan.org/pod/DateTime#How-DateTime-Math-Works
> [4] https://tree.se/t.org
>



[-- Attachment #2: range-report.emacs --]
[-- Type: text/plain, Size: 4091 bytes --]

;; Orgmode
(require 'org)
(add-to-list 'auto-mode-alist '("\\.org\\'" . org-mode))
(setq org-agenda-files (file-expand-wildcards "~/org/*.org"))
(setq org-export-docbook-xslt-stylesheet "/usr/share/xml/docbook/stylesheet/docbook-xsl/fo/docbook.xsl")
(setq org-export-docbook-xslt-proc-command "xsltproc --output %o %s %i")
(setq org-export-docbook-xsl-fo-proc-command "fop %i %o")
(setq org-time-clocksum-format
      '(:hours "%d" :require-hours t :minutes ":%02d" :require-minutes t))
;; Org-mode daily reports

(defun tstr (timeval)
  "Format date to string"
  (format-time-string (car org-time-stamp-formats) timeval))

(defun hmstr (timeval)
  "Format date to string"
  (format-time-string "%H:%M" timeval))

(defun timeuniq (timelist)
  (defun dup (ts timelist)
    (cond
     ((not timelist) nil)
     ((equal ts (car timelist)) t)
     (t (dup ts (cdr timelist)))))

  (defun tcheck (timelist duplist)
    (cond
     ((not timelist) nil)
     ((dup (car timelist) (cdr timelist))
      (tcheck (cdr timelist) (cons (car timelist) duplist)))
     ((dup (car timelist) duplist)
      (tcheck (cdr timelist) duplist))
     (t
      (cons (car timelist) (tcheck (cdr timelist) duplist)))))

  (tcheck timelist ()))

(defun org-clock-seb (&optional tstart tend)
  (interactive)
  (org-with-silent-modifications
   (let* ((re (concat "^[ \t]*"
		      org-clock-string
		      "[ \t]*\\(?:\\(\\[.*?\\]\\)-+\\(\\[.*?\\]\\)\\|=>[ \t]+\\([0-9]+\\):\\([0-9]+\\)\\)"))
	  ts te
	  tl)
     (if (stringp tstart) (setq tstart (org-time-string-to-seconds tstart)))
     (if (stringp tend) (setq tend (org-time-string-to-seconds tend)))
     (if (consp tstart) (setq tstart (org-float-time tstart)))
     (if (consp tend) (setq tend (org-float-time tend)))
     (save-excursion
       (goto-char (point-max))
       (while (re-search-backward re nil t)
	 (cond
	  ((match-end 1)
	   ;; Two time stamps
	   (setq ts (match-string 1)
		 te (match-string 2)
		 ts (org-float-time
		     (apply 'encode-time (org-parse-time-string ts)))
		 te (org-float-time
		     (apply 'encode-time (org-parse-time-string te)))
		 ts (if tstart (max ts tstart) ts)
		 te (if tend (min te tend) te)
		 tl (if (> (- te ts) 0) (append (list ts te) tl))))))
       (timeuniq (sort tl '<))))))

(defun org-dblock-write:rangereport (params)
  "Display day-by-day time reports."
  (let* ((ts (plist-get params :tstart))
	 (te (plist-get params :tend))
	 (block (plist-get params :block))
	 (ws (plist-get params :wstart))
	 (ms (plist-get params :mstart))
	 cc range-text)
    (org-clock-sum 0 1)
    (when block
      (setq cc (org-clock-special-range block nil t ws ms)
	    ts (car cc) te (nth 1 cc) range-text (nth 2 cc)))
    (setq ts (org-time-string-to-time ts))
    (setq te (org-time-string-to-time te))
    (defun add-day (timeval)
      "Add a day to the date"
      (setq curr (decode-time timeval))
      (setcar (nthcdr 3 curr) (+ (nth 3 curr) 1))
      (apply 'encode-time (butlast curr 3)))

    (unless range-text
      (setq range-text (concat "[" (tstr ts) " - " (tstr te) "]")))

    (insert "Time report " range-text "\n")
    (setq totalmin 0)
    (while (time-less-p ts te)
      (setq nday (add-day ts))
      (org-clock-sum (time-to-seconds ts) (time-to-seconds nday))
      (setq seb (org-clock-seb (time-to-seconds ts) (time-to-seconds nday)))
      (setq minutes org-clock-file-total-minutes)
      (setq totalmin (+ minutes totalmin))
      (setq tt (org-minutes-to-clocksum-string minutes))
      (if (not (string= "0:00" tt))
	 (progn
	  (setq seb (append (cons (car seb) (last seb)) (butlast (cdr seb))))
	  (setq sebstr (mapcar 'hmstr (mapcar 'seconds-to-time seb)))
	  (setq timestr (format "%s    [s %5s e %5s %15s]   [%5s (%0.2f)]\n"
				(tstr ts) (car sebstr) (car (cdr sebstr))
				(cdr (cdr sebstr)) tt (/ minutes 60.0)))
	  (insert timestr)))
      (setq ts nday )
      )
    (insert "Total time : " (org-minutes-to-clocksum-string totalmin)"\n")
    (insert "Total mins : " (format "%f"  totalmin) "\n")
    (insert "Total days : " (format "%f" (/ totalmin (* 7.7 60.0))))
    ))

Re: Recording Time - org-clock-sum

[Eric S Fraga]

On Monday, 17 Nov 2014 at 19:31, Fredrik Unger wrote:
> Hi,
>
> I implemented a couple of functions that provide me with the
> needed timestamps for my timereport [1].

Thanks for this.  Looks interesting.  I don't need this right now but I
will find it quite useful in 2.5 years' time (we need to clock our work
for a year every 3 years -- long story).  I've made a note of it.

Small point: good practice would be to name your emacs lisp code file
with a ".el" suffix.

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.1, Org release_8.3beta-558-g83d8a2

Re: Recording Time - org-clock-sum

[Fredrik Unger]

Hi,

I found a bug when testing larger files.

diff --git a/.emacs b/.emacs
index 3579b7a..9bb7e8c 100644
--- a/.emacs
+++ b/.emacs
@@ -255,7 +255,7 @@
                      (apply 'encode-time (org-parse-time-string te)))
                  ts (if tstart (max ts tstart) ts)
                  te (if tend (min te tend) te)
-                tl (if (> (- te ts) 0) (append (list ts te) tl))))))
+                tl (if (> (- te ts) 0) (append (list ts te) tl) tl)))))
         (timeuniq (sort tl '<))))))

  (defun org-dblock-write:rangereport (params)


keeps the list from being set to nil once a day comes that does
not account to the day.
In my simple example there were just one day.
There might be a better way, but these seems to work well for more 
complex examples as well.
It can possibly be improved for speed, but as I only use it on a per 
month basis, it is fast enough.

Fredrik Unger

On 11/18/2014 08:52 AM, Eric S Fraga wrote:
> On Monday, 17 Nov 2014 at 19:31, Fredrik Unger wrote:
>> Hi,
>>
>> I implemented a couple of functions that provide me with the
>> needed timestamps for my timereport [1].
>
> Thanks for this.  Looks interesting.  I don't need this right now but I
> will find it quite useful in 2.5 years' time (we need to clock our work
> for a year every 3 years -- long story).  I've made a note of it.
>
> Small point: good practice would be to name your emacs lisp code file
> with a ".el" suffix.
>