Date-centric Clocktable

Date-centric Clocktable

[Rasmus]

Hi, 

Is is possible to have a clocktabke with times in the left-most column?
The people I am doing some work for now prefer it that way for unknown
reasons. 

This is an example

| date                                           | Headline        | total |
|------------------------------------------------+-----------------+-------|
| [2011-08-19 Fri 00:28]--[2011-08-19 Fri 00:51] | Writing mails   |  0:23 |
| [2011-06-22 Wed 17:00]--[2011-06-22 Wed 17:45] | Data processing |  0:45 |

And so forth.

Thanks,
Rasmus

-- 
Sent from my Emacs

Re: Date-centric Clocktable

[Bernt Hansen]

Rasmus <rasmus@gmx.us> writes:

> Hi, 
>
> Is is possible to have a clocktabke with times in the left-most column?
> The people I am doing some work for now prefer it that way for unknown
> reasons. 
>
> This is an example
>
> | date                                           | Headline        | total |
> |------------------------------------------------+-----------------+-------|
> | [2011-08-19 Fri 00:28]--[2011-08-19 Fri 00:51] | Writing mails   |  0:23 |
> | [2011-06-22 Wed 17:00]--[2011-06-22 Wed 17:45] | Data processing |  0:45 |
>
> And so forth.

Hi Rasmus,

I'm not aware of any way to automatically get the clock report in that
format using existing org-mode functions but you can just manually move
the column before you publish it to whomever wants it in column 1.  Just
put point in the total column and M-<left-arrow> to shift the column
where you want it.

You can automate this table change with elisp if you're doing it often.

HTH,
Bernt

Re: Date-centric Clocktable

[Rasmus]

Bernt Hansen <bernt@norang.ca> writes:

> Rasmus <rasmus@gmx.us> writes:
>
>> Hi, 
>>
>> Is is possible to have a clocktabke with times in the left-most column?
>> The people I am doing some work for now prefer it that way for unknown
>> reasons. 
>>
>> This is an example
>>
>> | date                                           | Headline        | total |
>> |------------------------------------------------+-----------------+-------|
>> | [2011-08-19 Fri 00:28]--[2011-08-19 Fri 00:51] | Writing mails   |  0:23 |
>> | [2011-06-22 Wed 17:00]--[2011-06-22 Wed 17:45] | Data processing |  0:45 |
>>
>> And so forth.
>
> Hi Rasmus,
>
> I'm not aware of any way to automatically get the clock report in that
> format using existing org-mode functions but you can just manually move
> the column before you publish it to whomever wants it in column 1.  Just
> put point in the total column and M-<left-arrow> to shift the column
> where you want it.

Well, this would require me to collect some 40 logbook entries myself
and associate them with a headline.  What I need is a summary showing
what I did when.  In other words, the default clock table summarizes too
much.

It might be easier to process the file with some other tool and extract
logbook entries with a regexp.  Are there any specialized
Org/Emacs-command I could use for generating a custom clock table?

–Rasmus

-- 
Sent from my Emacs

Re: Date-centric Clocktable

[Carsten Dominik]

On 6.9.2011, at 23:36, Rasmus wrote:

> Bernt Hansen <bernt@norang.ca> writes:
> 
>> Rasmus <rasmus@gmx.us> writes:
>> 
>>> Hi, 
>>> 
>>> Is is possible to have a clocktabke with times in the left-most column?
>>> The people I am doing some work for now prefer it that way for unknown
>>> reasons. 
>>> 
>>> This is an example
>>> 
>>> | date                                           | Headline        | total |
>>> |------------------------------------------------+-----------------+-------|
>>> | [2011-08-19 Fri 00:28]--[2011-08-19 Fri 00:51] | Writing mails   |  0:23 |
>>> | [2011-06-22 Wed 17:00]--[2011-06-22 Wed 17:45] | Data processing |  0:45 |
>>> 
>>> And so forth.
>> 
>> Hi Rasmus,
>> 
>> I'm not aware of any way to automatically get the clock report in that
>> format using existing org-mode functions but you can just manually move
>> the column before you publish it to whomever wants it in column 1.  Just
>> put point in the total column and M-<left-arrow> to shift the column
>> where you want it.
> 
> Well, this would require me to collect some 40 logbook entries myself
> and associate them with a headline.  What I need is a summary showing
> what I did when.  In other words, the default clock table summarizes too
> much.
> 
> It might be easier to process the file with some other tool and extract
> logbook entries with a regexp.  Are there any specialized
> Org/Emacs-command I could use for generating a custom clock table?

Have you tried log mode in the agenda:

Make an agenda for the time intervar you are interested in,
day week month, or so.  Then press `C-u C-u v L'.  Maybe you can work from there?

- Carsten

Re: Date-centric Clocktable

[Bernt Hansen]

Rasmus <rasmus@gmx.us> writes:

> Bernt Hansen <bernt@norang.ca> writes:
>
>> Rasmus <rasmus@gmx.us> writes:
>>
>>> Hi, 
>>>
>>> Is is possible to have a clocktabke with times in the left-most column?
>>> The people I am doing some work for now prefer it that way for unknown
>>> reasons. 
>>>
>>> This is an example
>>>
>>> | date                                           | Headline        | total |
>>> |------------------------------------------------+-----------------+-------|
>>> | [2011-08-19 Fri 00:28]--[2011-08-19 Fri 00:51] | Writing mails   |  0:23 |
>>> | [2011-06-22 Wed 17:00]--[2011-06-22 Wed 17:45] | Data processing |  0:45 |
>>>
>>> And so forth.
>>
>> Hi Rasmus,
>>
>> I'm not aware of any way to automatically get the clock report in that
>> format using existing org-mode functions but you can just manually move
>> the column before you publish it to whomever wants it in column 1.  Just
>> put point in the total column and M-<left-arrow> to shift the column
>> where you want it.
>
> Well, this would require me to collect some 40 logbook entries myself
> and associate them with a headline.  What I need is a summary showing
> what I did when.  In other words, the default clock table summarizes too
> much.
>
> It might be easier to process the file with some other tool and extract
> logbook entries with a regexp.  Are there any specialized
> Org/Emacs-command I could use for generating a custom clock table?

Hi Rasmus,

Sorry I think I totally missed your point earlier - I thought all you
wanted was to move the total column, not get time detail in column 1.

As Carsten mentioned, the agenda view with logging enabled (and
appropriate tag filters) may get you closer to what you are looking
for.  I use a combination of that and C-u R in the daily / weekly agenda
to get a summary of clock detail lines.

Regards,
Bernt

Re: Date-centric Clocktable

[Rasmus]

Hi Carsten, 

>>>> This is an example
>>>> 
>>>> | date                                           | Headline        | total |
>>>> |------------------------------------------------+-----------------+-------|
>>>> | [2011-08-19 Fri 00:28]--[2011-08-19 Fri 00:51] | Writing mails   |  0:23 |
>>>> | [2011-06-22 Wed 17:00]--[2011-06-22 Wed 17:45] | Data processing |  0:45 |
> Have you tried log mode in the agenda:
>
> Make an agenda for the time intervar you are interested in,
> day week month, or so.  Then press `C-u C-u v L'.  Maybe you can work from there?

That is certainly better! One minor problem is that it displays total
time only, I need the time interval and total time.

I don't know whether the time sheets used by my university is very
different from everywhere else . . . As said, each entry requires an
interval, a total time and an entry text. Certainly the interval seems
silly as  hourly wage is flat rate, I believe. 

–Rasmus

-- 
Sent from my Emacs

Re: Date-centric Clocktable

[Rasmus]

Hi Bernt,

> As Carsten mentioned, the agenda view with logging enabled (and
> appropriate tag filters) may get you closer to what you are looking
> for.  I use a combination of that and C-u R in the daily / weekly agenda
> to get a summary of clock detail lines.

I am impressed with the possibilities of Org-agenda.  Although I have
used it extensively I know only a fraction of the possibilities, it
seems.  It only revels total time, it seems, which unfortunately isn't
enough cf. my other post.

Thanks,
Rasmus

-- 
Sent from my Emacs

Re: Date-centric Clocktable

[Carsten Dominik]

On Sep 7, 2011, at 10:07 AM, Rasmus wrote:

> 
> Hi Carsten, 
> 
>>>>> This is an example
>>>>> 
>>>>> | date                                           | Headline        | total |
>>>>> |------------------------------------------------+-----------------+-------|
>>>>> | [2011-08-19 Fri 00:28]--[2011-08-19 Fri 00:51] | Writing mails   |  0:23 |
>>>>> | [2011-06-22 Wed 17:00]--[2011-06-22 Wed 17:45] | Data processing |  0:45 |
>> Have you tried log mode in the agenda:
>> 
>> Make an agenda for the time intervar you are interested in,
>> day week month, or so.  Then press `C-u C-u v L'.  Maybe you can work from there?
> 
> That is certainly better! One minor problem is that it displays total
> time only, I need the time interval and total time.
> 
> I don't know whether the time sheets used by my university is very
> different from everywhere else . . . As said, each entry requires an
> interval, a total time and an entry text. Certainly the interval seems
> silly as  hourly wage is flat rate, I believe. 

I guess then your next option is to write your own clock report function,
starting from org-clocktable-write-default.

- Carsten

> 
> –Rasmus
> 
> -- 
> Sent from my Emacs
> 
> 

- Carsten

Re: Date-centric Clocktable

[Olaf Dietsche]

Rasmus <rasmus@gmx.us> writes:

> Is is possible to have a clocktabke with times in the left-most column?
> The people I am doing some work for now prefer it that way for unknown
> reasons. 
>
> This is an example
>
> | date                                           | Headline        | total |
> |------------------------------------------------+-----------------+-------|
> | [2011-08-19 Fri 00:28]--[2011-08-19 Fri 00:51] | Writing mails   |  0:23 |
> | [2011-06-22 Wed 17:00]--[2011-06-22 Wed 17:45] | Data processing |  0:45 |

Why don't you just use a simple (perl/python/...) script to collect your
data? Here's a quick hack in perl:

---8<--- cut here ---
#! /usr/bin/perl -n

use strict;
use warnings;

our $headline;

BEGIN {
print "| date                                           | Headline        | total |\n";
print "|------------------------------------------------+-----------------+-------|\n";
}

# save current headline
$headline = $1 if (m/^\s*\*+\s+(.+)/);

# dump clock line
if (m/^\s*CLOCK:\s+(.+?)\s+=>\s+(.+)/) {
	print "| $1 | $headline | $2 |\n";
}
--- cut here --->8---

Run the script as

$ perl -n collect-timestamps.pl /path/to/org/*.org >clocktable.txt

You must tweak the formatting and make it more robust, but you get the
idea. If you prefer python, there are some python libraries listed at
<http://orgmode.org/worg/org-tools/index.html>.

Regards, Olaf

Re: Date-centric Clocktable

[Rasmus]

Hi Olaf, 

> Why don't you just use a simple (perl/python/...) script to collect your
> data? Here's a quick hack in perl:

That was my plan if I was not able to do from within Org. To me it would
be a lot faster than hacking something together in emacs-lisp,
unfortunately. 

Thanks for the link to Pythonic flavors.

–Rasmus

-- 
Sent from my Emacs

Re: Date-centric Clocktable

[Olaf Dietsche]

Rasmus <rasmus@gmx.us> writes:

>> Why don't you just use a simple (perl/python/...) script to collect your
>> data? Here's a quick hack in perl:
>
> That was my plan if I was not able to do from within Org. To me it would
> be a lot faster than hacking something together in emacs-lisp,
> unfortunately. 

If you insist on elisp, maybe something along these (untested) lines
might work:

---8<--- cut here ---
(defvar clockstable)

(defun collect-clock-lines ()
  (org-narrow-to-subtree)
  (let ((re (concat "^[ \t]*" org-clock-string "[ \t]+\\(.+?\\)[ \t]+=>[ \t]+\\(.+\\)"))
	(headline (nth 4 (org-heading-components))))
    (while (re-search-forward re)
      (setq clockstable (concat clockstable (match-string 1) "|" headline "|" (match-string 2) "\n")))))

(defun summarize-clocks ()
  (interactive)
  (setq clockstable "| date | headline | total |\n|-----+----+----|\n")
  (org-map-entries collect-clock-lines nil 'agenda)
  (insert clockstable))
--- cut here --->8---

Regards, Olaf

Re: Date-centric Clocktable

[Carsten Dominik]

On Sep 7, 2011, at 12:16 PM, Olaf Dietsche wrote:

> Rasmus <rasmus@gmx.us> writes:
> 
>>> Why don't you just use a simple (perl/python/...) script to collect your
>>> data? Here's a quick hack in perl:
>> 
>> That was my plan if I was not able to do from within Org. To me it would
>> be a lot faster than hacking something together in emacs-lisp,
>> unfortunately. 
> 
> If you insist on elisp, maybe something along these (untested) lines
> might work:
> 
> ---8<--- cut here ---
> (defvar clockstable)
> 
> (defun collect-clock-lines ()
>  (org-narrow-to-subtree)
>  (let ((re (concat "^[ \t]*" org-clock-string "[ \t]+\\(.+?\\)[ \t]+=>[ \t]+\\(.+\\)"))
> 	(headline (nth 4 (org-heading-components))))
>    (while (re-search-forward re)
>      (setq clockstable (concat clockstable (match-string 1) "|" headline "|" (match-string 2) "\n")))))
> 
> (defun summarize-clocks ()
>  (interactive)
>  (setq clockstable "| date | headline | total |\n|-----+----+----|\n")
>  (org-map-entries collect-clock-lines nil 'agenda)
>  (insert clockstable))
> --- cut here --->8---

Hi Olaf,

this is great!  Maybe we should make this a little builtin function,
with a format specification to create the lines.
What is still missing, I think, is some sorting by time would.
Basically, use

  (org-float-time
   (apply 'encode-time (save-match-data (org-parse-time-string (match-string 1)))))

after the successful search for a clock string to get a floating
point number representing the starting time, collect the 
line you are creating into an alist with the times and sort
them before inserting into the buffer.

- Carsten

Re: Date-centric Clocktable

[Rasmus]

Olaf Dietsche <olaf+list.orgmode@olafdietsche.de> writes:
> If you insist on elisp, maybe something along these (untested) lines
> might work:

It just nicer to do text stuff from within Emacs but my personal Lisp
skill are surpassed by my Python skills.  That is not to say that any of
the skill sets are high. . . 

The function didn't work in my test.org in emacs -q. I will investigate
more later when I've got more time. 

> ---8<--- cut here ---
> (defvar clockstable)
>
> (defun collect-clock-lines ()
>   (org-narrow-to-subtree)
>   (let ((re (concat "^[ \t]*" org-clock-string "[ \t]+\\(.+?\\)[ \t]+=>[ \t]+\\(.+\\)"))
> 	(headline (nth 4 (org-heading-components))))
>     (while (re-search-forward re)
>       (setq clockstable (concat clockstable (match-string 1) "|" headline "|" (match-string 2) "\n")))))
>
> (defun summarize-clocks ()
>   (interactive)
>   (setq clockstable "| date | headline | total |\n|-----+----+----|\n")
>   (org-map-entries collect-clock-lines nil 'agenda)
>   (insert clockstable))
> --- cut here --->8---

Thanks again,
Rasmus

-- 
Sent from my Emacs

Re: Date-centric Clocktable

[Olaf Dietsche]

Rasmus <rasmus@gmx.us> writes:

> Olaf Dietsche <olaf+list.orgmode@olafdietsche.de> writes:
>> If you insist on elisp, maybe something along these (untested) lines
>> might work:
>
> It just nicer to do text stuff from within Emacs but my personal Lisp
> skill are surpassed by my Python skills.  That is not to say that any of
> the skill sets are high. . . 
>
> The function didn't work in my test.org in emacs -q. I will investigate
> more later when I've got more time. 
>
>> ---8<--- cut here ---
>> (defvar clockstable)
>>
>> (defun collect-clock-lines ()
>>   (org-narrow-to-subtree)
>>   (let ((re (concat "^[ \t]*" org-clock-string "[ \t]+\\(.+?\\)[ \t]+=>[ \t]+\\(.+\\)"))
>> 	(headline (nth 4 (org-heading-components))))
>>     (while (re-search-forward re)
>>       (setq clockstable (concat clockstable (match-string 1) "|" headline "|" (match-string 2) "\n")))))
>>
>> (defun summarize-clocks ()
>>   (interactive)
>>   (setq clockstable "| date | headline | total |\n|-----+----+----|\n")
>>   (org-map-entries collect-clock-lines nil 'agenda)
>>   (insert clockstable))
>> --- cut here --->8---

Look at C-h f org-map-entries RET. Maybe changing the scope from 'agenda
to 'file will help.

Regards, Olaf

Re: Date-centric Clocktable

[Giovanni Ridolfi]

Carsten Dominik <carsten.dominik@gmail.com> writes:

Hi, Rasmus !
> On Sep 7, 2011, at 10:07 AM, Rasmus wrote:
>
>> 
>> Hi Carsten, 
>> 
>>>>>> This is an example
>>>>>> 
>>>>>> | date                                           | Headline        | total |
>>>>>> |------------------------------------------------+-----------------+-------|
>>>>>> | [2011-08-19 Fri 00:28]--[2011-08-19 Fri 00:51] | Writing mails   |  0:23 |
>>>>>> | [2011-06-22 Wed 17:00]--[2011-06-22 Wed 17:45] | Data processing |  0:45 |
>> 
>> That is certainly better! One minor problem is that it displays total
>> time only, I need the time interval and total time.
>> 
You could use the clocktable properties (extremely useful patch by
Niels Giesen [1] )

#+BEGIN: clocktable :maxlevel 2 :scope file :properties ("CLOCK" ) :inherit-props t 
                             ^^^^
#+END:                         |
                               |
- be careful of the level -----|
- narrow the file to the date you're interested in ":scope file" search
  on the whole file

hth 
Giovanni

[1] http://article.gmane.org/gmane.emacs.orgmode/40160

Re: Date-centric Clocktable

[Olaf Dietsche]

Olaf Dietsche <olaf+list.orgmode@olafdietsche.de> writes:

> Rasmus <rasmus@gmx.us> writes:
>
>> That was my plan if I was not able to do from within Org. To me it would
>> be a lot faster than hacking something together in emacs-lisp,
>> unfortunately. 
>
> If you insist on elisp, maybe something along these (untested) lines
> might work:
>
> ---8<--- cut here ---
[buggy code]
> --- cut here --->8---

I finally came around and tested the code, but unfortunately it doesn't
work as expected. Here is another version:

---8<--- cut here ---
(defun collect-clock-lines ()
  (let ((re (concat "^[ \t]*" org-clock-string "[ \t]+\\(.+?\\)[ \t]+=>[ \t]+\\(.+\\)"))
	(headline (nth 4 (org-heading-components)))
	clocks)
    (org-narrow-to-subtree)
    (while (re-search-forward re nil t)
      (setq clocks (concat clocks (format "| %s | %s | %s |\n" (match-string 1) headline (match-string 2)))))
    (widen)
    clocks))

(defun summarize-clocks ()
  (interactive)
  (insert "| date | headline | total |\n|-----+----+----|\n")
  (let ((headings (org-map-entries 'collect-clock-lines nil 'file)))
    (mapc (lambda (clocks)
	    (mapc (lambda (line) (insert line)) clocks))
	    headings)))
--- cut here --->8---

This version works a little bit better, but has its own drawbacks.
Anyway, it's a start and I don't want to take all the fun away. ;-)

Regards, Olaf

Re: Date-centric Clocktable

[Olaf Dietsche]

Carsten Dominik <carsten.dominik@gmail.com> writes:

> On Sep 7, 2011, at 12:16 PM, Olaf Dietsche wrote:
>
>> Rasmus <rasmus@gmx.us> writes:
>> 
>>>> Why don't you just use a simple (perl/python/...) script to collect your
>>>> data? Here's a quick hack in perl:
>>> 
>>> That was my plan if I was not able to do from within Org. To me it would
>>> be a lot faster than hacking something together in emacs-lisp,
>>> unfortunately. 
>> 
>> If you insist on elisp, maybe something along these (untested) lines
>> might work:
>> 
>> ---8<--- cut here ---
[snip]
>> --- cut here --->8---
>
> this is great!

Thank you :-)

> Maybe we should make this a little builtin function,
> with a format specification to create the lines.
> What is still missing, I think, is some sorting by time would.
> Basically, use
>
>   (org-float-time
>    (apply 'encode-time (save-match-data (org-parse-time-string (match-string 1)))))
>
> after the successful search for a clock string to get a floating
> point number representing the starting time, collect the 
> line you are creating into an alist with the times and sort
> them before inserting into the buffer.

I don't know, wether adding small special purpose functions adds real
value, since we already have "org-map-entries". Maybe adding generic
functions to org or showing lisp snippets at worg would be more useful.

Regards, Olaf

Re: Date-centric Clocktable

[Rasmus]

Olaf Dietsche <olaf+list.orgmode@olafdietsche.de> writes:

>>> ---8<--- cut here ---
> [snip]
>>> --- cut here --->8---
>>
>> this is great!
>
> Thank you :-)

It is. 

> I don't know, wether adding small special purpose functions adds real
> value, since we already have "org-map-entries". Maybe adding generic
> functions to org or showing lisp snippets at worg would be more useful.

In my experience this is the kind of table most working class heroes
will need to hand in.  With Org-sort and you second script it's easy to
get the kind of table I at least need quite often.  Using the totaling
function of regular clock tables everything will probably be swell. 

If it shouldn't be incorporated in Org it should probably retire to
Worg.

Thanks a bunch!

–Rasmus

-- 
Sent from my Emacs

Re: Date-centric Clocktable

[Rasmus]

> ---8<--- cut here ---
> (defun collect-clock-lines ()
>   (let ((re (concat "^[ \t]*" org-clock-string "[ \t]+\\(.+?\\)[ \t]+=>[ \t]+\\(.+\\)"))
> 	(headline (nth 4 (org-heading-components)))
> 	clocks)
>     (org-narrow-to-subtree)
>     (while (re-search-forward re nil t)
>       (setq clocks (concat clocks (format "| %s | %s | %s |\n" (match-string 1) headline (match-string 2)))))
>     (widen)
>     clocks))
>
> (defun summarize-clocks ()
>   (interactive)
>   (insert "| date | headline | total |\n|-----+----+----|\n")
>   (let ((headings (org-map-entries 'collect-clock-lines nil 'file)))
>     (mapc (lambda (clocks)
> 	    (mapc (lambda (line) (insert line)) clocks))
> 	    headings)))
> --- cut here --->8---
>
> This version works a little bit better, but has its own drawbacks.
> Anyway, it's a start and I don't want to take all the fun away. ;-)

I think I found one of the drawbacks :)  

#+begin_src org
* COMMENT Organization Headline
** Collecting data
   :LOGBOOK:
   CLOCK: [2011-09-08 Thu 00:31]--[2011-09-08 Thu 00:59] =>  0:28
   :END:
* Clocktable
  :LOGBOOK:
  CLOCK: [2011-09-08 Thu 01:30]--[2011-09-08 Thu 01:37] =>  0:07
  :END:

Note that subitems are counted twice: 

| date                                           | headline                      | total |
|------------------------------------------------+-------------------------------+-------|
| [2011-09-08 Thu 00:31]--[2011-09-08 Thu 00:59] | COMMENT Organization Headline |  0:28 |
| [2011-09-08 Thu 00:31]--[2011-09-08 Thu 00:59] | Collecting data               |  0:28 |
| [2011-09-08 Thu 01:30]--[2011-09-08 Thu 01:37] | Clocktable                    |  0:07 |

#+end_src

I might look into it later.

–Rasmus

-- 
Sent from my Emacs

Re: Date-centric Clocktable

[A. Ryan Reynolds]

On Sep 7, 2011, at 5:53 PM, Rasmus wrote:
> In my experience this is the kind of table most working class heroes
> will need to hand in.

At my last job this was the sort of table I needed to produce, too.

--
A. Ryan Reynolds

Re: Date-centric Clocktable

[Bastien]

Hi Rasmus,

Rasmus <rasmus@gmx.us> writes:

> Is is possible to have a clocktabke with times in the left-most column?
> The people I am doing some work for now prefer it that way for unknown
> reasons. 
>
> This is an example
>
> | date                                           | Headline        | total |
> |------------------------------------------------+-----------------+-------|
> | [2011-08-19 Fri 00:28]--[2011-08-19 Fri 00:51] | Writing mails   |  0:23 |
> | [2011-06-22 Wed 17:00]--[2011-06-22 Wed 17:45] | Data processing |  0:45 |

This is not currently possible as such.  

In the meantime, playing with the :block, :tstart and :tend parameters
can help providing something *not* that far.

HTH (a bit),

-- 
 Bastien