Re: Estimate ranges in column view

[Carsten Dominik <carsten.dominik@gmail.com>]

Hi Michael,

I was about to apply the patch, but I don't think I got a reply from  
you regarding copyright issues.
Did I miss that, or did you not say anything about it?

Unfortunately the patch is larger than the maximum that I can take  
without copyright assignment.



- Carsten

On Jun 22, 2010, at 4:36 AM, Michael Gauland wrote:

> Carsten,
> Here is a patch for a new 'est+' summary type, including  
> corresponding changes for xemacs and the manual. I've done basic  
> testing on the GNU emacs version, but not the xemacs code.
>
> I'm not sure the change to the manual provides the right amount of  
> information
> in the right place; I'd be happy to re-write to make it find in
> better. Similarly, the name of the summary type is entirely up to you.
>
> I didn't know whether to send this directly to you, or to the list;  
> if it should go to the list I'd be happy to send it there directly.
>
> Thanks for the chance to contribute,
> Mike
> diff --git a/doc/org.texi b/doc/org.texi
> index 17615e0..529be4d 100644
> --- a/doc/org.texi
> +++ b/doc/org.texi
> @@ -4723,9 +4723,10 @@ optional.  The individual parts have the  
> following meaning:
>                 @{:min@}    @r{Smallest time value in column.}
>                 @{:max@}    @r{Largest time value.}
>                 @{:mean@}   @r{Arithmetic mean of time values.}
> -                @{@@min@}   @r{Minimum age (in days/hours/mins/ 
> seconds).}
> -                @{@@max@}   @r{Maximum age (in days/hours/mins/ 
> seconds).}
> -                @{@@mean@}  @r{Arithmetic mean of ages (in days/ 
> hours/mins/seconds).}
> +                @{@@min@}    @r{Minimum age (in days/hours/mins/ 
> seconds).}
> +                @{@@max@}    @r{Maximum age (in days/hours/mins/ 
> seconds).}
> +                @{@@mean@}   @r{Arithmetic mean of ages (in days/ 
> hours/mins/seconds).}
> +                @{est+@}    @r{Add low-high estimates.}
> @end example
>
> @noindent
> @@ -4733,6 +4734,22 @@ Be aware that you can only have one summary  
> type for any property you
> include. Subsequent columns referencing the same property will all  
> display the
> same summary information.
>
> +The 'est+' summary type requires further explanation. It is used for
> +combining task estimates, expressed as low-high ranges. For  
> example, instead
> +of estimating a particular task will take 5 days, you might  
> estimate it as
> +5-6 days if you're fairly confident you know how much woark is  
> required, or
> +1-10 days if you don't really know what needs to be done.  Both  
> ranges
> +average at 5.5 days, but the first represents a more predictable  
> delivery.
> +
> +When combining a set of such estimates, simply adding the lows and  
> highs
> +produces an unrealistically wide result. Instead, 'est+' adds the  
> statistical
> +mean and variance of the sub-tasks, generating a final estimate  
> from the sum.
> +For example, suppose you had ten tasks, each of which was estimated  
> at 0.5 to
> +2 days of work. Straight addition produces an estimate of 5 to 20  
> days,
> +representing what to expect if everything goes either extremely  
> well or
> +extremely poorly. In contrast, 'est+' estimates the full job more
> +realistically, at 10-15 days.
> +
> Here is an example for a complete columns definition, along with  
> allowed
> values.
>
> diff --git a/lisp/org-colview-xemacs.el b/lisp/org-colview-xemacs.el
> index 152d9fe..90bf4c4 100644
> --- a/lisp/org-colview-xemacs.el
> +++ b/lisp/org-colview-xemacs.el
> @@ -917,7 +917,8 @@ around it."
>     ("@max" max_age max (lambda (x) (- org-columns-time x)))
>     ("@mean" mean_age
>      (lambda (&rest x) (/ (apply '+ x) (float (length x))))
> -     (lambda (x) (- org-columns-time x))))
> +     (lambda (x) (- org-columns-time x)))
> +    ("est+" estimate org-estimate-combine))
>   "Operator <-> format,function,calc  map.
>  Used to compile/uncompile columns format and completing read in
>  interactive function org-columns-new.
> @@ -1206,6 +1207,7 @@ Don't set this, this is meant for dynamic  
> scoping.")
> (defun org-columns-number-to-string (n fmt &optional printf)
>   "Convert a computed column number to a string value, according to  
> FMT."
>   (cond
> +   ((memq fmt '(estimate)) (org-estimate-print n printf))
>    ((not (numberp n)) "")
>    ((memq fmt '(add_times max_times min_times mean_times))
>     (let* ((h (floor n)) (m (floor (+ 0.5 (* 60 (- n h))))))
> @@ -1250,9 +1252,9 @@ Don't set this, this is meant for dynamic  
> scoping.")
> 	    (setq sum (+ (string-to-number (pop l)) (/ sum 60))))
> 	  sum))
>        ((memq fmt '(checkbox checkbox-n-of-m checkbox-percent))
> -	(if (equal s "[X]") 1. 0.000001))
> -       (t (string-to-number s)))
> -    0))
> +        (if (equal s "[X]") 1. 0.000001))
> +       ((memq fmt '(estimate)) (org-string-to-estimate s))
> +       (t (string-to-number s)))))
>
> (defun org-columns-uncompile-format (cfmt)
>   "Turn the compiled columns format back into a string  
> representation."
> @@ -1693,6 +1695,42 @@ This will add overlays to the date lines, to  
> show the summary for each day."
>       (format "%dd %02dh %02dm %02ds" days hours minutes seconds))
>     ""))
>
> +(defun org-estimate-mean-and-var (v)
> +  "Return the mean and variance of an estimate."
> +  (let* ((low (float (car v)))
> +         (high (float (cadr v)))
> +         (mean (/ (+ low high) 2.0))
> +         (var (/ (+ (expt (- mean low) 2.0) (expt (- high mean)  
> 2.0)) 2.0)))
> +    (list  mean var)
> +    ))
> +
> +(defun org-estimate-combine (&rest el)
> +  "Combine a list of estimates, using mean and variance.
> +The mean and variance of the result will be the sum of the means
> +and variances (respectively) of the individual estimates."
> +  (let ((mean 0)
> +        (var 0))
> +    (mapc (lambda (e)
> +	    (let ((stats (org-estimate-mean-and-var e)))
> +	      (setq mean (+ mean (car stats)))
> +	      (setq var (+ var (cadr stats)))))
> +	  el)
> +    (let ((stdev (sqrt var)))
> +      (list (- mean stdev) (+ mean stdev)))
> +    ))
> +
> +(defun org-estimate-print (e &optional fmt)
> +  "Prepare a string representation of an estimate, as two numbers  
> with a '-' in between them."
> +  (if (null fmt) (set 'fmt "%.0f"))
> +  (format "%s" (mapconcat (lambda (n) (format fmt n))  e "-")))
> +
> +(defun org-string-to-estimate (s)
> +  "Convert a string to an estimate. The string should be two  
> numbers joined with a '-'."
> +  (if (string-match "\\(.*\\)-\\(.*\\)" s)
> +      (list (string-to-number (match-string 1 s)) (string-to- 
> number(match-string 2 s)))
> +    (list (string-to-number s) (string-to-number s))
> +    ))
> +
>
> (provide 'org-colview)
> (provide 'org-colview-xemacs)
> diff --git a/lisp/org-colview.el b/lisp/org-colview.el
> index c820be0..af7eef5 100644
> --- a/lisp/org-colview.el
> +++ b/lisp/org-colview.el
> @@ -746,7 +746,8 @@ around it."
>     ("@max" max_age max (lambda (x) (- org-columns-time x)))
>     ("@mean" mean_age
>      (lambda (&rest x) (/ (apply '+ x) (float (length x))))
> -     (lambda (x) (- org-columns-time x))))
> +     (lambda (x) (- org-columns-time x)))
> +    ("est+" estimate org-estimate-combine))
>   "Operator <-> format,function,calc  map.
> Used to compile/uncompile columns format and completing read in
> interactive function org-columns-new.
> @@ -1031,6 +1032,7 @@ Don't set this, this is meant for dynamic  
> scoping.")
> (defun org-columns-number-to-string (n fmt &optional printf)
>   "Convert a computed column number to a string value, according to  
> FMT."
>   (cond
> +   ((memq fmt '(estimate)) (org-estimate-print n printf))
>    ((not (numberp n)) "")
>    ((memq fmt '(add_times max_times min_times mean_times))
>     (let* ((h (floor n)) (m (floor (+ 0.5 (* 60 (- n h))))))
> @@ -1054,28 +1056,30 @@ Don't set this, this is meant for dynamic  
> scoping.")
>       (format "[%d/%d]" n m)
>     (format "[%d%%]"(floor (+ 0.5 (* 100. (/ (* 1.0 n) m)))))))
>
> +
> (defun org-columns-string-to-number (s fmt)
>   "Convert a column value to a number that can be used for column  
> computing."
>   (if s
>       (cond
>        ((memq fmt '(min_age max_age mean_age))
> -	(cond ((string= s "") org-columns-time)
> -	      ((string-match
> -		"\\([0-9]+\\)d \\([0-9]+\\)h \\([0-9]+\\)m \\([0-9]+\\)s"
> -		s)
> -	       (+ (* 60 (+ (* 60 (+ (* 24 (string-to-number (match-string  
> 1 s)))
> -				    (string-to-number (match-string 2 s))))
> -			   (string-to-number (match-string 3 s))))
> -		  (string-to-number (match-string 4 s))))
> -	      (t (time-to-number-of-days (apply 'encode-time
> -						(org-parse-time-string s t))))))
> +        (cond ((string= s "") org-columns-time)
> +              ((string-match
> +                "\\([0-9]+\\)d \\([0-9]+\\)h \\([0-9]+\\)m \\([0-9]+ 
> \\)s"
> +                s)
> +               (+ (* 60 (+ (* 60 (+ (* 24 (string-to-number (match- 
> string 1 s)))
> +                                    (string-to-number (match-string  
> 2 s))))
> +                           (string-to-number (match-string 3 s))))
> +                  (string-to-number (match-string 4 s))))
> +              (t (time-to-number-of-days (apply 'encode-time
> +                                                (org-parse-time- 
> string s t))))))
>        ((string-match ":" s)
> -	(let ((l (nreverse (org-split-string s ":"))) (sum 0.0))
> -	  (while l
> -	    (setq sum (+ (string-to-number (pop l)) (/ sum 60))))
> -	  sum))
> +        (let ((l (nreverse (org-split-string s ":"))) (sum 0.0))
> +          (while l
> +            (setq sum (+ (string-to-number (pop l)) (/ sum 60))))
> +          sum))
>        ((memq fmt '(checkbox checkbox-n-of-m checkbox-percent))
> -	(if (equal s "[X]") 1. 0.000001))
> +        (if (equal s "[X]") 1. 0.000001))
> +       ((memq fmt '(estimate)) (org-string-to-estimate s))
>        (t (string-to-number s)))))
>
> (defun org-columns-uncompile-format (cfmt)
> @@ -1492,6 +1496,43 @@ This will add overlays to the date lines, to  
> show the summary for each day."
>       (format "%dd %02dh %02dm %02ds" days hours minutes seconds))
>     ""))
>
> +(defun org-estimate-mean-and-var (v)
> +  "Return the mean and variance of an estimate."
> +  (let* ((low (float (car v)))
> +         (high (float (cadr v)))
> +         (mean (/ (+ low high) 2.0))
> +         (var (/ (+ (expt (- mean low) 2.0) (expt (- high mean)  
> 2.0)) 2.0)))
> +    (list  mean var)
> +    ))
> +
> +(defun org-estimate-combine (&rest el)
> +  "Combine a list of estimates, using mean and variance.
> +The mean and variance of the result will be the sum of the means
> +and variances (respectively) of the individual estimates."
> +  (let ((mean 0)
> +        (var 0))
> +    (mapc (lambda (e)
> +              (let ((stats (org-estimate-mean-and-var e)))
> +                (setq mean (+ mean (car stats)))
> +                (setq var (+ var (cadr stats)))))
> +            el)
> +    (let ((stdev (sqrt var)))
> +      (list (- mean stdev) (+ mean stdev)))
> +    ))
> +
> +(defun org-estimate-print (e &optional fmt)
> +  "Prepare a string representation of an estimate, as two numbers  
> with a '-' in between them."
> +  (if (null fmt) (set 'fmt "%.0f"))
> +  (format "%s" (mapconcat (lambda (n) (format fmt n))  e "-"))
> +  )
> +
> +(defun org-string-to-estimate (s)
> +  "Convert a string to an estimate. The string should be two  
> numbers joined with a '-'."
> +  (if (string-match "\\(.*\\)-\\(.*\\)" s)
> +      (list (string-to-number (match-string 1 s)) (string-to- 
> number(match-string 2 s)))
> +    (list (string-to-number s) (string-to-number s))
> +    ))
> +
>
> (provide 'org-colview)
>

- Carsten