Re: overloading of internal priority calculations in agenda

[Adam Spiers <orgmode@adamspiers.org>]

Hi again,

On Wed, Dec 02, 2020 at 02:20:53PM +0000, Adam Spiers wrote: 
>Hi all,
>
>I'm currently working on adding a feature to org-agenda which allows 
>manual ordering of entries in combination with the existing automatic 
>ordering (as dictated by `org-agenda-sorting-strategy'). 
>
>During my investigations I noticed that while `org-get-priority' converts 
>[#B] style cookies into a numeric priority which is a multiple of 
>1000, further adjustments are made in functions like 
>`org-agenda-get-scheduled' before adding this numeric priority as a 
>text property on the entry: 
>
>    'priority (if habitp (org-habit-get-priority habitp)
>                (+ 99 diff (org-get-priority item)))
>
>In this case `diff' refers to the number of days between now and when 
>the item was scheduled.  A slightly different calculation is made in 
>`org-agenda-get-timestamps': 
>
>    (org-add-props item props
>      'priority (if habit?
>                    (org-habit-get-priority (org-habit-parse-todo))
>                  (org-get-priority item))
>
>I further noticed that this overloading of the internal priority by 
>including timestamp and habit data causes disruption to the behaviour 
>I imagine most users would expect from `org-agenda-sorting-strategy'. 
>For example, if you have `priority-down' as the first entry in the 
>`agenda' section and `category-keep' as the second, then differences 
>in the SCHEDULED timestamp are included in the priority calculation 
>and can therefore prevent sorting of two adjacent [#B] items by 
>category.  This seems like a bug to me, or at least breaks the 
>Principle of Least Surprise. 

[snipped]

>Given that `org-agenda-sorting-strategy' now supports all manner of 
>sorting criteria, many of which are time-sensitive, I would like to 
>know if there is any reason not to remove this overloading of the 
>priority calculation, i.e. decoupling it to depend purely on the 
>result of `org-get-priority' and `org-habit-get-priority'? 
>
>If fact, perhaps we could go one step further and add support for new 
>habit-priority-{up,down} sorters to `org-agenda-sorting-strategy', so 
>that the priority-{up,down} sorters sort purely by the priority cookie 
>and nothing else? 

Gently bumping this as I didn't get any replies yet.  I would like to 
continue working on a solution, but obviously don't want to waste time 
on something which would be rejected. 

If it is considered important to preserve the exact behaviour 
currently offered by `org-agenda-sorting-strategy' then I would 
propose the following: 

- Keep the existing priority-{up,down} which combine priority cookies
   with timestamp data and the result from `org-habit-get-priority',
   but probably also deprecate it and remove it from the default value.

- Introduce new priority-cookie-{up,down} sorters which operate purely
   on [#A] and [#1] style priority cookies and nothing else.

This would facilitate decoupling of the sortable criteria whilst 
remaining backwards compatible. 

Does this sound reasonable?  I am keen to proceed very soon (ideally 
over the Xmas break).  I have already written some new ert tests for 
`org-agenda-sorting-strategy' which would be included in any submitted 
patches. 

Thanks!
Adam