Agenda search: setting sort-order

Agenda search: setting sort-order

[Nathan Neff]

[-- Attachment #1: Type: text/plain, Size: 830 bytes --]

Hi all,

I'm using org-mode version 9.1.13 from elpa, and
have a custom agenda command:


(setq org-agenda-custom-commands
   '(
     ("z" search ""
       ((org-agenda-sorting-strategy '(timestamp-up))))))

And I have an org-file with this data:

* TODO [#C] Test1
:foo:
<2018-04-24 Tue 19:30>
* TODO [#A] Test2
:foo:
<2018-05-24 Thu 19:30>
* TODO [#B] Test3
:foo:
<2017-12-24 Sun 23:33>

The data is not sorted according to timestamp.  I cannot seem to
get the data sorted in any fashion when using the "search" agenda mode.

However, if I change "search" to "tags" and search for the "foo" tag, then
it appears that the sorting works.  I've tested it with priority and
timestamp
using the above data and it sorts the items correctly.

Why is my "search" agenda command ignoring the org-agenda-sorting-strategy?

Thanks,
--Nate

[-- Attachment #2: Type: text/html, Size: 1572 bytes --]

Re: Agenda search: setting sort-order

[Nathan Neff]

[-- Attachment #1: Type: text/plain, Size: 1369 bytes --]

Bumping this question to see if anyone has a workaround -
I recently have been wishing to search for keywords and order
the results in reverse timestamp order.

This sounds like a fairly useful and common feature, so I think
there must be something I'm configuring incorrectly.  I am able
to reproduce the problem using a fairly clean Emacs install (a
VM with no customizations).

Thanks,
--Nate



On Wed, May 30, 2018 at 4:57 PM, Nathan Neff <nathan.neff@gmail.com> wrote:

> Hi all,
>
> I'm using org-mode version 9.1.13 from elpa, and
> have a custom agenda command:
>
>
> (setq org-agenda-custom-commands
>    '(
>      ("z" search ""
>        ((org-agenda-sorting-strategy '(timestamp-up))))))
>
> And I have an org-file with this data:
>
> * TODO [#C] Test1
> :foo:
> <2018-04-24 Tue 19:30>
> * TODO [#A] Test2
> :foo:
> <2018-05-24 Thu 19:30>
> * TODO [#B] Test3
> :foo:
> <2017-12-24 Sun 23:33>
>
> The data is not sorted according to timestamp.  I cannot seem to
> get the data sorted in any fashion when using the "search" agenda mode.
>
> However, if I change "search" to "tags" and search for the "foo" tag, then
> it appears that the sorting works.  I've tested it with priority and
> timestamp
> using the above data and it sorts the items correctly.
>
> Why is my "search" agenda command ignoring the org-agenda-sorting-strategy?
>
> Thanks,
> --Nate
>

[-- Attachment #2: Type: text/html, Size: 2494 bytes --]

Re: Agenda search: setting sort-order

[Nathan Neff]

[-- Attachment #1: Type: text/plain, Size: 4722 bytes --]

I had a look at org-agenda.el, and took a first stab at "coding" :)
I fixed the sorting problem when using agenda-search-view, and
I have a DIFF/patch at the bottom of this e-mail [1] that provided the fix.

Would this be of interest to the org-mode project?  This code fixes the
problem,
but it is duplicated in the function org-agenda-get-todos, and seems
redundant,
therefore a more "long term" fix would probably not resemble the minor patch
below.  Also, I have no experience with lisp, nor the org-mode codebase :-O

Here's steps I used to find the problem.  Feel free to skip to [1]
for the DIFF/patch.

I found that if I perform an agenda-search, and change
the sort method to priority or tag, then the agenda items
matching my search are indeed sorted in the correct order.

This lead me to think to inspect the code that is performing
the time/date comparison of entries found by the org-search-view.

I put a debug-watch on the function org-cmp-ts and set
the sort method to "ts-down".  I ran an agenda search, and I
noticed that none of the entries matching my search had a 'ts-date
property.

I ran another agenda search (by tag) and noticed that the entries
being compared in the org-cmp-ts function did indeed have a 'ts-date
property.

So, I came to the conclusion that org-search-view does not give matching
entries a 'ts-date property.  Also, the 'type property of these entries
needed
the ts-date-type appended to their 'type property (e.g. "search timestamp"
instead of just "search").  Otherwise, the org-cmp-ts function would think
the two entries it was comparing did not have matching timestamp types
and simply return nil, indicating both entries were "equal" in sort order.

Thanks,
--Nate

[1]
I ran a DIFF of org-agenda.el (mine) against org-agenda.el (9.1.13) and
here's the DIFF.

diff --git a/lisp/org-agenda.el b/lisp/org-agenda.el
index 322480c..5af6743 100644
--- a/lisp/org-agenda.el
+++ b/lisp/org-agenda.el
@@ -4567,6 +4567,9 @@ is active."
                        (goto-char beg)
                        (setq marker (org-agenda-new-marker (point))
                              category (org-get-category)
+                  ts-date-pair (org-agenda-entry-get-agenda-timestamp
(point))
+                  ts-date (car ts-date-pair)
+                  ts-date-type (cdr ts-date-pair)
                              level (make-string (org-reduced-level
(org-outline-level)) ? )
                              inherited-tags
                              (or (eq org-agenda-show-inherited-tags
'always)
@@ -4584,10 +4587,12 @@ is active."
                        (org-add-props txt props
                          'org-marker marker 'org-hd-marker marker
                          'org-todo-regexp org-todo-regexp
+              'ts-date ts-date
+              'ts-date-type ts-date-type
                          'level level
                          'org-complex-heading-regexp
org-complex-heading-regexp
                          'priority 1000
-                         'type "search")
+              'type (concat "search" ts-date-type))
                        (push txt ee)
                        (goto-char (1- end))))))))))
        (setq rtn (nreverse ee))


On Tue, Jun 12, 2018 at 3:07 PM, Nathan Neff <nathan.neff@gmail.com> wrote:

> Bumping this question to see if anyone has a workaround -
> I recently have been wishing to search for keywords and order
> the results in reverse timestamp order.
>
> This sounds like a fairly useful and common feature, so I think
> there must be something I'm configuring incorrectly.  I am able
> to reproduce the problem using a fairly clean Emacs install (a
> VM with no customizations).
>
> Thanks,
> --Nate
>
>
>
> On Wed, May 30, 2018 at 4:57 PM, Nathan Neff <nathan.neff@gmail.com>
> wrote:
>
>> Hi all,
>>
>> I'm using org-mode version 9.1.13 from elpa, and
>> have a custom agenda command:
>>
>>
>> (setq org-agenda-custom-commands
>>    '(
>>      ("z" search ""
>>        ((org-agenda-sorting-strategy '(timestamp-up))))))
>>
>> And I have an org-file with this data:
>>
>> * TODO [#C] Test1
>> :foo:
>> <2018-04-24 Tue 19:30>
>> * TODO [#A] Test2
>> :foo:
>> <2018-05-24 Thu 19:30>
>> * TODO [#B] Test3
>> :foo:
>> <2017-12-24 Sun 23:33>
>>
>> The data is not sorted according to timestamp.  I cannot seem to
>> get the data sorted in any fashion when using the "search" agenda mode.
>>
>> However, if I change "search" to "tags" and search for the "foo" tag, then
>> it appears that the sorting works.  I've tested it with priority and
>> timestamp
>> using the above data and it sorts the items correctly.
>>
>> Why is my "search" agenda command ignoring the
>> org-agenda-sorting-strategy?
>>
>> Thanks,
>> --Nate
>>
>
>

[-- Attachment #2: Type: text/html, Size: 7311 bytes --]

Re: Agenda search: setting sort-order

[Nicolas Goaziou]

Hello,

Nathan Neff <nathan.neff@gmail.com> writes:

> I had a look at org-agenda.el, and took a first stab at "coding" :)
> I fixed the sorting problem when using agenda-search-view, and
> I have a DIFF/patch at the bottom of this e-mail [1] that provided the fix.
>
> Would this be of interest to the org-mode project?

IMO, the Org mode project is interested in having its bugs fixed ;)

> This code fixes the
> problem,
> but it is duplicated in the function org-agenda-get-todos, and seems
> redundant,
> therefore a more "long term" fix would probably not resemble the minor patch
> below.  Also, I have no experience with lisp, nor the org-mode
> codebase :-O

A more "long term" fix would be to rewrite the agenda (asynchronous,
more scalable, better API for external use).

> Here's steps I used to find the problem.  Feel free to skip to [1]
> for the DIFF/patch.

Would you mind sending it using `git format-patch'?

Thank you.

Regards,

-- 
Nicolas Goaziou

Re: Agenda search: setting sort-order

[Nathan Neff]

[-- Attachment #1: Type: text/plain, Size: 1686 bytes --]

On Tue, Jun 19, 2018 at 3:20 PM, Nicolas Goaziou <mail@nicolasgoaziou.fr>
wrote:

> Hello,
>
> Nathan Neff <nathan.neff@gmail.com> writes:
>
> > I had a look at org-agenda.el, and took a first stab at "coding" :)
> > I fixed the sorting problem when using agenda-search-view, and
> > I have a DIFF/patch at the bottom of this e-mail [1] that provided the
> fix.
> >
> > Would this be of interest to the org-mode project?
>
> IMO, the Org mode project is interested in having its bugs fixed ;)
>
> > This code fixes the
> > problem,
> > but it is duplicated in the function org-agenda-get-todos, and seems
> > redundant,
> > therefore a more "long term" fix would probably not resemble the minor
> patch
> > below.  Also, I have no experience with lisp, nor the org-mode
> > codebase :-O
>
> A more "long term" fix would be to rewrite the agenda (asynchronous,
> more scalable, better API for external use).
>
> > Here's steps I used to find the problem.  Feel free to skip to [1]
> > for the DIFF/patch.
>
> Would you mind sending it using `git format-patch'?
>

Hi Nicolas,

I'm a bit leery of submitting these lines of code - one reason is that
the sort feature still really doesn't work very well.  IIRC, the results
are globally sorted by timestamp, but there's results with NULL or
no timestamps mixed in.

In other words, results are something like:

* foo 12/1/2001
* bar 12/2/2002
* baz <no timestamp>
* blech 12/2/2003
* another <no timestamp>

So, it's not really a fix, and I'm not sure this is even good code :)
If someone wants to grab the fix and jump from here, that's fine, it's only
a few lines.

Thanks,
--Nate


>
> Thank you.
>
> Regards,
>
> --
> Nicolas Goaziou
>

[-- Attachment #2: Type: text/html, Size: 2766 bytes --]