ANN: org-super-agenda

ANN: org-super-agenda

[Adam Porter]

Hi friends,

I put together another little tool I've been thinking about for a while.
I'm calling it org-super-agenda at the moment, but org-divided-agenda
might be more descriptive; feedback on the name is welcome.

Before I forget and have to have Grant cover for me, you can find it
here: https://github.com/alphapapa/org-super-agenda

This is a prototype for a “supercharged” Org daily/weekly agenda. The
idea is to divide items into sections, rather than having them all in
one big list.

Now you can sort-of do this already with custom agenda commands, but
when you do that, you lose the daily/weekly aspect of the agenda: items
are no longer shown based on deadline/scheduled timestamps, but are
shown no-matter-what.

So this org-super-agenda command essentially copies the org-agenda-list
command, but right before it inserts the agenda items, it runs them
through a set of user-defined filters that separate them into
sections. Then the sections are inserted into the agenda buffer, and any
remaining items are inserted at the end.

The end result is your standard daily/weekly agenda, but arranged into
sections defined by you. You might put items with certain tags in one
section, habits in another section, items with certain todo keywords in
another, and items with certain priorities in another. The possibilities
are only limited by the filter functions (which you can easily add to,
and more will be added here).

The org-super-agenda command works as a custom agenda command, so you
can add it to your org-agenda-custom-commands list. You can also test it
quickly like this:

#+BEGIN_SRC elisp
((org-agenda-custom-commands
       (list (quote ("u" "SUPER Agenda"
                     org-super-agenda ""
                     ((super-filters
                       '((osa/separate-by-any-tags ("bills"))
                         osa/separate-by-habits
                         (osa/separate-by-todo-keywords "WAITING")
                         (osa/separate-by-todo-keywords ("SOMEDAY" "TO-READ" "CHECK")
                         (osa/separate-by-priorities "A")
                         (osa/separate-by-priorities "B")
                         (osa/separate-by-priorities "C")))
                      (org-agenda-span 'day)))))))
  (org-agenda nil "u"))
#+END_SRC

Here's a screenshot:
https://github.com/alphapapa/org-super-agenda/raw/master/screenshot.png

I consider this a functional prototype or basic first-version, but it's
very usable right now.  Feedback is welcome, and I plan to continue
making improvements before eventually submitting it to MELPA.

Thanks,
Adam

Re: ANN: org-super-agenda

[Adam Porter]

Seems like the harder I try to be thorough, the more likely I am to mess
something up.  The instructions for trying it out were slightly
incorrect, but they're outdated now, anyway.  Please see the repo for
current instructions.  Thanks.

Re: ANN: org-super-agenda

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 245 bytes --]

Thanks.  This also looks rather useful.  Will try it out soon (busy with
deadlines at the moment and the current agenda view is already good
enough to nag me about those ;-)).

-- 
: Eric S Fraga via Emacs 26.0.50, Org release_9.0.9-573-g09e612

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: ANN: org-super-agenda

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 554 bytes --]

Hi Adam,

have now given this a try.  Looks really nice.  Some questions, if I
may.  I would like to have the following sections:

1. items that have deadlines, any deadline for now and in the future,
   whether scheduled or not.
2. items that have a specific TODO type whether scheduled or not.
3. a clock summary (equivalent to what one gets with "v c" in standard
   view) 
4. a clock report at the end (and really the end, after all other items)

Are these possible?

Thanks,
eric

-- 
: Eric S Fraga via Emacs 26.0.50, Org release_9.0.9-573-g09e612

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: ANN: org-super-agenda

[Adam Porter]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> Hi Adam,
>
> have now given this a try.  Looks really nice.  Some questions, if I
> may.  I would like to have the following sections:

Hi Eric,

Thanks, I appreciate your feedback.

> 1. items that have deadlines, any deadline for now and in the future,
>    whether scheduled or not.
> 2. items that have a specific TODO type whether scheduled or not.

Well, as it stands, this package is mainly a way to group together items
that the standard agenda view (i.e. the org-agenda-list function) has
gathered.  The standard agenda view uses org-agenda-get-day-entries for
that.  What I think you're wanting is already provided by, e.g. the
org-todo-list function.  So you could add that as a separate section in
a custom agenda series command.

Now if you're wanting to use the grouping ability on the global TODO
list provided by org-todo-list, that is a possibility, but it would
require my essentially copying the org-todo-list function and making it
available under a new name that does the grouping.  This is what I did
with the org-agenda-list function: I just copied it and made one change
that calls my grouping/inserting functions instead of inserting results
directly.

The problem with this approach in general is that I end up copying large
Org agenda functions (and they are large) and making minor changes to
them, and then they will get out of sync when those functions change in
Org.

So the question becomes, how many Org agenda functions do I want to copy
into my package and be responsible for fixing?

I considered an alternative approach of post-processing agenda buffers,
instead of doing the filtering and inserting myself, but considering all
the special blocks that agenda buffers can have (e.g. clockreports,
timelines, etc), I decided that down that path lies madness.  ;)

So I'm not sure how far I should go with adding these other agenda types
to this package.  It would be nice if the agenda functions in Org itself
could be modified to accept grouping functions like this (it's a minor
change, really, to call a function to do insertion instead of doing it
directly), and that's something that I'd eventually like to propose to
the maintainers.

> 3. a clock summary (equivalent to what one gets with "v c" in standard
>    view)

I confess that I have never used this feature before you mentioned it.
:)  I tried to investigate this, but I've hit somewhat of a dead end:
the org-agenda-show-log-scoped variable, which is bound by
org-agenda-list, for some reason becomes unbound in org-super-agenda,
which doesn't seem to make sense, since it is just a copy of
org-agenda-list.  And even using edebug and stepping through everything
I can find, I can't figure out why it becomes unbound.

...And, having written that, it suddenly occurred to me: I wonder if
org-agenda-show-log-scoped is defined with defvar...and I see that it is
not...and so I tried doing it...and it seems to have fixed that
problem.  I guess I don't realy know anything until I explain it to
someone else.  :)

Now I can go ahead and add that defvar to my package for now, but it
should probably be added to Org itself.  It might be considered a bug
fix, but on the other hand, if it only happens with my non-standard
package...

Anyway, I will add this to my package and upload it shortly.  Please let
me know if it fixes the clockcheck command for you.

> 4. a clock report at the end (and really the end, after all other
> items)

As far as I know, this already works.  Just press "R" in the agenda
buffer.  Please let me know if it doesn't work for you.

After thinking about this some more, I have an idea: if I go ahead and
import the agenda-building functions into my package and modify them, I
can use a minor mode to override the standard functions with advice.
That way users would only have to define the groups variable, and all
the agenda commands would pick it up automatically, from the user's
perspective.  That would prevent users from having to define new agenda
commands that call special functions; they could just use their existing
custom agenda commands.

I will look into this and push it to the repo if it works.  Please let
me know if you have any other ideas or suggestions.

Thanks,
Adam

Re: ANN: org-super-agenda

[Adam Porter]

Adam Porter <adam@alphapapa.net> writes:

> After thinking about this some more, I have an idea: if I go ahead and
> import the agenda-building functions into my package and modify them, I
> can use a minor mode to override the standard functions with advice.
> That way users would only have to define the groups variable, and all
> the agenda commands would pick it up automatically, from the user's
> perspective.  That would prevent users from having to define new agenda
> commands that call special functions; they could just use their existing
> custom agenda commands.

FYI, this idea worked out well.  In fact, I only have to override one
function, org-agenda-finalize-entries.  I haven't fully tested all the
different agenda commands yet, but it's working well so far.  I'd
appreciate help testing further.  :)

Re: ANN: org-super-agenda

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 2468 bytes --]

Hi Adam,

thanks for the long email explaining the process and issues.

I've updated to latest version.

> Well, as it stands, this package is mainly a way to group together items
> that the standard agenda view (i.e. the org-agenda-list function) has
> gathered.

Understood.  I guess my challenge is understanding how to specify what I
want in your super agenda specifications.  For instance, how do I get a
list of tasks with deadlines as a separate entry?  How do I get
un-prioritised TODO entries listed?  These all come at the end, in the
"catch-all" section, for me which is not what I want.  The same with the
clock report: it would be good to be able to place this where I want
it.

I have my own customised view where I have the following:

#+begin_src emacs-lisp
  ("v" "Startup View"
   (
    (todo "INPR" (quote(org-agenda-overriding-header "Currently active tasks: ")))
    (todo "WAIT" (quote(org-agenda-overriding-header "Tasks waiting on others: ")))
    (todo "POST" (quote(org-agenda-overriding-header "Tasks for sometime later: ")))
    (agenda "" ((org-agenda-time-grid nil)
                (org-deadline-warning-days 365)
                (org-agenda-entry-types (quote (:deadline)))
                (org-agenda-skip-entry-if (quote scheduled))
                (org-agenda-ndays 1)
                (org-agenda-overriding-header "Unscheduled upcoming deadlines:")))
    (agenda "" ((org-agenda-ndays 1)
                (org-deadline-warning-days 1)))
    (todo "" (quote(org-agenda-overriding-header "Unscheduled No Deadline TODO: ")))))
#+end_src 

and I would like to mimick some of this behaviour.  Mind you, I can
simply use my view, of course!

> So the question becomes, how many Org agenda functions do I want to copy
> into my package and be responsible for fixing?

Probably not too many :-)

>> 3. a clock summary (equivalent to what one gets with "v c" in standard
>>    view)
>
> I confess that I have never used this feature before you mentioned it.

It's a useful feature but annoying in the default behaviour in that it
is either this or not at all.  (view log will display clocking
information but also task completion etc. which is more info than I
usually want [I know which jobs I finished])

> I will look into this and push it to the repo if it works.  Please let
> me know if you have any other ideas or suggestions.

Thanks.  I'll keep playing on and off.

-- 
: Eric S Fraga via Emacs 26.0.50, Org release_9.0.9-573-g09e612

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: ANN: org-super-agenda

[Adam Porter]

Hi Eric,

Thanks for your email.  First off, the package is on MELPA now, and I've
added several more features, so feel free to give it a shot.  :)

> I guess my challenge is understanding how to specify what I want in
> your super agenda specifications.  For instance, how do I get a list
> of tasks with deadlines as a separate entry?

That's the :deadline selector.

> How do I get un-prioritised TODO entries listed?

I don't have a selector for unprioritized TODO entries...yet.  I'll look
into that.  :)

> These all come at the end, in the "catch-all" section, for me which is
> not what I want.

Ok, I think I see the problem.  You see, your example command is
actually a series of custom commands, and the "todo" custom command
selects items without regard for their timestamp, without regard for
whether they'd appear in an daily/weekly "agenda" command.  So you may
need to define org-super-agenda-groups for individual commands in the
series.

Just to be sure we're on the same page, this package is for grouping
together items that are already selected by an agenda command (and the
:discard selector can be used as a filter, but that's not the package's
primary purpose).

> The same with the clock report: it would be good to be able to place
> this where I want it.

That's actually handled by Org itself, not this package.  To modify
where that goes, I think, would require modifying the agenda commands,
like org-agenda-list, to change when in the insertion sequence it adds
the clock report.  That would be a nice feature, but it's something that
should be addressed in Org itself.

I apologize for the confusion.  Since you wrote that email, I refactored
the code so that I no longer override the agenda commands, but just the
org-agenda-finalize-entries function, which is much simpler.  So while I
could have changed the clock report position before, now I can't.  But
it's better this way...for me, at least.  ;)

I don't know if this is quite what you're looking for, but maybe this
will give you some ideas:

#+BEGIN_SRC el
(let ((org-agenda-span 'day)
      (org-agenda-custom-commands
       '(("v" "Startup View"
          ((todo "TODO" ((org-agenda-overriding-header "Currently active tasks: ")))
           (todo "WAITING" ((org-agenda-overriding-header "Tasks waiting on others: ")))
           (todo "SOMEDAY" ((org-agenda-overriding-header "Tasks for sometime later: ")))
           (agenda "" ((org-agenda-time-grid nil)
                       (org-deadline-warning-days 365)
                       (org-agenda-entry-types (quote (:deadline)))
                       (org-agenda-skip-entry-if (quote scheduled))
                       (org-agenda-ndays 1)
                       (org-agenda-overriding-header "Unscheduled upcoming deadlines:")))
           (agenda "" ((org-agenda-ndays 1)
                       (org-deadline-warning-days 1)))
           (todo "" ((org-agenda-overriding-header "Unscheduled No Deadline TODO: "))))
          ;; Settings for all commands in the set
          ((org-super-agenda-groups
            '((:deadline t))))))))
  (org-agenda nil "v"))
#+END_SRC

Thanks for your feedback!

Re: ANN: org-super-agenda

[Adam Porter]

Adam Porter <adam@alphapapa.net> writes:

Hi friend,

FYI, this package is now on MELPA, and has had significant changes since
I announced it.  Please let me know any feedback you may have.  Thanks.

https://github.com/alphapapa/org-super-agenda

Re: ANN: org-super-agenda

[Adam Porter]

Adam Porter <adam@alphapapa.net> writes:

>> How do I get un-prioritised TODO entries listed?
>
> I don't have a selector for unprioritized TODO entries...yet.  I'll look
> into that.  :)

Silly me, you can already do that, like this:

#+BEGIN_SRC
(let ((org-agenda-files (list "~/test.org"))
      (org-agenda-span 'day)
      (org-super-agenda-groups
       '((:name "Unprioritized"
                :not (:priority>= "C")))))
  (org-agenda nil "a"))
#+END_SRC

Apparently I can't even remember what I did yesterday.  :)

Re: ANN: org-super-agenda

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 1091 bytes --]

On Saturday, 29 Jul 2017 at 19:41, Adam Porter wrote:
>> I guess my challenge is understanding how to specify what I want in
>> your super agenda specifications.  For instance, how do I get a list
>> of tasks with deadlines as a separate entry?
>
> That's the :deadline selector.

Yes, I assumed so and had tried that but I had not added the 't' you
showed in the example you now sent.  That probably makes some
difference!  Thanks.

> I don't have a selector for unprioritized TODO entries...yet.  I'll look
> into that.  :)

I look forward to seeing this!

> Just to be sure we're on the same page, this package is for grouping
> together items that are already selected by an agenda command (and the
> :discard selector can be used as a filter, but that's not the package's
> primary purpose).

Yes, I realise.  I showed my "v" custom agenda to show the type of
information I would like to see.  I do realise that mine is undated; I
want the dated equivalent using your super agenda.  Almost there!

Thanks again,
eric

-- 
: Eric S Fraga via Emacs 26.0.50, Org release_9.0.9-573-g09e612

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: ANN: org-super-agenda

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 840 bytes --]

On Saturday, 29 Jul 2017 at 23:11, Adam Porter wrote:
> Adam Porter <adam@alphapapa.net> writes:
>
>>> How do I get un-prioritised TODO entries listed?
>>
>> I don't have a selector for unprioritized TODO entries...yet.  I'll look
>> into that.  :)
>
> Silly me, you can already do that, like this:
>
> #+BEGIN_SRC
> (let ((org-agenda-files (list "~/test.org"))
>       (org-agenda-span 'day)
>       (org-super-agenda-groups
>        '((:name "Unprioritized"
>                 :not (:priority>= "C")))))
>   (org-agenda nil "a"))
> #+END_SRC

Seems to work but not really tested as I don't have many tasks... (about
to go on holiday ;-)).

> Apparently I can't even remember what I did yesterday.  :)

I can't remember what I did this morning... ;-)

Thanks again,
eric

-- 
: Eric S Fraga via Emacs 26.0.50, Org release_9.0.9-573-g09e612

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: ANN: org-super-agenda

[Adam Porter]

FYI, I just pushed an update to org-super-agenda that lets you
automatically sort agenda items into groups based on an "agenda-group"
Org property, which can be set for entire subtrees with inheritance
(which is enabled by default).  This makes configuration very easy, as
you only need one selector, like:

#+BEGIN_SRC elisp
(let ((org-super-agenda-groups '((:auto-groups t))))
  (org-agenda-list))
#+END_SRC

I should be able to do something similar for categories, which would
provide another way to easily sort automatically.

Everything still seems to be working, but please let me know if you find
any bugs.  Thanks.