Temporarily setting agenda files list, cleaning up

Temporarily setting agenda files list, cleaning up

[Christoph Groth]

[-- Attachment #1: Type: text/plain, Size: 2379 bytes --]

Hello,

I would like to expand my use of Org for notes, and to this end spread
project-specific org files across my home directory (currently I'm using
a central directory with one agenda file per year).  For obvious
reasons, I can't possibly include all these org files in
org-agenda-files permanently and have them open in Emacs all the time.

Instead, I'm thinking of a setup as follows: I'll reserve
org-agenda-files for active projects only.  But there will be many other
org files (for projects that are finished or dormant, general notes,
etc.), that I would like to be able to search as well.

I saw that some people have solved this problem with a custom search
solution, even using a database [1], but I think that for my purposes a
simpler solution mostly using Org's built-in functionality will do.  I'm
also aware of org-agenda-text-search-extra-files, keeping all my extra
files there is not satisfactory, because the search possibilities are
limited (and otherwise it poses the same questions that I will ask
further below).

Instead, I imagine a custom Emacs command to launch an agenda with
org-agenda-files that is temporarily set to a list of files that depends
on the current context.  For starters, this list could contain all the
org files under the current directory:

(split-string (shell-command-to-string "find `pwd`/* -name '*.org' -type f") "\n" t)

This way, I could easily explore the org files that belong to specific
subsets of my activities and interests.  I could also search *all* my
notes, when needed (for now the time this takes should not be a
problem).

I wonder if anyone has tried a similar setup and would be willing to
share his experience and ideas.  Specifically, I wonder about the
following points:

* Do you see any obvious problems with the above idea?

* In Elisp, what is the best way to temporarily set org-agenda-files and
  later reset it back to the standard value?

* How to automatically close unneeded org buffers that were opened for
  the temporarily agenda?  The command org-agenda-exit is not
  satisfactory, because it also kills the buffers that one has just
  found due to the search, and also it's not automatic.  Perhaps one
  could close the buffers immediately once the agenda view has been
  created?

Thanks
Christoph

[1] https://kitchingroup.cheme.cmu.edu/blog/2017/01/03/Find-stuff-in-org-mode-anywhere/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Temporarily setting agenda files list, cleaning up

[John Kitchin]

[-- Attachment #1: Type: text/plain, Size: 3922 bytes --]

For running code that should close all buffers that were opened, I use a
macro like this:

(defmacro with-no-new-buffers (&rest body)
  "Run BODY, and kill any new buffers created.
Returns whatever BODY would return."
  (let ((current-buffers (buffer-list)))
    `(prog1
(progn
  ,@body)
       (mapc (lambda (buf)
      (unless (-contains? ',current-buffers buf)
(kill-buffer buf)))
    (buffer-list)))))

For the other kinds of things you listed, check out
https://github.com/jkitchin/scimax/blob/master/scimax-notebook.org,
especially the nb-agenda function. It is more than you want, you can choose
a project, and then get an agenda for the org-files in that project.

Another example is at
https://github.com/jkitchin/scimax/blob/master/scimax-journal.el#L284 where
I get an agenda for journal entries in a date range. You can use this idea
generally, you let-bind the org-agenda-files variable to be the list you
want, and then call org-agenda.

These are a work in progress, but I use them pretty often.

You can restrict the agenda to a file, and even select search with
something like this:

(org-agenda nil "s" "<")




John

-----------------------------------
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu



On Mon, May 27, 2019 at 8:11 AM Christoph Groth <christoph@grothesque.org>
wrote:

> Hello,
>
> I would like to expand my use of Org for notes, and to this end spread
> project-specific org files across my home directory (currently I'm using
> a central directory with one agenda file per year).  For obvious
> reasons, I can't possibly include all these org files in
> org-agenda-files permanently and have them open in Emacs all the time.
>
> Instead, I'm thinking of a setup as follows: I'll reserve
> org-agenda-files for active projects only.  But there will be many other
> org files (for projects that are finished or dormant, general notes,
> etc.), that I would like to be able to search as well.
>
> I saw that some people have solved this problem with a custom search
> solution, even using a database [1], but I think that for my purposes a
> simpler solution mostly using Org's built-in functionality will do.  I'm
> also aware of org-agenda-text-search-extra-files, keeping all my extra
> files there is not satisfactory, because the search possibilities are
> limited (and otherwise it poses the same questions that I will ask
> further below).
>
> Instead, I imagine a custom Emacs command to launch an agenda with
> org-agenda-files that is temporarily set to a list of files that depends
> on the current context.  For starters, this list could contain all the
> org files under the current directory:
>
> (split-string (shell-command-to-string "find `pwd`/* -name '*.org' -type
> f") "\n" t)
>
> This way, I could easily explore the org files that belong to specific
> subsets of my activities and interests.  I could also search *all* my
> notes, when needed (for now the time this takes should not be a
> problem).
>
> I wonder if anyone has tried a similar setup and would be willing to
> share his experience and ideas.  Specifically, I wonder about the
> following points:
>
> * Do you see any obvious problems with the above idea?
>
> * In Elisp, what is the best way to temporarily set org-agenda-files and
>   later reset it back to the standard value?
>
> * How to automatically close unneeded org buffers that were opened for
>   the temporarily agenda?  The command org-agenda-exit is not
>   satisfactory, because it also kills the buffers that one has just
>   found due to the search, and also it's not automatic.  Perhaps one
>   could close the buffers immediately once the agenda view has been
>   created?
>
> Thanks
> Christoph
>
> [1]
> https://kitchingroup.cheme.cmu.edu/blog/2017/01/03/Find-stuff-in-org-mode-anywhere/
>

[-- Attachment #2: Type: text/html, Size: 5326 bytes --]

Re: Temporarily setting agenda files list, cleaning up

[Thomas Plass]

Christoph Groth wrote at 14:04 on May 27, 2019:

: Instead, I imagine a custom Emacs command to launch an agenda with
: org-agenda-files that is temporarily set to a list of files that depends
: on the current context.  For starters, this list could contain all the
: org files under the current directory:

This will use a private set of 'org-agenda-files and kill their
buffers after executing `org-agenda-list':

(defun Groth/agenda-list (&optional dir)
  (interactive)
  (let* ((project-dir (or dir default-directory))
         (org-agenda-files (directory-files-recursively project-dir "\\.org$"))
         tmp-agenda-buffer)
    (unwind-protect
        ;; FIXME: set org-agenda-list args as necessary: &optional ARG START-DAY SPAN WITH-HOUR) 
        (org-agenda-list)
      (mapc
       (lambda (f)
         (and (setq tmp-agenda-buffer (find-buffer-visiting f))
              (kill-buffer tmp-agenda-buffer)))
       org-agenda-files))))

Call it like this:

  M-x Groth/agenda-list

or eval 

  (Groth/agenda-list "/path/to/dir")

I just whipped this up and it might need improvements such as optional
prompting for a search directory and a set of arguments to
`org-agenda-list' that fit your needs (refer to its docstring).  Note
that buffers for files on the old value of 'org-agenda-files that are
already open will be killed, too.

Regards

Thomas

Re: Temporarily setting agenda files list, cleaning up

[Christoph Groth]

[-- Attachment #1: Type: text/plain, Size: 1906 bytes --]

Many thanks to John and Thomas for their suggestions! 

Thomas Plass wrote:

> This will use a private set of 'org-agenda-files and kill their
> buffers after executing `org-agenda-list':
>
> (...)

This function is a very good prototype of what I had in mind.  The
details like it closing buffers that were already open before do not
matter for now.  (John's post offers a solution to this particular
problem.)  I replaced the call to org-agenda-list with one to
org-agenda, in order to get the full thing, and this works.

The prototype reveals a problem with the approach: if the buffers with
the agenda files are closed immediately (to avoid cluttering the buffer
list with possibly hundreds of buffers), most of the agenda
functionality no longer works.  It's not possible to jump to entries,
reschedule them, etc.

I guess that modifying orgmode so that agenda buffers open files as
needed would by rather complicated.  The simpler alternative should be
to keep the buffers open and only kill them when the associated agenda
buffer itself gets killed.

I suppose that this can be done by using kill-buffer-hook.  It seems
that the buffer killing can be made smarter by storing the
buffer-modified-tick at the time of the agenda construction for all the
buffers that are to be closed.  This information can then be used later
to only kill buffers that haven't been touched since.  Ideally, one
should also check whether it is still the *same* buffer, because it
might have been closed and reopened on purpose by the user.  (I have no
good idea how to check for that, other than storing opening times of org
buffers using some hook and using that.)

By the way, the function directory-files-recursively takes 1.5 s to
search my (huge) work directory for org files.  Using
shell-command-to-string to launch the 'find' shell command takes only
0.2 s.

Christoph

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]