Re: Culling org files

[TRS-80 <lists.trs-80@isnotmyreal.name>]

On 2021-03-03 16:59, Samuel Wales wrote:
> along lines of reducing logbook entries

I guess you must have picked up on my comment in another recent
thread.  :)

> i often want to reduce org
> files, and i wonder if anybody already had the same desire.
> 
> here are some random ideas.  my org files are so
> large i might have written this list a few times....
> 
>   1) list links to duplicate headlines
>   2) list links to duplicate body text
>   3) list links to duplicate entries
>   4) list links to duplicate entries, body text, or
>      headlines using fuzzy matching
>      - suppose you captured an email slightly differently a
>        few times
>   5) show in agenda the biggest few tasks so you can go to
>      them and reduce them or doneify them
>   6) (waves hands) git magic to find old entries that might
>      be stale
>   7) show in agenda the tasks with biggest logbook drawers
>      so you can go to them and reduce them
>   8) find similar body text that are in distant subtrees
>      that might be candidates for refactoring using org-id
>      linking
>   9) show in agenda deepest olpath levels
>   10) indicate deep, shallow, text-filled, etc. top levels
>   11) show in agenda entries with most children
>   12) archive logbook drawer entries older than 1 year
>       - get rid of drawer if empty
>       - put the drawer entries into a logbook drawer in a
>         new task, with a similar header, that then gets
>         doneified.  then that gets archived when you archive
>         stuff.
>   13) operate on lines matching a pattern
>       - e.g. "* [2021-02-17 Wed 20:35]  whatever" lines
>         might be insubstantial notes that do not need to
>         clutter the inactive timestamp display in the agenda
>         and thus should be moved to a target location with
>         query
>       - that target location would presumably not be in an
>         agenda file
>   14) function to lint all agenda files
>   15) reduce false positives in lint
> 
> well, idk if htese are good ideas.  just thought maybe we
> could form a cult of "don't let org files get too big".

I have come to similar conclusion about "don't let org files get too
big."  Besides agenda speed, I think it is just easier to
conceptualize things when each file covers only a limited scope, trees
are more shallow, etc.

So, lately (last year or more), I have been trying a "many small (up
to perhaps medium)" instead of "few big" files approach (along with
some custom tooling) and it has been working /a lot/ better for me.  I
really feel on top of things for the first time in a long time.  My
agenda is not cluttered.  I can focus on important things, while not
losing track of the rest, etc.

I could write a whole lot about my "custom tooling" but as that is an
entire package in its own right (still in experimental development and
thus unreleased), I will limit my comments here only to the "archival"
portion of this problem.

I realized, at least in my case, after mulling this over for some
time, that there seem to be a few distinct cases which would need to
be handled by a custom archival function:

- If the TODO is still active, and the number of logbook entries
   exceed some (definable) threshold, either move the older entries to
   a similarly named archive file/heading, or (also definable) simply
   delete them.  This would cover things like habits and other
   recurring tasks that tend to generate lots and lots of entries over
   time.  This is perhaps the part I mentioned in the other thread
   recently.

- If the TODO is completed (and perhaps also after it becomes a
   certain (again, definable) age), then move the whole TODO to a
   similarly named archive file.

- There was another, but I think it was for the case where the entire
   file is a project (which is a bit specific to my own setup).

Ideally, this custom function would handle all the above cases, and
could be called with point at each headline, so it would be easy to
map over a file or even a directory full of files, in order to
automate the archival process (perhaps annually?).

Cheers,
TRS-80