Re: Culling org files (notion of Types, many small vs few big files)

[TRS-80 <lists.trs-80@isnotmyreal.name>]

On 2021-03-04 16:11, Samuel Wales wrote:
>> tim> naming convention ... to determine what is included
> 
> this is also what i do.  org-agenda-files is just set at startup
> according to basename pattern.

I find it very interesting that all three of us seem to have
independently arrived at some of same conclusions.

Originally I did not want to go off on this tangent, but now I wonder
how close what I am doing is to what you guys are doing.

In my case, I came up with some notion of "Types."  Each Org file MAY
use one of a list of defined Types at the beginning of the file name
(which is also the first top level headline in the file, starting at
position 0), followed by a delimiter (currently ": ").[0]

I keep experimenting with my list of Types (I probably have too many),
but there are a few that definitely seem useful so far.  For example:

- Project: Pretty self-explanitory.

- Area: A concept lifted straight from PARA Method ("a sphere of
   activity with a standard to be maintained over time").[1]

   - Equipment: A special type of Area: that pertains to a single major
     piece of equipment (like a vehicle) or some group of related
     equipment (e.g. "small shop equipment" or "home appliances",
     etc.).

- HowTo: Literally "how to do x" which is great for remembering those
   obscure command line invocations (or whatever) that you only use 2x
   per year.  Combined with headline level completing-read search (see
   below) this becomes very powerful/handy.

So then, by default, any of Org files starting with either Area:,
Equipment:, or Project: are the only ones that are considered "active"
for purposes of agenda and scanned for TODOs (implemented as a simple
`directory-files' function and a regexp).  I use my system as a
combination of TODO and PIM[2], so this makes a nice logical split
where all those PIM "random notes" do not impact the agenda
performance whatsoever.

I have some other custom agenda functions as well, for things like
periodic reviews (in the GTD sense) and others.  Org's Agenda really
is essentially just like a database query engine when you get right
down to it (except storing in plain text of course).

>> trs> [smaller files] My agenda is not cluttered.
> 
> it is not clear to me why more smaller files and shallower trees in
> the outline would improve the agenda.  sounds good though.

I somewhat addressed this above with Types (which improve
performance), but as to your specific point (clutter)...

OK, so maybe not /directly/.  But rather the whole system have
improved my engagement, by way of no longer feeling lost/overwhelmed
as I did with very deep trees in only a few files.  I think it is just
easier to reason about some small subset of the whole at one time, as
represented in a single file.  In theory, I guess you could accomplish
the same by narrowing subtrees or other methods, but for whatever
reason separate files seem to appeal more to me than those other ways
(probably because they are also faster to navigate, among other
benefits).  However, to each their own here, I suppose.

I think I was also responding to some specific comment you made about
time stamps (re: "cluttered").

There is also this whole "inter linking" / "atomicity" thing.  I came
to Orgmode from TiddlyWiki, and that was the only thing I missed, this
notion of many small "atomic" nodes, which could then be put back
together in many different ways (links, tags, etc.) as opposed to a
(usually single, large) tree which (at least somewhat) imposes a
particular structure and implicit categorization.  Nowadays the
Zettelkasten stuff have become quite popular, but it is exactly the
same notion.  Tree knowledge structures are great when many people
must share the same info, for example law codes.  Or reference
manuals.  But in our own PIM, we should be more free to link
information together in whatever way suits our own brains.  And be
able to link it back together in multiple, sometimes differing, ways.
I seem to also recall some discussion of research even supporting the
idea that our brains actually function more like an "interconnected
web" than a "tree" structure (can you tell I am a bit of a PIM geek
and have been interested in this subject for some years now?).  :D

Thinking further, I guess my usage has also become possible by some of
the search and other tools I have built /around/ my directory full of
small files, which obviate some of the reasons for keeping things in
"one or a few big files."

One example is my custom headline search function (which uses grep
under the hood)[3].  It has been very helpful in being able to locate
things.  Now I have a completing-read search over all headlines in all
my files (which will jump to that location upon selection).  I have
found that by carefully constructing headlines that this works "well
enough" for almost all my search needs so far.[4]

> On 3/4/21, Tim Cross <theophilusx@gmail.com> wrote:
>> 
>> My use pattern also constantly evolves as my requirements and 
>> priorities
>> change. It is and probably always will be, a work in progress!

I consider mine an ongoing experiment as well, which is why I have not
(as of yet) published anything.  But I'm glad I decided to discuss it
on the list, as now I see I'm not the only one to reach some of same
conclusions!

Cheers,
TRS-80

[0] But I need to change that because colon I learned is an illegal
character on exFAT file system (i.e., larger SD cards, which becomes a
problem when syncing to mobile).

[1] https://fortelabs.co/blog/para/

[2] Personal Information Management

[3] So it doesn't need to open a bunch of files in Emacs / Org.  It's
also fast enough (so far) to use "live."

[4] I originally planned to implement additional full text search
functions using things like org-ql, [rip-]grep, and/or Recoll, but so
far this has proved to be unnecessary.