Re: One vs many directories

[Jean Louis <bugs@gnu.support>]

* Texas Cyberthal <texas.cyberthal@gmail.com> [2020-11-21 03:35]:
> Having a tall directory tree with many leaves and branches is against
> Org's philosophy.

Thank you for your nice ideas.

Here is your fellow classifier of information.

I don't know what is Org's strategy but if you mean many deep
subheadings my deepest is with 7 stars ******* and quite I have 7 even
of them in 850 Kb file and there are thousands of Org files on my
system.

Point is that apparently it becomes harder and hard to organize if one
does not have enhanced system of organizing. Org is rather for notes
and it not for everything

> Here is my argument that such a structure is objectively correct for
> personal info management:
> 
> https://github.com/cyberthal/10-Bins-template
> 
> For the record, Org works fine with this, although I had to do a bit
> of config to get around the default philosophy.

Your system is understandable and great idea.

Some of first ideas on filing files and notes and any piece of
knowledge in an organized manner have been presented by Doug
Engelbart, forfather of modern computing and I recommend reviewing his
work for your further thinkering.

Personally I have created similar filing system and now I may upgrade
by using parts of your offered paradigm. Maybe you pick up some of
these ideas as well.

In my life I deal with people including me and groups of people. One
person can belong to various groups of people. As both groups and
people can have same names all of them have their unique ID
numbers. And I work both with the file system and the PostgreSQL
database in background.There are just few directories that I define at
beginning, those are:

- Directories for groups or accounts how I call it since years, for
  example /home/me/data/accounts/By-ID and
  /home/me/data/accounts/By-Name

- Directories for individuals for example /home/me/data/people/By-ID
  and /home/me/data/people/By-Name

Those By-Name directories are symlinks to By-ID/UNIQUE-ID and those
symlinks need to be updated from time to time as people change their
names and there may be duplicate names, so name can look like
John-Doe-1 or John-Doe-2, those are just helpers

After that I forget anything about the above structure, so I do not
work any more with remembering the structure, not at all.

Then there are helper programs in Dired and outside of Emacs that
should work with any file manager. If file manager supports
external scripts such as Rox filer or Midnight Commander or Nautilus,
that is fine.

Let us say that I wish to file something related to the Civil War, at
first such account should exist in the database. So if it does not
exist it is created very quickly at moment of filing.

Accounts have their location in terms of the address not quite in
terms of a continent or they may be without location.

I would press s-f in Dired and choose the proper account by using
completion, normally helm. Then the file is filed under date of filing
there.

During that process I know nothing where the file is located
exactly. That is what program does for me, I am not remembering the
location neither I know where it is but I could find it when I wish.

Files goes there:

/home/me/data/accounts/By-ID/102/2020/11/2020-11-21/file-here.org and
/home/me/data/accounts/By-Name/Civil-War/2020/11/2020-11-21/file-here.org
pointing as symlink to above unique ID

That is all, after that I forget about the file.

When it comes the time to research the subject again, I just browse or
find the account "Civil War" and I say: Go to files

Then the files of Civil War are opened in front of me and I can find
it by using find-dired, I can find it by using system locate tool that
is very flexible as I can search by its specific directory, right?

Let us say I deal with person Joe Doe, normally I look into the
database for the person, and database also have location
information. There are sometimes Joe Does in various countries in
various cities so I have to quickly locate the right Joe Doe.

Once I find the Joe Doe, at a click of the key F4 I am already in the
Org file for Joe Doe. If Org file does not exist it is created on the
file and contains all the basic information I usually need to manage
for Joe Doe.

As if I am sending tasks to Joe Doe and organization X has relation to
Joe Doe, then why should I keep Joe Doe's information in one Org file?
Or few that may be personal?

Personal Org file I keep under my ID's directory. Org file related to
specific person I keep in that specific person's ID directory. My
personal Org file may point or hyperlink to some persons. Though I
find reference to such hyperlinks through communication and various
lists, and not through the list I make in my personal Org file.

I can just save the file and forget about it. But I never remember
where the file is located because that is what computer does for me. I
do not know the name of the file. That is what computer does for
me.

Another user of the computer would not know the location of file but
user would be able to always work with the file and to locate quickly
the file.

The paradigm of the PostgreSQL and general relational database entries
is reflected on the file system that becomes relational file system.

When searching in the database for anything, I am not looking where
such entry is stored, I am looking for the name or other meaning
related to the entry. Then I can find all other pieces of information
related to that entry.

Filing on file system and general filing of any pieces of information
should be related to each other as much as necessary so that by
finding one piece of information one may browse or come to those other
pieces of information through its relations.

When I search for Joe Doe, I can press F3 and one Org file is opened,
constructed on the fly, that shows me all needed relations of this
contacts:
		    ________________________________

		     PROFILE FOR CONTACT ID: 239917

			       NAME HERE
		    ________________________________


		   Saturday, November 21 2020, 08:08


Table of Contents
_________________

1. Profile for Joe Doe, contact ID: 239917
2. Pictures for contact 239917
3. Notes for contact ID: 239917
4. SMS list
5. Mailing list subscriptions
6. Mailings received
7. Interactions
8. Emails
9. Calls

In every subheading of such automatically on-the-fly created Org file
I have hyperlinks pointing to pieces of information related to that
person or actual notes, SMSs inside. Emails are in email folders that
can be quickly opened without thinking where such email folder is
located. 

Treefactor has paradigm of filing into 10 bins. I do not find it is
idiosyncratic even if it is peculiar to your system of filing. It is
so much similar to the ideas of Doug Engelbart how things should be
filed that other users can also access files by X type of queries and
that you as main user can find it easily.

> 1 Make info easily retrievable when needed. The next time you need
>  the info, you will think of it in X way. So file it at X location,
>  to convenience future you.

That is right.

By general principle to make it easily retrievable when needed I agree
on that. My searching goes by group names and by individual names and
locations and other attributes of those groups and individuals. I may
search for the country name and I am ending up in the folder related
to countries. But from countries I may not go to individuals. As if I
file a note about country then it is about legalities like various
laws, security, geology and similar. When the need arises in future I
can easily link information.

The SQL table `countries` is similar to following:

 countries_id 
 countries_name
 countries_code
 countries_prefix (phone prefix)

There are also tables `region` and `continents`, and then there are
groups of countries, for example there is group Europe and there is
group European Union and Schengen Zone.

Then if I need some particular field I can just add it to the table
`countries` such as:

 countries_apostille

to designate if country does provide international legalization by
Hague convention.

Then for reach table there shall be known what is the "name" part of
the able, in this case: countries_name, but sometimes it could be
countries_title or counties_hid (human ID) or similar. Once the setup
is made, the file system structure can be automatically derived.

All I want is to file this law into country profile, but not into
database, so I may invoke filing by country, continent, region or
city, you name it. Program is that decides WHERE it will be filed by
using those various identifiers.

In general I am not filing by location if such information is personal
or belongs to specific group. This is because that specific group
already has its location defined in the database and specific
individuals may have location in their profile and because my files
mostly related to people I interact with, they are many times sent by
those people to me.

If files are sent by specific date, such as image, then I am filing it
by choosing the date. Otherwise files are filed under the currend date
of the today under the file belonging to the group or individual.

Notes of person are filed in the database and they can also be on file
system. When opening the profile Org file I have access to all notes
in the same one view or one file (that does not exist on the file
system). From there I see all notes of the person such as notes in the
database and notes in the file system. If necessary I can save such
meta Org file or automatically update it next time.

> These rules sound complicated in theory, but they're easy in practice:

> 1 Useful stuff goes up.
> 2 Useless stuff goes down.
> 3 Put things where you'll find them.

At time of filing a piece of information it is hard to know what will
be useful in future and it is hard to know what and why is useful in
present time.

For this reason various grouping or lists or subsets are important
where one can file other piece of information.

For example there is group FOLLOW-UP where I put people I have to
follow-up until they are not followed up.Person could be member of the
group FRIENDS but not a friend that I follow up with communication or
bring through certain stage or process. Some people are in the group
of AGENTS and some have been sent BIRTHDAY-CARD, some are relatives of
person ABC and so on, all those relations once established should then
be quickly accessed from one piece of information to other.

My ranking is  thus based on various groups that designate ranking.

There is basic and on the fly created group of last contacts.

M-x cf-last-contacts

shows me all the last 200-300 last contacts entered into the
database by their group, country, by the date they are entered.

> 0-Inbox is first because its existence is a red flag. If it exists,
> then you should stop and refile it before anything else. Otherwise,
> arbitrary headings can be lost indefinitely in bypassed inboxes
> buried in the tree!

Good concept.

My inbox for files is usually $HOME directory or $HOME/TO-SORT or
$HOME/TO-DO

Those files need to be filed and notes created for future, or maybe
deleted. When filed they get its use, become useful.

Example is the mining license pertaining to specific person, if not
filed under that person due to nature that I am looking for people and
not files, I would not find the file that is not filed. When I speak
next time to that person I would not know what the person speaks
about:

"Hey, how do you find the licensed location?"

- now if I know the name or email of person, I can quickly search and
  find all notes, files and the license. I can be able to answer
  quickly. When dealing with thousands of people that is the way to
  go. I will know what to answer if I have filed people, groups, files
  in proper place. If I have not, I am lost and relation may get
  broken because I am not following up and not knowing what person is
  talking about. And that is business that I am losing if I am not
  organized.

> 1-Outbox is next because it is effectively the Inbox for another
> repo. A similar danger applies, although at least you know the info
> doesn't belong in the current repo.

I support this idea though personally I have not get other
repositories. I have external hard disks with videos that I cannot
possibly keep internally and I have external hosting space with
files. They are (should be) all sorted through the dynamic knowledge
repository that is named HyperScope.

If the file is under subject "Hobby" at location of external disk ABC,
then such is indexed in HyperScope. All I need to do is to click on
the file. If disk is not connected, program has to tell me where to
find the disk with the specific designation. If disk is connected and
not mounted it has to get mounted when I need it and file is opened.

All that time I need not know exactly where such file is located. All
I know is either AUTHOR name or SUBJECT of video or VIDEO FILE NAME
that I see in front of me, maybe I can see description and some tags
or marks. But where the file is really located I do not need to see.

So I spare all the time to search for files by browsing the file
system. I search for author or name and files are found
automatically.

> 2-Codex is next. Its content is terse and dense. You don't want your
> code snippets buried in your prose.

Idea is good. Personally not so important.

My software is filed under some directory and various subjects.

If it belongs to person it may be filed under specific person. Let us
say there is group: Emacs Package Authors, it is quite easy to index
authors, to file their software into their place which I do not
consciously need to know where it is, to extract the README or package
commentary and form an index. Then by using HyperScope I can quickly
access the index or update their files let us say by using `git
pull`. Or I could automate the process.

In software world authors are almost invisible as their names are
buried, maybe that is customarily I do not know but not that I agree
to that fact. I do like to know the author's name at least as there is
quite a difference if software is written by Thierry Volpiatto or by
`jtbm37`. The difference is related to trust, quality and familiarity
of software. Especially free software is used often by relation to the
community or by some principles of software creation. Those principles
originate from developers and their philosophies such as OpenBSD or
DragonflyBSD principles or philosophies, or GNU and Richard Stallman's
philosophy, or freedom pushing GNU Hyperbola/Linux-libre and GNU Guix
operating systems.

> Info also varies by retrievability. The more costly the retrieval,
> the less profitable the info.  Maintenance cost also reduces info
> profitability.

Not that I can relate to this. Retrieval is not for me important
attribute to note.

> Stable info has high retrievability and low maintenance cost. The next
> four directories are ranked primarily by stability.

> 4-Time is first, since both money and time are precisely and
> objectively quantifiable. Both are precious.

That is one necessary factor for classifications. I am using it and it
is done automatically by the date of filing or semi-automatically when
I wish to designate specific date to the filing.

> 5-Location is next, because geography is quite stable and
> fundamental, as is architecture. Both are a bit more ambiguous and
> dynamic than time and money, though.

As locations are related to people and accounts (entities, groups,
lists) my search for file under specific location goes over account or
group, not over location to account, but that is also possible.

I am searching for the group "Transport" with country "Uganda", so I
find transporters in Uganda or in city Kampala and their files are
quickly opened without me knowing where the files are, and their
emails are opened without me seeing their email address, it is just by
click as things are hyperlinked.

While I do agree to file by location, I think it is not manageable
without using the database. There are continents, 248 countries in my
database system, there are so many regions, cities, etc. My people are
located in 229 countries. Do I really want their data located under
countries? I do not think so. That would be too much geographical
information on the file system. I can search for country and find
peple in that country or that city, but not that I wish to have too
many geographical attributes on the file system as that is repetition
not necessary. My database of people has 196159 people. Majority of
them have their location, it is mostly country, often state and
city.

I can find a mining engineer in country Senegal if I wish so, that has
some work experience and I can see files pertaining to this
person. But not that I would make file system directory Senegal to
find the files for this person, as Senegal as attribute is already
related to specific subset of people in the database.

While I am using similar system of searching by location, I am not
filing necessarily by location. Unless that filing relates to the
location itself as a group such as USA/Colorado/Laws/LLC-Law.txt

Same for objects.

> 7-Name is next. Names are more ephemeral than objects. Your personal
> nomenclature evolves gradually.

Quite clear.

> The last two directories follow the priority-density rule again.

> 8-Action is next. Plans are more ephemeral than names, but more
> important than background info.

> 9-Background is last. It contains huge quantities of
> poorly-retrievable info, most of it easily replaceable.

I have plans in the database. There are plans as stragegical plans,
tactical plans, projects that fulfill specific plan step, tasks that
fullfil specific project's step, goals, purposes, etc.

When person belongs to project, person is related to that project.

If I then wish to look into the plan or project, I can also see which
people are related to plan or project and act upon it.

Set of hyperlinks in HyperScope can provide me a hyperlink to the file
where by such hyperlink is related to specific contact 123 and
specific project 234.

Hyperlinks belonging to one subset of things are then organized
together.

HyperScope is my main dynamic knowledge repository of many things,
various hyperlinks.

It can create dynamic on the fly meta Org file that points out to all
the other Org files, any kind of files as hyperlinks (not only WWW),
PDFs, movies, images, notes, people and groups, etc.

> This improves iterational velocity, accuracy, creativity, and
> effective intelligence.

While I have expressed some personal preferences and differences, I am
in full agreement that systems of knowledge organizing such as
Treefactor are useful and helpful.

And I think that every person is able to develop such for oneself. It
is not that hard or complex. It is just a paradigm that one wish and
wants to adopt and apply in life.

When used for the benefit of the group such paradigm increases the
collective IQ.

The system of organizing knowledge you have described would be so much
more useful if it would be practically used by people of the same
group. That is where it increases collective IQ tremendously.

References:

Doug Engelbart Institute - Boosting mankind's capability for coping with complex, urgent problems
https://www.dougengelbart.org/

Draft OHS-Project Plan
https://web.archive.org/web/20070704185333/http://www.bootstrap.org/augdocs/bi-2120.html

TECHNOLOGY TEMPLATE PROJECT OHS Framework 
https://www.dougengelbart.org/content/view/110/460/

Addressing Hyperdocument Content And Structure
https://www.dougengelbart.org/content/view/110/460/#2b1

Toward High-Performance Organizations A Strategic Role for Groupware 
https://www.dougengelbart.org/content/view/116/#7j

About Dynamic Knowledge Repositories (DKR)
https://www.dougengelbart.org/content/view/190/163/