Re: One vs many directories

[briangpowell <briangpowellms@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 16007 bytes --]

* Strongly suggest looking into Emacs' vlf-mode and the newer vlfi-mode

** That is Very-Large-File-Mode & Very-Large-File-Improved-Mode for issues
you're experiencing & if not, simply because they're very useful &
interesting & fun Emacs Modes to explore & put into your toolbox

https://www.emacswiki.org/emacs/VLF

https://github.com/m00natic/vlfi

* You mentioned other types of GREP, I second that, and the indexing
suggestion--I remember long ago using SGREP which is Simple-GREP, used
indexing & was much faster than the usual grep implementations for some
things; but, this is at the expense of the fancier & more elaborate GREP
functions

** You mention RipGrep--thanks for that, very interesting

* Which brings me to my main suggestion to you & why:

Emacs, believe it or not, has the FASTEST ENGINE available, without
augmentation in any way, for INTERACTIVE SEARCH--since the whole engine is
designed to be optimized to search-while-editing

But for many other searches, more elaborate searches, fancier GREP
searches, it's a VERY BAD choice of ways or programs to use for searching

What I mean is, say you're editing a file, and you search for your
"ProviderBuilderFactory"

Suggest you try opening a huge file--even MULTI-GIGABYTE FILES--huge files
in Emacs VLF-Mode--Very Large File-mode {which I believe can be done as a
sub-mode to/with Org-Mode}

And you can do this fearlessly since vlf-mode works by dividing the files
up for you in the background, etc.--while you're editing--but uses the same
built-in Emacs engine, optimized for such searches

And then you type:

Control-s

And start to type the first letters of "ProviderBuilderFactory"

This will interactive-search HUGE files, very quickly, and in near "Real
Time"--since this is what Emacs (implemented in C) is optimized to do--its
optimized for initial-character-searching "as you type them"--most other
engines are NOT

IF FOR NO OTHER REASON THAN IT SOUNDS LIKE FUN! And you might use vlf-mode
for other tasks you may face in the future.

Please try it out & tell how you like it--you'll never avoid opening huge
files again is one benefit

Beyond that, suggest you look into using LEX, it's as fast as you can get
for some things too.

Everything has its niche in the *nix world--which is where GREP came from,
Bell Labs, etc.--that's the Unix philosophy, Emacs & LEX tools came from
that world & the work of thousands of contributors--suggest you try these
tools too for these issues

Lastly, say you want to search for things without opening a file, you can
still use Emacs in batch-mode, at the command line, without opening a full
emacs session



















On Sat, Nov 21, 2020 at 8:34 PM Jean Louis <bugs@gnu.support> wrote:

> * Dr. Arne Babenhauserheide <arne_bab@web.de> [2020-11-22 01:48]:
> > > So in general I never need to use some general search through Org
> > > files or any other files as my way of thinking begins with People or
> > > Groups and that narrows what has to be searched.
> >
> > How do you deal with stuff that applies to several people?
>
> From database viewpoint there are
>
> - accounts (which means companies, groups, entities, like "People who
>   wish to get employed in Europe")
>
> - there are contacts, that may belong to account, additionally belong
>   to company (also account), additionally be member of account, so
>   there are 3 groupings for each contact how that contact may be
>   related to account. If it is main account such as "Welders" or if
>   maybe under "Company" is written "welders" (not quite correct) in
>   reality it does not matter.
>
> - then there are lists to which other lists belong. Account A and
>   account B, C, D can belong to list 01. Various accounts can be put
>   together in uniting lists. Those lists are encompassing other lists,
>   not individual people but people in the list (account) usually
>   unless there is only one in the account. Those lists I am using for
>   mailing them or informing them by letter, SMS, etc. Geologists and
>   mining engineers and metallurgists are 3 different accounts but if
>   all of them speak Swahili both in Kenya and Tanzania and are in the
>   related branch of economy so they can be sent same type of
>   information.
>
> Then there are groups, which is just another name for a new list. Then
> there are tags. I can freely tag account, contact or anything else. By
> tags I can finely select specific people belonging to specific group.
>
> There are account types and group types.
>
> Tags by itself have its own description or purpose to name it type.
>
> Some people introduce other people, few of them introduced
> thousands. So contacts have a column "introduced by". That becomes
> very handy when talking to somebody and it also helps in awarding
> introduces. It helps when people place their hyperlinks and become
> automated introducers (lead generation).
>
> When I know that person belongs to some group of people and I have to
> write email and I know it is better to inform everybody, then there is
> action to assign identity from which I am sending email. It could be
> public or internal identity with different email address. Emails to
> that person would always go from designated email address without me
> thinking about it. Then there are Cc and Bcc fields and in those
> fields I could designate: to inform same contact to each of email
> addresses with same message, and to inform group of people each
> time. Thus if writing to one contact all others get informed through
> same message. But I am not thinking about who belongs to the group,
> and what are their email addresses, that gets inserted
> automatically. Email composition is usually inside of Emacs.
>
> Sending files? If contact is in the group and others need to see the
> file, then Cc and Bcc fields work in the same way, file sent to one
> person is sent to other members of the group.
>
> Sometimes contact tells me to please exclude some people from Cc, so I
> just go into definition of identities for contact and exclude such.
>
> SMS I am sending by using kdeconnect.el package so any SMS I send this
> way it is getting recorded to the contact. If I need to send SMS to
> the group, then same SMS could be sent to whole group.
>
> If the note on one contact is related to other people then is trivial
> to automate to inform other people on that particular note.
>
> If any group of people is there, filing files under group does not
> make much sense to me. I would not file it as Group/ABC/file-01.pdf
>
> I would file it under By-ID/Contact-ID-123/Group/file-01.pdf as files
> are normally sent by one email address of one person or under one
> name. I deal with people not with empty groups that do not
> communicate. But group is important, so there can be /Group/ directory
> on the file system where all contacts of one group can be symlinked
> automatically if it happens that I need to search by Group on file
> system, but I don't. I search for groups in the database and see list
> of contacts there and then jump to contact.
>
> Same thing with invoices, they are written either to company or to
> individual or some individual in some company. I will not file or act
> based on invoice. I have to act based on authorirative or maybe
> hierarchically higher object first.
>
> In general is always good to make a list of things and then lists are
> foundation for dealing easier with whatever groups of anything.
>
> > > it comfortable. My way of thinking is always People or Groups, and
> > > from there various searches are performed and that narrows drastically
> > > the subject that has to be searched.
> >
> > That does sound like it should speed up searching by directory.
>
> You remember that I do not search by directory.
>
> Computer stuff I search by directory, like
> Work/Computer/Devices/Dictaphone or Work/Computer/Graphics/shadow
>
> Stuff related to any entity, organization, group of people, mailing
> list or list that encompasses other lists, I am locating in the
> database but the file system location is automatically expanded from
> the unique IDs. So I need not know where is exactly directory for the
> group ABC, but I know if I press F8 that I am right there. Then files
> are sorted for the group or account normally by date, but could be by
> License number or something else. It is mixture of database relational
> jump to base directory for specific account/company/group and
> sub-directories. Base directory I do not need to know, those
> sub-directories are few and search becomes very fast. No special
> database is needed for this as one can keep hundreds of groups and
> people in simple Lisp structures saved in files.
>
> > My mind works differently here: I remember some concept and need to
> > find stuff connected to that, including people, but also additional
> > ideas, instructions, and just bullet points with info I might need
> > again later (which multiple times saved many hours of searching
> > already).
>
> What I learn is that concept of management of information in computer
> better works by the concept of associations or relations as that is
> similar how mind works. I do not see much difference here in thinking
> how things information should be stored.
>
> I have some hobby and there are paper books, images, videos, DVDs,
> digital books related to that hobby. All that is sorted by Authors as
> that is one attribute that those things have. Additionally there are
> subjects. Authors's individuals works are symlinked to various
> multiple subjects as multiple subjects (tags) related to same Author's
> work. If author produces only for one subject then his name is
> symlinked to that subject. Sometimes there are 2 or more authors
> working together, so their directory is also symlinked to individual
> authors.
>
> Similar concept applies to music or videos, it is sorted by authors
> and bands, and then individual works are sorted by its type of music
> or genres of videos.
>
> Images are sorted by people who are on the image. But I do not think
> of that, I just say "Adrian" and image is sorted there in appropriate
> date folder for that person. Computer thinks WHERE is the location,
> not me. All other files are also being sorted like that and it is work
> in progress.
>
> > The one thing that keeps me from that is that I often file under
> > specific projects, and the active projects are shifting
> > constantly. For that I enjoy it a lot that I only need to customize
> > the capture templates to add a project.
>
> In the dynamical knowledge repository backed by PostgreSQL I have
> subtrees and each subtree has its name. I am more familiar with
> structured data in the database. So if I wish to have equivalent of
> capture it becomes trivial, press key, choose set and insert whatever
> note or task or article, anything. It just has little different
> structures.
>
> I do not need to setup extra capture configuration as there are
> already sets or subtree names. I am quickly locating appropriate
> substree name and filing under.
>
> Something I wish to file need not be Org mode, but it can be. A note
> can be written in any text mode and filed in the set or
> subtree. Markdown, Koutliner, it can be Perl snippet, one liner,
> excerpt from email or message mode note, why not. SQL snippet, Emacs
> Lisp or Scheme. Normal text. I feel more free this way.
>
> Hyperdocuments have its types like WWW, Gopher, Gemini, FTP, SFTP,
> Note, Task, Video to be played at exact time, local file, YouTube
> video, Dired directory, Program Launch, PDF, PDF by query, PDF by page
> number, Emacs Lisp, Org Heading, Org file, Message ID, Set, and so
> on.
>
> These days I am thinking to make self-defining types and I mean to
> place the code inside of the hyperdocument type entry that defines how
> that entry should be handled. User accessing database at University
> Makerere could access hyperdocument that designates that not only
> video file has to be opened in the browser, but that window has to be
> split so that annotations related to video file has to be shown in
> same time. But user accessing such hyperdocument need not necessarily
> know that its viewing or launching is customized by the type of the
> hyperdocument defined somewhere else, not in software directly. It
> sounds similar to Javascript launched in web browsers with full
> liberty in computing. Emacs Lisp snippets would be pulled from
> database directly and executed on users' computer. File transfers are
> non existent and file system is not there. Nothing gets really
> downloaded or saved, just loaded into memory and executed.
>
> Just thinking of a hyperdocument type or object type "comment" or
> "feedback". When activating such user loads few functions, write the
> comment, functions can also say what will be done with the
> comment. School teacher or head teacher acts upon it or comments are
> simply published.
>
> Sets for now do not have types, but why not. If set has type of topics
> and topics allow comments there is automatic forum and discussion.
>
> > I for one am still working on my workflow, and I guess that 10 years
> > from now it won’t be the same as today. I hope that learning about other
> > ways to work with org will help me a lot in future.
>
> Same on my side.
>
> Summary for me:
>
> - navigation by hierarchy is one way, we all use it to search and file
>   documents
>
> - searching through indexed database is different way, it is not good
>   for filing anything
>
> - direct relational access where computer locates the file is third
>   way for knowledge management
>
> - it is better designing filing, sorting and retrievals by concepts of
>   mind or how mind works
>
> - any objects or pieces of information shall have its actions that act
>   upon it and that can hyperpush the user to other pieces of
>   information
>
> - general increase of hyperlinking by relations helps in creation of
>   better systems.
>
> Here is how I have implemented simple versioning system that is
> decided by program and not the database. It was just before hour.
>
> (defun hyperscope-vc (table column id &optional description)
>   "Simple version system."
>   (let* ((value (rcd-db-get-entry table column id *hs*)) ;; takes DB value
>          (value (sql-escape-string value)) ;; prepares value for SQL
>          (description (if description (sql-escape-string description)
> "NULL"))
>          (sql (format "INSERT INTO vc (vc_table,
>                                       vc_column,
>                                       vc_tableid,
>                                       vc_value,
>                                       vc_description) VALUES ('%s', '%s',
> %s, %s, %s)
>                          RETURNING vc_id"
>                       table column id value description))
>          (id (rcd-sql sql *hs*)))
>     (if id id nil)))
>
> Above function takes the value from any table, any column, by ID and
> inserts the value into the `vc` table.
>
> Then comes this function below that edits hyperdocument as Org file
> with Org mode. Before updating it is using `hyperscope-vc` version
> function to basically save the old value in the separate table. If I
> want to see diffs I think it is also trivial to do.
>
> (defun hlink-edit-org (id)
>   (let* ((blob (hlink-description-value id))
>          (blob (if blob blob  ""))
>          (buffer-name (format "HyperScope Editing ID: %d" id))
>          (new-blob (read-from-buffer blob buffer-name 'org-mode)))
>     (hyperscope-vc "hlinks" "hlinks_description" id)
>     (rcd-db-update-entry "hlinks" "hlinks_description" "text" id new-blob
> *hs*)))
>
>
>

[-- Attachment #2: Type: text/html, Size: 18217 bytes --]