* org-id-locations as a large-scale database store?
@ 2024-03-12 5:03 Laurence von Bottorff
2024-03-12 16:49 ` yeti
0 siblings, 1 reply; 8+ messages in thread
From: Laurence von Bottorff @ 2024-03-12 5:03 UTC (permalink / raw)
To: emacs-orgmode Mailinglist
[-- Attachment #1: Type: text/plain, Size: 2296 bytes --]
I've been using a package called org-brain that is using, I'm all but
certain, the org-id-locations file as its database to store graph-like
relationships between org files and org headings. org-brain is a sort of
graph database which adds a PROPERTIES drawer with just the ID field with a
UUID to an org file, as well as to the org headings in each file. So I
opened this 127k file residing in my .emacs.d directory and apparently
whatever is in my org directory and has a PROPERTIES drawer with ID field
winds up in .org-id-locations represented as plists in one big list, one
for each file that has one or more drawers. But you knowledgeable people
here know all this already. However, it surprised me since such a format as
this
(...("~/Dropbox/org/orgb_fassungen/FassungenDesc.org"
"1ba78dfc-d6c3-46c5-827d-37db94fa9053"
"6809202d-9a4c-4ed0-87fb-798676f448fe"
"f2149157-6ba1-4e24-81c3-4e070fcc5de8"
"5b7c2d9a-2df8-450a-a0c8-9d3b8e703044"
"f3031b9f-3708-493d-9546-f489835c9f17")
("~/Dropbox/org/orgb_fassungen/Fassungen.org"
"e9b683af-90b1-4ece-9afd-18dc28a9fab2")...)
where, e.g., inside the file FassungenDesc.org there are many org headings,
each with a PROPERTIES/ID drawer -- and they're all associated
to FassungenDesc.org with that simple list. So I'm asking if this is a good
design for a really big graph database? I'd like to start using org files
and their containing headings as vertices. But I'm guessing this is not
really all that scalable, not really intended for such a big thing. I saw
another emacs graph database (https://github.com/toshism/netz) that uses
h.el hash to store graph vertices and edges. So again, is this org-brain
approach DOA for anything big? I plan on having potentially hundreds of org
files with very many heading entries each (over a thousand perhaps per
file) representing vertices. What would be a better way of databasing org
files and headings? The PROPERTIES/ID drawer idea is fine, but not throwing
it all in .org-id-locations, right?
Ancillary: If I move or delete .org-id-locations, Emacs will rebuild it,
right? What is the idea behind having such a file structured this way --
other than a simple database? What else uses it?
--
⨽
Lawrence Bottorff
Grand Marais, MN, USA
borgauf@gmail.com
[-- Attachment #2: Type: text/html, Size: 2771 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: org-id-locations as a large-scale database store?
2024-03-12 5:03 org-id-locations as a large-scale database store? Laurence von Bottorff
@ 2024-03-12 16:49 ` yeti
2024-03-12 17:00 ` Rick Lupton
2024-03-12 17:20 ` Ihor Radchenko
0 siblings, 2 replies; 8+ messages in thread
From: yeti @ 2024-03-12 16:49 UTC (permalink / raw)
To: emacs-orgmode
Laurence von Bottorff <borgauf@gmail.com> writes:
> So again, is this org-brain approach DOA for anything big?
I like it, but even my small pool of notes is slow with org-brain.
-------------------------------8<---------------------------------------
~ $ hash-table-size $org-id-locations
1322
------------------------------->8---------------------------------------
--
I do not bite, I just want to play.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: org-id-locations as a large-scale database store?
2024-03-12 16:49 ` yeti
@ 2024-03-12 17:00 ` Rick Lupton
2024-03-12 17:20 ` Ihor Radchenko
1 sibling, 0 replies; 8+ messages in thread
From: Rick Lupton @ 2024-03-12 17:00 UTC (permalink / raw)
To: Y. E.
You might be interested in https://www.orgroam.com/ which is a broadly similar idea to org-brain but uses a SQLite database to track id locations and links.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: org-id-locations as a large-scale database store?
2024-03-12 16:49 ` yeti
2024-03-12 17:00 ` Rick Lupton
@ 2024-03-12 17:20 ` Ihor Radchenko
2024-03-12 18:37 ` yeti
1 sibling, 1 reply; 8+ messages in thread
From: Ihor Radchenko @ 2024-03-12 17:20 UTC (permalink / raw)
To: yeti; +Cc: emacs-orgmode
yeti <yeti@tilde.institute> writes:
> Laurence von Bottorff <borgauf@gmail.com> writes:
>
>> So again, is this org-brain approach DOA for anything big?
>
> I like it, but even my small pool of notes is slow with org-brain.
>
> -------------------------------8<---------------------------------------
> ~ $ hash-table-size $org-id-locations
> 1322
> ------------------------------->8---------------------------------------
Does org-brain have anything to do with org-id-locations?
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: org-id-locations as a large-scale database store?
2024-03-12 17:20 ` Ihor Radchenko
@ 2024-03-12 18:37 ` yeti
2024-03-12 18:47 ` Ihor Radchenko
0 siblings, 1 reply; 8+ messages in thread
From: yeti @ 2024-03-12 18:37 UTC (permalink / raw)
To: emacs-orgmode
Ihor Radchenko <yantar92@posteo.net> writes:
> yeti <yeti@tilde.institute> writes:
>
>> Laurence von Bottorff <borgauf@gmail.com> writes:
>>
>>> So again, is this org-brain approach DOA for anything big?
>>
>> I like it, but even my small pool of notes is slow with org-brain.
>>
>> -------------------------------8<---------------------------------------
>> ~ $ hash-table-size $org-id-locations
>> 1322
>> ------------------------------->8---------------------------------------
>
> Does org-brain have anything to do with org-id-locations?
I think so...
E.g. org-brain.el 788-794:
-------------------------------8<---------------------------------------
(defun org-brain-entry-from-id (id)
"Get entry from ID."
(unless org-id-locations (org-id-locations-load))
(when-let ((path (gethash id org-id-locations)))
(list (org-brain-path-entry-name path)
(org-brain-headline-at (org-id-find id t))
id)))
------------------------------->8---------------------------------------
--
I do not bite, I just want to play.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: org-id-locations as a large-scale database store?
2024-03-12 18:37 ` yeti
@ 2024-03-12 18:47 ` Ihor Radchenko
2024-03-12 20:57 ` Laurence von Bottorff
0 siblings, 1 reply; 8+ messages in thread
From: Ihor Radchenko @ 2024-03-12 18:47 UTC (permalink / raw)
To: yeti; +Cc: emacs-orgmode
yeti <yeti@tilde.institute> writes:
>> Does org-brain have anything to do with org-id-locations?
>
> I think so...
>
> E.g. org-brain.el 788-794:
>
> -------------------------------8<---------------------------------------
> (defun org-brain-entry-from-id (id)
> "Get entry from ID."
> (unless org-id-locations (org-id-locations-load))
> (when-let ((path (gethash id org-id-locations)))
> (list (org-brain-path-entry-name path)
> (org-brain-headline-at (org-id-find id t))
> id)))
That's redundant. Calling `org-id-find' would suffice (org-id-find
internally arranges org-id-locations to be updated).
In any case, loading and searching id locations scales well for
thousands of entries - I currently have 60+k entries there without
issues.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: org-id-locations as a large-scale database store?
2024-03-12 18:47 ` Ihor Radchenko
@ 2024-03-12 20:57 ` Laurence von Bottorff
2024-03-12 21:16 ` Ihor Radchenko
0 siblings, 1 reply; 8+ messages in thread
From: Laurence von Bottorff @ 2024-03-12 20:57 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: yeti, emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 1307 bytes --]
Any docs on what org-id-locations is about, who/what uses it?
On Tue, Mar 12, 2024 at 1:44 PM Ihor Radchenko <yantar92@posteo.net> wrote:
> yeti <yeti@tilde.institute> writes:
>
> >> Does org-brain have anything to do with org-id-locations?
> >
> > I think so...
> >
> > E.g. org-brain.el 788-794:
> >
> > -------------------------------8<---------------------------------------
> > (defun org-brain-entry-from-id (id)
> > "Get entry from ID."
> > (unless org-id-locations (org-id-locations-load))
> > (when-let ((path (gethash id org-id-locations)))
> > (list (org-brain-path-entry-name path)
> > (org-brain-headline-at (org-id-find id t))
> > id)))
>
> That's redundant. Calling `org-id-find' would suffice (org-id-find
> internally arranges org-id-locations to be updated).
>
> In any case, loading and searching id locations scales well for
> thousands of entries - I currently have 60+k entries there without
> issues.
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <https://liberapay.com/yantar92>
>
>
--
⨽
Lawrence Bottorff
Grand Marais, MN, USA
borgauf@gmail.com
[-- Attachment #2: Type: text/html, Size: 2207 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: org-id-locations as a large-scale database store?
2024-03-12 20:57 ` Laurence von Bottorff
@ 2024-03-12 21:16 ` Ihor Radchenko
0 siblings, 0 replies; 8+ messages in thread
From: Ihor Radchenko @ 2024-03-12 21:16 UTC (permalink / raw)
To: Laurence von Bottorff; +Cc: yeti, emacs-orgmode
Laurence von Bottorff <borgauf@gmail.com> writes:
> Any docs on what org-id-locations is about, who/what uses it?
It is an internal variable.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2024-03-12 21:13 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-03-12 5:03 org-id-locations as a large-scale database store? Laurence von Bottorff
2024-03-12 16:49 ` yeti
2024-03-12 17:00 ` Rick Lupton
2024-03-12 17:20 ` Ihor Radchenko
2024-03-12 18:37 ` yeti
2024-03-12 18:47 ` Ihor Radchenko
2024-03-12 20:57 ` Laurence von Bottorff
2024-03-12 21:16 ` Ihor Radchenko
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).