Re: Is Org really so simple?

[Jean Louis <bugs@gnu.support>]

* Ihor Radchenko <yantar92@gmail.com> [2020-11-29 10:16]:
> 
> Jean Louis <bugs@gnu.support> writes:
> > Rather the query that is made by the user should be remembered as such
> > and be made available as a command, macro or similar that can be saved
> > for later.
> 
> Users can explicitly define custom searches using
> org-agenda-custom-commands.

That is definitely useful feature. Just not as accessible and
understandable. It needs chunk of effort to learn. I would rather
prefer something like:

"I wish to find [Headings] in [file] from [date] to [date] etc"
                [TODO Headings]
		[TAGS]
		[Headings with tags]
                etc.
		^^^^^^^^^^^^^
		drop down

Not for me, but for future of Org mode I would find dropdowns more
useful for users who would simply say what they want and choose it
from drop downs "I wish to get list of ....... with attributes
......."

And multiple such examples could be offered and user could customize
them. As C in (org-agenda) function shows this:

Hide Org Agenda Custom Commands:
INS DEL Choice: Value Menu Command series, all agenda files:
            Access Key(s): n
            Description  : Agenda and all TODOs
            Component:
            INS DEL Choice: Value Menu Agenda:
                        Local settings for this command.  Remember to quote values:
                        INS
            INS DEL Choice: Value Menu TODO list (all keywords):
                        Local settings for this command.  Remember to quote values:
                        INS
            INS
            Settings for entire command set:
            INS
            [ ] Export:
                INS

Which is simply not user friendly. I would prefer it as:

"I wish to find [Headings] in [file] from [date] to [date] etc"
                [TODO Headings]
		[TAGS]
		[Headings with tags]
                etc.
		^^^^^^^^^^^^^
		drop down

where [date] is not important but generation and construction of a
human readable expression that user may easier relate to it.

INS - what is it? Sure I know, but user only wants to see
agenda. Emacs is advanced editor but we invite non-advanced users to
become advanced. Those advanced are already using it.

This design being too hackish be major block for an organization to
adopt Emacs without previous knowledge about it.

            INS DEL Choice: Value Menu Agenda:
                        Local settings for this command.  Remember to quote values:
                        INS
            INS DEL Choice: Value Menu TODO list (all keywords):
                        Local settings for this command.  Remember to quote values:
                        INS

Agenda in the real world is never that complicated and never
complicated in same difficult manner.

Paper based organization or even hand writing is easier than using the
Org mode. It is for advanced users and it would be great to see
simplifications and integrations bringing it closer to beginners.

In comparison to Emacs based agenda views, I find it easier to use
SQL:

SELECT notes FROM table WHERE notes_tags LIKE 'TODO';

Something like that is more readable and human friendly.

I find it funny that in customizations somebody actually put effort to
tell users "Remember to quote values:".

> Though format of that variable is indeed quite complex (similar to
> org-capture-templates).

I think that wizard or code generation function is very easy to make
in Emacs, think like this:

- press C or other key to generate your own query
- User is asked "Press a key you wish to use for this query?"
- User press "n"
- Then user is offered values in a visible way to choose as Choice
   such as those in customization
- "Do you want to add more choices to search from? yes/no"
- "Do you want to export?"

etc.

When designing computer programs one should think to make things
faster, easier, more useful than what people would otherwise do let us
say on paper. If paper tasks have been spread in various groups, those
would be already containing headlines that may be used for agenda.

> > Example is when I search for TODO entries with special TODO kwd, then
> > I may search for 3-4 keywords or 7-10 keywords during one
> > day. Then there would be repetition to search again for those
> > keywords. There is history value access in the minibuffer but that
> > does not remember searches for the next Emacs session. Having
> > rememberable customized searches for users would be useful.
> 
> Take a look at savehist-mode. Enabling it should save all the minibuffer
> history by default. However, the history of your agenda searches will be
> mixed with global minibuffer history. If you do not like this behaviour,
> feel free to open feature request to make org-agenda keep history
> separately. Then, savehist can be configured to save that across Emacs
> sessions.

savehist-mode is all time enabled on my side. I am not sure if I can
see here in the function `completing-read-multiple' that it does uses
history.

        (setq org-select-this-todo-keyword
              (mapconcat #'identity
                         (let ((crm-separator "|"))
                           (completing-read-multiple
                            "Keyword (or KWD1|KWD2|...): "
                            (mapcar #'list kwds) nil nil))
                         "|")))

The short function's definition is here:

(completing-read-multiple PROMPT TABLE &optional PREDICATE
REQUIRE-MATCH INITIAL-INPUT HIST DEF INHERIT-INPUT-METHOD)

and 6th element should be history variable. I do not see the 6th
element there and that is why it is not remembered. It uses history
from anything, but not history specifically to Org files.

> > - if text file is internally related to Joe Doe, then by clicking on
> >   the text file such as Org file, I could automatically get various
> >   hyperlinks to anything related to Joe Doe: Joe's emails list, Joe's
> >   SMS list, Joe's contact information, I could click and initiate a
> >   call with my mobile phone and just write notes without thinking on a
> >   phone number, I could click in Org file and send SMS to Joe that
> >   will be saved in computer without thinking on Joe's phone number, I
> >   could see relatives of Joe and find his sister and again have all
> >   the hyperlinks and relations to various other pieces of information
> >   related to Joe.
> >
> > - and if I am in an Org file that has relation to other objects I
> >   would not need to construct hyperlinks by hand, they would be or
> >   could be constructed semi-automatically because the relation is
> >   already known.
> 
> What you are describing appears to be easy to achieve in org files if
> one wishes to:
> 
> some_file.org:
> 
> #+begin_src org
> Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Donec
> hendrerit tempor tellus. Donec pretium posuere tellus. Proin quam nisl,
> tincidunt et, mattis eget, convallis nec, purus. Cum sociis natoque
> # clicking on the link with bring you to Joe Doe's heading in Joe Doe.org
> [[id:joe-id][Joe Doe]]
> penatibus et magnis dis parturient montes, nascetur ridiculus mus. Nulla
> posuere. Donec vitae dolor. Nullam tristique diam non turpis. Cras
> placerat accumsan nulla. Nullam rutrum. Nam vestibulum accumsan nisl.
> #+end_src
> Contact information:
> 100 Street, City, Country
> # Clicking the following link would start the phone/skype call
> Tel: [[call:11111111111][call to 11111111111]]
> [[sms:111111111][send sms to 111111111]]
> 
> ,** SMS messages
> ,*** [[id:message1-id][Message 1]]
> ,*** [[id:message2-id][Message 2]]
> 
> ,** Related
> ,*** [[id:antony-id][Antony Joe]]
> ,*** [[id:something][Something else]]
> ,*** [[id:yetmore][Yet another one]]
> #+end_src

Thank you for pointing that out. You are very right that it is
possible. Only it is tedious without users' functions to accomplish
the task easier. Phone numbers are typed, not that heading is related
to the phone number. Typing is error prone. Hyperlinks need so much
more developed for users to automatically create them. Something like
open menu, insert phone number as hyperlink. No thinking about which
phone number, which person. If person's ID is already there, computer
should find it.

I am aware that by hand work one can construct such relations as you
have demonstrated, just that it is hard work in the end.

> > When I work with the database and wish to edit something equivalent to
> > a headline than the unique ID is generated automatically and I do not
> > need to think of it. It is made by database design.
> 
> Note that not all users would want this by default. But you can simply
> set org-id-link-to-org-use-id to 't and the unique ids will be created
> every time you create a link to heading.

That is true and not all people use it. I was used to it from database
work. I will soon abolish them all and move tasks to meta level
including notes, then when I need Org to be distributed I can export
in a second to Org file.

> > Instead of customizing Emacs to insert IDs, customizing Org,
> > customizing specific files,
> 
> It is an advantage for me. Not all the people need the specific workflow
> you like, but it can thankfully be changed. Same as any other part of
> Emacs.

That is like bicycle and motorcycle, as bicycle is advantage for
walkers but motorcycle for bicycle drivers. We can observe people
having problems with execution of functions, speed, queries,
customizations, etc. The design of Org mode in its root as plain text
is very fine and good. But when it jumped into management of
structured data that is where Org mode became something different. Now
the strategy of Org mode is to hard code and to provide functions that
databases provided for decades.

When user edits its file (Org), after a while there is nice
enlightenment, satisfaction and pleasure. This is because it helps to
keep things organized. But try to share it with your fellows, friends,
family who do not know about Org mode. And then is good to draw
conclusions from difficulties they are faced to as those will be
speaking of usability.

> > ... going over specific files to find which
> > has unique ID which one does not have, having duplicates all over the
> > place,
> 
> Why do you need to find it yourself? It is all handled by org.
> Duplicates are rare and mostly happen in archives/trash according to my
> experience.

I have 50 duplicates now. And it comes from copy and paste of
subtrees. As user I am warned to look into *Messages* buffer to help
me. Then I find there:

Duplicate ID "2759bfcf-73fb-4f8a-9a44-04553797185e"
Duplicate ID "b981c204-ecf7-41a3-a6f4-f40c22f6823f"
Duplicate ID "8e507f2b-8c43-46d0-903f-e667c914e358" [2 times]
Duplicate ID "c1565202-9f74-4c70-91b6-3e7bf55af802" [2 times]

which tells me nothing. Do not mind these thoughts for me personally,
I am fine. These thoughts describe where Org mode need enhancement.

When I was making my own functions it was trivial to make for example
separate buffer where messages from async-shell-command are displayed
so that I can easier access them. They are not scattered all over
not-related messages in the put-it-all buffer named *Messages*. Think
about usability for others, not for me. I am transitioning from Org
and doing it on meta level. So I am liberated from various designs
that do not help me.

That there is possibility to have duplicate ID is already enough for
me. I like CREATE TABLE id SERIAL NOT NULL PRIMARY KEY and finished. I
never even think of ID numbers, they are made for me, stable, and
always unique.

Back to {M-x org-id-update-id-locations} as that function should or
could open up the separate buffer and use the Org mode in that file to
construct hyperlinks to those duplicate IDs. Why is Org mode function
like {M-x org-id-update-id-locations} using *Messages* buffer to
display errors is very hard for me to understand. It breaks all my
logic down. Why I do not get any relation from the duplicate ID to the
file where it should be found?

Think about usability. How about making a policy to use Org read-only
buffers for Org agenda, as good example? Expand the Org agenda buffer
with hyperlinks:

- if user press a it is for week or day
                                    ^^^
- but if user clicks on "day" it will show only day

- use hyperlinks to explain more to user or guide one
				    
We already talked that the buffer for org-agenda is really way to
rigid and asks for closing the buffer, opening it, closing, until user
finds its way.

> > ... having pretty ugly and not meaningfull unique ID always shown
> > on screen and more or less to a degree disturbing,
> 
> org-custom-properties ;)

Sure and I use it, like :ASSIGNED: and I also use database IDs. It is
a lot of work for many Org nodes to be handled like that. Each time
assign, insert custom ID, invoke this, that. No, I do not do that. I
count (about) the number of keystrokes and repetitive actions by
observing what I am doing and why and then design faster workflow.

Often I have to export an Org node to ASCII or Unicode, so I have it
as Emasc bookmark.

{C-x r b stag TAB RET C-c C-e C-s t A C-x h M-w C-x k RET C-x b mutt-pro TAB RET C-e RET RET C-y}

That above series of commands may be automatically invoked by using
GNU Hyperbole and action key {M-RET}

GNU Hyperbole
https://www.gnu.org/s/hyperbole

What I do, I jump into Org file, to specific heading, export to plain
text, copy and move to email where I insert it for the person to
receive it.

The workflow I prefer is just:

{s-t stag RET}

I find Emacs {C-x r b} to pull a bookmark also tedious. Function such
as `org-insert-heading-from-pool-as-ascii' where heading is quickly
located by using completion would be useful.

> > ... instead of having
> > them easily editable and changeable and error prone,
> 
> Agree. This is the price of keeping everything in plain text.

I keep everything in plain text too. I just like relating whatever
chunks of plain text to other chunks of plain text. Org does need more
relational approach by any means. It has IDs, custom IDs, properties,
tags, those are all relational attributes. It provides hyperlinks to
related objects. Org is about relations. What is currently lacking is
what partially has been developed and that is integration with other
objects within Emacs or maybe outside, like automated ID generation,
heading wizards, automated hyperlink generation, back links and so on.

Heading wizard could ask user:

- is it related to person?
- organization?
- another task?
- is it assigned to some user?

then after few questions the new Org heading could be created pulling
all those relations into the heading.

** TODO Wash all cloths collected on the first floor
   [Organization name] as hyperlink
   [Manager name]: [Manager phone]
   [Responsible for floor]: [Name]
   Related to task: [3232]

> > ... instead of many
> > actions related to that unique ID,
> 
> Sorry, I do not understand which actions you refer to.

Let us say I am in heading related to organization ABC, there are many
actions connected like calling, contacting people there, reviewing
list of their people, reviewing their emails.

Maybe I am looking on the heading

** Organization ABC

   [IN THIS SPACE]

in that space could be profile of organization. I click [tasks] and
Org file gets nicely ordered. Or maybe [Agenda] and I get only what
relates to this Organization be it in this file or other file, I click
[People] and get people belonging in this organization, [Call] [SMS]
[Send email] [Web] [Schedule meeting], all hyperlinks there without me
thinking.

> I refer to such features as integration. 
> > - last USER who ever modified the heading, note, task, URL, anything
> >   related to that ID, I can find that last user. Hyperlink could be
> >   automatically generated button that points to on the fly generated
> >   Org buffer showing me such modification.
> 
> I do not understand how you determine the user editing something
> in the database. Or do I misunderstand?

Database like PostgreSQL is accessed by using username,
password. Database know that query is conducted by use "Joe Doe" and
by decision of database designer that may remain as record in some of
database tables.

In my opinion any hypertext system should be multi-user based by its
design. Org mode is customarily single user based. With version
control systems it could be multi user but it did not go to that
direction.

It could be concurrently or simultaneously edited as well. 

> > - There is DATE in the hlinks table, so it is possible to have date as
> >   hyperlink button from where I can click to find other same date
> >   related entries, or I could invoke list to find nearby dates.
> 
> FYI. Clicking on timestamps in org automatically invokes agenda view
> listing all the headings containing timestamps from the same day.

Well what I would not know without you I have hundreds of timestamps
and years of using it and I never clicked on it. And now I speak how I
wish to have more hyperlinks. No, I was not aware really those are
hyperlinks. I never clicked on them. Misunderstanding. While they do
look as hyperlinks I was not thinking they are.

> >  That is my button. I never create those hyperlinks again. This
> >  relates only to the tags, but hyperlinks can be for any other type
> >  of relation.
> 
> FYI. When you have multiple tags in your heading, clicking on the tags
> will automatically show agenda view listing all the headings with
> matching tags.

Thank you. Definitely extremely useful feature!

> > - the heading of meta Org can have a status defined by user anyhow and
> >   I do not mean TODO/DONE but could be anything that changes later
> >   actions of the user, or automatically generates hyperlinks.
> 
> The keywords in org are not limited to TODO/DONE. One can create
> arbitrary set of keywords (representing the status) and even define them
> locally in buffer (see "5.2.5 Setting up keywords for individual files"
> in the manual).

Yes, I am aware of that and use it since years. One group is meant as
done, other as not done, and users can specify it. I am aware of
similarities and that is why I said "I do not mean TODO/DONE" but I
did not mean only status of action. Even keywords like that in Org
mode do not need necessarily to mean status of action for the user,
even though they do mean as when "DONE" is chosen then there is date
when it is "CLOSED".

When types are defined in the database it becomes trivial extending
features, instead of type relating to task such as task being closed
or not closed, pending, etc.

** DONE My task
   CLOSED: [2020-11-29 Sun 14:14]

Example of a different type could relate to article such as REVISION

** REVISED My article
   VERSION: [2020-11-29 Sun 14:14]:[1.3]:[1.2]

Where 1.2 could be hyperlink to previous revision, while 1.3 would be
present revision. But keywords like TODO/DONE and others are designed
for action type only. Version control on file system is related to
whole file, but Org is like a book and it can contain various
structured information, project planning that may be revised and where
users wish to see previous versions.

Because of the Org structure that is hard coded one cannot just
invent, one has to make complex enhancements in Emacs Lisp to again
hard code the type.

> > - possible EXPIRATION entry is there, it could be automatically
> >   hyperlinked to other expired items
> 
> Generally, any database field can be represented in org using property
> drawers (note that you can even set global property drawer at the top of
> the org file before the first heading). For the expiration specifically,
> there is even a package in org - org-expity ;)

Yes, and I use it now rather in reverse. Not that I create it by hand,
I let database create it. Then somebody could participate and edit Org
file, send it back or upload for merging or update.

On export to Org those database fields I can attach to headings, tags
I can attach to headings and similar. But not that I wish to work by
hand and edit it and put all those relations manually. So Org becomes
derivation of meta level organization. I am using the headings for
global property drawer, scarcely, like list of people to which tasks
could be assigned.

If a hyperdocument has some attribute related such as person's name,
then on export there can be property with the hyperlink to that
person, from where one can see profile of the person, SMS, notes,
tasks, people introduced by that person, etc. All those become
hyperlinks just very similar to web based applications where HTML
isautomatically generated with many hyperlinks to related objects. We
work in Emacs so we do not work with HTML and there is more
extensibility allowed then just by doing it with HTML.

There are hundreds of thousands of people in my database, thousands
are used in hyperlinks. Imagine if I need to go into their
information, then copy phone number, construct hyperlink to SMS, copy
person who introduced another person, paste into Org file, construct
link, format and align it, test and verify, go into list of notes,
construct link to notes, insert it. That deserves automation.

> Some things may not be very clear for an arbitrary reader though. For
> example, it took me quite a lot of time to wrap my head around the idea
> of using databases - I was thinking too much in terms of org-mode files.
> Would you consider making a blog post showing your workflows in more
> consistent way with examples and possibly gif demonstrations?

What I see is that you and other people are already using database
that is managed by Org. The more properties, tags, timestamps and
other relations are used the more relational it becomes. You would not
have hard time to understand extensing Org with meta data in the
database. Such extension would liberate Org file from meta data and
make meta data reliable and redundable.

Yes, I will have features posted as videos. Then it will come to GNU
ELPA.
 
> I personally approach this by keeping my files as attachments to org
> headings. Then, the files can be references simply by referencing the
> relevant heading (equivalent of your record in the database).

That is good approach. It makes references static and the reference
modifiable. Such solutions help when we come to complexities. If we
have simpler information they may not be necessary.

> Have you ever been in scenario when you want to file something to
> multiple locations?

Many times. One can use hard links or symbolic links on a file
system. Better is using it by relations and defining groups. If group
of sciences consists of biology and physics, maybe photosintesis
belongs to both of them.

If one has to file same thing in multiple Org files on a meta level
there is easy solution. Just creating a new node that points to other
node. Action on a hyperdocument that means "other hyperdocument" would
be the same as on that other hyperdocument. But referencing
hyperdocument would just have ID identifier not a content inside if
that content should be centrally editable.

I have table emails and table pages. If I wish to send page by email I
just tell into the row for the email that the email with ID 3 has the
body text that is actually the page body from page ID 234. When email
is sent, basically page body is sent.

When I define a page I can also say "this page is other page" and this
page acts as other page.

If I define a group of people like Joe, James, Jane, with the ID 4
then I could choose to file under ID 4 as that belongs to group of
those 3 people. Accessing any of those people individually would give
me listing of files filed to the group where person belong and
hyperlinks to those other persons.

Better is when filing of anything has its own ID but is related to the
group as then the filing or document has its own title, annotation and
maybe set of other hyperlinks to hyperdocuments, not only files.

Right now is for me trivial to file into multiple categories when
using database as meta level Org file preparation. For Org file itself
is maybe not trivial.

I can display the list of desired sets or search for them, mark the set and it gets remembered in the list like:

Marked ID 34739 ((34739))
Marked ID 34740 ((34740 34739))
Marked ID 34741 ((34741 34740 34739))

Then what I do with those ID numbers is up to me. I can say update TAG
for each of them, modify their timestamps, close them all at once,
schedule them together, generate group of people, delete them if
necessary, send them by email, anything imaginable. I have already
many such functions and they are simple as they deal with IDs and not
parsing of text.

This is function that marks multiple contacts and creates a group for
those contacts. Because groups are generic they could be groups of
anything like groups of pages, tasks, quick tasks, hyperdocuments,
invoices, opportunities, variables, units, currencies, statistics,
SMS, emails, payment methods, videos, etc.

(defun hyperscope-marked-contacts-to-group ()
  (interactive)
  (let ((id (tabulated-list-get-id)))
    (when id
      (let ((group (rcd-db-get-entry "hlinks" "hlinks_groups" id *hs*)))
	(when group
	  (let* ((type (rcd-db-get-entry "groups" "groups_grouptypes" group *hs*))
		 (table (rcd-db-get-entry "grouptypes" "grouptypes_table" type *hs*)))
	    (when (string= "contacts" table)
	      (let ((array (rcd-sql-int-list-to-array hyperscope-marked-items))
		    (sql (format "UPDATE groups SET groups_array = %s WHERE groups_id = %s" array group)))
		(rcd-sql sql *hs*)))))))))

That would work this way:

- make a group name
  
- filter contacts to get let us say Joe, James, Jane but it could be in multiple accesses.

- press `m' to mark contacts_id 

- invoke function to update contacts for that group

In this case it is group that is related to hyperdocument on which I
am located within tabulated-list-mode, it would be updating group of
people to which this note would be related.

Org way of doing things is to parse everything together each time a
new which essentially is same what database is doing only blazing fast
and error prone. There can be many Joes, James, etc. It is
distinguished clearly due to unique IDs and other attributes.

Now when you have a group defined in a similar speedy manner like
marking files in Dired you can file then anything to that group be it
file, WWW, note, TODO, action, just anything. Additionally
hyperdocument can be related to various other objects such as contact,
person that this contact is assigned to, even group that it is
assigned to. Hyperdocument such as TASK can be related to group of 3
people but assigned to group of other 4 people who need to handle the
deal. In the same time TASK can belong to company ABC, under business
XYZ, as company ABC has multiple businesses. And it could represent a
note to business opportunity X, that belongs in planning and project
Y.

When editing such relations in Org file I would be repetitively spend
time writing, typing, aligning text, formatting, invoking key bindings
and so on. But such editing is not plain text editing any more and we
manage Org and speak of plain text when it is not any more, it
requires editing of structured data. The more data there is the more
complicated it becomes.

Would Emacs have GDBM database built-in I am very sure that Org would
be using the database from its inception and many other packages as
Emacs is already now using various files as replacement for databases
such as bookmarks, saved history, desktop and similar. It needs
external memory that persist even when it is not running. Interesting
it that such can be accessed by any programming language than.

> > c a - I am editing author's name in free text. If author is in the
> >       database I would choose author. Author's name can be hyperlinked to
> >       find other entries under same author. No need to type it by hand.
> 
> Do you need to set the link to author every single time you create a new
> entry authored/related by/to that author? Or do you somehow create the
> link to the author automatically?

I have 2 columns, one is hlink_author related to table `contacts'
having priority and other is hlink_authorname which may be used when
author is not in the contacts table.

If there is hyperdocument "Book ABC" and I need to enter Book XYZ
under same node, I would simple `C' to duplicate the node which
duplicates all meta data and I just curate the name of the book and
actual hyperlink (URI) to the book. This duplicates author.

Otherway is to filter various items belonging to author, mark those
items and then update author for all of them at once.

Once value is updated, the description of the hyperdocument is shown
similar like this below and I can click on Doug Engelbart to get all
entries related to author. But I do not create the hyperlink to author
any more. Anything that is already assigned as relation should have
hyperlinks automatically to help in faster access to information.

Doug Engelbart's Design
=======================
   Author: Doug Engelbart
     Type: Set ➜
Hyperlink: Doug Engelbart's Design

> > T - I am editing tags freely, I can enter: emacs lisp todo review; and
> >     those words would become tags. After ENTER I can immediately see
> >     hyperlinks to those tags.
> 
> Are your tags also uniquely defined using relational database? Say, you
> want to change some tag name for some reason. Will changing the name
> automatically update all the tags in the entries marked with that tag?

Good question. Answer is yes and not yet as I have `tags' table since
years and anything can be inside.

                                    Table "public.tags"
      Column      |  Type   | Collation | Nullable |                Default                
------------------+---------+-----------+----------+---------------------------------------
 tags_id          | integer |           | not null | nextval('tags_tags_id_seq'::regclass)
 tags_name        | text    |           | not null | 
 tags_description | text    |           |          | 

 But hyperdocuments are now tagged freely as a string with spaces as separator:

 emacs-lisp gnu lisp

 would be 3 tags. Upper case does not matter.

 I could as well enter the loop where I choose multiple tags from the
 table `tags' and relate it to the table. What I wish to say is that
 when one works with database tables, unique IDs, things become very
 simple and many checks, verifications, and data or variable handlings
 become redundant.

> > If the task is assigned to Communication Officer or senior who has to
> > advise Joe how to do it, then on the next key press I could get
> > various hyperlinks because there is CID-1 that relates to Joe Doe:
> > 
> > * Joe to purchase ticket in Kahama and travel on Saturday to Dar es Salaam  :CID-1:
> > 
> >   [Joe's files] [Initiate call] [SMS to Joe] [Send this task to Joe] [Plain email to Joe]
> 
> This is an interesting idea. You mentioned it in the past, but this
> example made it much more clear.
> 
> Something similar is implemented in org-ref
> (https://github.com/jkitchin/org-ref). Instead of adding links like
> [Joe's files] directly into the text, org-ref shows various
> possibilities when one clicks on the link itself - there is a completion
> dialogue suggesting actions related to the link.

That is great. I have seen pictures and that sounds how it should be
to help user obtain references and do various actions with them. It
also again shows the need for more structured and more relational
approach. Only due to experience with SQL and easy of organizing I
have the viewpoint that Org developers work much and people put much
efforts in construction of something that is generally and have been
constructed before with database approaches. Without experience with
PostgreSQL I would not know it.

The more and more enhancements are there for Org, but I do not get
excited really. To people looking on it, it looks great. I am not
excited about that.

org-reg demonstrates good integration to bibtex entries.. It provides
necessary relations of list of objects (table in database) to text and
other objects. It creates relations and for that reason provides
better integration and use.

The BibTeX entries belong to TeX and LaTeX. There are also Emacs
packages for editing such.
https://en.wikipedia.org/wiki/BibTeX#Entry_types

My table hlinks (hyperlinks) contains one column being array:

hlinks_array text[]

which means that such node can be defined as BibTeX type and that such
type has entry types stored in the array just as described:

Article: author, title, journal, year, volume
Book: author, editor, publisher, year, title
etc.

It could in itself as whole table be defined to be BibTeX compatible
or it could just use the table column hlinks_array which would hold the BibTeX entry:

hlinks_type could be "BibTeX Article" with the title being TITLE,
AUTHOR being referenced to table `contacts' or `authors', JOURNAL
field could also reference bunch of journals, YEAR can be entered
freely and volume, etc.

BibTeX is prime example where database editing gets its use. No need
to repetition as if author has been entered once then such author may
be chosen in other 23 entries by using just few letters to complete
author's name and find the unique ID. In fact hlinks table is somewhat
similar but does not complete BibTeX specification. It could complete
if I work 30 minutes on it.

> Since org-ref is specifically designed for referencing scientific
> articles, it implements only a single set of suggestions like: (1)
> open pdf of the article; (2) show notes; (3) email article to
> someone; (4) open url page of the article, ...

That is good observation. If org-ref would be designed on meta level
it would not hold only BibTeX, it could hold anything and become a new
Memex similar to HyperScope becoming Memex in its definition. For now
it only helps in providing references by providing simple relation to
BibTeX entries. But it does not fixes those relations.

> > Because specific paragraph referencing is now not easily possible I am
> > using database approach of editing chunks of the Org file where then
> > any piece of Org can become referential.
> 
> It is actually possible to reference arbitrary chunks of text in org:
> 
> "4.2 Internal Links" section of the manual:
> 
>      1. one item
>      2. <<target>>another item
>      Here we refer to item [[target]].

Somebody referred that to me, maybe you, before few weeks as I wanted
to export Org file that I am producing on the fly with fine
referencing. That is where I realized that it is not built in for
paragraphs or items.

** Exported 
1. one item
2. <<target>>another item
Here we refer to item [[target]].

Exported would look like this:

1. one item
2. another item
Here we refer to item 2.

So it loses its references in export. Not that I find it amusing. I
would like to have unique references in export and also shown to
people.

I was considering to go over all the nodes or headings and to inject
<<target>> for every paragraph to get a finely referenced
document. But that is again hackish and complex and makes documents
ugly. So I have decided to transition into multi-object structures.

That approach for me to parse text and inject targets does not comply
to:

TECHNOLOGY TEMPLATE PROJECT OHS Framework 
https://www.dougengelbart.org/content/view/110/460/

See:

Hyperdocument Content and Structure [2a]
https://www.dougengelbart.org/content/view/110/460/#2a

You may also observe how Doug Engelbart's Institute website references
hyperdocuments finely grained by using [2a] reference in this
case. When teaching people in the field of mining I have need to
reference specifics, otherwise people get confused.

,----
|  Elementary Objects [2a1a]
| 
| Objects are basic content packets of an arbitrary, user and developer
| extensible nature. Types of elementary objects could contain text,
| graphics, equations, tables, spreadsheets, canned-images, video,
| sound, code elements, etc.
`----

That is what I have created. It is very simple:

1. Create table for types that can be user and developer extensible
2. Create table for elementary objects. In my case called hlinks

Finished there. If type is spreadsheet, image, video, etc. it is
designated upon creation of elementary object.

Org file is going in reverse. Org has not been defined that way from
ground up, but developers try to fit headings into elementary objects,
but it does not work well.

,----
| Mixed-Object Documents [2a1b]
| 
| Documents are a coherent entity made up of an arbitrary mix of
| elementary objects bundled within a common "envelope" to be stored,
| transmitted, read, printed, or otherwise be operated on.  2a1b1
| 
| the MIME specification is an example of a document definition based on
| a higher level collection of elementary objects.  2a1b2
`----

On my side any node can be also hyperdocument. But if I wish to have
hyperdocument consisting of many other mixed objects that is what I
have defined as a "Set" which is Mixed-Object Document. I can export
mixed-object document into Org file as example. I could as well export
it as directory, or export it as tar package or full HTML page that
contains all those objects hyperlinked to each other, or actually
construct MIME message and send everything together to somebody.

,----
| Objects 
| 
| Objects and the documents made out of them are shareable among members
| of a networked community: they may be simultaneously viewable,
| manipulable, and changeable according to rules appropriate to their
| nature and supported in access permissions and restrictions contained
| in their definitions.  2a1c1
`----

Hyperscope is PostgreSQL database backed so multi users can access one
database or use software to access multiple databases or copy from one
to each other various objects or reference from one database to other
those objects, or export objects into file system or into some type of
files like Org or HTML where objects can be again re-used , shared,
viewed, edited.

Database has concurrency support, all users can edit it in the same
time with version control. Access permission and restrictions are
defined on the database level and on the software level. For example
if user has access to modify some value I have a version control on
the software level (for now) but user could skip the version control
and modify value without having previous version stored in the
database.

,----
| Object ID-Time
| 
| Each creation or modification of an object automatically results in
| the creation of a stamp containing information concerning the date,
| time, date, and user identification associated with that modification.
| 2a1d1
| 
| Users can filter and control the portrayal of this information and
| their associated content objects
`----

This requirement becomes trivial. When creating hyperdocument `hlink'
entry there is automated entry date-created, date-modified,
user-created, user-modified and I have now version control where
history of modifications can be stored as well.

This becomes important in multi user environment, as then agenda gets
additional meanings:

- tasks modified by Joe since February 20th

- tasks entered by Jane, modified by James

- tasks never modified in past

,----
| Personal Signature Encryption/Verification/Authentication of Objects and Documents
| 
| Users can affix personal signatures to a document, or a specified
| collections of objects within the document, using private signature
| keys. Users can verify that the signature is authentic and that no bit
| of the signed document or document segment has been altered since it
| was signed. Signed document segments can be copied or moved in full
| without interfering with later signature verification.  2a1e1
`----

I keep: `hlinks_signature' table entry for this. Now I see that I have
to add signature column to version control table.

,----
| ALTER TABLE vc ADD COLUMN vc_signature TEXT;
| ALTER TABLE
`----

Done.

This would mean that version table could optionally receive GPG/PGP
signature of the previous version of some single value. Not the whole
document in this case. But actual document could contain the
signature. If it is file, the file could be signed. If it is "Set"
then name and ID of the set could be signed. If it is note, the note
in memory can be signed. If it is task, then it can be signed. This is
important feature especially in remote transactions.

Not one time it happened that staff member tells other to disburse
money when it is not ordered. If there is rule that money cannot be
disbursed unless digitally signed and verified than incidents may be
prevented.

,----
| https://www.dougengelbart.org/content/view/110/460/#2b1a
| 
| Global, Human-Understandable, Object Addresses
| 
| Every object that someone might validly want/need to cite or otherwise
| access should have an unambiguous address, capable of being portrayed
| in a human readable and interpretable manner. Most common existing
| spreadsheet programs have provisions similar to this for cell
| addressibility.  2b1a1
`----

Now we come to <<targets>> as I can place them but they are only meant
for HTML export. I do not get targets in ASCII export. But I do need
it as they are useful to locate the specific section that has to be
found. They could be wrapped in something [[123]] to be easily found
by searching [[123 for example.

If I wish to provide specific target for everything in Org file it
would not be readable any more, rather ugly. Paragraph may change or
be split in two paragraphs, parsers would provide new <<target>> not
being unambiguous.

If preparing structure on meta level, specific paragraphs could be
split but their reference and meaning would remain under one unique
ID.

Let us say somebody wish to write a document and chunk it into
referencable paragraphs. One could write one node as a document of 4
pages. Once document has been finished, finalized, then it could be
automatically chunked into paragraphs each having its reference.

,----
| Linking/Filtering/Access Control
| 
| The Basic "Hyper" Characteristics
| 
| Embedded objects called links can point to any arbitrary object within
| the document, or within another document in a specified domain of
| documents. Links can be actuated by a user or an automatic process to
| "go see what is at the other end," or "bring the other-end object to
| this location," or "execute the process identified at the other end."
| These executable processes may control peripheral devices such as CD
| ROM, video-disk players, etc.  2b2a1
`----

- can I point to any other object within the document? I can if I use
  multi-object document.

  I could also construct link that points to specific lines or
  searches within same document similar like:
  {M-x highlight-lines-matching-regexp RET hyper RET hi-yellow RET}

  Who is using GNU Hyperbole would know be able to press{M-RET} on the
  above hyperlink within {M-x ...} and would see the lines highlighted
  in yellow.

  Other systems within Emacs may be used to point out to same parts of
  the text within one specific document.

,----
| Hyperdocument "Back-Links"
| 
| Information about other objects pointing to a specific object in a document is available.
| 2b2b1
`----

As long as hyperlinks are in my database or other database that is
accessible to me, a search in the database could locate hyperlinks to
specific objects and find their IDs and provide list of
back-links. Otherwise without search and without reporting that
somebody is pointing to it, I do not know how would I implement this
feature.

Other objects pointing to one hyperdocument signify there is some
relation involved and are necessary to be discovered.

,----
| Access Control
| 
| Hyperdocuments in personal, group, and library files can have access
| restrictions down to the object level based on individual identity or
| organizational role.  2b2c1
| 
| Access controls can include, for example, allowing or disallowing a
| range of access types such as read, write, or knowledge about
| existence.  2b2c2
`----

PostgreSQL is great as it does allow row security policies:
https://www.postgresql.org/docs/13/ddl-rowsecurity.html

My first surprise question when I discovered Org mode was:

,----
|   why I can assign or delegate a task but cannot share it to those
|   people to which I have delegated it?
`----

  This question becomes more surprising when one can see mobile
  devices have solved the problem of sharing and integrated it very
  well while desktop and notebook computers did not. It should be
  built-in feature if we speak of tasks.

> Or using #+NAME: keyword
> 
>      #+NAME: My Target
>      | a  | table      |
>      |----+------------|
>      | of | four cells |
> 
> Unfortunately, it only works inside a single file. Though I imagine that
> org-id can be modified to support global referencing.
> 
> You can send a feature request about this if you are interested.

In general I am satisfied with Org. Our brainstorming could be used by
those who may read it to enhance it how they wish. My comments are
there for thinking on what could be useful.

Recently I found very useful that `browse-url-handlers' can be
customized so that Emacs can open any kind of hyperlinks as long as
`goto-address-mode' is turned on:

gopher://example.com appears as hyperlink

mid:mailbox:message-ID would appear as hyperlink (I do not know if this
is proper usage)

gemini://example.com can be defined to appear as hyperlink

This makes things very simple and allows my hyperdocuments to be
opened without using Org mode. For example comments in Emacs Lisp can
have plethora of hyperlinks without Org mode and I do not say
necessary WWW hyperlinks. It could be message ID hyperlinks which
explain better why some function has been created.

By using GNU Hyperbole Emacs Package, one can use M-RET to active this hyperlink:
{C-h v thing-at-point-uri-schemes RET} to see the URI schemes that
Emacs can support.

Its value is

("aaa://" "about:" "acap://" "apt:" "bzr://" "bzr+ssh://"
"attachment:/" "chrome://" "cid:" "content://" "crid://" "cvs://"
"data:" "dav:" "dict://" "doi:" "dns:" "dtn:" "feed:" "file:/"
"finger://" "fish://" "ftp://" "geo:" "git://" "go:" "gopher://"
"h323:" "http://" "https://" "im:" "imap://" "info:" "ipp:" "irc://"
"irc6://" "ircs://" "iris.beep:" "jar:" "ldap://" "ldaps://" "magnet:"
"mailto:" "mid:" "mtqp://" "mupdate://" "news:" "nfs://" "nntp://"
"opaquelocktoken:" "pop://" "pres:" "resource://" "rmi://" "rsync://"
"rtsp://" "rtspu://" "service:" "sftp://" "sip:" "sips:" "smb://"
"sms:" "snmp://" "soap.beep://" "soap.beeps://" "ssh://" "svn://"
"svn+ssh://" "tag:" "tel:" "telnet://" "tftp://" "tip://" "tn3270://"
"udp://" "urn:" "uuid:" "vemmi://" "webcal://" "xri://"
"xmlrpc.beep://" "xmlrpc.beeps://" "z39.50r://" "z39.50s://" "xmpp:"
"fax:" "man:" "mms://" "mmsh://" "modem:" "prospero:" "snews:"
"wais://")

And one could define one owns URIs. Isn't it great? Then it works in
every buffer and every mode with `goto-address-mode' as minor mode.

Write text, on the end of text you may include glossary to words.

Glossary
========
Emacs: dict://dict.org&emacs

Or give instructions which packages to install:

apt:install:emacs (do not know if usage is proper but handler can be
made to make it anyhow)

or embed picture into buffer:

data:base64ababbac (huge)

or access VPS servers one by one to maintain them:

fish://user@example.com

or download map of user's location:

geo:1.2,1.1 (wrong usage here, just demonstration)

or contact person

jabber:louis@gnusocial.club

or

xmpp:louis@gnusocial.club

or call or SMS person:

tel:+1234567890

sms:+1234567890

or open manual:

man:bash and this hyperlink actually work without me setting up handler.

> > Heading A can be one node, edited by user one time. It need not be
> > duplicated for other file, rather only a pointer to Heading A can be
> > included and the Heading then appears in a new Org file.
> 
> That would be a great feature indeed. This idea is actually getting a
> lot of attention recently:
> https://github.com/alphapapa/transclusion-in-emacs

Those are logical user demands that come from their practice. When
using database that becomes trivial:

1. Assign the source node to be of type "Org heading" with its "body".
  
2. Assign the symbolical node to be symbolical to point to the node 1.

If I try to implement the centralized database like approach, then
this could be workflow:

1. Create node
   
2. Refile it as copy to specific Org file for source nodes. Such could
   be designated in the form of a hyperlink.
   
3. Assign property to symbolic nodes in the other Org file that it
   becomes clear they are symbolic to the original. Symbolic node can
   be anything but it would convert itself into the source node.

4. By using key the symbolic node could inject itself into the
   original.

** Source node
   :PROPERTIES:
   :TEE-ID:   1
   :END:

Some text

** Symbolic node, call it as you wish as it would replace itself with source node
   :PROPERTIES:
   :SYMBOLIC-TO: ~/Documents/Org/Original-File.org::1
   :END:

   Anything here.

** For function

(defun org-fetch-insert-tee-heading ()
  "Replaces Org node with SYMBOLIC-TO property pointing to
FILE::ID with the Org node from FILE and property TEE-ID being ID"
  (interactive)
  (let* ((symbolic-to (org-property-values "SYMBOLIC-TO"))
	(source (split-string (car symbolic-to) "::"))
	(source-file (car source))
	(source-id (cadr source)))
    (when (and source-file source-id)
      (let* ((heading (with-temp-buffer
			(insert-file source-file)
			(org-mode)
			(goto-char (org-find-property "TEE-ID" source-id))
			(org-get-entry)
			(buffer-string)))
	     (replacement (with-temp-buffer
			    (insert heading)
			    (org-mode)
			    (goto-char (org-find-property "TEE-ID" source-id))
			    (org-delete-property "TEE-ID")
			    (org-set-property "SYMBOLIC-TO" (concat source-file "::" source-id))
			    (buffer-string))))
	(org-get-heading)
	(org-cut-special)
	(insert replacement)))))

Function works and it is concept. It can or should be improved.

Workflow is:

1. Designate property in a file with TEE-ID (remind me of `tee'
   shell command) and assign the ID as you wish.

2. Provide property in other nodes with :SYMBOLIC-TO: to point to file
   and the ID.

3. Assign function to find all symbolical nodes to replace itself with
   the original node.

4. Another function could help to "upload" the source node with
   whatever has been edited in the in symbolical nodes.