Re: Completely hide properties drawer in 9.6

[Jean Louis <bugs@gnu.support>]

Hello Sterling,

* Sterling Hooten <hooten@gmail.com> [2022-12-14 08:39]:
> I'm trying to clone Lotus notes in Org mode.

What features do you try to implement?

Or are you trying to import your own notes into the Org?

Org does not have functions of sharing, while it could have,
it doesn't, it is not considered important. 

Org is designed for personal notes, not really for
organizational. 

Org is not meant to be related to many people as Lotus Notes is.

Rather, Org mode is centric, and meant to be related to mainly
to the person using it. Other people are not recognized as
objects in Org, despite the attempts by single individuals to
integrate such.

Complexity of Org and Emacs prevents new users adapting
themselves to the system. 

User of Lotes Notes would expect to see the integrated
interface of the application, while Org has mainly menu
interface for editing, not for activities. Two activities such
as Agenda and Export are there.

Org users constantly tend to have organized structured
collection of data, but because of it being text in the first
place it becomes (only apparently) impossible to convert
itself into the database. 

There is no such thing as problem solving and product creation
in Org. Lotus Notes is for that. Track calls related to
people, identify problems, resolve the problems, assign the
next step to next person solving it. All those system
processes can of course be done by hand, on the paper, and I
was doing it. And of course it can be done by using Org mode,
including by using any text editor. 

Though Org does not have fundamental organized structured
collection of data that can easily be accessed, manipulated or
updated -- in other word the database.

Building Lotus Notes inside of Org is thus impossible task.
Fighting windmills may be more feasible.

In my workflow, I have many headings as in Org and "headings"
similar to Org. Each heading may be sent to other people by
using SMS, XMPP, E-mail, or letter, or communicated directly
face to face, or by phone, conference. There is record that
heading ABC has been communicated to person ID 123. What if
heading ABC has been communicated to 4000 people? It is not
feasible to keep such information in Org properties, it slows
down the user to review most important information. But in
organizational management we do need to track which person
received which document, what day, why, for what reason.

We have to be able to see which person is related to which
headings. 

Org in its form how it is not, as it is based on text, is not
yet scalable to become similar to application such as Lotus
Notes.

Reference to review what Lotus Notes does:
https://archive.org/details/notedemo_SoftwareLibrary

In Org mode that would imply that there is some list of people
which is separate list and that any other Org heading or Org
file or external documents are shared to people and that
record of such sharing shall be stored in some file related to
people only. While in same time Org has not been taught to
know about "people", "sharing", and "relations". 

Sharing expertise with each other by using Org is currently
not supported. It could be. There are techniques and methods
by connecting Org to online databases, not really hard. But it
is not yet integrated. 

I wish to see org-sharing.el coming up for various types of
communication channels.

Org as Lotus Notes works on Emacs and Emacs being compatible
with various systems makes it equally well compatible. 

Though for application like Lotus Notes, the Emacs Menu should
be partially amended with the application specific menu that
relates to functions similar to Lotus Notes.

And then if we speak of "groupware", Org is far from that.

One way to add unlimited properties to any Org heading, but
then also to any other text file in Emacs is to first figure
out how to reference headings and any other types. 

Org heading may be duplicated in single file and in multiple files:

* Business
** List of items
* Family
** List of items

Where "List of items" could be duplicate in same file.
Reference cannot be "List of items" as that could be easily
renamed "List of stuff" to cause havoc in organization of
data. And it cannot be reference to tree node, as nodes may
also move from place to place in single file and to other
files.

The way to go is to have a separate and external list of UUIDs
to make sure that UUIDs are unique.  Then in every Org file
one may generate UUIDs as references to headings and other objects.

External list of UUIDs is necessary Org's UUID does not have
UNIQUE check like in relational databases such as PostgreSQL.
Any UUID generated in the Org file could                                               
Default                      Nullable Collation Type             Column           be 
compared to
external UUID only to check if UUID already exists. External
list of UUIDs represents the pointer to Org heading.

That all may use simple text. It could use the Emacs 29
built-in SQLite database or PostgreSQL.

By creating special hypothetical uuid-reference-mode one could
then insert UUIDs anywhere in the next and make them not
visible, but referencable by uuid-reference-mode.

It would be an invisible markup.

* Heading like this one, could have it's UUID f22fb1a8-4bcb-4592-a955-070bac158f50

And it could be either in the same line, previous or following line.

The hypothetical uuid-reference-mode would make such UUIDs invisible.

In the next step one could design "properties" table in
SQLite, that could contain anything related to that specific UUID.

Sample `properties' table:
                                                      Table "public.properties"
┌───────────────────────────┬─────────────────────────────┬───────────┬──────────┬───────────────────────────────────────────────────┐
│          Column           │            Type             │ Collation │ Nullable │                      Default                      │
├───────────────────────────┼─────────────────────────────┼───────────┼──────────┼───────────────────────────────────────────────────┤
│ properties_id             │ integer                     │           │ not null │ nextval('properties_properties_id_seq'::regclass) │
│ properties_datecreated    │ timestamp without time zone │           │ not null │ CURRENT_TIMESTAMP                                 │
│ properties_datemodified   │ timestamp without time zone │           │          │                                                   │
│ properties_usercreated    │ text                        │           │ not null │ CURRENT_USER                                      │
│ properties_usermodified   │ text                        │           │ not null │ CURRENT_USER                                      │
│ properties_name           │ text                        │           │ not null │                                                   │
│ properties_description    │ text                        │           │          │                                                   │
│ properties_value          │ text                        │           │          │                                                   │
│ properties_hyobjects      │ integer                     │           │          │                                                   │
│ properties_people         │ integer                     │           │          │                                                   │
│ properties_referenceduuid │ uuid                        │           │          │                                                   │
│ properties_uuid           │ uuid                        │           │ not null │ gen_random_uuid()                                 │
│ properties_lastfile       │ text                        │           │          │		                                     │
└───────────────────────────┴─────────────────────────────┴───────────┴──────────┴───────────────────────────────────────────────────┘
Indexes:
    "properties_pkey" PRIMARY KEY, btree (properties_id)
Check constraints:
    "properties_not_empty_reference_check" CHECK (properties_hyobjects IS NOT NULL OR properties_people IS NOT NULL)
Foreign-key constraints:
    "properties_properties_hyobjects_fkey" FOREIGN KEY (properties_hyobjects) REFERENCES hyobjects(hyobjects_id)
    "properties_properties_people_fkey" FOREIGN KEY (properties_people) REFERENCES people(people_id)
Triggers:
    insert_username_properties BEFORE INSERT OR UPDATE ON properties FOR EACH ROW EXECUTE FUNCTION insert_username('properties_usermodified')
    properties_moddatetime BEFORE UPDATE ON properties FOR EACH ROW EXECUTE FUNCTION moddatetime('properties_datemodified')

It's "design" took me less than few minutes. Sample entry in
properties table:

		     ID   1
	   Date created   "2022-12-16 11:32:38.424024"
	  Date modified   "2022-12-16 11:47:08.008854"
	   User created   "maddox"
	  User modified   "maddox"
		   Name   "SUMMARY"
	    Description   nil
		  Value   "Good to learn"
	  Hyperdocument   "Emacs Lisp Cookbook"
	      People ID   nil
	Referenced UUID   nil
		   UUID   "4ab34c24-f52a-4c03-8cae-372a5fb5b673"
     Last found in file   "/home/user/my-notes.org"

- Properties table thus could reference an UUID

- That UUID could be heading in Org, and it could get same properties
  just as defined in Org

- But beyond the Org, the UUID could be in any kind of the file. One
  could place references in text or C file, Emacs Lisp files and so
  on, and add properties to such UUID reference.

- Properties table or file could remember where the UUID was last time
  found.  Special crawler could search for UUID in all files to update
  it.

- Hypothetical uuid-reference-mode would make sure that UUIDs do not
  disturb, and that they are not visible within Emacs. Function
  `facemenu-set-invisible' makes things invisible, but available, that
  could be one way of hiding the UUID.

> This has been discussed multiple times before in this list. The
> general sentiment has been to reject adding it to Org as hiding things
> ompletely from the user might be confusing, but it's also been
> brought up enough times independently to demonstrate a latent demand
> for this behavior, and a working example would be very useful for
> those of us who wish to have a cleaner interface and editing
> experience.

We are in computing subjects, and Org is definitely an interesting and
useful programming subject, it is simply not the pure "text editing"
subject. 

When feature is considered useful, use it, add it.

If any feature is disturbing users, consider making it invisible and
not disturbing.

But is not good to say that "hiding" things would be confusing. That
invalidates users. 

File creation date, modification, ownership, group ownership is basic
file information that is also not shown to user at all times, and we
could say it is "hidden" while it is not realy hidden.

Nothing in free software is really hidden to the user. Hiding
properties completely or folding it is almost equal to each other.

I am already long time user of my Dynamic Knowledge Repository named
RCD Notes for Emacs and system is logically based on headings (Sets)
and the tree (nodes) like many other similar systems. 

What I wish to see in the first place is the name of the node or
heading. If I personally wish to see any timestamps, I press TAB on
the heading and see time stamps, clocked-in, out, scheduled times and
similar. 

Other key shows me people related to specific documents, who is
assigned to the task, who has received the task by e-mail, SMS, or
how, and simple "E" key gives me to see and edit search status,
starting dates and times, UUID, markup type, type, subtype, hyperlink,
arguments, description of the heading, internal information, author,
revision, language of heading, it's size, time length in case of media
like audio and video media, width, height, it's hash, PGP signature,
related people, priority, publisher information, action status like
TODO, related URIs, template, WWW domain, slug, license, report, major
mode for it, minor modes, related country, physical location, lead
source, currency, sales flow stage, communication line, geographic
location, SKU, value or price, introducer, etc.

I could say all such information is "hidden" while it is not, it is
matter of pressing single key and teaching users how to do it.

Org files are confusing with more properties, they are legible with
less properties. 

Same thing for tags, all documents may be tagged, but I do not like
watching the tags every time close to the heading, that is what really
disturbs. Tags are separate. If I wish to see them I press "i" for
info and see the tag. If I press on the tag, I can see all documents
tagged by it.

┌─────────────────────────────┬────────┐
│         Description         │ Count  │
├─────────────────────────────┼────────┤
│ Total tags: Hyperdocuments  │ 126801 │
│ Total tags: People          │   2426 │
│ Total number of unique tags │   1706 │
└─────────────────────────────┴────────┘

Thus it is very beneficial to separate any kind properties from that
what really matters, which is the text that is written.

> Part of the complication comes from updating modified timestamps in
> the properties of each heading. This drives the need for hiding to be
> fast so as not to bog down the editing experience when triggered by
> `after-change-functions'.
> 
> The constant updating of a modification timestamp system puts some
> constraints on the property hiding system. 

Org is not a database. Connect it to database and you can do that
all. Assign UUIDs only and by each update, update the properties such
as modified times in the database automatically. That speeds up your
workflow possibly.

> Any help would be greatly appreciated as things have changed in 9.6
> and I'm not sure where the opportunities are to leverage the
> improvements (e.g., are text properties now the way? Or should I still
> implement it with faces?).

You are trying to make Org become database. Strategy will generate
waste of time and efforts for something it was not designed for.

Instead, use Org for writing pieces of information within a database,
and use the database to keep properties.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/