Completely hide properties drawer in 9.6

Completely hide properties drawer in 9.6

[Sterling Hooten]

First of all, thanks to everyone for working on 9.6!

I'm trying to clone Lotus notes in Org mode. This requires certain
properties to be generated for every heading (e.g., ID, creation time,
modification time, etc.). But these are mainly used for searching and
sorting, and rarely of interest to me during editing. For this reason
I would like to /completely/ hide the properties box (not just the
individual properties).

This has been discussed multiple times before in this list. The
general sentiment has been to reject adding it to Org as hiding things
ompletely from the user might be confusing, but it's also been
brought up enough times independently to demonstrate a latent demand
for this behavior, and a working example would be very useful for
those of us who wish to have a cleaner interface and editing
experience.

Alternative suggestions have been to change the face to make the
properties box appear smaller, but this interferes with movement from
the heading to the body, and the ability to toggle the visibility of a
property box would make this is redundant.

In <https://list.orgmode.org/87zh9ecfgc.fsf@localhost/> Ihor suggested
adding `org-custom-properties-hide-emptied-drawers', this was
rejected, but is pretty close to the desired behavior, except it's
still global. I was unable to get this working in release 9.6.

Prior to 9.6 I overwrote org-cycle-heading to hide the properties box
using overlays. This worked alright but had slowdowns on larger files
and affected typing cadence noticeably. Org 9.6 is now relying on text
properties instead of overlays, and I would like help replicating (and
improving) that behavior.

Part of the complication comes from updating modified timestamps in
the properties of each heading. This drives the need for hiding to be
fast so as not to bog down the editing experience when triggered by
`after-change-functions'.

The constant updating of a modification timestamp system puts some
constraints on the property hiding system. From working in 9.5 I
believe the relevant requirements are:

• Ability to add or modify properties with `org-entry-put' and
  maintain the state of being hidden or visible
  • E.g., Updating the ":MODIFIED:" property to a hidden property
    drawer should keep the drawer hidden.
• Toggle on and off hiding globally (perhaps in a minor mode)
• Moving headings around with `org-metaup' etc preserves the
  visibility state
  • A hidden property drawer should stay hidden when the outline is
    rearranged
• Resistant to changes with undo etc
  • Undoing a change should cause the contet to revert to the same
    visibility state it had at time of change.
• Don't overwrite when editing normally
  • With the point at the end of a heading and immediately before an
    ellipsis, typing "j" should result in the character being inserted
    with the face of the heading, and not overwrite or otherwise
    disturb the invisible text. `org-fold-catch-invisible-edits' might
    be relevant here.
• Reveal or hide just the property drawer of the heading at point and
  not the whole buffer
  • This is partially for speed reasons (to handle modified timestamp
    updates), but also because revealing all property drawers is
    extremely annoying when the surrounding headings in the buffer
    have headings with >10 properties.
• Fast enough to use with `after-change-functions'
  • The modification timestamp setup I'm using is triggered by buffer
    changes, and thus when typing runs after every keystroke. To keep
    this smooth and not be a bottleneck it would need `(benchmark 100
    'org-hide-properties)' to be less than .1s on an Org file with
    >20,000 lines and >1,000 headings.
• Ability to reveal just a subset of property lines
  • e.g., `org-bibtex' creates many properties to store bibliographic
    data. It would be useful to have enough control over hiding of
    properties such that if only ":BIB_FIELD:" properties of.
• Hide ellipsis unless body text is present
  • If all the headings have properties then the default folding
    behavior will place ellipses at the end of each heading, thus
    destroying the informational value of the ellipsis as an
    indication of folded but displayable content. It would be useful
    if instead the ellipsis would only be present when there's body
    text.

Any help would be greatly appreciated as things have changed in 9.6
and I'm not sure where the opportunities are to leverage the
improvements (e.g., are text properties now the way? Or should I still
implement it with faces?).

Thanks!

Re: Completely hide properties drawer in 9.6

[Ihor Radchenko]

Sterling Hooten <hooten@gmail.com> writes:

> In <https://list.orgmode.org/87zh9ecfgc.fsf@localhost/> Ihor suggested
> adding `org-custom-properties-hide-emptied-drawers', this was
> rejected, but is pretty close to the desired behavior, except it's
> still global. I was unable to get this working in release 9.6.

I do have this implemented in local branch. Just did not push it
upstream as it was rejected.

> Prior to 9.6 I overwrote org-cycle-heading to hide the properties box
> using overlays. This worked alright but had slowdowns on larger files
> and affected typing cadence noticeably. Org 9.6 is now relying on text
> properties instead of overlays, and I would like help replicating (and
> improving) that behavior.

Note that Emacs 29 does not have issues with large numbers of overlays.

> Part of the complication comes from updating modified timestamps in
> the properties of each heading. This drives the need for hiding to be
> fast so as not to bog down the editing experience when triggered by
> `after-change-functions'.

This is handled by org-fold-core.el library.
Note that you can use the library to introduce new fold types.
Editing inside folds is optimized. All other concerns are also handled
within org-fold-core.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Completely hide properties drawer in 9.6

[Sterling Hooten]

> On 2022-12-14, at 06:15, Ihor Radchenko <yantar92@posteo.net> wrote:
> 
> I do have this implemented in local branch. Just did not push it
> upstream as it was rejected.

Could you share that here so I could test it?

> Note that Emacs 29 does not have issues with large numbers of overlays.

I’m using the MacOS version and not building Emacs myself so I can’t try this out at the moment.

> This is handled by org-fold-core.el library.
> Note that you can use the library to introduce new fold types.
> Editing inside folds is optimized. All other concerns are also handled
> within org-fold-core.

I will read through this and try it out.

Re: Completely hide properties drawer in 9.6

[Ihor Radchenko]

Sterling Hooten <hooten@gmail.com> writes:

>> On 2022-12-14, at 06:15, Ihor Radchenko <yantar92@posteo.net> wrote:
>> 
>> I do have this implemented in local branch. Just did not push it
>> upstream as it was rejected.
>
> Could you share that here so I could test it?

https://github.com/yantar92/org/tree/feature/org-font-lock-element

>> This is handled by org-fold-core.el library.
>> Note that you can use the library to introduce new fold types.
>> Editing inside folds is optimized. All other concerns are also handled
>> within org-fold-core.
>
> I will read through this and try it out.

I tried my best to explain the design in the top comment. You may also
look into `org-fold-initialize'.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Completely hide properties drawer in 9.6

[Sterling Hooten]

I was able to get a very rough version working to fold and hide 
the properties drawer completely at point.

#+begin_src emacs-lisp
  (defun swh-org-hide-properties-drawer ()
    "Completely hide the property drawer in heading at point."
    (org-fold-add-folding-spec 'org-hide-property-drawer
			       '((:ellipsis . nil)
				 (:isearch-open . t))
			       nil
			       'append)
    (let* ((block (org-get-property-block))
	   ;; Extremely hacky way to get the whole property drawer
	   (beg (- (car block) 14))
	   (end (+ (cdr block) 5)))
      (with-silent-modifications
	(org-fold-region beg end t 'org-hide-property-drawer))))
#+end_src

Is there a better way to get the beginning and end 
of the property drawer more robustly?

This is a step in the right direction, but it seems too sensitive to accidental editing.

This should be impervious to most types of user interaction,
and prevent accidentally deleting or abutting anything
up against either the start or end of the properties box.

I tried setting `org-fold-catch-invisible-edits' to different values, but even
with set to nil I can backward delete and erase some invisible
text. Is this dependent on their being an ellipsis? Or is there
another way to make this impervious to most edits?

When adding a property with `org-property-put’ this would need
to be called again, is there a way to do that robustly?

> On 2022-12-14, at 12:14, Ihor Radchenko <yantar92@posteo.net> wrote:
> 
> I tried my best to explain the design in the top comment. You may also
> look into `org-fold-initialize'.

Re: Completely hide properties drawer in 9.6

[Ihor Radchenko]

Sterling Hooten <hooten@gmail.com> writes:

>     (let* ((block (org-get-property-block))
> 	   ;; Extremely hacky way to get the whole property drawer
> 	   (beg (- (car block) 14))
> 	   (end (+ (cdr block) 5)))
>       (with-silent-modifications
> 	(org-fold-region beg end t 'org-hide-property-drawer))))
> #+end_src
>
> Is there a better way to get the beginning and end 
> of the property drawer more robustly?

org-element-at-point

> This is a step in the right direction, but it seems too sensitive to accidental editing.

See :fragile in org-fold-core--specs.

> This should be impervious to most types of user interaction,
> and prevent accidentally deleting or abutting anything
> up against either the start or end of the properties box.

which is part of the reason why the proposal was rejected.
Though Org tries hard in org-catch-invisible-edits.

> I tried setting `org-fold-catch-invisible-edits' to different values, but even
> with set to nil I can backward delete and erase some invisible
> text. Is this dependent on their being an ellipsis? Or is there
> another way to make this impervious to most edits?

It's just because only a fixed set of folds is caught in the invisible
edits check now.

> When adding a property with `org-property-put’ this would need
> to be called again, is there a way to do that robustly?

I should not need to be called again once you set
:front-sticky/:rear-sticky folding spec property.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Completely hide properties drawer in 9.6

[Jean Louis]

Hello Sterling,

* Sterling Hooten <hooten@gmail.com> [2022-12-14 08:39]:
> I'm trying to clone Lotus notes in Org mode.

What features do you try to implement?

Or are you trying to import your own notes into the Org?

Org does not have functions of sharing, while it could have,
it doesn't, it is not considered important. 

Org is designed for personal notes, not really for
organizational. 

Org is not meant to be related to many people as Lotus Notes is.

Rather, Org mode is centric, and meant to be related to mainly
to the person using it. Other people are not recognized as
objects in Org, despite the attempts by single individuals to
integrate such.

Complexity of Org and Emacs prevents new users adapting
themselves to the system. 

User of Lotes Notes would expect to see the integrated
interface of the application, while Org has mainly menu
interface for editing, not for activities. Two activities such
as Agenda and Export are there.

Org users constantly tend to have organized structured
collection of data, but because of it being text in the first
place it becomes (only apparently) impossible to convert
itself into the database. 

There is no such thing as problem solving and product creation
in Org. Lotus Notes is for that. Track calls related to
people, identify problems, resolve the problems, assign the
next step to next person solving it. All those system
processes can of course be done by hand, on the paper, and I
was doing it. And of course it can be done by using Org mode,
including by using any text editor. 

Though Org does not have fundamental organized structured
collection of data that can easily be accessed, manipulated or
updated -- in other word the database.

Building Lotus Notes inside of Org is thus impossible task.
Fighting windmills may be more feasible.

In my workflow, I have many headings as in Org and "headings"
similar to Org. Each heading may be sent to other people by
using SMS, XMPP, E-mail, or letter, or communicated directly
face to face, or by phone, conference. There is record that
heading ABC has been communicated to person ID 123. What if
heading ABC has been communicated to 4000 people? It is not
feasible to keep such information in Org properties, it slows
down the user to review most important information. But in
organizational management we do need to track which person
received which document, what day, why, for what reason.

We have to be able to see which person is related to which
headings. 

Org in its form how it is not, as it is based on text, is not
yet scalable to become similar to application such as Lotus
Notes.

Reference to review what Lotus Notes does:
https://archive.org/details/notedemo_SoftwareLibrary

In Org mode that would imply that there is some list of people
which is separate list and that any other Org heading or Org
file or external documents are shared to people and that
record of such sharing shall be stored in some file related to
people only. While in same time Org has not been taught to
know about "people", "sharing", and "relations". 

Sharing expertise with each other by using Org is currently
not supported. It could be. There are techniques and methods
by connecting Org to online databases, not really hard. But it
is not yet integrated. 

I wish to see org-sharing.el coming up for various types of
communication channels.

Org as Lotus Notes works on Emacs and Emacs being compatible
with various systems makes it equally well compatible. 

Though for application like Lotus Notes, the Emacs Menu should
be partially amended with the application specific menu that
relates to functions similar to Lotus Notes.

And then if we speak of "groupware", Org is far from that.

One way to add unlimited properties to any Org heading, but
then also to any other text file in Emacs is to first figure
out how to reference headings and any other types. 

Org heading may be duplicated in single file and in multiple files:

* Business
** List of items
* Family
** List of items

Where "List of items" could be duplicate in same file.
Reference cannot be "List of items" as that could be easily
renamed "List of stuff" to cause havoc in organization of
data. And it cannot be reference to tree node, as nodes may
also move from place to place in single file and to other
files.

The way to go is to have a separate and external list of UUIDs
to make sure that UUIDs are unique.  Then in every Org file
one may generate UUIDs as references to headings and other objects.

External list of UUIDs is necessary Org's UUID does not have
UNIQUE check like in relational databases such as PostgreSQL.
Any UUID generated in the Org file could                                               
Default                      Nullable Collation Type             Column           be 
compared to
external UUID only to check if UUID already exists. External
list of UUIDs represents the pointer to Org heading.

That all may use simple text. It could use the Emacs 29
built-in SQLite database or PostgreSQL.

By creating special hypothetical uuid-reference-mode one could
then insert UUIDs anywhere in the next and make them not
visible, but referencable by uuid-reference-mode.

It would be an invisible markup.

* Heading like this one, could have it's UUID f22fb1a8-4bcb-4592-a955-070bac158f50

And it could be either in the same line, previous or following line.

The hypothetical uuid-reference-mode would make such UUIDs invisible.

In the next step one could design "properties" table in
SQLite, that could contain anything related to that specific UUID.

Sample `properties' table:
                                                      Table "public.properties"
┌───────────────────────────┬─────────────────────────────┬───────────┬──────────┬───────────────────────────────────────────────────┐
│          Column           │            Type             │ Collation │ Nullable │                      Default                      │
├───────────────────────────┼─────────────────────────────┼───────────┼──────────┼───────────────────────────────────────────────────┤
│ properties_id             │ integer                     │           │ not null │ nextval('properties_properties_id_seq'::regclass) │
│ properties_datecreated    │ timestamp without time zone │           │ not null │ CURRENT_TIMESTAMP                                 │
│ properties_datemodified   │ timestamp without time zone │           │          │                                                   │
│ properties_usercreated    │ text                        │           │ not null │ CURRENT_USER                                      │
│ properties_usermodified   │ text                        │           │ not null │ CURRENT_USER                                      │
│ properties_name           │ text                        │           │ not null │                                                   │
│ properties_description    │ text                        │           │          │                                                   │
│ properties_value          │ text                        │           │          │                                                   │
│ properties_hyobjects      │ integer                     │           │          │                                                   │
│ properties_people         │ integer                     │           │          │                                                   │
│ properties_referenceduuid │ uuid                        │           │          │                                                   │
│ properties_uuid           │ uuid                        │           │ not null │ gen_random_uuid()                                 │
│ properties_lastfile       │ text                        │           │          │		                                     │
└───────────────────────────┴─────────────────────────────┴───────────┴──────────┴───────────────────────────────────────────────────┘
Indexes:
    "properties_pkey" PRIMARY KEY, btree (properties_id)
Check constraints:
    "properties_not_empty_reference_check" CHECK (properties_hyobjects IS NOT NULL OR properties_people IS NOT NULL)
Foreign-key constraints:
    "properties_properties_hyobjects_fkey" FOREIGN KEY (properties_hyobjects) REFERENCES hyobjects(hyobjects_id)
    "properties_properties_people_fkey" FOREIGN KEY (properties_people) REFERENCES people(people_id)
Triggers:
    insert_username_properties BEFORE INSERT OR UPDATE ON properties FOR EACH ROW EXECUTE FUNCTION insert_username('properties_usermodified')
    properties_moddatetime BEFORE UPDATE ON properties FOR EACH ROW EXECUTE FUNCTION moddatetime('properties_datemodified')

It's "design" took me less than few minutes. Sample entry in
properties table:

		     ID   1
	   Date created   "2022-12-16 11:32:38.424024"
	  Date modified   "2022-12-16 11:47:08.008854"
	   User created   "maddox"
	  User modified   "maddox"
		   Name   "SUMMARY"
	    Description   nil
		  Value   "Good to learn"
	  Hyperdocument   "Emacs Lisp Cookbook"
	      People ID   nil
	Referenced UUID   nil
		   UUID   "4ab34c24-f52a-4c03-8cae-372a5fb5b673"
     Last found in file   "/home/user/my-notes.org"

- Properties table thus could reference an UUID

- That UUID could be heading in Org, and it could get same properties
  just as defined in Org

- But beyond the Org, the UUID could be in any kind of the file. One
  could place references in text or C file, Emacs Lisp files and so
  on, and add properties to such UUID reference.

- Properties table or file could remember where the UUID was last time
  found.  Special crawler could search for UUID in all files to update
  it.

- Hypothetical uuid-reference-mode would make sure that UUIDs do not
  disturb, and that they are not visible within Emacs. Function
  `facemenu-set-invisible' makes things invisible, but available, that
  could be one way of hiding the UUID.

> This has been discussed multiple times before in this list. The
> general sentiment has been to reject adding it to Org as hiding things
> ompletely from the user might be confusing, but it's also been
> brought up enough times independently to demonstrate a latent demand
> for this behavior, and a working example would be very useful for
> those of us who wish to have a cleaner interface and editing
> experience.

We are in computing subjects, and Org is definitely an interesting and
useful programming subject, it is simply not the pure "text editing"
subject. 

When feature is considered useful, use it, add it.

If any feature is disturbing users, consider making it invisible and
not disturbing.

But is not good to say that "hiding" things would be confusing. That
invalidates users. 

File creation date, modification, ownership, group ownership is basic
file information that is also not shown to user at all times, and we
could say it is "hidden" while it is not realy hidden.

Nothing in free software is really hidden to the user. Hiding
properties completely or folding it is almost equal to each other.

I am already long time user of my Dynamic Knowledge Repository named
RCD Notes for Emacs and system is logically based on headings (Sets)
and the tree (nodes) like many other similar systems. 

What I wish to see in the first place is the name of the node or
heading. If I personally wish to see any timestamps, I press TAB on
the heading and see time stamps, clocked-in, out, scheduled times and
similar. 

Other key shows me people related to specific documents, who is
assigned to the task, who has received the task by e-mail, SMS, or
how, and simple "E" key gives me to see and edit search status,
starting dates and times, UUID, markup type, type, subtype, hyperlink,
arguments, description of the heading, internal information, author,
revision, language of heading, it's size, time length in case of media
like audio and video media, width, height, it's hash, PGP signature,
related people, priority, publisher information, action status like
TODO, related URIs, template, WWW domain, slug, license, report, major
mode for it, minor modes, related country, physical location, lead
source, currency, sales flow stage, communication line, geographic
location, SKU, value or price, introducer, etc.

I could say all such information is "hidden" while it is not, it is
matter of pressing single key and teaching users how to do it.

Org files are confusing with more properties, they are legible with
less properties. 

Same thing for tags, all documents may be tagged, but I do not like
watching the tags every time close to the heading, that is what really
disturbs. Tags are separate. If I wish to see them I press "i" for
info and see the tag. If I press on the tag, I can see all documents
tagged by it.

┌─────────────────────────────┬────────┐
│         Description         │ Count  │
├─────────────────────────────┼────────┤
│ Total tags: Hyperdocuments  │ 126801 │
│ Total tags: People          │   2426 │
│ Total number of unique tags │   1706 │
└─────────────────────────────┴────────┘

Thus it is very beneficial to separate any kind properties from that
what really matters, which is the text that is written.

> Part of the complication comes from updating modified timestamps in
> the properties of each heading. This drives the need for hiding to be
> fast so as not to bog down the editing experience when triggered by
> `after-change-functions'.
> 
> The constant updating of a modification timestamp system puts some
> constraints on the property hiding system. 

Org is not a database. Connect it to database and you can do that
all. Assign UUIDs only and by each update, update the properties such
as modified times in the database automatically. That speeds up your
workflow possibly.

> Any help would be greatly appreciated as things have changed in 9.6
> and I'm not sure where the opportunities are to leverage the
> improvements (e.g., are text properties now the way? Or should I still
> implement it with faces?).

You are trying to make Org become database. Strategy will generate
waste of time and efforts for something it was not designed for.

Instead, use Org for writing pieces of information within a database,
and use the database to keep properties.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Completely hide properties drawer in 9.6

[Ihor Radchenko]

Jean Louis <bugs@gnu.support> writes:

> * Sterling Hooten <hooten@gmail.com> [2022-12-14 08:39]:
>> I'm trying to clone Lotus notes in Org mode.
>
> What features do you try to implement?
>
> Or are you trying to import your own notes into the Org?
>
> Org does not have functions of sharing, while it could have,
> it doesn't, it is not considered important. 
> ....

Jean, while I understand that Org does not completely fit your usage
patterns, please do not assume that every other user who appears to be
needing somewhat similar functionality also needs all other things you
are using.

This message is first asking if a user needs something and then goes
ahead as if the answer was "yes".

Honestly, it sounds more like advertisement for Hyperscope rather than an
attempt to help a particular user in particular question discussed in
particular thread. Especially for long term Org mailing list subscribers
who did see your other similar messages.

Please consider staying closer to the actual topic in the thread.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Completely hide properties drawer in 9.6

[Jean Louis]

Message was on topic as it provided design directions on how to avoid
properties, without technical actual program.

Numerous people will express numerous directions on how specific
problem may be solved. 

Take the life as it is.

I don't appreciate this kind of slap and wining about nothing!

* Ihor Radchenko <yantar92@posteo.net> [2022-12-16 14:46]:
> Jean Louis <bugs@gnu.support> writes:
> 
> > * Sterling Hooten <hooten@gmail.com> [2022-12-14 08:39]:
> >> I'm trying to clone Lotus notes in Org mode.
> >
> > What features do you try to implement?
> >
> > Or are you trying to import your own notes into the Org?
> >
> > Org does not have functions of sharing, while it could have,
> > it doesn't, it is not considered important. 
> > ....
> 
> Jean, while I understand that Org does not completely fit your usage
> patterns, please do not assume that every other user who appears to be
> needing somewhat similar functionality also needs all other things you
> are using.
> 
> This message is first asking if a user needs something and then goes
> ahead as if the answer was "yes".
> 
> Honestly, it sounds more like advertisement for Hyperscope rather than an
> attempt to help a particular user in particular question discussed in
> particular thread. Especially for long term Org mailing list subscribers
> who did see your other similar messages.
> 
> Please consider staying closer to the actual topic in the thread.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Completely hide properties drawer in 9.6

[Ihor Radchenko]

Jean Louis <bugs@gnu.support> writes:

> Message was on topic as it provided design directions on how to avoid
> properties, without technical actual program.
>
> Numerous people will express numerous directions on how specific
> problem may be solved. 
>
> Take the life as it is.

I agree that your message is on topic. Just, as I pointed, I feel like
it is a bit too broad to be useful. I may be wrong, of course.

> I don't appreciate this kind of slap and wining about nothing!

It is not "nothing" for me. I'd prefer to have helpful discussion
happening in Org ML. Your message might be helpful, but I feel like it
might not.

I understand that my message might be frustrating, but I wanted to see
if others feel similar. If nobody else support my concern, I will
correct myself and accept that my definition of "too broad" diverges
from most other Org ML subscribers.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Completely hide properties drawer in 9.6

[Jean Louis]

* Ihor Radchenko <yantar92@posteo.net> [2022-12-16 18:21]:
> I agree that your message is on topic. Just, as I pointed, I feel like
> it is a bit too broad to be useful. I may be wrong, of course.

You may take it personally, but you shouldn't, as people like me, and
others, may not always discuss exactly that what you would like it to
be.

You and people may wish to develp new Emacs packages and extend
features. What you extend is what you decide, some other people make
different packages, why then so much resistance and opposition just
because you personally don't like idea. If you don't like it, don't
engage in it. If you think it is wrong, than say why is it technically
wrong, and if there is intention to extend some features, let me do
it, what you don't like, you will not use anyway, and I did not tell
to inject any code into main Org.

When some proposal is not helpful, then feel free to give argumented
analysis why it would not be helpful, instead of generalizations,
personal antagonism and calling mob on me. Thanks for attention so
far. Better we stop here about it. 🛑

In my opinion, my proposals were either too abstract for you, or out
of your reality, and you cannot do nothing with it. Personally I use
UUIDs for quick references to whatever it may be. While majority of
people in many other software packages do not use UUIDs to reference
to things universally, it is already in Org mode when ID is
generated. It is not really new.

To me it is very practical to have this simple heading with a drawer
that is completely hidden, which references by using UUID to a complex
set of properties. To somebody else, that may not be useful or is
abstract to understand.

In my work I would not like having more properties for Org heading but
one, like this one:

* My heading
  :PROPERTIES:
  :ID: 944334aa-acab-4e6a-8dd6-ebfd144eac6b
  :END:

And that alone would be a reference to any other property that I
personally may need. It is a link from heading to relations to many
different pieces of information. 

If you have maybe followed some latest Hyperbole discussion, using
UUIDs with Hyperbole has been shown useful. UUID itself may decide to
what it is referenced. Simple click on UUID and one may be in whatever
object it can be.

Let us make it visual, here is the video how it works when Org
headline has only UUID. Video is mock up of how properties can be
removed from Org file, it uses package Hyperbole to quickly jump to
properties related to UUID:

https://gnu.support/images/2022/12/2022-12-18/2022-12-18-02:07:35.ogv

Instead of Hyperbole, one can simply use keybindings as usual on the
function which will show the properties.

Following the thought of separating properties, to add a property, one
would of course use a key, and add property in about the same manner
as usual. To delete a property, one jumps to property list and press
key like `d', to edit the property one uses either Org functions or
direct editing and selecting like shown in video.

Emacs 29 has SQLite built-in, there is PostgreSQL module and also
rcd-hash-edit package that works in similar way as shown on
video. Hashes may easily be stored in text files and read from text files. 

I use following functions to store data to file:

;;;; LISP DATA FUNCTIONS

(defun string-to-file-force (string file)
  "Prints string into file, matters not if file exists. Return FILE as file name."
  (with-temp-file file
    (insert string))
  file)

(defun file-to-string (file)
  "File to string function"
  (with-temp-buffer
    (insert-file-contents file)
    (buffer-string)))

(defun data-to-file (data file)
  "PRIN1 Emacs Lisp DATA to FILE"
  (string-to-file-force (prin1-to-string data) file))

(defun data-from-file (file)
  "Reads and returns Emacs Lisp data from FILE"
  (condition-case nil
      (car (read-from-string
	    (file-to-string file)))
    (error nil)))

Visual hash editing is one of options to store properties when
database is not available. Editing of the hash looks just the same as
on the video. Hash can store any kind of data.

Saving hash in text file goes like this:

(let ((my-hash (make-hash-table :test #'equal)))
  (puthash "SUMMARY" "This is my summary of the Org heading." my-hash)
  (data-to-file my-hash (concat (file-name-as-directory (getenv "HOME")) "my-hash.el")))

Then hash get saved:
#s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("SUMMARY" "This is my summary of the Org heading."))

And can be easily read back:

(let ((my-properties (data-from-file (concat (file-name-as-directory (getenv "HOME")) "my-hash.el"))))
  (gethash "SUMMARY" my-properties)) ➜ "This is my summary of the Org heading."

And by thinking that, it is also possible to make Org functions that save properties in separate Org file automatically. 

In case of the heading like this:

* My heading
  :PROPERTIES:
  :ID: 944334aa-acab-4e6a-8dd6-ebfd144eac6b
  :END:

One could have a separate Org file that is used only to write properties, and which could contain something like following:

* 944334aa-acab-4e6a-8dd6-ebfd144eac6b
  :PROPERTIES:
  :DATE-CREATED: 2022-12-18
  :PERSON-RELATED: Ihor Radchenko
  :END:

There are many reasons why properties should be hidden or separate
from Org file, apart from aesthetics and readability. I have been
sharing Org files and headings too many times to staff members
(without export), and then why should other person see clocked or
scheduled properties if such are not meant for that person?

The above eloboration demonstrates that properties could not only be
hidden from main view, but become structural and organized collection
of data. Storage is possible into PostgreSQL by using emacs-libpq
module or PostgreSQL packages, in Emacs 29 by using the built-in
SQLite or SQLite packages, or cdb database or key/value databases, or
by storing hashes into files, or by storing properties into separate
Org files.

Implementation is close to fingertips.

Generally, using properties and UUIDs as references to properties,
also liberates properties from Org mode and gives the power of
referencing to any kind of text or mode.

That means, having Markdown file with commented UUID, could allow the
user to have the same properties as in Org mode, one could freely tag,
clock-in, clock-out, schedule, deadline, etc. any kind of Markdown
heading, whatabout Asciidoc, truly plain text, HTML pieces, Wiki, and
other formats and files.

In normal text files, I would then make UUIDs invisible inside of
Emacs, when necessary:

(defun rcd-uuid-reference-make-invisible ()
  "Make all UUIDs in buffer invisible."
  (interactive)
  (save-excursion
    (goto-char (point-min))
    (while (search-forward-regexp thing-at-point-uuid-regexp nil t)
      (when (thing-at-point 'uuid)
	(facemenu-set-invisible (match-beginning 0) (match-end 0))))))

;; df2c9f4d-77ab-4697-842e-d7aa31ffeee3

If UUIDs are disturbing in the output, then I would have a filter to
export the file without UUIDs, when necessary.

And there is no need to go to property drawer to find properties, as
there is the function (org-property-values "ID") ➜
("944334aa-acab-4e6a-8dd6-ebfd144eac6b") which may be used in a
function and by using key binding one could quickly jump to properties
just as shown on the video, but without moving cursor to the UUID.

How to hide Org properties, I do not know, as the following function
can't work with font-lock in Org mode, that is something where you
could say how to make org-property-drawer-re invisible?

(defun rcd-org-drawer-make-invisible ()
  "Make Org property drawers invisible."
  (save-excursion
    (goto-char (point-min))
    (while (search-forward-regexp org-property-drawer-re nil t)
      (facemenu-set-invisible (match-beginning 0) (match-end 0)))))


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

[Feature] Store heading properties remotely, outside the Org file (was: Completely hide properties drawer in 9.6)

[Ihor Radchenko]

Below is the idea how one can store heading properties in a separate
file or even database.

I think it might be of interest for users who hate property drawer
clutter yet needed to store some metadata associated with Org headings.

To be consistent with Org internal logic, we may, instead of using ID,
use a separate :INCLUDE-PROPERTIES: heading property to define where the
additional heading properties are stored.

If a heading has :INCLUDE-PROPERTIES:, the properties will be calculated
according to: (1) heading property drawer; (2) INCLUDE-PROPERTIES value;
(3) parent headings, if property inheritance is enabled.

The value of INCLUDE-PROPERTIES might be a file:/id: link that may point to
another Org heading, or, potentially (maybe via third-party packages),
to SQL database.

Jean Louis <bugs@gnu.support> writes:

> To me it is very practical to have this simple heading with a drawer
> that is completely hidden, which references by using UUID to a complex
> set of properties. To somebody else, that may not be useful or is
> abstract to understand.
>
> In my work I would not like having more properties for Org heading but
> one, like this one:
>
> * My heading
>   :PROPERTIES:
>   :ID: 944334aa-acab-4e6a-8dd6-ebfd144eac6b
>   :END:
>
> And that alone would be a reference to any other property that I
> personally may need. It is a link from heading to relations to many
> different pieces of information. 
> ...
> Emacs 29 has SQLite built-in, there is PostgreSQL module and also
> rcd-hash-edit package that works in similar way as shown on
> video. Hashes may easily be stored in text files and read from text files. 
> ...
> And by thinking that, it is also possible to make Org functions that save properties in separate Org file automatically. 
>
> In case of the heading like this:
>
> * My heading
>   :PROPERTIES:
>   :ID: 944334aa-acab-4e6a-8dd6-ebfd144eac6b
>   :END:
>
> One could have a separate Org file that is used only to write properties, and which could contain something like following:
>
> * 944334aa-acab-4e6a-8dd6-ebfd144eac6b
>   :PROPERTIES:
>   :DATE-CREATED: 2022-12-18
>   :PERSON-RELATED: Ihor Radchenko
>   :END:
>
> There are many reasons why properties should be hidden or separate
> from Org file, apart from aesthetics and readability. I have been
> sharing Org files and headings too many times to staff members
> (without export), and then why should other person see clocked or
> scheduled properties if such are not meant for that person?
>
> The above eloboration demonstrates that properties could not only be
> hidden from main view, but become structural and organized collection
> of data. Storage is possible into PostgreSQL by using emacs-libpq
> module or PostgreSQL packages, in Emacs 29 by using the built-in
> SQLite or SQLite packages, or cdb database or key/value databases, or
> by storing hashes into files, or by storing properties into separate
> Org files.
>
> Implementation is close to fingertips.
>
> Generally, using properties and UUIDs as references to properties,
> also liberates properties from Org mode and gives the power of
> referencing to any kind of text or mode.
>
> That means, having Markdown file with commented UUID, could allow the
> user to have the same properties as in Org mode, one could freely tag,
> clock-in, clock-out, schedule, deadline, etc. any kind of Markdown
> heading, whatabout Asciidoc, truly plain text, HTML pieces, Wiki, and
> other formats and files.
>
> In normal text files, I would then make UUIDs invisible inside of
> Emacs, when necessary:
>
> (defun rcd-uuid-reference-make-invisible ()
>   "Make all UUIDs in buffer invisible."
>   (interactive)
>   (save-excursion
>     (goto-char (point-min))
>     (while (search-forward-regexp thing-at-point-uuid-regexp nil t)
>       (when (thing-at-point 'uuid)
> 	(facemenu-set-invisible (match-beginning 0) (match-end 0))))))
>
> ;; df2c9f4d-77ab-4697-842e-d7aa31ffeee3
>
> If UUIDs are disturbing in the output, then I would have a filter to
> export the file without UUIDs, when necessary.


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Completely hide properties drawer in 9.6

[Sterling Hooten]

So I've got a solution somewhat working but I'd like some help simplifying it.

The good news is that it's significantly faster than the previous Org
implementation I was using (the one you wrote on stack exchange), and
can now handle folding a file with 30k lines in around .065s for
hiding and .005s for showing. It does seem to be dependent on garbage
collection settings though.

I'm just going to walk through my solution and then ask some specific
questions about how to fix it.

Most of the work is actually in dealing with invisible edits and then
handling =org-return= when called in a heading with a hidden property
drawer.

Starting with a basic minor mode:

#+begin_src emacs-lisp
  (define-minor-mode org-hide-properties-mode
    "Toggle `org-hide-properties-mode'. Hide all property drawers in buffer and any further created."
    :init-value nil
    :lighter (" HP")
    (if org-hide-properties-mode
	(progn 
	  (org-fold-add-folding-spec 'property-drawer-hidden-edges
				     '((:ellipsis . nil)
				       (:isearch-open . t)
				       (:front-sticky . nil)
				       (:rear-sticky . nil))
				     nil
				     'append)
	  (org-fold-add-folding-spec 'property-drawer-hidden-interior
				     '((:ellipsis . nil)
				       (:isearch-open . t)
				       (:front-sticky . t)
				       (:rear-sticky . t))
				     nil
				     'append)
	  (advice-add 'org-insert-property-drawer :after #'org-fold-hide-property-drawer)
	  (org-fold-hide-property-drawer-all))
      (org-fold-show-property-drawer-all)
      (advice-remove 'org-insert-property-drawer #'org-fold-hide-property-drawer)))
#+end_src

In order to both have newly added properties automatically adopt the
invisibility text-property the interior characters of the properties
box needs to be sticky. But this conflicted with being able to type
with the point before the hidden text. To satisfy both these
requirements I applied =property-drawer-hidden-interior= as a folding
spec which was sticky to all but the first two and last two characters
of the property drawer. Then I applied =property-drawer-hidden-edges=
to these two remaining chunks. In this way you can add in new
properties and they'll be invisible, but typing at the edges appears
visibly.

Is there an alternative way of doing this without two separate folding
specs? Maybe an additional key for ":interior-sticky" which is sticky
on all characters sharing the same spec, but not sticky at the front
and rear?

I couldn't quite figure out what the :fragile setting was.

#+begin_src emacs-lisp
(defun org-fold--hide-property-drawers (begin end)
	"Hide all property drawers betweeen BEGIN and END."
	(org-with-wide-buffer
	  (goto-char begin)
	  (while (and (< (point) end)
		      (re-search-forward org-property-drawer-re end t))
    (with-silent-modifications
	     ;; Hide interior of drawer to be sticky so org-entry-put will respect invisibility
	     (org-fold-region (+ (match-beginning 0) 1) (- (match-end 0) 2) t 'property-drawer-hidden-interior)
	     ;; Hide first two and last two characters without sticky
	     (org-fold-region (- (match-beginning 0) 1) (+ (match-beginning 0) 1) t 'property-drawer-hidden-edges)
	     (org-fold-region (- (match-end 0) 2) (match-end 0) t 'property-drawer-hidden-edges)))))

(defun org-fold--show-property-drawers (begin end)
	"Unhide all property drawers in buffer between BEG and END"
	(org-fold-region begin end nil
			 'property-drawer-hidden-edges)
	;; SWH 2022-12-15 HACK because I'm using two diferent folding specs
	(org-fold-region begin end nil
			 'property-drawer-hidden-interior))

(defun org-fold-hide-property-drawer-all ()  
	"Hide all property drawers in visible part of current buffer."
	(org-fold--hide-property-drawers (point-min) (point-max)))
      
(defun org-fold-show-property-drawer-all ()
	"Unhide all property drawers visible part of current buffer"
	(org-fold--show-property-drawers (point-min) (point-max)))

(defun org-fold-toggle-property-drawer-all ()
	"Show or hide all property drawers in buffer."
	(interactive)
	(require 'org-macs)
	(if org-properties-hidden-p
	    (progn 
	      (setq org-properties-hidden-p nil)
	      (org-fold-show-property-drawer-all))
	  (setq org-properties-hidden-p t)
	  (org-fold-hide-property-drawer-all)))
#+end_src

Folding headings or subtrees:

#+begin_src emacs-lisp
      (defun org-fold-show-property-drawer (&optional arg)
	"Unhide property drawer in heading at point.
    With non-nil ARG show the entire subtree."
	(org-with-wide-buffer 
	 (org-back-to-heading)
	 (let* ((beg (point))
		(end (if arg (org-element-property :end (org-element-at-point))
		       (org-next-visible-heading 1)
		       (1+ (point)))))
	   (org-fold-region beg end nil
			    'property-drawer-hidden-edges)
	   (org-fold-region beg end nil
			    'property-drawer-hidden-interior))))

      (defun org-fold-hide-property-drawer (&optional arg)
	"Completely hide the property drawer in heading at point.
  With non-nil ARG hide the entire subtree."
	(org-with-wide-buffer 
	 (org-back-to-heading)
	 (let* ((beg (point))
		(end (if arg (org-element-property :end (org-element-at-point))
		       (org-next-visible-heading 1)
		       (1+ (point)))))
	   (org-fold--hide-property-drawers beg end))))
#+end_src

Is there a faster or better way of getting the boundary points for a
heading without the subheadings? I've wanted this feature in a few
other situations but don't know a clean way to fetch it. I suppose you
could also just have that the regex stop searching for the beginning
of the property box after 2 lines past the heading, since the
properties box needs to be in the first two lines.

These next sections are more opinionated in how to handle the edge
cases when dealing with hidden text that shouldn't be moved during
normal editing (e.g., if the property drawer has an empty line between
it and the headline, it isn't recognized as such).

My intention was to match default behavior as much as possible without
breaking the parsing of the property drawer or needing to reveal the
invisible text.

The default =org-end-of-meta-data= will go past any blank lines and
only stop at a non-empty line.  I would prefer for hitting <RETURN> on
a headline to act as if the planning line and properties box were
immutable, and just pass directly after them. For this I wrote a
version that will stop after the logbooks or property drawer.

Is there a better way to do this? I tried using the match-data but it
seemed unreliable (if there's nothing but blank lines after the meta
data it matches on the next heading).

#+begin_src emacs-lisp
  (defun swh-org-end-of-meta-data (&optional full)
    "Skip planning line and properties drawer in current entry.

  Differs from `org-end-of-meta-data' in that it stops after the first logbook.

  When optional argument FULL is t, also skip planning information,
  clocking lines and any kind of drawer.

  When FULL is non-nil but not t, skip planning information,
  properties, clocking lines and logbook drawers."
    (org-back-to-heading t)
    (forward-line)
    ;; Skip planning information.
    (when (looking-at-p org-planning-line-re) (forward-line))
    ;; Skip property drawer.
    (when (looking-at org-property-drawer-re)
      (goto-char (match-end 0))
      (forward-line))
    ;; When FULL is not nil, skip more.
    (when (and full (not (org-at-heading-p)))
      (catch 'exit
	(let ((end (save-excursion (outline-next-heading) (point)))
	      (re (concat "[ \t]*$" "\\|" org-clock-line-re)))
	  (while (not (eobp))
	    (cond ;; Skip clock lines.
	     ((looking-at-p re) (forward-line))
	     ;; Skip logbook drawer.
	     ((looking-at-p org-logbook-drawer-re)
	      (if (re-search-forward "^[ \t]*:END:[ \t]*$" end t)
		  ;; SWH 2022-12-19 Exit after finding the /first/ logbook
		  (progn (forward-line) (throw 'exit t))
		(throw 'exit t)))
	     ;; When FULL is t, skip regular drawer too.
	     ((and (eq full t) (looking-at-p org-drawer-regexp))
	      (if (re-search-forward "^[ \t]*:END:[ \t]*$" end t)
		  (forward-line)
		(throw 'exit t)))
	     (t (throw 'exit t))))))))
#+end_src

This tries to deal with hitting <RETURN> on a headline in a natural way:

#+begin_src emacs-lisp
      (defun org-return-newline-at-heading-skip-drawers (fun &rest args)
	"If at heading do nothing, if on heading skip the planning line,
  property drawer, logbook, and insert a newline after the last
  drawer or right before the first non-empty line."
	;; FIXME Should also handle the case when on planning line
	(if (not (org-at-heading-p))
	    (apply fun args)
	  ;; Store point so undo will put it back in heading
	  (push (point) buffer-undo-list)
	  ;; HACK to get entry to unfold when hidden from org-cycle
	  (org-fold-show-entry 'hide-drawers)
	  (swh-org-end-of-meta-data 'logbook)
	  (if (not (org-at-heading-p))
	      (if (eobp)
		  (org-return)
		(beginning-of-line)
		(newline)
		(forward-line -1))
	    (beginning-of-line)
	    (newline)
	    (forward-line -1))))

    (advice-add 'org-return :around #'org-return-newline-at-heading-skip-drawers)
#+end_src

I wanted the editing to be natural and the hidden text unobtrusive. I
added a new =org-catch-invisible-edits= flag of ='swh= and overwrote
=org-fold-check-before-invisible-edit= to accomodate this. To allow
for insertion before and after the hidden text I increased the
lookback for invisible characters from one to two. This allows for
having a trailing newline folded after the :END: and thus you can type
without appending to the end of property drawer (and messing up the
syntax). Deletion is still not allowed after the hidden text.

This is ugly but it seems to be working at the moment.

#+begin_src emacs-lisp
(defun org-fold-check-before-invisible-edit (kind)
  "Check if editing KIND is dangerous with invisible text around.
The detailed reaction depends on the user option
`org-fold-catch-invisible-edits'."
  ;; First, try to get out of here as quickly as possible, to reduce overhead
  (when (and org-fold-catch-invisible-edits
	     (or (not (boundp 'visible-mode)) (not visible-mode))
	     (or (org-invisible-p)
		 (org-invisible-p (max (point-min) (1- (point))))
                 ;; SWH 2022-12-19 Searching 2 characters back to allow for newlines in hidden property drawers
                 (org-invisible-p (max (point-min) (- (point) 2)))))
    ;; OK, we need to take a closer look.  Only consider invisibility
    ;; caused by folding of headlines, drawers, and blocks.  Edits
    ;; inside links will be handled by font-lock.
    
    ;; SWH 2022-12-14 Add property-drawer-hidden-edges to allow for completely hiding properties
    (let* ((invisible-at-point (org-fold-folded-p (point) '(headline drawer block property-drawer-hidden-edges)))
	   (invisible-before-point
	    (and (not (bobp))
	         (org-fold-folded-p (1- (point)) '(headline drawer block property-drawer-hidden-edges))))
           (invisible-before-point-2
	    (and (not (bobp))
	         ;; SWH 2022-12-19 counting invisble things 2 characters away to allow for newlines in hidden property drawers
                 (org-fold-folded-p (- (point) 2) '(headline drawer block property-drawer-hidden-edges))))
	   (border-and-ok-direction
	    (or
	     ;; Check if we are acting predictably before invisible
	     ;; text.
	     (and invisible-at-point (not invisible-before-point)
		  (memq kind '(insert delete-backward)))
             ;; Check if we are acting predictably after invisible text
             ;; This works not well, and I have turned it off.  It seems
             ;; better to always show and stop after invisible text.
             ;; (and (not invisible-at-point) invisible-before-point
             ;;  (memq kind '(insert delete)))
             )))
      (when (or invisible-at-point invisible-before-point invisible-before-point-2)
        (when (and invisible-before-point-2 (memq kind '(delete-backward)))
          (user-error "Can't delete after invisible areas is prohibited, make them visible first"))
        ;; (when (and invisible-before-point-2 (not invisible-before-point) (memq kind '(insert)))
        ;;   (message "SWH allowing modifying two places after invisible point editing"))
        (when (and (eq org-fold-catch-invisible-edits 'swh)
                   invisible-at-point (memq kind '(delete)))
          (user-error "Show hidden properties before editing invisible"))
	(when (or (eq org-fold-catch-invisible-edits 'error)
                  (and invisible-before-point (eq org-fold-catch-invisible-edits 'swh)))
	  (user-error "Editing in invisible areas is prohibited, make them visible first"))
	(if (and org-custom-properties-overlays
		 (y-or-n-p "Display invisible properties in this buffer? "))
	    (org-toggle-custom-properties-visibility)
	  ;; Make the area visible
          (if (and (eq org-fold-catch-invisible-edits 'swh)
		   border-and-ok-direction)
              (message "Allowing modifying before invisible point editing")
            (save-excursion
	      (org-fold-show-set-visibility 'local))
            (when invisible-before-point
              (org-with-point-at (1- (point)) (org-fold-show-set-visibility 'local)))
	    (cond
	     ((eq org-fold-catch-invisible-edits 'show)
	      ;; That's it, we do the edit after showing
	      (message
	       "Unfolding invisible region around point before editing")
	      (sit-for 1))
	     ((and (eq org-fold-catch-invisible-edits 'smart)
		   border-and-ok-direction)
	      (message "Unfolding invisible region around point before editing"))
             ((and invisible-before-point-2 (not invisible-before-point) (memq kind '(insert delete)))
              (message "Allowing modifying two places after invisible point editing"))
             (t
	      ;; Don't do the edit, make the user repeat it in full visibility
              (user-error "Edit in invisible region aborted, repeat to confirm with text visible")))))))))
#+end_src

I am also experiencing a weird (overlay?) problem with i-search. After
implementing when I search for things it will make all of the windows
greyed out, and then only show the matching characters.  I can see the
text again if I =mark-whole-buffer=. But I don't know what's causing
this.

Thanks again for your help, this is trickier than I initially thought
but I think it's coming together.

Re: Completely hide properties drawer in 9.6

[Ihor Radchenko]

Sterling Hooten <hooten@gmail.com> writes:

> In order to both have newly added properties automatically adopt the
> invisibility text-property the interior characters of the properties
> box needs to be sticky. But this conflicted with being able to type
> with the point before the hidden text. To satisfy both these
> requirements I applied =property-drawer-hidden-interior= as a folding
> spec which was sticky to all but the first two and last two characters
> of the property drawer. Then I applied =property-drawer-hidden-edges=
> to these two remaining chunks. In this way you can add in new
> properties and they'll be invisible, but typing at the edges appears
> visibly.
>
> Is there an alternative way of doing this without two separate folding
> specs? Maybe an additional key for ":interior-sticky" which is sticky
> on all characters sharing the same spec, but not sticky at the front
> and rear?

In theory, it is possible to add something like :interior-sticky.
However, it will be against how Emacs' sticky properties work.

An alternative approach you may use is hooking into font-lock.
Fontification will be re-applied upon editing, and you can hide only the
property drawer and nothing else. And you will not need to deal with
edge cases with inserting text.

> I couldn't quite figure out what the :fragile setting was.

:fragile makes sure that things are unfolded when a drawer is no longer
a drawer. Imagine:

<point>:drawer:<begin fold>
Some text.
:end:
<end fold>

Then, you delete ":":

drawer:
Some text.
:end:

:fragile defines rules allowing to auto-unfold the drawers (or anything
else) as needed.

We would not want the drawer above to remain folded as you would not be
able to use <TAB> to unfold any more - tab will no longer see a "drawer"
at point.

> #+begin_src emacs-lisp
>       (defun org-fold-show-property-drawer (&optional arg)
> 	"Unhide property drawer in heading at point.
>     With non-nil ARG show the entire subtree."
> 	(org-with-wide-buffer 
> 	 (org-back-to-heading)
> 	 (let* ((beg (point))
> 		(end (if arg (org-element-property :end (org-element-at-point))
> 		       (org-next-visible-heading 1)
> 		       (1+ (point)))))
> 	   (org-fold-region beg end nil
> 			    'property-drawer-hidden-edges)
> 	   (org-fold-region beg end nil
> 			    'property-drawer-hidden-interior))))
>
>       (defun org-fold-hide-property-drawer (&optional arg)
> 	"Completely hide the property drawer in heading at point.
>   With non-nil ARG hide the entire subtree."
> 	(org-with-wide-buffer 
> 	 (org-back-to-heading)
> 	 (let* ((beg (point))
> 		(end (if arg (org-element-property :end (org-element-at-point))
> 		       (org-next-visible-heading 1)
> 		       (1+ (point)))))
> 	   (org-fold--hide-property-drawers beg end))))
> #+end_src
>
> Is there a faster or better way of getting the boundary points for a
> heading without the subheadings? I've wanted this feature in a few
> other situations but don't know a clean way to fetch it. I suppose you
> could also just have that the regex stop searching for the beginning
> of the property box after 2 lines past the heading, since the
> properties box needs to be in the first two lines.

You can use the following at or inside a drawer:
 (org-element-lineage (org-element-at-point) '(section))

:begin and :end will contain current section boundaries. "section" is
all the text under the parent heading, up to the first child. The AST is:

(heading
 (section
   [(planning)]
   [(property-drawer)]
   [(paragraph1)]
   ...)
 (subheading)
 ...)

> The default =org-end-of-meta-data= will go past any blank lines and
> only stop at a non-empty line.  I would prefer for hitting <RETURN> on
> a headline to act as if the planning line and properties box were
> immutable, and just pass directly after them. For this I wrote a
> version that will stop after the logbooks or property drawer.
>
> Is there a better way to do this? I tried using the match-data but it
> seemed unreliable (if there's nothing but blank lines after the meta
> data it matches on the next heading).

You can just use `org-element-at-point'. At planning, `org-element-type'
will be 'planning. At property drawer - 'property-drawer. At drawer -
'drawer. :end property will include blank lines, but you can also check
:post-blank that will contain the number of blank lines at the end.
Check out `org--at-headline-data-p' implementation.

> I am also experiencing a weird (overlay?) problem with i-search. After
> implementing when I search for things it will make all of the windows
> greyed out, and then only show the matching characters.  I can see the
> text again if I =mark-whole-buffer=. But I don't know what's causing
> this.

No idea here. Need more information.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>