Ihor Radchenko writes: > Ihor Radchenko writes: > >> Instead of fontifying elements individually via regexps, we can leverage >> org-element-map, org-element-parse-buffer, org-element-cache, and >> jit-lock-mode. Each type of Org element/object can be assigned with a >> fontification function accepting a single argument - the element datum. > > I have been working on the new fontification library for the last > several months and I have noticed several confusing things in the > current fontification settings. It has been a while since the last update on the fontification... for a reason. As the fontification should be very tightly optimized to avoid Emacs freezes, I stumbled upon a need to improve the performance of Org parser further than we already have. This triggered a number of significant changes in listp/org-element.el. So many that the current logic of the library becomes a mess. In order not to turn org-element into another org-agenda, I am proposing to factor out Org syntax tree API into a separate file org-element-ast.el and add a number of breaking changes how the syntax tree is structured. The most important changes are the following: 1. Frequently used element properties will no longer be stored directly in the property list. Instead, they will be placed inside a special vector that is much faster to access. `org-element-property' and other accessor functions are changed accordingly to inline the property queries into `aref' calls, when possible. This is a major breaking change. 2. Org parser will now be able to parse elements partially, with some parts of the parser executed later, only when necessary. The downside is relying upon the original buffer to be live even after parsing. This is also a major breaking change. 3. The order of multiple affiliated keywords in Org parse tree will be reversed. The main purpose here is to avoid special cases when fine details of Org syntax had to be accounted for when traversing Org parse trees. This is a breaking change. 4. Org cache will be available even when `org-element-use-cache' is non-nil. This is not a breaking change and simplifies Org code greatly without compromising performance. 5. The usage of regular expressions is by Org parser is now tightly optimized. See the discussion in https://debbugs.gnu.org/cgi/bugreport.cgi?bug=63225 This is also not a breaking change. 6. Memory footprint is reduced for Org AST. This is achieved using shared string objects in Org parser. With low probability, this might cause problems if third-party code modifies these string objects by side effect. This refactoring is large, spanning >80 commits. The full commit log can be found at https://git.sr.ht/~yantar92/org-mode/log/feature/org-element-ast-tidy Here, I am attaching ORG-NEWS diff, summarizing important changes. I am also adding the commentary section of org-element-ast.el with more details about the new syntax tree structure and about the new concept of deferred parsed values. I am sure that I missed things, so comments are welcome.