Re: #4 Org mode profiling meetup on Sat, May 7

[Ihor Radchenko <yantar92@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 378 bytes --]

Ihor Radchenko <yantar92@gmail.com> writes:

> Ihor Radchenko <yantar92@gmail.com> writes:
>
>> The time will be the same: 9pm SG time (4pm Kyiv; 2pm London; 9am New
>> York). Sat, May 7
>>
>> I will post the link to the meeting one hour before the meeting start.
>
> Meeting link: https://teamjoin.de/Org-dev-profiling-20220507-d708k

Attaching the meeting notes.

Best,
Ihor


[-- Attachment #2: 2022-05-07-summary.org --]
[-- Type: application/vnd.lotus-organizer, Size: 5058 bytes --]

# Created 2022-05-08 Sun 14:18
#+title: Ihor Radchenko [ML:Org mode] (2022) #4 Org mode profiling meetup on Sat, May 7
#+date: May 7, 2022
#+author: Ihor Radchenko

* Discussion summary

- org-ql (https://github.com/alphapapa/org-ql) contributions
  - I maintain a fork (https://github.com/yantar92/org-ql/) making use of recent org-element-cache improvements
  - A function to use org-ql as ~org-agenda-skip-function~
    See ~yant/org-agenda-skip-org-ql~ in https://github.com/yantar92/emacs-config/blob/master/config.org#archiving
- Max mentioned about =yank-handled-properties:= [[info:elisp#Yanking][elisp#Yanking]] to be used instead of =filter-buffer-substring-function= to handle copying invisible parts of Org buffers into non-Org buffers.
- Discussed weird behaviour of M-<RET> and C-<RET> when point is inside folded headlines in Org (reasons unclear)
- Discussed possible improvements to ~org-babel-tangle~ performance
  1. Not overwriting existing files when they are not going to be changed
     https://list.orgmode.org/878rrcws6q.fsf@localhost/T/#u
  2. Caching hash of the src blocks in .org buffer and not tangling them if the code inside remains unchanged
     - This will not work in cases when src blocks contain code with side effects:
       #+begin_src emacs-lisp
              # this code may yield different results depending on contents
              # of /tmp folder even when this code remains unchanged
              ls /tmp/
       #+end_src

* =org-fold-core=: Universal folding library for Emacs


Yup, it is not just for Org mode. It can be used by third-party code and has minimum dependencies from Org.

1. Can use text properties or overlays to hide text
   - Text properties are significantly faster than overlays in Emacs display engine
   - Attempts to optimise overlays are ongoing...
     https://lists.gnu.org/archive/html/emacs-devel/2016-09/msg00582.html
     ... and ongoing
     https://lists.gnu.org/archive/html/emacs-devel/2018-03/msg00565.html
     ... and still ongoing
     https://lists.gnu.org/archive/html/emacs-devel/2019-12/msg00115.html
     ... and it is already [2022-05-07 Sat]

!! *Read the commentary of org-fold-core.el*

1. Features
   - Folding/unfolding regions of text
   - Searching and examining boundaries of folded text (high-level)
   - Interactive searching in folded text (via isearch)
   - Handling edits in folded text
     - When using text properties, default functions like ~insert~ do not inherit text properties
       We need special handling of buffer edits to keep text inserted in the middle of the fold invisible
   - Killing/yanking (copying/pasting) of the folded text
     - We do not want folded text to carry over the hidden state when copying to other buffers
     - Or sometimes we do
   - Fontification of the folded text
     - Large chunks of hidden text are extremely slow to fontify

   *All the above features are configurable using different fold types*: see ~org-fold-core--specs~ docstring.

2. API symbols to check out
   - Initialisation
     ~org-fold-core-style~; ~org-fold-core--specs~; ~org-fold-core-initialize~
   - Folding/unfolding
     ~org-fold-core-region~
   - Working with folds
     ~org-fold-core-folded-p~; ~org-fold-core-region-folded-p~;
     ~org-fold-core-get-folding-spec~; ~org-fold-core-get-folding-specs-in-region~;
     ~org-fold-core-get-region-at-point~
     ~org-fold-core-next-visibility-change~; ~org-fold-core-next-folding-state-change~;
     ~org-fold-core-search-forward~;
   - Editing folds
     When we hide text using text properties, extra care should be taken when inserting text inside
     - org-fold takes care about this, but at the cost of slowdown
       - See =:front-sticky=, =:rear-sticky= in ~org-fold-core--specs~
     - Ideally, use ~insert-and-inherit~ + ~org-fold-core-ignore-modifications~ to preserve folding state around
     - Sometimes, we want to attract user attention when edits cause major change related to folded text
       - =:fragile= in ~org-fold-core--specs~
   - Performance optimisations
     ~org-fold-core--optimise-for-huge-buffers~
     - What can be done to reduce load on Emacs redisplay engine in huge (10Mb+) buffers
     ~org-fold-core-ignore-fragility-checks~
     ~org-fold-core-ignore-modifications~
     - Macros to know about for developers
   - Isearch integration (isearch does not search inside 'invisible text properties by default)
     - See ~org-fold-core--isearch-setup~
     - =:isearch-ignore= and =:isearch-open= in ~org-fold-core--specs~
   - Clearing folded state when text is copied interactively
     When using text properties, hidden text will remain hidden when inserted in other buffers
     Implementation: ~org-fold-core--buffer-substring-filter~
     Use ~filter-buffer-substring~ instead of usual ~buffer-substring~
     - Also, see =:global= in ~org-fold-core--specs~
   - Font-lock interaction
     It does not make sense to fontify folded text
     - ~org-fold-core-fontify-region~
     - Also see =:font-lock-skip= in ~org-fold-core--specs~