[SUMMARY] #7 [[bbb:OrgMeetup]] on Wed, Apr 24, 19:00 UTC+3

[Ihor Radchenko <yantar92@posteo.net>]

Here is a summary of the discussed topics + comment log:

- Tim reported Org mode hanging (potential infinite loop) when setting
  tags in a new capture near the end of a big Org mode buffer.

  After ~M-x toggle-debug-on-quit~, the potential offender is
  ~org-element-cache-map~:

  : Debugger entered--Lisp error: (quit)
  : org-element-at-point(1935572)
  : org-element-cache-map(#f(compiled-function (el) #<bytecode -0x9c38e0188a800ce>))
  : org-get-buffer-tags()
  : org-set-tags-command(nil)
  : funcall-interactively(org-set-tags-command nil)
  : command-execute(org-set-tags-command)

  The bug is intermittent - hard to reproduce on demand (so, not possible
  to debug during the meetup via screen share).

  - This kind of bugs is not easy to debug without a reproducer
  - Since the bug was reported on stable version of Org mode, the
    potential things to try (and hope for the best):
    1. Try the latest dev version of Org mode
    2. Try ~(setq org-element-use-cache nil)~
    3. If nothing helps, report to the mailing list where we can try
       harder to fix the problem

- Kathink reported a cache error when expanding =yasnippet= snippets for
  LaTeX fragments.

  - At this point, the cache does not have /obvious/ bugs. The remaining
    are pretty peculiar and are often caused by some specific changes
    in the cache state.
  - Org mode provides a way to record cache state log, so that we can
    diagnose these difficult bugs. One can enable it by adding the
    following to Emacs config

    : (setq org-element--cache-self-verify 'backtrace)

    Then, the cache warning message will be supplied by very, /very/
    long cache log. This log is to be submitted to Org developers.

    Preferably, if it is possible to trigger the bug simply by
    tinkering around the problematic Org file, also do ~M-x
    org-element-cache-reset~ before triggering the bug, so that the log
    size (which can be a few Mbs easily) is not as overwhelming.

- Karthink also reported lags when he edits an Org mode buffer with
  LaTeX equations.

  ~M-x profiler-start~ / ... / ~M-x profiler-report~ revealed that the cause
  is Org mode's LaTeX fontification.

  This is a known problem happening in /some/ Org files where Emacs
  regexp engine has a particularly bad time matching
  ~org-latex-and-related-regexp~ against the buffer text.

  The fix would be either (1) simplifying
  ~org-latex-and-related-regexp~, although not clear how - someone
  experiencing this particular problem would need to play around with
  it; (2) waiting until we rewrite Org mode fontification to use
  parser: https://orgmode.org/list/87ee7c9quk.fsf@localhost.

- Karthink further asked about his bug report about slowdown when
  killing Org buffers: https://list.orgmode.org/orgmode/874jbz9xvl.fsf@gmail.com/
  - I already replied on the list
  - I suspect that there is some O((N^2) complexity manifesting itself
    when using the new LaTeX preview system (Kathink is the developer
    of the new preview)
    - The new LaTeX preview generates a large number of cache entries,
      while org-persist library looks up for cache duplicates using a
      simple ~memq~ leading to O(N^2)
    - See ~cl-pushnew~ call in ~org-persist-write-all~
    - But first need to know the actual number of cache entries to be
      sure that my suspicion is right

- I briefly mentioned the latest status of upstreaming the new LaTeX
  preview system.  Karthink is the co-author of it.

  I am starting a formal review of the preview code, piece by piece
  (the LaTeX preview branch introduces a number of completely new
  sub-features): https://list.orgmode.org/87a5llzmco.fsf@localhost/T/#u
  The first set of comments was regarding changes in the core Org
  export API.
  
  Karthink is not the author of this particular change though. So, we
  are still waiting for Timothy to reply.

- Tim (I think it was Tim) asked if it is possible to execute, say,
  all *bash* src blocks in Org buffer. Or, more generally, all the src
  blocks with a particular language.

  I quickly wrote a modification to the existing
  ~org-babel-execute-buffer~, adding a "DWIM" behavior
  "When point is at src block, only execute src block with the same
  language."

  If such addition is of interest to other Org users, we might
  consider something similar to be upstreamed.

  #+begin_src emacs-lisp
  (defun org-babel-execute-buffer-dwim (&optional arg)
    "Execute source code blocks in a buffer.
  Prefix argument ARG is passed to `org-babel-execute-src-block'.
  Call `org-babel-execute-src-block' on every source block in
  the current buffer.
  When point is at src block, only execute src block with the same language."
    (interactive "P")
    (org-babel-eval-wipe-error-buffer)
    (let* ((current-element (org-element-context))
  	 (current-src-block-language
            (and (memq (org-element-type current-element) '(src-block inline-src-block))
  	       (org-element-property :language current-element))))
      (org-save-outline-visibility t
        (org-babel-map-executables nil
  	(when (or (not current-src-block-language)
  		  (equal current-src-block-language
  			 (org-element-property :language (org-element-context))))
  	  (if (memq
  	       (org-element-type (org-element-context))
                 '(babel-call inline-babel-call))
                (org-babel-lob-execute-maybe)
              (org-babel-execute-src-block arg)))))))
  #+end_src

- I did a demo, showing my workflow merging a feature request for Org mode

  1. I started from looking at my agenda view that lists all the
     recent emails/links related to Org mode development.
     One of the todo items was a new feature submitted for oc-biblatex library: 
     https://list.orgmode.org/87a5lgux9z.fsf@localhost/T/#t

  2. I clocked in, and opened the linked email submitted to our
     mailing list (https://orgmode.org/worg/org-mailing-list.html,
     https://orgmode.org/worg/org-contribute.html) in Emacs, using
     notmuch.el (MUA I am using).

  3. I applied the patch attached to the email using M-x piem-am
     command, provided by https://git.kyleam.com/piem/about/ package.

     The command automatically creates a new branch in my local
     org-mode repository, applies the patch on top of it, and opens
     Magit buffer displaying the branch.

     - Karthink commented that there is an alternative approach to
       apply patches from emails. One can simply use M-| (M-x shell-command-on-region)
       from inside the email buffer, and enter git am command. This is the same as
       doing cat patch | git am from command line.

  4. Now, I can conveniently review the patch, jumping to the full Org
     mode source code if necessary or run tests on the local modified branch.

  5. I can verify that the patch did not break anything using C-x p c
     (M-x project-compile) make test command, making use of the
     built-in project.el to run Org mode tests.

  6. The patch I was looking into did not have the correctly formatted
     Changelog entries
     (https://orgmode.org/worg/org-contribute.html#commit-messages),
     so I went ahead with modifying the commit message: c w ([c]ommit
     [r]eword) from Magit status buffer.

     While editing the commit message, I could use "C" binding from
     the diff window displayed alongside the commit message to insert
     automatically formatted ChangeLog entries into the commit message
     (see https://orgmode.org/worg/org-contribute.html#orga05f6ae)
     The ChangeLog entries have the following format:

     : * <file changed> (<function name>):

     Example:

     : * lisp/org-capture.el (org-capture-set-plist):

     - Kathink asked how Magit knows the function names from git

       - AFAIK, git itself has a database of regular expression to
         determine the function name for various programming
         languages. Including Elisp.

       - That said, I still modify the defaults a little bit, to
         include ;;; Comment headers and el-patch-... macros, and
         produce ChangeLog for changed .org files

	 #+begin_src conf
	 [diff "lisp"]
	 xfuncname = "^(((;;;+ )|\\(|([ \t]+\\(((cl-|el-patch-)?def(un|var|macro|method|custom)|gb/))).*)$"

	 [diff "org"]
	 xfuncname = "^(\\*+ +.*)$"
         #+end_src

  7. The patch did not break any tests, but the manual review revealed
     that the patch did not have ORG-NEWS entry and a bit vague
     addition to the manual, despite introducing a new feature.

     So, I switched away from the patch-specific branch in my local
     Org repo and replied to the original email asking to make modifications.

  8. I captured the reply email using https://orgmode.org/manual/Capture-templates.html
     into my inbox, marked it as "WAITING" (for a reply), and
     immediately refiled
     (https://orgmode.org/manual/Refile-and-Copy.html) it under my Org
     mode development subtree in the notes.
     
     - I use a custom helm+org-ql command to quickly select refile
       target. See ~yant/helm-org-ql-refile~ in https://github.com/yantar92/emacs-config/blob/master/config.org#helm-org-ql

     - Karhink asked about the refile target criteria I am using
       - It is a custom org-ql query
	 : '(and (not (path "rss.org" "schedule.org"))
         :       (not (tags-local "NOREFILE"))
         :       (tags-local "REFILE" "goal"))
	 - The main idea is to mark refile targets explicitly with a =REFILE= tag
	 - I usually do not set the refile tag manually - it is simply
           a part of my capture templates for projects/areas of interest:
	   https://github.com/yantar92/emacs-config/blob/master/config.org#area-of-interest

	   - The concept of "Area of interest" is coming from GTD and
             P.A.R.A. methods
             (https://tasshin.com/blog/implementing-a-second-brain-in-emacs-and-org-mode/)
	     Unlike "projects" that have a defined finish criteria,
	     "areas" are a continuous effort that should be sustained.
	     Examples: maintaining Org mode, maintaining health,
	     keeping the house clean, keeping track of important world
	     news, etc

- Near the end, we deviated a bit from Org mode, and discussed orderless completion framework

  - I am now playing around with orderless
    (https://github.com/oantolin/orderless) and mct
    (https://protesilaos.com/emacs/mct) in place of Helm
    - What I found annoying is the completion in ~find-file~ that puts
      longer matching file names before shorter files. For example,
      when matching =org= in Org mode repository, I had =org-capture.el=
      listed before =org.el=
      - I solved with using a custom ~completions-sort~ function that
        sorts by Levenshtein distance from the minibuffer input:
	https://github.com/yantar92/emacs-config/blob/master/config.org#show-recent-completions-first-then-by-levenshtein-distance-to-input

      - Kathink commented that he has no such problem when using
        vertico (https://github.com/minad/vertico)

- At the very end there was a complaint that the meetup was not listed in Emacs News
  - Something to ping Sacha next time.


:comments:
[18:51] Welcome to <b>[[bbb:OrgMeetup]]</b>!<br /><br />For help on using BigBlueButton see these (short) <a href="https://www.bigbluebutton.org/html5" target="_blank"><u>tutorial videos</u></a>.<br /><br />To join the audio bridge click the phone button.  Use a headset to avoid causing background noise for others.<br /><br />This server is running <a href="https://docs.bigbluebutton.org/" target="_blank"><u>BigBlueButton</u></a>.
[18:51] Ihor Radchenko : The latest Emacs News: https://sachachua.com/blog/2024/04/2024-04-22-emacs-news/
[18:51] Ihor Radchenko : (I accidently started a session in different room. Oops)
[18:52] Ihor Radchenko : Official start in 10 minutes :)
[18:59] Tim : Good evening, this is Tim.
[19:01] karthink : Hello all
[19:04] Tim : Debugger entered--Lisp error: (quit)
    org-element-at-point(1935572)
    org-element-cache-map(#f(compiled-function (el) #<bytecode -0x9c38e0188a800ce>))
    org-get-buffer-tags()
    org-set-tags-command(nil)
    funcall-interactively(org-set-tags-command nil)
    command-execute(org-set-tags-command)
[19:11] Ihor Radchenko : 1. Try the latest dev version of org
2. Try ~(setq org-element-use-cache nil)~
[19:31] karthink : I got an org-element backtrace for the yasnippet error
[19:40] Ihor Radchenko : (defun org-babel-execute-buffer-dwim (&optional arg)
  "Execute source code blocks in a buffer.
Prefix argument ARG is passed to `org-babel-execute-src-block'.
Call `org-babel-execute-src-block' on every source block in
the current buffer.
When point is at src block, only execute src block with the same language."
  (interactive "P")
  (org-babel-eval-wipe-error-buffer)
  (let* ((current-element (org-element-context))
	 (current-src-block-language
          (and (org-element-type-p current-element '(src-block inline-src-block))
	       (org-element-property :language current-element))))
    (org-save-outline-visibility t
      (org-babel-map-executables nil
	(when (or (not current-src-block-language)
		  (equal current-src-block-language
			 (org-element-property :language (org-element-context))))
	  (if (org-element-type-p
	       (org-element-context) '(babel-call inline-babel-call))
              (org-babel-lob-execute-maybe)
            (org-babel-execute-src-block arg)))))))
[19:41] Ihor Radchenko : The above is for the latest dev version
[19:42] Ihor Radchenko : (defun org-babel-execute-buffer-dwim (&optional arg)
  "Execute source code blocks in a buffer.
Prefix argument ARG is passed to `org-babel-execute-src-block'.
Call `org-babel-execute-src-block' on every source block in
the current buffer.
When point is at src block, only execute src block with the same language."
  (interactive "P")
  (org-babel-eval-wipe-error-buffer)
  (let* ((current-element (org-element-context))
	 (current-src-block-language
          (and (memq (org-element-type current-element) '(src-block inline-src-block))
	       (org-element-property :language current-element))))
    (org-save-outline-visibility t
      (org-babel-map-executables nil
	(when (or (not current-src-block-language)
		  (equal current-src-block-language
			 (org-element-property :language (org-element-context))))
	  (if (memq
	       (org-element-type (org-element-context))
               '(babel-call inline-babel-call))
              (org-babel-lob-execute-maybe)
            (org-babel-execute-src-block arg)))))))
[19:42] Ihor Radchenko : This version works on Org stable
[19:49] Ihor Radchenko : https://git.kyleam.com/piem/about/
[19:58] Ihor Radchenko : #+begin_src conf
[diff "lisp"]
  xfuncname = "^(((;;;+ )|\\(|([ \t]+\\(((cl-|el-patch-)?def(un|var|macro|method|custom)|gb/))).*)$"

[diff "org"]
  xfuncname = "^(\\*+ +.*)$"

#+end_src
[20:32] karthink : Thank you
[20:33] Ihor Radchenko : Thanks everyone!
:end:

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>