* #8 [[bbb:OrgMeetup]] on Wed, June 12, 19:00 UTC+3
@ 2024-05-22 14:30 Ihor Radchenko
2024-06-22 8:32 ` [SUMMARY] " Ihor Radchenko
0 siblings, 1 reply; 6+ messages in thread
From: Ihor Radchenko @ 2024-05-22 14:30 UTC (permalink / raw)
To: emacs-orgmode
Dear all,
Another OrgMeetup will be scheduled on the second Wednesday of June,
in three weeks.
Previous meetup notes:
https://list.orgmode.org/orgmode/87edanyzme.fsf@localhost/T/#u
URL: https://bbb.emacsverse.org/b/iho-h7r-qg8-led
Time & Date: <2024-06-12 Wed 19:00-21:00 @+03,Europe/Istanbul>
The room will be open half an hour before the official start.
During the meetup, we can:
- Give advice to new users
- Showcase Org configs or workflows
- Demo/discuss interesting packages
- Troubleshoot each-other's issues
- Discuss Org mode development
- Discuss "Org mode" section of Emacs news (https://sachachua.com/blog/)
- Discuss anything else Org-related
Everyone is free to join the discussion/chat or lurk around silently,
listening.
We will _not_ do any recording by default.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 6+ messages in thread
* [SUMMARY] #8 [[bbb:OrgMeetup]] on Wed, June 12, 19:00 UTC+3
2024-05-22 14:30 #8 [[bbb:OrgMeetup]] on Wed, June 12, 19:00 UTC+3 Ihor Radchenko
@ 2024-06-22 8:32 ` Ihor Radchenko
2024-06-27 12:45 ` Pedro
0 siblings, 1 reply; 6+ messages in thread
From: Ihor Radchenko @ 2024-06-22 8:32 UTC (permalink / raw)
To: emacs-orgmode
It was a long discussion this time (almost 3 hours).
We ended up talking in-depth about Org mode development and some very
technical aspects of it.
- As usual, we started from posting Sacha's News, but with a bit of a twist
- Since recently, https://orgmode.org shows Org mode-related news on the front page
- Mailing list feature discussions, polls, and announcements
- Org mode section of Sacha's News
- Suhail asked about the status of 3 bugs reports posted on Org mailing list recently
- https://yhetil.org/orgmode/87h6e134sp.fsf@gmail.com/
- This one is about completion of :async header argument in
"shell" blocks, where "shell" is not literally shell, but
"bash", "csh", etc. The completion only works for "shell"
blocks, but not for other blocks.
- This is simply an omission in ob-shell.el
- ob-shell is implemented in the following way
1. Generic "shell" backend is defined
2. Other backends like "bash", "csh", "fish" (full list in
~org-babel-shell-names~) are defined based on the generic
backend in ~org-babel-shell-initialize~
3. ~org-babel-shell-initialize~ fails to define
~org-babel-header-args:<shell name>~ - variable used to
compute completions.
4. The fix is easy; just need time to get there.
- [2024-06-22 Sat] Fixed.
- https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=e666660c
- https://yhetil.org/orgmode/87cyop3445.fsf@gmail.com/
- This is a similar issue related to ~org-babel-shell-initialize~
- This time, ~org-babel-<shell name>-initiate-session~ is not
defined for specific shells
- The problem is more severe than completion though. It renders
=C-u C-c '= (~org-edit-special~) unusable.
- ~org-edit-special~ /without/ prefix argument opens code block at point for editing
- ~org-edit-special~ /with/ prefix argument should instead open
associated session for code block with =:session ...= header arg
- Because of the bug, this does not currently work for shell blocks
- [2024-06-22 Sat] Fixed.
- https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=5b366a73
- https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=5e9ac146
- https://yhetil.org/orgmode/87cyonhuq3.fsf@gmail.com/
- This report is about regression of =#+bind: ...= keywords in the new Org 9.7
- Fixed. https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=96113f3b5
- Karl Voit (publicvoit) shared his recent bug report for helm-org-rifle
- https://github.com/alphapapa/org-rifle/issues/85
- The problem he experiences is with parts of outline path being
invisible in the candidates buffer as long as they are folded
- The likely reason with the way folding is done in Org mode 9.6.
Org 9.6 (and Org 9.7 for Emacs <29) use 'invisible text property
to fold headings. If such folded headings are copied verbatim from
the Org buffer, they may remain invisible.
- If my guess is right, setting ~org-fold-core-style~ to ~overlays~
should make things work again in helm-org-rifle
- Note that Org 9.7 now moves towards restoring the old approach -
using overlays to fold staff. This is because built-in isearch.el,
query-replace-regexp, and a number of external packages (like
evil) only support searching (and revealing!) invisible text when
it is hidden via overlays. (I was hoping that it can be possible
to work around such limitation, but constant stream of bug reports
and growing pile of fragile workarounds proved that text hidden
via text properties cannot reliably searched in practice; alas...)
- As an unfortunate side effect of this, I had to revert feature
introduced in Org 9.6 - searching inside hidden parts of the
links
- See https://orgmode.org/Changes.html (It is no longer possible
to reveal hidden parts of the links during isearch)
- We went on talking about org-ql vs. helm-org-rifle
- org-ql is generally more flexible as it allows more than just
plain text + outline search
- In org-ql, one can quickly match headings by specific
properties, tags, timestamps, etc
- In addition, org-ql, since recently, has integration with
built-in completion frameworks, and their extensions like vertico.
- Of course, helm interface is also there - helm-org-ql
- However, as it turns out, org-ql lacks one important feature Karl
particularly likes.
In helm-org-rifle, when you enter search string interactively, if
the match is inside a heading, the line containing match is
displayed in addition to the outline path to the heading. This
provides a valuable additional context - you can briefly look what
exactly inside a heading is matched.
- [2024-06-15 Sat] alphapapa, in
https://github.com/alphapapa/org-rifle/issues/85, pointed out
that one can set ~org-ql-completing-read-snippet-function~ to
~org-ql-completing-read-snippet-function~ - this will replicate
what helm-org-rifle does in ~org-ql-completing-read~ (but not in
~helm-org-ql~).
- For ~helm-org-ql~ one may need to fiddle with the source code of
~helm-org-ql-source~ to add this kind of feature
- Tim continued his struggle with an elusive bug he experiences when
setting tags (C-c C-q) in large Org buffers - setting tags takes
forever for him. (There is infinite loop likely lurking somewhere)
- His last observation was that setting ~org-element-use-cache~ to nil
or running =M-x org-element-cache-reset= makes settings tags work
again and not hang (for some time, at least)
- I can guess two possibilities now:
1. Some problem with org-element-cache mishandling headline boundaries
2. Infinite loop in ~org-element-cache-map~
https://list.orgmode.org/orgmode/CH3PR84MB342450FE12DC716A9FA65EADC50D2@CH3PR84MB3424.NAMPRD84.PROD.OUTLOOK.COM/
- For now, I asked Tim to enable cache self-checks (~org-element--cache-self-verify~)
- If something shows up there, we got a problem in the cache
- Otherwise, it is likely ~org-element-cache-map~
- In any case, ~org-element-cache-map~ should be rewritten and simplified sooner than later
- Karl shared his config snippet that allows updating link description
according to its source
https://graz.social/@publicvoit/112582948662128779
- For Urls, it fetches the web page title
- For Org id: links, it looks up the linked heading title
- He had a technical question about how to update the link description from Elisp
- Now, he uses interactive
: (org-insert-link nil nil myheadingtitle);; add description
- The above works, but it is too interactive - this particular
call to ~org-insert-link~ will query for the link path, happening
to use path at point as the default. So, Karl has to hit <RET>
<RET> needlessly for confirmation.
- To make ~org-insert-link~ work non-interactively (from Elisp), one
needs to supply two optional arguments: LINK-LOCATION and DESCRIPTION
: (org-insert-link &optional COMPLETE-FILE LINK-LOCATION DESCRIPTION)
See the docstring for details.
- To retrieve link location at point, it is possible to use Org element API
https://orgmode.org/worg/dev/org-element-api.html
: (org-element-property :raw-link (org-element-context))
- Also, I noticed that another part of the "auto-description" code
uses a function that is not designed for Elisp API:
~org-open-at-point~
- ~org-open-at-point~ fiddles with window arrangement and using is
from Elisp may cause unexpected results
- Instead, it is better to use ~org-link-open~ that only jumps to
the link target (as defined by :follow link parameter):
: (org-link-open (org-element-context))
- Since karthink (one of the authors of the WIP async LaTeX preview
feature) was around, I shared my major WIP work on Org mode development.
(My work will affect their WIP branch)
For the last half year, I am working on a large refactoring
touching all parts of Org mode code. The main objectives are:
1. Avoid situations when loading almost *any* part of Org mode
requires loading the *whole Org mode codebase*
2. Make Org mode libraries less scary to contribute to by reducing
their size.
3. Minimize uses of global variables when they are not strictly
necessary. Or, at least, document them.
In practice, it means splitting Org mode into smaller sub-libraries
and carefully reviewing all the ~require~, ~declare-function~, and
dynamically scoped ~defvar~ statements. (this may sound easy, I
initially hoped to spend a few weeks max on this task; but here we
are, 6 months later, 400+ commits ahead of main)
The WIP branch is available at
https://git.sr.ht/~tartan92/org-mode/log/feature/refactor-deeps-v2
(yup, it is already the second attempt; first several tries failed
miserably with Org mode in completely broken state)
- The most visible changes in the branch are many, *many* new
libraries. I separated huge old files like =org.el=, =org-agenda.el=,
=org-table.el=, =org-list.el=, =ob-core.el=, =org-clock.el=, and
=org-compat.el= into many smaller helper libraries.
- This includes the existing LaTeX preview code being put into a
separate =org-preview-latex.el=, which may create problems when
merging WIP async preview branch and refactored Org version.
- Overall 127 Org libraries will be turned into 263 (WIP!) smaller
libraries. The whopping +135 libraries more...
- Org major mode will leave in separate =org-mode.el=, which only
loads strictly necessary libs, without options user commands or
extra Org functionality. All the extras will be autoloaded
instead - when a key binding is pressed.
- =org.el= will still load everything, so that old packages and
configs that do ~(require 'org)~ or ~(use-package org...)~ are not
broken. Eventually, we will ask users to avoid loading =org.el=
and instead load more specific libraries.
- Many external optional libraries will also be loaded just-in-time,
when their functionality is needed
- For example, =ol-gnus.el= - the library implementing links to GNUS
buffers, will no longer load the whole GNUS when Org mode is
loaded. It will instead do it when we need to follow an actual
gnus: link.
- The way it is done is placing ~(require '<third-party-lib>)~ not
directly as a top-level form in =org-lib.el=, but inside
individual functions in that lib.
- Suhail commented that top-level ~(require ...)~ are treated
differently by the byte-compiler implying that something may
not be right if we move things inside functions
- This is not a problem though. The main effect of top-level
require is that byte-compiler loads that library at the
compile time, and knows which variables and functions are
defined there.
- The main effect of putting requires into functions is that
we get a bunch of compile-time warnings as byte-compiler
does not know functions/variables coming from third-party
libs and complains about them
- It can be suppressed by ~(defvar <name>)~ or
~(declare-function ...)~ forms.
- karthink also commented that runtime ~(require '<library>)~ may
hit performance because is must search through the whole
~features~ list (list of all the libraries loaded into current
Emacs session)
- It is not a big deal in practice though
: (benchmark-run 10000 (featurep 'does-not-exist))
: ;; => 0.1 sec or 10 µs per require
- Even if it were, we could ask Emacs devs to optimize this by
caching loading status in the library symbol itself
: (put 'foo :loaded t)
: (get 'foo :loaded) ; => t
- One more aspect of the changes is attempting yet another fix for
the everlasting problem with "mixed" Org mode installations we
face every major Org release - when some Org libs are loaded from
built-in Org mode version while other libs are loaded from newer
Org. This often results in random breakage people keep reporting.
- Emacs 30 recently introduced a tool to address this problem somewhat:
~require-with-check~. Unlike simple ~require~, ~require-with-check~
checks if a different version of the requested library has been
added to the ~load-path~, and re-loads it as necessary. This way,
if newer Org gets loaded, it can make sure that the versions of
Org sub-libraries are all coming from the correct version of Org mode.
- The problem with this idea is that Org codebase does not
currently use ~require~ everywhere. Instead, many places in Org
simply rely upon other Org libraries already loaded
(~declare-function~ statements are everywhere).
And we cannot simply add more ~require-with-check~ everywhere -
many Org libs struggle from circular dependencies. For
example, =org-agenda.el= requires =org.el= to be loaded and =org.el=
expects =org-agenda.el= to be loaded.
- Untangling these circular deps has proven to be the hardest
part of the ongoing refactoring.
- Splitting Org into multiple small libraries is also a step forward
toward more modular Org and to converting some Org-specific
libraries to proper Emacs libraries, as requested by RMS:
https://list.orgmode.org/E1kIkxv-0007iy-Av@fencepost.gnu.org/
- Karthink asked which libraries would be good candidates for upstreaming.
Here is the list, naming the existing libraries, and the
libraries that are a part of WIP branch:
- org-agenda-undo :: undoing changes in multiple buffers at once
This is the way undo works in agenda buffers, undoing changes
in multiple headlines changed from agenda, even when they live
in different Org files.
- org-capture :: Adding quick notes is not limited to adding
them to Org mode; in fact, org-capture is a development of the original remember.el, which is a part of Emacs core.
See https://lists.gnu.org/archive/html/emacs-orgmode/2023-12/msg00412.html
- org-clock-notify :: System notifications
- org-diary-lib :: Org extensions for diary-lib.el
- org-duration :: Library to manipulate durations in
human-readable format (like 1d 3h 5min)
- org-fold-core :: Generic folding API
- org-idle :: Advanced idle time tracking (not just for Emacs
idleness, but for system idle time as well)
- org-macs :: has many small helper functions; some may be of
interest upstream
- org-persist :: generic cache API
- org-preview-image :: Image preview. Does not have to be
limited to Org mode buffers.
- org-preview-latex :: Same thing.
- org-protocol :: Processing external input into Emacs (from
command line, from clicked Urls as url handler). Unlike
running commands directly (which is unsafe), =org-protocol= can
be used for safe interaction with external processes
- org-time :: Extra functions to work with Emacs time data.
Some functions might be of interest upstream.
- org-timer :: (almost) generic timer for Emacs
- org-track-markers :: Keeping relative positions of markers
when cutting/pasting text in buffers. This is the way agenda
does not get broken by cutting/pasting/refiling the underlying
headings around.
- In the context of =org-agenda.el= split into multiple smaller libs,
Ilya Chernyshov commented about sub-optimality of agenda searches,
when agenda looks for timestamps.
- Org mode has parser cache, but this cache only goes down to
paragraph level. Markup inside paragraphs (like timestamps) is not
cached.
- Such caching is planned to be done in future, although even now,
the agenda search is fairly fast in practice.
- In the past, agenda struggled from accounting for property/tag
inheritance, but this is no longer a problem because properties
and tags are parts of headings, which do get cached.
- Ilya also commented that org-ql searches for timestamps very fast
- Agenda is actually on par with org-ql these days wrt to timestamps.
In fact, AFAIR, the way agenda searches timestamps was one of
the things that inspired org-ql.
- Ilya further asked about plans to merge org-ql in Org core
- We do have such plans
- See https://github.com/alphapapa/org-ql/issues/409
- Alphapapa wants to merge ts.el to Emacs first
- https://github.com/alphapapa/ts.el/issues/29
- As usual, everything is a subject of free time available to the
contributors. So, things are progressing, but slowly.
- visuwesh reported a bug with WIP async LaTeX preview branch
- It does not work with pdf-tools, because pdf-tools is using
~org-create-formula-image~. The function is supposed to work,
(although it will be obsoleted), but it does not.
- Karthink suggested reporting the details to Org ML or to TEC's github repo
- https://github.com/tecosaur/org-latex-preview-todos/issues/34
- Original bug report: https://github.com/vedang/pdf-tools/issues/283
- We looked inside pdf-tools to figure out how pdf-tools has
anything to do with latex preview
- ~pdf-annot-print-annotation-latex-maybe~ is using ~org-create-formula-image~
- ... and the way it use it is sub-optimal
- It is not the call to ~org-create-formula-image~, but the fact that they have to
: (require 'org) ;; org-create-formula-image
Load the whole(!) Org mode just for that function.
- It clearly shows why splitting Org libraries is a good idea
- visuwesh also commented that similar situation is with
comint-mime package (https://github.com/astoff/comint-mime):
it uses Org's LaTeX preview functionality
https://github.com/astoff/comint-mime/blob/ca278064dac431b8c3f72ab1b860bf9c9b8d8d60/comint-mime.el#L179
- In future, the idea is to discourage ~(require 'org)~ in
third-party packages and instead load more specific libraries
where the necessary functions live
- Jeff Trull asked about using =org-element= in Org tables
- I first misunderstood the question and started talking about
parsing table formulas (it is done using ancient regexp-based
code) and went ahead talking about new parsing features in Emacs
like PEG https://elpa.gnu.org/packages/peg.html
- PEG is an alternative way to describe syntax, especially
recursive syntax (which Org mode is)
- And PEG has become a part of Emacs since a few months ago.
(it took over 2 years, but we are finally there:
https://yhetil.org/emacs-devel/875yvtbbn3.fsf@ericabrahamsen.net/)
- However, the actual question was not about table formulas, but
about how Org mode edits tables in Org buffers (swap rows/columns;
alter cells; re-aligns tables; writes formula results, etc)
- Currently, everything is done very manually, hard-coding for the
current table syntax and not using Org parser
- Ideally, we indeed want to make use of the parser as much as
possible, so that we can extend formula syntax more easily
- However, Org element API currently lacks sophisticated functions
to edit Org text in buffers
- We have very rudimentary ~org-element-interpret-data~, but not
much more
- Various parts of Org instead employ various kludges to edit staff
- =org-list= has semi-independent parser and editing API
- =org-table= uses old regexp-based approach
- headline editing is using parser partially only
- So, a proper editing API is necessary before we can properly
use Org parser to edit things
- One inspiration may be https://github.com/ndwarshuis/org-ml,
although there are questions to the performance of that
library
- Karthink asked about technical details of how Org mode parser code
is able to respond to C-g even when it is stuck (that is: not just
exit, but interact with the user asking to press C-g multiple times
in order to force-terminate the parser).
- Karthink assumed that Org mode is using ~while-no-input~ Emacs macro
- Org mode does not use ~while-no-input~ - ~while-no-input~ terminates
on any input event, not just on C-g. Instead, Org mode let-binds
~inhibit-quit~ around critical parts of the parser that may trash
the parser state if exited without cleanup. Then, to retain the
ability for users to terminate the parser if there is a bug in it
(there used to be many bugs related to cache here a few releases
back), Org checks ~quit-flag~ to see if C-g was pressed, displays a
message, and only runs full parser cleanup when
~org-element--cache-interrupt-C-g-max-count~ is reached.
See ~org-element--parse-to~ for the details.
- More generally, Org mode parser implements old-school
asynchronisity using timers. Curious users may explore the
docstrings of ~org-element-cache-sync-idle-time~,
~org-element-cache-sync-duration~, and ~org-element-cache-sync-break~.
- Jeff Trull commented that he made a custom extension to Org mode
parser last year. We did not go into the details, but went on
discussing extending the Org mode syntax
- The entry point is the commentary in org-element.el that details
what needs to be done
- In short, one needs to write a new parser function, a new
interpreter function, add the new element/object to
org-element-all-elements/objects variable, and modify
org-element--current-element to fit the new syntax element into the
parser priorities.
- We currently have one new major change in Org syntax being
discussed: "inline special blocks" (tentative name)
https://list.orgmode.org/orgmode/875xwqj4tl.fsf@localhost/
- The idea is to create something similar to special blocks
(paragraph-level element) that can represent custom type of
paragraph data (notes, sidebars, boxes, alerts, etc: see
https://github.com/alhassy/org-special-block-extras that
demonstrates how flexible it can be)
- Unlike special blocks, inline special blocks are, well...,
inline. They represent custom markup
- The plan is to allow in-document definition of inline special
block exports; custom faces; etc
- In a sense, they are like custom link types, but without
limitation that links cannot contain other links and that
link path is interpreted verbatim
- They will also be able to contain attributes, which will
solve long-standing problem that we cannot pass export
attributes to images on export (like image width)
- also, a lot more; all the details in the linked mailing list
thread
- We already have experimental branch to play around, but
still discussing the details of the syntax and the features
that will need to be added
- (I do not remember whom here, probably Karthink :]) but we also
discussed the potential to extend async latex preview to other kinds
of text: latex, images, html files, pdf files, typst code snippets, etc
- We have one unfinished (but working) attempt to preview html inline in Org mode
https://github.com/alejandrogallo/org-inline-webkit
It would be nice is somebody take that repo and finish it to the
state ready for merging into Org mode
- HTML previews will move towards feature parity with ipython and
similar notebooks that can produce interactive graphs and
visualizations
- Also, the way WIP async preview library works (in its core) is not
limited to previewing latex. One can implement other kinds of
previews, like typst. All that is needed is a way to qeury typst
fragment at point in some way
- In fact, we already have some proof of concept code that goes even
further and extends LaTeX (for now, just LaTeX) preview
functionality _outside_ Org mode, in arbitrary buffers
https://list.orgmode.org/orgmode/87edbhljr7.FSF@hyperspace/
- Ideally, the preview might be made almost completely independent
from what major mode is currently in use
- This might be done using ~thingatpt~ Emacs library (built-in)
- A recall Yuan Fu talking about integrating thingatpt with tree-sitter
https://yhetil.org/emacs-devel/5E7F2A94-4377-45C0-8541-7F59F3B54BA1@gmail.com/
- And we can integrate Org mode parser with thingatpt as well
(we already partially do, see ~org--link-at-point~ and
~org--bounds-of-link-at-point~)
- P.S. It has been really long discussion :P
:comments:
[17:44] Ihor Radchenko : Official start in 15 minutes
[17:44] Ihor Radchenko : Meanwhile, check out the latest Org news at https://orgmode.org/
[17:44] Ihor Radchenko : We added Sacha's News to the front page for convenience :)
[17:57] Tim : Hi, this is Tim
[18:01] suhail : nothing on my side, but if we're short on topics, i'll mention 3 recent items i posted in the mailing list:
- https://yhetil.org/orgmode/87h6e134sp.fsf@gmail.com/
- https://yhetil.org/orgmode/87cyop3445.fsf@gmail.com/
- https://yhetil.org/orgmode/87cyonhuq3.fsf@gmail.com/
[18:02] suhail : one bug report, and the other 2 it's not clear if they are bugs or if user-error
[18:03] Karl Voit (publicvoit) : I just wrote a feature request for org-rifle: https://github.com/alphapapa/org-rifle/issues/85 maybe somebody has an answer so that we could collaboratively close this with a proper answer?
[18:11] Ihor Radchenko : org-fold-core-style
[18:14] Ihor Radchenko : https://github.com/alphapapa/org-ql/tree/master provides helm and vertico interfaces
[18:14] Ihor Radchenko : and allows tag/outline filtering
[18:14] Ihor Radchenko : and much more
[18:17] Tim : 78170 99% - command-execute
78124 99% - funcall-interactively
78124 99% - org-set-tags-command
78124 99% - org-get-buffer-tags
68151 86% - org-element-cache-map
38554 49% - org-element-at-point
3839 4% - org-element--parse-to
2030 2% - org-element--cache-find
678 0% #<compiled -0x1b52c4432bc3eb83>
21 0% org-knuth-hash
99 0% #<compiled 0x8f2ae6f02c727dd>
44 0% org-element--cache-active-p
117 0% org-element--cache-active-p
101 0% derived-mode-p
41 0% org-element--cache-sync
39 0% org-element--cache-verify-element
2881 3% - #<compiled 0x13daf1653b872335>
[...]
[18:18] Ihor Radchenko : (setq org-element--cache-self-verify t)
[18:24] Ihor Radchenko : helm-org-ql-source
[18:34] Ilya Chernyshov : Hi
[18:35] Ihor Radchenko : https://orgmode.org/worg/dev/org-element-api.html
[18:38] Ihor Radchenko : org-link-open
[18:40] karthink : I see
[18:40] karthink : Can you provide more details about what you're planning to do?
[18:41] karthink : Most of the changes in our feature branch are limited to the new org-latex-preview.el file
[18:41] karthink : M-1
[18:41] karthink : Use M-1 or M-2
[18:43] karthink : This is a great idea
[18:43] Karl Voit (publicvoit) : Sorry, I need to leave already - thanks for all the input I need to digest in the next days ;-)
[18:44] Ilya Chernyshov : If I remember correctly, org-element hasn't cached active/inactive timestamps inside body (except for scheduled, deadline). Has this been changed in the latest release? That could increase the speed of subsequent search in org-agenda
These problems are already solved in org-ql, the subsequent search results are instant
I also heard that you and alphapapa are planning to merge org-ql to org core. Do you think it will happen this year?
[19:02] suhail : i was under the impression that the byte-compiler treats top-level requires especially. and so if a require is within a defun, it's handled somewhat differently
[19:02] suhail : and that the latter is to be avoided (for reasons independent from performance considerations)
[19:09] Ihor Radchenko : https://github.com/alphapapa/org-ql/issues/409
[19:09] Ihor Radchenko : https://github.com/alphapapa/ts.el/issues/29
[19:17] oylenshpeegul : I have another meeting coming up. Thanks, Ihor! Thanks, all!
[19:23] karthink : I can hear you Ihor
[19:23] Welcome to <b>[[bbb:OrgMeetup]]</b>!<br /><br />For help on using BigBlueButton see these (short) <a href="https://www.bigbluebutton.org/html5" target="_blank"><u>tutorial videos</u></a>.<br /><br />To join the audio bridge click the phone button. Use a headset to avoid causing background noise for others.<br /><br />This server is running <a href="https://docs.bigbluebutton.org/" target="_blank"><u>BigBlueButton</u></a>.
[19:26] visuwesh : Does the org-create-formula-image function in org-compat in the async latex preview branch need more work? It seems like org--get-display-dpi is accidentally removed, and there are other incompatibilities like new format options in org-preview-latex-default-process, fg=auto not being correctly handled, etc.
This function is still used in pdf-tools, and comint-mime. I am not sure how pdf-tools can handle the new async code since it needs to show the image in a tooltip.
[19:35] visuwesh : Yea, its there but it odesnt work
[19:35] visuwesh : yes and some more
[19:35] visuwesh : The problem is the old user option is an alias to the new option
[19:35] visuwesh : so everything breaks
[19:36] visuwesh : And you might need to look at handling auto in uhh
[19:36] visuwesh : one min
[19:36] visuwesh : yea, i will send a mail to the ML
[19:37] visuwesh : emacs-orgmode@gnu.org?
[19:37] Ihor Radchenko : Yup, just report this to Org mailing list
[19:37] Ihor Radchenko : https://orgmode.org/manual/Feedback.html#Feedback
[19:37] karthink : Or here:
https://github.com/tecosaur/org-latex-preview-todos/issues
[19:37] karthink : But mailing list is fine
[19:37] visuwesh : ah didn't know about the github repo, thx for the link
[19:38] Ihor Radchenko : https://github.com/vedang/pdf-tools/issues/283
[19:38] visuwesh : When creating the tooltip for text annotations
[19:38] visuwesh : seee pdf-annot-print-annotation-latex-maybe
[19:38] visuwesh : Yea
[19:40] visuwesh : Will the async code work nicely with tooltips though?
[19:41] visuwesh : AHHH
[19:41] visuwesh : okkk, thanks for the hint
[19:42] visuwesh : btw, the same problem also exists in comint-mime but the latex functionality there is disabled by default
[19:42] visuwesh : https://github.com/astoff/comint-mime/blob/ca278064dac431b8c3f72ab1b860bf9c9b8d8d60/comint-mime.el#L179
[19:43] visuwesh : yep
[19:43] visuwesh : these two were the packages that burnt me when i switched to the async code mainly because of the new api
[19:47] visuwesh : yea, im happy to not have to wait for like 15 mins when i need to regen previews for whatever reason
[19:48] visuwesh : (when i was using the old sync code)
[19:52] visuwesh : afair i dont think so
[19:52] visuwesh : its just a temp buffer iirc
[19:52] visuwesh : yep
[19:54] visuwesh : okay thank you. i will have a look at it when i have the itme
[19:56] Jeff Trull : I didn't understand everything but I enjoyed it :)
[19:57] Jeff Trull : No, interesting things!
[19:57] Jeff Trull : magic
[19:57] Tim : And again: Thanks for your work!
[19:58] Jeff Trull : What's the state of org-element in tables?
[19:58] Jeff Trull : It seems like some parts like formulas are treated as a string IIRC
[20:00] Jeff Trull : Also it seemed like the table code used a lot of regex
[20:01] karthink : I have a somewhat technical question: where do you use while-no-input in the org-element parsing code? I couldn't find it. (Except when it gets stuck, creating the parse tree appears to be fully interruptible, it never slows down Emacs)
[20:01] Jeff Trull : Would it make sense to integrate element more into org-table.el?
[20:03] Ihor Radchenko : https://elpa.gnu.org/packages/peg.html
[20:04] Jeff Trull : ooh I have a completely different use for this. When was it added?
[20:04] karthink : It was added a couple of months ago I think
[20:04] visuwesh : This year iirc
[20:04] Ihor Radchenko : https://github.com/ndwarshuis/org-ml
[20:05] Jeff Trull : I see. org-element is kind of static
[20:05] Jeff Trull : Perfect for exporters, maybe not as good for editing
[20:06] Ihor Radchenko : org-element-interpret-data
[20:06] Jeff Trull : I got my answer
[20:06] Jeff Trull : I made a custom parser for something last year
[20:07] Ihor Radchenko : peg is a part of Emacs since few months ago
[20:16] Ihor Radchenko : https://list.orgmode.org/orgmode/875xwqj4tl.fsf@localhost/
[20:16] Ihor Radchenko : inline special blocks
[20:23] Ihor Radchenko : https://github.com/alejandrogallo/org-inline-webkit
[20:23] Ihor Radchenko : preview html
[20:23] Ihor Radchenko : inline, with rendering
[20:23] Ihor Radchenko : (proof of concept - it would be nice if someone finishes it up to be ready for inclusion)
[20:25] Ihor Radchenko : links are supported by thingatpt
[20:33] Ihor Radchenko : org--link-at-point
[20:33] Ihor Radchenko : org--bounds-of-link-at-point
[20:34] Jeff Trull : thanks for the interesting content I'll see you all next time
[20:35] karthink : I have to leave as well, thank you for the meetup Ihor
:end:
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [SUMMARY] #8 [[bbb:OrgMeetup]] on Wed, June 12, 19:00 UTC+3
2024-06-22 8:32 ` [SUMMARY] " Ihor Radchenko
@ 2024-06-27 12:45 ` Pedro
2024-06-27 13:31 ` Ihor Radchenko
0 siblings, 1 reply; 6+ messages in thread
From: Pedro @ 2024-06-27 12:45 UTC (permalink / raw)
To: emacs-orgmode
[-- Attachment #1.1.1.1: Type: text/plain, Size: 3616 bytes --]
On 6/22/24 10:32, Ihor Radchenko wrote:
> - Tim continued his struggle with an elusive bug he experiences when
> setting tags (C-c C-q) in large Org buffers - setting tags takes
> forever for him. (There is infinite loop likely lurking somewhere)
I have the same bug, but I said nothing because I don't have right now
time to report. When I read it, I just wanted to share that this also
happens to me.
I work with huge diary files and I discovered that since the cache is
activated by default, when I remove or move certain subtrees/parts looks
like makes the cache corrupted (I could not reproduce exactly how), not
only affects tag completion, that yeah, it takes forever, it also works
weird on org-captures applied to huge buffer files. So I have a current
diary, that tries to get a record of last weeks. When I could not
archive it (like right now), it has 5 weeks from now, that means 24k
lines in a file, when I can clean it up, goes to an archive file (so
then, alleviates org-agenda-files processing), the 2023, which is pretty
archived, is 263k lines.
From user-view perspective, it was uncomfortable to see it enabled by
default, specially when you have problems, specially, when you try to
deal with a solution, and is not really documented on the workarounds,
etc [0]. It took me nearly 2 hours to stabilize again my workflow so I
agree that some information should be explained as stated in a recent
conversation [1]
My vicious workflow regretably involves opening a lot of duplicated
buffers through emacsclient, and at some point, it becomes slow, so I do
this [2]
Being said all, I am very happy with how emacs and orgmode are getting
progress done on managing large files, etc. I just wanted to let you
know about a problem that I am experimenting, and maybe other people too.
I will note in my agenda on the #9 meeting [3] and I will try to join
Cheers,
pinmacs
[0] In my notes it says:
#+name: sourceblock_created_2024-06-27_14-32-44
#+begin_src text
how to disable Element cache? I am having warnings but I don't have time
to do bug reporting ( https://orgmode.org/Changes.html )
found the solution (setq org-element-use-cache nil) src
https://emacs.stackexchange.com/questions/42006/trouble-with-org-mode-cache-find-error
[ I don't remember where I got this, but I think it should appear in the
manual saying that this is a workaround until it is fixed
https://orgmode.org/org.html ]
org-element-cache-reset
#+end_src
[1] email Subject: Please document the caching and its user options,
From: Eli Zaretskii, Date: 2024-06-12 11:38:05 AM CEST
[2]
#+name: sourceblock_created_2024-06-27_14-33-28
#+begin_src emacs-lisp
;; I use this program when it works slow
(defun kill-duplicate-frames ()
"Kills duplicate frames of buffers."
(interactive)
(let ((buffers '()))
(dolist (frame (frame-list))
(with-selected-frame frame
(let ((buffer (current-buffer)))
(when (member buffer buffers)
(delete-frame))
(add-to-list 'buffers buffer)))))
;; after doing stuff, cache goes slow, but with this, it improves
;; so this is arbitrary program, that my brain has associated with
"make emacs faster again"
(org-element-cache-reset)
(message "Killed duplicate frames of buffers."))
#+end_src
[3] email Subject: #9 ((bbb:OrgMeetup)) on Wed, July 10, 19:00 UTC+3,
From: Ihor Radchenko <yantar92@posteo.net>, Date: 2024-06-26 5:48:46 PM CEST
[-- Attachment #1.1.1.2: Type: text/html, Size: 4724 bytes --]
[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 2847 bytes --]
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [SUMMARY] #8 [[bbb:OrgMeetup]] on Wed, June 12, 19:00 UTC+3
2024-06-27 12:45 ` Pedro
@ 2024-06-27 13:31 ` Ihor Radchenko
2024-06-27 14:11 ` Pedro
0 siblings, 1 reply; 6+ messages in thread
From: Ihor Radchenko @ 2024-06-27 13:31 UTC (permalink / raw)
To: Pedro; +Cc: emacs-orgmode
Pedro <pinmacs@cas.cat> writes:
> On 6/22/24 10:32, Ihor Radchenko wrote:
>> - Tim continued his struggle with an elusive bug he experiences when
>> setting tags (C-c C-q) in large Org buffers - setting tags takes
>> forever for him. (There is infinite loop likely lurking somewhere)
>
> I have the same bug, but I said nothing because I don't have right now
> time to report. When I read it, I just wanted to share that this also
> happens to me.
To be clear, I have no idea what kind of bug Tim experiences. I just
had a _guess_. It may or may not be the same bug.
Same for your case. I cannot tell without seeing the details.
> I work with huge diary files and I discovered that since the cache is
> activated by default, when I remove or move certain subtrees/parts looks
> like makes the cache corrupted...
Did it start happening recently?
If you think that the cache is corrupted, please add
(setq org-element--cache-self-verify t) to you config and watch the
warnings.
> ... not
> only affects tag completion, that yeah, it takes forever, it also works
> weird on org-captures applied to huge buffer files
Again, please report these things separately.
I can see that you have problems, but I cannot help you without seeing
more details.
> From user-view perspective, it was uncomfortable to see it enabled by
> default, specially when you have problems, specially, when you try to
> deal with a solution, and is not really documented on the workarounds,
> etc [0].
The cache was enabled by default after I (0) rewrote the old buggy
implementation (1) enabled the new version in dev builds (Org 9.6-pre)
and fixed all the reported bugs; (2) released Org 9.6 with
org-element--cache-self-verify set to t and fixed the inflow of all the
reports about corrupted cache; (3) disabled cache verification to nil
when the reports stopped coming.
So, at this point, cache issues are extremely rare. (and note how your
[0] link is from a 6-years-old post; I rewrote that old cache version)
> ... It took me nearly 2 hours to stabilize again my workflow so I
> agree that some information should be explained as stated in a recent
> conversation [1]
That conversation is tangent. It is not about cache being enabled. It is
about various caches (not just org-element) being saved to disk.
> My vicious workflow regretably involves opening a lot of duplicated
> buffers through emacsclient, and at some point, it becomes slow, so I do
> this [2]
> ;; after doing stuff, cache goes slow, but with this, it improves
> ;; so this is arbitrary program, that my brain has associated with
Please report this as a bug separately. Ideally, with a reproducer.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [SUMMARY] #8 [[bbb:OrgMeetup]] on Wed, June 12, 19:00 UTC+3
2024-06-27 13:31 ` Ihor Radchenko
@ 2024-06-27 14:11 ` Pedro
2024-06-27 14:16 ` Ihor Radchenko
0 siblings, 1 reply; 6+ messages in thread
From: Pedro @ 2024-06-27 14:11 UTC (permalink / raw)
To: emacs-orgmode
[-- Attachment #1.1.1: Type: text/plain, Size: 489 bytes --]
Thanks Ihor
I am experiencing it since end of July 2023 when I upgraded to the
version 9.6.15, but my situation now is stable with that workaround and
I have no time now for reporting the problem right now.
I expect to have some time at the end of July 2024, and I hope to
upgrade again orgmode and/or emacs to the latest stable version, then I
will try again to see if I stop getting cache errors without regularly
running =(org-element-cache-reset)=
Cheers,
pinmacs
[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 2847 bytes --]
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [SUMMARY] #8 [[bbb:OrgMeetup]] on Wed, June 12, 19:00 UTC+3
2024-06-27 14:11 ` Pedro
@ 2024-06-27 14:16 ` Ihor Radchenko
0 siblings, 0 replies; 6+ messages in thread
From: Ihor Radchenko @ 2024-06-27 14:16 UTC (permalink / raw)
To: Pedro; +Cc: emacs-orgmode
Pedro <pinmacs@cas.cat> writes:
> I am experiencing it since end of July 2023 when I upgraded to the
> version 9.6.15, but my situation now is stable with that workaround and
> I have no time now for reporting the problem right now.
FYI, we did have a couple of fixes related to cache in Org 9.7.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2024-06-27 14:16 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-05-22 14:30 #8 [[bbb:OrgMeetup]] on Wed, June 12, 19:00 UTC+3 Ihor Radchenko
2024-06-22 8:32 ` [SUMMARY] " Ihor Radchenko
2024-06-27 12:45 ` Pedro
2024-06-27 13:31 ` Ihor Radchenko
2024-06-27 14:11 ` Pedro
2024-06-27 14:16 ` Ihor Radchenko
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).