* Large source block causes org-mode to be unusable
@ 2021-06-21 18:27 Léo Ackermann
2021-06-21 18:43 ` John Hendy
` (3 more replies)
0 siblings, 4 replies; 21+ messages in thread
From: Léo Ackermann @ 2021-06-21 18:27 UTC (permalink / raw)
To: emacs-orgmode
[-- Attachment #1.1: Type: text/plain, Size: 918 bytes --]
Dear all,
I am working in an org-file of reasonable size (<2000 lines): my first
paper written in org-mode. Everything fine (and fast) until I started to
add `#+BEGIN_proof / #+END_proof` within my .org to make my .pdf export
prettier. This caused the editing of the proofs to be very slow: navigation
within the proof is fast but adding/removing any char takes around 4s per
char.
It seems that the fontify function is responsible for that (see
screenshot). As far as I understand, this function tries to fontify the
whole block as soon as a single char is modified. In my case, it then tries
to fontify a whole proof (~4 pages in my .pdf, with many LaTeX formulas)
several times per second...
Is there a way to make this fontify function to act "around my cursor" ?
Best,
Leo
--
Here the profiler's report when adding a single char
[image: image.png]
Spec.
- Emacs 27.2 (doom emacs 2.0.9) (I just updated them)
[-- Attachment #1.2: Type: text/html, Size: 1189 bytes --]
[-- Attachment #2: image.png --]
[-- Type: image/png, Size: 115679 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: Large source block causes org-mode to be unusable 2021-06-21 18:27 Large source block causes org-mode to be unusable Léo Ackermann @ 2021-06-21 18:43 ` John Hendy 2021-06-21 18:57 ` Sébastien Miquel ` (2 subsequent siblings) 3 siblings, 0 replies; 21+ messages in thread From: John Hendy @ 2021-06-21 18:43 UTC (permalink / raw) To: Léo Ackermann; +Cc: emacs-orgmode [-- Attachment #1.1: Type: text/plain, Size: 2679 bytes --] Seems known, assuming this is the same thing. I have not experienced the phenomenon or used proof blocks, so I'm just going by google-fu matching (which finds a lot): https://emacs.stackexchange.com/questions/46561/org-mode-9-too-slow-with-long-code-blocks https://emacs.stackexchange.com/questions/62376/slow-markdown-mode-as-emacs-spends-lots-of-time-fontifying https://emacs.stackexchange.com/questions/46110/help-needed-to-debug-very-slow-markdown-mode https://www.reddit.com/r/emacs/comments/c28svb/has_emacs_always_been_this_slow_at_highlighting/ https://lists.gnu.org/archive/html/emacs-orgmode/2015-07/msg00215.html As with all things like this, I'd work through the old information first to understand the true problem, if any improvements/workarounds exist, etc. Then this lands in some bucket of outcomes: - it's a novel find and therefore a bug (due diligence searching says this is not the case) - it's known bad, but not addressed (due to time or inability), but it can be worked around (nuggets exist in the links above as well as other results if you search "emacs fontify proof block slow") - a true "fix" is possible, in which case this would require pinpointing the exact problem and asking for alternatives or proposing a better/faster way to fontify Hope that helps, John P.S. This looks independent to Org, so I'm not sure if this is the best place for the eventual resolution as this may have nothing to do with Org? I'm no emacs guru, but from my skim I'm guessing org just pulls in some other package vs. having it's own fontification engine. On Mon, Jun 21, 2021 at 1:31 PM Léo Ackermann <leo.komba@gmail.com> wrote: > Dear all, > > I am working in an org-file of reasonable size (<2000 lines): my first > paper written in org-mode. Everything fine (and fast) until I started to > add `#+BEGIN_proof / #+END_proof` within my .org to make my .pdf export > prettier. This caused the editing of the proofs to be very slow: navigation > within the proof is fast but adding/removing any char takes around 4s per > char. > It seems that the fontify function is responsible for that (see > screenshot). As far as I understand, this function tries to fontify the > whole block as soon as a single char is modified. In my case, it then tries > to fontify a whole proof (~4 pages in my .pdf, with many LaTeX formulas) > several times per second... > > Is there a way to make this fontify function to act "around my cursor" ? > > Best, > Leo > > -- > Here the profiler's report when adding a single char > [image: image.png] > Spec. > - Emacs 27.2 (doom emacs 2.0.9) (I just updated them) > > > [-- Attachment #1.2: Type: text/html, Size: 3911 bytes --] [-- Attachment #2: image.png --] [-- Type: image/png, Size: 115679 bytes --] ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Large source block causes org-mode to be unusable 2021-06-21 18:27 Large source block causes org-mode to be unusable Léo Ackermann 2021-06-21 18:43 ` John Hendy @ 2021-06-21 18:57 ` Sébastien Miquel 2021-06-21 19:22 ` John Kitchin 2021-06-22 6:10 ` Timothy 3 siblings, 0 replies; 21+ messages in thread From: Sébastien Miquel @ 2021-06-21 18:57 UTC (permalink / raw) To: Léo Ackermann, emacs-orgmode Hi Léo, Léo Ackermann writes: > I am working in an org-file of reasonable size (<2000 lines): my first > paper written in org-mode. Everything fine (and fast) until I started > to add `#+BEGIN_proof / #+END_proof` within my .org to make my .pdf > export prettier. This caused the editing of the proofs to be very > slow: navigation within the proof is fast but adding/removing any char > takes around 4s per char. > It seems that the fontify function is responsible for that (see > screenshot). As far as I understand, this function tries to fontify > the whole block as soon as a single char is modified. In my case, it > then tries to fontify a whole proof (~4 pages in my .pdf, with many > LaTeX formulas) several times per second... You can try setting org-highlight-latex-and-related to '(latex) instead of '(native). Even with this setting, latex fontification in special blocks is slow. The reason being that the whole block has the `font-lock-multiline' property, hence every char insertion triggers refontification of the whole block. In the function `org-do-latex-and-related', I commented out the second condition of the `cond' form, which makes calls to `face-at-point'. This yields a significant speedup, and was enough to make things bearable in my cases. You can also try to simplify the latex regexp. If you try these, I'd be interested to hear how much of an improvement theymake. Regards, -- Sébastien Miquel ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Large source block causes org-mode to be unusable 2021-06-21 18:27 Large source block causes org-mode to be unusable Léo Ackermann 2021-06-21 18:43 ` John Hendy 2021-06-21 18:57 ` Sébastien Miquel @ 2021-06-21 19:22 ` John Kitchin 2021-06-21 19:36 ` John Hendy 2021-06-22 6:10 ` Timothy 3 siblings, 1 reply; 21+ messages in thread From: John Kitchin @ 2021-06-21 19:22 UTC (permalink / raw) To: Léo Ackermann; +Cc: emacs-orgmode A quick and dirty way to fix this might be an include file, i.e. move the block out of your manuscript file into a separate org file, and then just include it. Léo Ackermann <leo.komba@gmail.com> writes: > Dear all, > > I am working in an org-file of reasonable size (<2000 lines): my first > paper written in org-mode. Everything fine (and fast) until I started to > add `#+BEGIN_proof / #+END_proof` within my .org to make my .pdf export > prettier. This caused the editing of the proofs to be very slow: navigation > within the proof is fast but adding/removing any char takes around 4s per > char. > It seems that the fontify function is responsible for that (see > screenshot). As far as I understand, this function tries to fontify the > whole block as soon as a single char is modified. In my case, it then tries > to fontify a whole proof (~4 pages in my .pdf, with many LaTeX formulas) > several times per second... > > Is there a way to make this fontify function to act "around my cursor" ? > > Best, > Leo -- Professor John Kitchin Doherty Hall A207F Department of Chemical Engineering Carnegie Mellon University Pittsburgh, PA 15213 412-268-7803 @johnkitchin http://kitchingroup.cheme.cmu.edu Pronouns: he/him/his ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Large source block causes org-mode to be unusable 2021-06-21 19:22 ` John Kitchin @ 2021-06-21 19:36 ` John Hendy 2021-06-21 20:41 ` Tom Gillespie 2021-06-22 7:54 ` Eric S Fraga 0 siblings, 2 replies; 21+ messages in thread From: John Hendy @ 2021-06-21 19:36 UTC (permalink / raw) To: John Kitchin; +Cc: emacs-orgmode, Léo Ackermann [-- Attachment #1: Type: text/plain, Size: 1978 bytes --] Indeed. The top google hit for "emacs fontify proof block slow" I provided says exactly that :) https://emacs.stackexchange.com/questions/46561/org-mode-9-too-slow-with-long-code-blocks """ That said, I think keeping 2000 lines of source code inside an org src block is neither a standard use case nor a reasonable idea. You can use the #+INCLUDE ... src ... directive or split the block into a number of more reasonably sized cells. If you insist on keeping the entire block, then disable native highlighting of src code blocks. """ On Mon, Jun 21, 2021 at 2:23 PM John Kitchin <jkitchin@andrew.cmu.edu> wrote: > A quick and dirty way to fix this might be an include file, i.e. move > the block out of your manuscript file into a separate org file, and then > just include it. > > Léo Ackermann <leo.komba@gmail.com> writes: > > > Dear all, > > > > I am working in an org-file of reasonable size (<2000 lines): my first > > paper written in org-mode. Everything fine (and fast) until I started to > > add `#+BEGIN_proof / #+END_proof` within my .org to make my .pdf export > > prettier. This caused the editing of the proofs to be very slow: > navigation > > within the proof is fast but adding/removing any char takes around 4s per > > char. > > It seems that the fontify function is responsible for that (see > > screenshot). As far as I understand, this function tries to fontify the > > whole block as soon as a single char is modified. In my case, it then > tries > > to fontify a whole proof (~4 pages in my .pdf, with many LaTeX formulas) > > several times per second... > > > > Is there a way to make this fontify function to act "around my cursor" ? > > > > Best, > > Leo > > > -- > Professor John Kitchin > Doherty Hall A207F > Department of Chemical Engineering > Carnegie Mellon University > Pittsburgh, PA 15213 > 412-268-7803 > @johnkitchin > http://kitchingroup.cheme.cmu.edu > Pronouns: he/him/his > > [-- Attachment #2: Type: text/html, Size: 2738 bytes --] ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Large source block causes org-mode to be unusable 2021-06-21 19:36 ` John Hendy @ 2021-06-21 20:41 ` Tom Gillespie 2021-06-22 4:48 ` Tim Cross 2021-06-22 7:54 ` Eric S Fraga 1 sibling, 1 reply; 21+ messages in thread From: Tom Gillespie @ 2021-06-21 20:41 UTC (permalink / raw) To: John Hendy; +Cc: emacs-orgmode, Léo Ackermann, John Kitchin > That said, I think keeping 2000 lines of source code inside an > org src block is neither a standard use case nor a reasonable idea. I would say that it certainly is a standard use case for people who want to keep everything in a single file (e.g. to simplify reproducibility and avoid the mess of trying to distribute multiple files to non-technical users). #+INCLUDE is not a substitute if you are going to be tangling files, breaks many workflows, and as a result should rarely be recommended as a solution when src blocks are involved. Org should definitely be able to handle this case because there is no reason why performance should be any worse than having a 2000 line file in another buffer. Org babel has many basic interactivity performance pitfalls that need to be investigated. I personally have many workarounds for bad emacs performance degradations related to code executing in the event loop because I need to get on with the task at hand, but they need to be fixed, not dismissed. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Large source block causes org-mode to be unusable 2021-06-21 20:41 ` Tom Gillespie @ 2021-06-22 4:48 ` Tim Cross 0 siblings, 0 replies; 21+ messages in thread From: Tim Cross @ 2021-06-22 4:48 UTC (permalink / raw) To: emacs-orgmode Tom Gillespie <tgbugs@gmail.com> writes: >> That said, I think keeping 2000 lines of source code inside an >> org src block is neither a standard use case nor a reasonable idea. > > I would say that it certainly is a standard use case for people who > want to keep everything in a single file (e.g. to simplify > reproducibility and avoid the mess of trying to distribute multiple > files to non-technical users). #+INCLUDE is not a substitute if you > are going to be tangling files, breaks many workflows, and as a result > should rarely be recommended as a solution when src blocks are > involved. Org should definitely be able to handle this case because > there is no reason why performance should be any worse than having a > 2000 line file in another buffer. > While I agree with the sentiment, I disagree with the conclusion "there is no reason why performance should be any worse than having a 2000 line file in another buffer.". Org mode is pushing right up against the limits of Emacs' architecture with what it is doing. The way modes and font-locking work in Emacs was never designed to support multiple modes and multiple font-locking schemes within a single buffer. The fact org is able to do this is a testament to the power of Emacs, but this comes at a cost and that cost is primarily with respect to performance. You can experience similar performance degradation when you use other 'modes' which try to support multiple modes in a single buffer (like mmm-mode). This is not specific to org mode, but shows up more due to the larger user base for org. > Org babel has many basic interactivity performance pitfalls that need > to be investigated. I personally have many workarounds for bad emacs > performance degradations related to code executing in the event loop > because I need to get on with the task at hand, but they need to be > fixed, not dismissed. I think the big challenge here is that the 'fixes' needed by org mode really need to be at a deeper level inside Emac core architecture. It isn't something which can be effectively addressed at the org level. This makes it a much bigger and more complex task which is further hampered by the need to maintain compatibility with all the existing modes and libraries. From what I've seen in the org-devel list, there is on-going work which is focused on improvements to font-lock efficiency and I think support for multiple modes is also being considered in some of this work. Other developments, such as native compilation and pure gtk implementations may also help in this area. Unfortunately, it may take some time before we see the benefits of this work as it is complex and non-trivial work. In the meantime, we need to consider all possible work-arounds and tweaks which can help and in some cases, accept some compromise. Having source blocks which are 2000 lines long is, at this time, a challenge for org to handle given existing facilities. There are no quick and easy fixes, but there are some tweaks, such as turning off native fontification or using include files which can make the system usable. While not ideal, we have ot work within the limits of the current architecture while we wait for improvements, which can be expected to be incremental and slow. -- Tim Cross ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Large source block causes org-mode to be unusable 2021-06-21 19:36 ` John Hendy 2021-06-21 20:41 ` Tom Gillespie @ 2021-06-22 7:54 ` Eric S Fraga 2021-06-22 11:20 ` Léo Ackermann 1 sibling, 1 reply; 21+ messages in thread From: Eric S Fraga @ 2021-06-22 7:54 UTC (permalink / raw) To: emacs-orgmode On Monday, 21 Jun 2021 at 14:36, John Hendy wrote: > [...] or split the block into a number of more reasonably sized > cells. This is what I do in practice. I then use noweb syntax to bring everything together. In my case, the long LaTeX blocks tend to be tikz pictures. Using (native) for org-highlight-latex-and-related also helps in most cases. For me, the remaining performance issue is working with big tables and I've not found a solution to this. -- : Eric S Fraga via Emacs 28.0.50, Org release_9.4.6-566-gf0198e : Latest paper written in org: https://arxiv.org/abs/2106.05096 ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Large source block causes org-mode to be unusable 2021-06-22 7:54 ` Eric S Fraga @ 2021-06-22 11:20 ` Léo Ackermann 2021-06-22 12:13 ` Eric S Fraga 0 siblings, 1 reply; 21+ messages in thread From: Léo Ackermann @ 2021-06-22 11:20 UTC (permalink / raw) To: Org Mode List [-- Attachment #1: Type: text/plain, Size: 1538 bytes --] Hi all, Many thanks for those advices! I am new to org-mode but, is there a reason why #+BEGIN_proof #+END_proof and other org-latex-special-block are treated as block ? I mean; those #+... aim, as far as I understand, to give tips to org-export for prettier exports but nothing else (between those #+... you still need to write in org). I highlight that org-latex-special-block have nothing to do (in my point of view) with #+BEGIN_latex / #+END_latex that allow a user to insert some LaTeX within the document (e.g. Tikz pictures). Therefore, it might a great idea to take #+BEGIN_proof & co into account only when it comes to export the document. The proofs/definitions/theorems will no longer be considered as block, and fast fontification may follow :) What do you think of this ? Best, Leo Le mar. 22 juin 2021 à 09:55, Eric S Fraga <e.fraga@ucl.ac.uk> a écrit : > On Monday, 21 Jun 2021 at 14:36, John Hendy wrote: > > [...] or split the block into a number of more reasonably sized > > cells. > > This is what I do in practice. I then use noweb syntax to bring > everything together. In my case, the long LaTeX blocks tend to be tikz > pictures. > > Using (native) for org-highlight-latex-and-related also helps in most > cases. > > For me, the remaining performance issue is working with big tables and > I've not found a solution to this. > -- > : Eric S Fraga via Emacs 28.0.50, Org release_9.4.6-566-gf0198e > : Latest paper written in org: https://arxiv.org/abs/2106.05096 > > [-- Attachment #2: Type: text/html, Size: 2041 bytes --] ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Large source block causes org-mode to be unusable 2021-06-22 11:20 ` Léo Ackermann @ 2021-06-22 12:13 ` Eric S Fraga 2021-06-22 12:32 ` Léo Ackermann 0 siblings, 1 reply; 21+ messages in thread From: Eric S Fraga @ 2021-06-22 12:13 UTC (permalink / raw) To: Léo Ackermann; +Cc: Org Mode List On Tuesday, 22 Jun 2021 at 13:20, Léo Ackermann wrote: > I am new to org-mode but, is there a reason why #+BEGIN_proof #+END_proof > and other org-latex-special-block are treated as block ? I am not sure what you are asking. What specific treatment are you referring to? -- : Eric S Fraga via Emacs 28.0.50, Org release_9.4.6-567-g22bf80 : Latest paper written in org: https://arxiv.org/abs/2106.05096 ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Large source block causes org-mode to be unusable 2021-06-22 12:13 ` Eric S Fraga @ 2021-06-22 12:32 ` Léo Ackermann 2021-06-22 13:03 ` Eric S Fraga 2021-06-23 16:40 ` Maxim Nikulin 0 siblings, 2 replies; 21+ messages in thread From: Léo Ackermann @ 2021-06-22 12:32 UTC (permalink / raw) To: Org Mode List [-- Attachment #1: Type: text/plain, Size: 1460 bytes --] > I am not sure what you are asking. What specific treatment are you referring to? I suggest that special block in LateX export ( https://orgmode.org/manual/Special-blocks-in-LaTeX-export.html) may be treated differently. Once again, I'm new to org-mode, but here is my intuition: - What is happening now. The #+BEGIN_proof / #+END_proof keywords cause my proof to be seen as a block. Therefore, babel fontification is called to highlight syntax of my proof within my buffer. Timothy already discussed here how inefficient the babel fontification is. - What could happen. Changing (somewhere) the treatment of #BEGIN_proof & co the following way: Assign them to a new face "latex_export_sugar" and ignore them while editing. Doing this, the fontification of the org proof will be made as usual and will be fast enough. While exporting, replace #+BEGIN_proof by \begin{proof} and so on... Do you see what I mean ? Best, Le mar. 22 juin 2021 à 14:13, Eric S Fraga <e.fraga@ucl.ac.uk> a écrit : > On Tuesday, 22 Jun 2021 at 13:20, Léo Ackermann wrote: > > I am new to org-mode but, is there a reason why #+BEGIN_proof #+END_proof > > and other org-latex-special-block are treated as block ? > > I am not sure what you are asking. What specific treatment are you > referring to? > > -- > : Eric S Fraga via Emacs 28.0.50, Org release_9.4.6-567-g22bf80 > : Latest paper written in org: https://arxiv.org/abs/2106.05096 > [-- Attachment #2: Type: text/html, Size: 2017 bytes --] ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Large source block causes org-mode to be unusable 2021-06-22 12:32 ` Léo Ackermann @ 2021-06-22 13:03 ` Eric S Fraga 2021-06-22 13:32 ` Léo Ackermann 2021-06-23 16:40 ` Maxim Nikulin 1 sibling, 1 reply; 21+ messages in thread From: Eric S Fraga @ 2021-06-22 13:03 UTC (permalink / raw) To: Léo Ackermann; +Cc: Org Mode List On Tuesday, 22 Jun 2021 at 14:32, Léo Ackermann wrote: > Do you see what I mean ? I do but I guess I must have a differently configured system as I do not see any special fortification due to being in a special block. However, I see nothing in my configuration that affects special blocks. -- : Eric S Fraga via Emacs 28.0.50, Org release_9.4.6-567-g22bf80 : Latest paper written in org: https://arxiv.org/abs/2106.05096 ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Large source block causes org-mode to be unusable 2021-06-22 13:03 ` Eric S Fraga @ 2021-06-22 13:32 ` Léo Ackermann 0 siblings, 0 replies; 21+ messages in thread From: Léo Ackermann @ 2021-06-22 13:32 UTC (permalink / raw) To: Org Mode List [-- Attachment #1.1: Type: text/plain, Size: 823 bytes --] Neither do I (concerning the special fontification). Nevertheless, if you look at the profiler report (1st mail), the fontification process differ from the usual one (see attached screenshot) *because* the proof is considered as a block by org-mode. [image: image.png] Best, Le mar. 22 juin 2021 à 15:03, Eric S Fraga <e.fraga@ucl.ac.uk> a écrit : > On Tuesday, 22 Jun 2021 at 14:32, Léo Ackermann wrote: > > Do you see what I mean ? > > I do but I guess I must have a differently configured system as I do not > see any special fortification due to being in a special block. > > However, I see nothing in my configuration that affects special blocks. > > -- > : Eric S Fraga via Emacs 28.0.50, Org release_9.4.6-567-g22bf80 > : Latest paper written in org: https://arxiv.org/abs/2106.05096 > [-- Attachment #1.2: Type: text/html, Size: 1302 bytes --] [-- Attachment #2: image.png --] [-- Type: image/png, Size: 91453 bytes --] ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Large source block causes org-mode to be unusable 2021-06-22 12:32 ` Léo Ackermann 2021-06-22 13:03 ` Eric S Fraga @ 2021-06-23 16:40 ` Maxim Nikulin 2021-06-23 19:42 ` Gennady Uraltsev 1 sibling, 1 reply; 21+ messages in thread From: Maxim Nikulin @ 2021-06-23 16:40 UTC (permalink / raw) To: emacs-orgmode On 22/06/2021 19:32, Léo Ackermann wrote: > > I suggest that special block in LateX export > (https://orgmode.org/manual/Special-blocks-in-LaTeX-export.html) may be > treated differently. Generally I agree with you that #+begin_proof should be a thin wrapper unlike a source code block. However I am not familiar with fontification, so I can not reason if it is feasible. Maybe the code is watching if something like #+end_proof appeared, so block finished. On the other hand my impression was that \begin{proof} environment is intended to emphasize a paragraph or two to separate it from regular text on the same page. My expectation is that 4-page long proof should be formatted as ordinary sections and subsections. If proof environment is still required, maybe it is possible to customize export backend to have proofs as a headings with some property in the source document. E.g. beamer backend converts headings to frames. I have never tried such tricks, so I am unaware how much work is required for such approach. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Large source block causes org-mode to be unusable 2021-06-23 16:40 ` Maxim Nikulin @ 2021-06-23 19:42 ` Gennady Uraltsev 2021-06-24 7:54 ` Eric S Fraga 0 siblings, 1 reply; 21+ messages in thread From: Gennady Uraltsev @ 2021-06-23 19:42 UTC (permalink / raw) To: Maxim Nikulin; +Cc: emacs-orgmode Hello Everyone, I wanted to chime in to say I encountered a very similar problem. I author my math papers in LaTeX but I really wanted to use org to have an interconnected corpus of math notes with definitions, proofs, ideas, attempted computations, etc. Then they could be exported to LaTeX or even better: HTML to make for quite nice interactive notes (with popups backlinking to definitions of mathematical objects, etc...) (I was writing the latter in some JS). Unfortunately, HTML (+mathjax or whatever) is much more flexible than PDFs >I suggest that special block in LateX export (https://orgmode.org/manual/Special-blocks-in-LaTeX-export.html) may be treated differently. Once again, I'm new to org-mode, but here is my intuition: >- What is happening now. The #+BEGIN_proof / #+END_proof keywords cause my proof to be seen as a block. Therefore, babel fontification is called to highlight syntax of my proof within my buffer. Timothy already discussed here how inefficient the >babel fontification is. >- What could happen. Changing (somewhere) the treatment of #BEGIN_proof & co the following way: Assign them to a new face "latex_export_sugar" and ignore them while editing. Doing this, the fontification of the org proof will be made as usual and >will be fast enough. While exporting, replace #+BEGIN_proof by \begin{proof} and so on... Exactly! Before HTML export becomes relevant, I if I tried to use #+BEGIN_proof #+END_proof or #+BEGIN_definition #+END_definition insertion in these environments slowed very quickly down to a crawl. If I remember correctly the profiler told me that it was due to fontification. For me, the slowness in fontification INSIDE actual latex math blocks (the only place where I need fontification, and I would prefer to use AucTeX to do that, since it is really good at it) is generally not an issue since I mostly have latex previews and use the edit source command to work on the formulae. It would allow for a greater variety of workflows if one could specify that some blocks lines should be ignored for fontification purposes and the contents be treated as usual org-mode text. If you think it can be useful I can try to dig up my (abandoned) attempts and try to provide test cases to improve performance. > If proof environment is still required, maybe it is possible to > customize export backend to have proofs as a headings with some property > in the source document. Unfortunately this is not really fitting with my workflow (and I think this is not only my issue). In Org Mode, headings cannot "terminate" i.e. only a new section can stop a previous one. So, essentially, one would need to do a very non-elegant trick similar to one used by the "inline TODOs" i.e. having a very deep level of heading (10?) that "starts" a proof and then an artificial one that "ends" it. Also, the point of having "proof" blocks and "example" blocks that often, when reading math, you do not want to go through the proof on the first reading. One issue that many colleagues have agreed when discussing is that it would be so great to have a "collapsible" environment or block. When we write notes or explanations of material, often we are tasked with making a call: do we do computations step by step and risk being verbose, or do we write only the main steps and leave it to the reader to fill in the intermediate ones. Both these approaches have downsides. The former makes seeing the main points of the reasoning difficult to catch and forces a reader to commit significant effort to completely read the paper or notes (even if they just want the gist of it). The other approach may frustrate the reader interested in details because he/she has to work through the material to fill in all the gaps, often spending a lot of time on omitted intermediate steps. I was actually trying to come up with a natural system through org and HTML export to avoid this dilemma and have sections that are hidden by default but can be "expanded" on request. > On the other hand my impression was that \begin{proof} environment is > intended to emphasize a paragraph or two to separate it from regular > text on the same page. My expectation is that 4-page long proof should > be formatted as ordinary sections and subsections. Unfortunately this goes against the standard of mathematical writing (that I at least am familiar with). Proofs are often long and technical and can span multiple pages. If there are self-contained results in the proofs you separate them out in statements (lemmata) with their own (shorter) proofs. Text outside proofs is usually explanations and "glue" that explains the process, the idea, but does so on a more "informal" level. Having "proof" environments is also very useful as a method of having "namespaces", often you need to introduce auxiliary objects, expressions, operations, that are relevant only for the proof of the result but that should not be "polluting" the full paper or writeup. For example, if you introduce something inside a proof and denote it with a symbol (e.g. $\phi$) then as soon as you hit the QED symbol at the end of the proof, that symbol is freed and can be used in other proofs/constructions. It is very bad form, for example, to reference objects inside proofs of theorems outside the proofs themselves. If something is an object of general interest then it should be "defined" (included in a definition) on its own. I apologize for going on a tangent. Thanks again to everyone for the great package that is ORG, that I am progressively trying to incorporate in more and more of my day-to-day work and life. Best! Gena -- Gennady Uraltsev <gennady.uraltsev@gmail.com> (https://guraltsev.gitlab.io) ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Large source block causes org-mode to be unusable 2021-06-23 19:42 ` Gennady Uraltsev @ 2021-06-24 7:54 ` Eric S Fraga 2021-06-26 14:10 ` Léo Ackermann 0 siblings, 1 reply; 21+ messages in thread From: Eric S Fraga @ 2021-06-24 7:54 UTC (permalink / raw) To: Gennady Uraltsev; +Cc: emacs-orgmode On Wednesday, 23 Jun 2021 at 15:42, Gennady Uraltsev wrote: > In Org Mode, headings cannot "terminate" i.e. only a new section can > stop a previous one. True. What I do for cases where I want "separation" for visibility etc. is use headings at the same level but tell the exporter to ignore the actual headings when exporting. So I frequently have something like this: #+begin_src org ,* Section heading ,** introduction to section :ignoreheading: ,** statement of problem :ignoreheading: ,** proof :ignoreheading: ,#+latex: \begin{proof} proof goes here ,#+latex: \end{proof} ,** continuation of section :ignoreheading: ,* Next section #+end_src also showing how you can introduce a proof environment without using special blocks and hence maybe help with speed. (noting that you could do the same for HTML.) I have a filter for export that strips headings that have the :ignoreheading: tag. I believe there's something similar built-in now? This approach makes for easier navigation in the document (for me) and helps keep that bird's eye view of what I am writing. -- : Eric S Fraga via Emacs 28.0.50, Org release_9.4.6-571-gc591be : Latest paper written in org: https://arxiv.org/abs/2106.05096 ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Large source block causes org-mode to be unusable 2021-06-24 7:54 ` Eric S Fraga @ 2021-06-26 14:10 ` Léo Ackermann 2021-06-28 8:28 ` Sébastien Miquel 2021-06-28 10:40 ` Eric S Fraga 0 siblings, 2 replies; 21+ messages in thread From: Léo Ackermann @ 2021-06-26 14:10 UTC (permalink / raw) To: Org Mode List [-- Attachment #1: Type: text/plain, Size: 2039 bytes --] Hi, As I found no convenient solution, I will try to write a small module that adress this problem. It will be a good occasion to discover Elisp :). It is thought as an alternative to org-special-latex-block: efficient fontification using default face, folding environment to hide proofs, pretty latex export... @EricSFrada, would you mind sharing your code for your proof sections ? If you've got ideas for this module, or/and want to participate to its development, please let me know :) Best, Le jeu. 24 juin 2021 à 09:59, Eric S Fraga <e.fraga@ucl.ac.uk> a écrit : > On Wednesday, 23 Jun 2021 at 15:42, Gennady Uraltsev wrote: > > In Org Mode, headings cannot "terminate" i.e. only a new section can > > stop a previous one. > > True. > > What I do for cases where I want "separation" for visibility etc. is use > headings at the same level but tell the exporter to ignore the actual > headings when exporting. So I frequently have something like this: > > #+begin_src org > ,* Section heading > ,** introduction to section :ignoreheading: > ,** statement of problem :ignoreheading: > ,** proof :ignoreheading: > ,#+latex: \begin{proof} > proof goes here > ,#+latex: \end{proof} > ,** continuation of section :ignoreheading: > ,* Next section > #+end_src > > also showing how you can introduce a proof environment without using > special blocks and hence maybe help with speed. (noting that you could > do the same for HTML.) > > I have a filter for export that strips headings that have the > :ignoreheading: tag. I believe there's something similar built-in now? > > This approach makes for easier navigation in the document (for me) and > helps keep that bird's eye view of what I am writing. > -- > : Eric S Fraga via Emacs 28.0.50, Org release_9.4.6-571-gc591be > : Latest paper written in org: https://arxiv.org/abs/2106.05096 > > [-- Attachment #2: Type: text/html, Size: 2680 bytes --] ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Large source block causes org-mode to be unusable 2021-06-26 14:10 ` Léo Ackermann @ 2021-06-28 8:28 ` Sébastien Miquel 2021-06-28 10:42 ` Eric S Fraga 2021-06-28 10:40 ` Eric S Fraga 1 sibling, 1 reply; 21+ messages in thread From: Sébastien Miquel @ 2021-06-28 8:28 UTC (permalink / raw) To: Léo Ackermann, Org Mode List [-- Attachment #1: Type: text/plain, Size: 493 bytes --] Hi, Léo Ackermann writes: > @EricSFrada, would you mind sharing your code for your proof sections ? This functionality is now built-in: headings with an `ignore' tag do not get exported (their contents do). For very large proof, this seems like the right thing to do. In small to moderate sized blocks, the delay can still be noticeable and ought to be fixed. Attached is a patch that seems to resolve this issue. I haven't noticed any drawbacks so far. Regards, -- Sébastien Miquel [-- Attachment #2: 0001-WIP-do-not-refontify-special-blocks.patch --] [-- Type: text/x-patch, Size: 6303 bytes --] From d843bdc5887a6e50a57e349128ebbe032086dc17 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu> Date: Sun, 27 Jun 2021 16:24:22 +0200 Subject: [PATCH] WIP : do not refontify special blocks --- lisp/org.el | 99 ++++++++++++++++++++++++++++++++++------------------- 1 file changed, 64 insertions(+), 35 deletions(-) diff --git a/lisp/org.el b/lisp/org.el index 1bd9e02eb..9fd3f8514 100644 --- a/lisp/org.el +++ b/lisp/org.el @@ -5265,22 +5265,32 @@ by a #." (org-fontify-meta-lines-and-blocks-1 limit) (error (message "Org mode fontification error in %S at %d" (current-buffer) - (line-number-at-pos))))) + (line-number-at-pos)))) + nil) (defun org-fontify-meta-lines-and-blocks-1 (limit) "Fontify #+ lines and blocks." - (let ((case-fold-search t)) - (when (re-search-forward - (rx bol (group (zero-or-more (any " \t")) "#" - (group (group (or (seq "+" (one-or-more (any "a-zA-Z")) (optional ":")) - (any " \t") - eol)) - (optional (group "_" (group (one-or-more (any "a-zA-Z")))))) - (zero-or-more (any " \t")) - (group (group (zero-or-more (not (any " \t\n")))) - (zero-or-more (any " \t")) - (group (zero-or-more any))))) - limit t) + (let* ((case-fold-search t) + (fl-beg (point)) + (fl-end limit) + (fbeg (when (and (> fl-beg (point-min)) + (get-text-property (1- fl-beg) 'font-lock-multiline-block)) + (or (previous-single-property-change + fl-beg 'font-lock-multiline-block) + (point-min))))) + (when fbeg (goto-char fbeg)) + (while (and (< (point) limit) + (re-search-forward + (rx bol (group (zero-or-more (any " \t")) "#" + (group (group (or (seq "+" (one-or-more (any "a-zA-Z")) (optional ":")) + (any " \t") + eol)) + (optional (group "_" (group (one-or-more (any "a-zA-Z")))))) + (zero-or-more (any " \t")) + (group (group (zero-or-more (not (any " \t\n")))) + (zero-or-more (any " \t")) + (group (zero-or-more any))))) + limit t)) (let ((beg (match-beginning 0)) (end-of-beginline (match-end 0)) ;; Including \n at end of #+begin line will include \n @@ -5318,7 +5328,7 @@ by a #." (remove-text-properties beg end-of-endline '(display t invisible t intangible t))) (add-text-properties - beg end-of-endline '(font-lock-fontified t font-lock-multiline t)) + beg end-of-endline '(font-lock-fontified t font-lock-multiline-block t)) (org-remove-flyspell-overlays-in beg bol-after-beginline) (org-remove-flyspell-overlays-in nl-before-endline end-of-endline) (cond @@ -5327,7 +5337,8 @@ by a #." (add-text-properties bol-after-beginline block-end '(src-block t))) (quoting (add-text-properties - bol-after-beginline beg-of-endline + (max bol-after-beginline (or fl-beg bol-after-beginline)) + (min beg-of-endline (or fl-end beg-of-endline)) (list 'face (list :inherit (let ((face-name @@ -5426,26 +5437,41 @@ by a #." (add-text-properties closing-start end '(invisible t))) t))))) -(defun org-fontify-extend-region (beg end _old-len) - (let ((begin-re "\\(\\\\\\[\\|\\(#\\+begin_\\|\\\\begin{\\)\\S-+\\)") - (end-re "\\(\\\\\\]\\|\\(#\\+end_\\|\\\\end{\\)\\S-+\\)") - (extend - (lambda (r1 r2 dir) - (let ((re (replace-regexp-in-string - "\\(begin\\|end\\)" r1 - (replace-regexp-in-string - "[][]" r2 - (match-string-no-properties 0))))) - (re-search-forward (regexp-quote re) nil t dir))))) - (save-match-data - (save-excursion - (goto-char beg) - (back-to-indentation) - (cond ((looking-at end-re) - (cons (or (funcall extend "begin" "[" -1) beg) end)) - ((looking-at begin-re) - (cons beg (or (funcall extend "end" "]" 1) end))) - (t (cons beg end))))))) +(defun org-fontify-extend-region (bego endo _old-len) + (let* ((beg bego) (end endo) + (bol (save-excursion (goto-char beg) (point-at-bol))) + (eol (save-excursion (goto-char end) (point-at-eol)))) + (let ((before-multi (and (> bol (point-min)) + (get-text-property (1- bol) 'font-lock-multiline-block))) + (after-multi (get-text-property eol 'font-lock-multiline-block))) + (if before-multi + (unless after-multi + (setq beg (or (previous-single-property-change + bol 'font-lock-multiline-block) + (point-min)))) + (when after-multi + (setq end (or (text-property-any eol (point-max) + 'font-lock-multiline-block nil) + (point-max)))))) + (let ((begin-re "\\(\\\\\\[\\|\\(#\\+begin_\\|\\\\begin{\\)\\S-+\\)") + (end-re "\\(\\\\\\]\\|\\(#\\+end_\\|\\\\end{\\)\\S-+\\)") + (extend + (lambda (r1 r2 dir) + (let ((re (replace-regexp-in-string + "\\(begin\\|end\\)" r1 + (replace-regexp-in-string + "[][]" r2 + (match-string-no-properties 0))))) + (re-search-forward (regexp-quote re) nil t dir))))) + (save-match-data + (save-excursion + (goto-char beg) + (back-to-indentation) + (cond ((looking-at end-re) + (cons (or (funcall extend "begin" "[" -1) beg) end)) + ((looking-at begin-re) + (cons beg (max end (or (funcall extend "end" "]" 1) end)))) + (t (cons beg end)))))))) (defun org-activate-footnote-links (limit) "Add text properties for footnotes." @@ -5772,6 +5798,9 @@ needs to be inserted at a specific position in the font-lock sequence.") '(org-font-lock-keywords t nil nil backward-paragraph)) (setq-local font-lock-extend-after-change-region-function #'org-fontify-extend-region) + (setq-local font-lock-extra-managed-props + (append font-lock-extra-managed-props + '(font-lock-multiline-block))) (kill-local-variable 'font-lock-keywords) nil)) -- 2.32.0 ^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: Large source block causes org-mode to be unusable 2021-06-28 8:28 ` Sébastien Miquel @ 2021-06-28 10:42 ` Eric S Fraga 0 siblings, 0 replies; 21+ messages in thread From: Eric S Fraga @ 2021-06-28 10:42 UTC (permalink / raw) To: Sébastien Miquel; +Cc: Org Mode List, Léo Ackermann On Monday, 28 Jun 2021 at 08:28, Sébastien Miquel wrote: > Léo Ackermann writes: >> @EricSFrada, would you mind sharing your code for your proof sections ? > This functionality is now built-in: headings with an `ignore' tag do > not get exported (their contents do). For very large proof, this seems > like the right thing to do. Ah, maybe this is what Leo was referring to. I may have misunderstood. Yes, it seems that org now does process :ignore: tags, I guess. I've been using my own code to ignore headings for so long that I did not realise this! -- : Eric S Fraga via Emacs 28.0.50, Org release_9.4.6-571-gc591be : Latest paper written in org: https://arxiv.org/abs/2106.05096 ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Large source block causes org-mode to be unusable 2021-06-26 14:10 ` Léo Ackermann 2021-06-28 8:28 ` Sébastien Miquel @ 2021-06-28 10:40 ` Eric S Fraga 1 sibling, 0 replies; 21+ messages in thread From: Eric S Fraga @ 2021-06-28 10:40 UTC (permalink / raw) To: Léo Ackermann; +Cc: Org Mode List On Saturday, 26 Jun 2021 at 16:10, Léo Ackermann wrote: > @EricSFrada, would you mind sharing your code for your proof sections ? > If you've got ideas for this module, or/and want to participate to its > development, please let me know :) I have no code for proofs. Sorry! I was simply illustrating that you can put in LaTeX code directly should special blocks be too large. -- : Eric S Fraga via Emacs 28.0.50, Org release_9.4.6-571-gc591be : Latest paper written in org: https://arxiv.org/abs/2106.05096 ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Large source block causes org-mode to be unusable 2021-06-21 18:27 Large source block causes org-mode to be unusable Léo Ackermann ` (2 preceding siblings ...) 2021-06-21 19:22 ` John Kitchin @ 2021-06-22 6:10 ` Timothy 3 siblings, 0 replies; 21+ messages in thread From: Timothy @ 2021-06-22 6:10 UTC (permalink / raw) To: Léo Ackermann; +Cc: emacs-orgmode [-- Attachment #1.1: Type: text/plain, Size: 1196 bytes --] Hi Léo, I think I can actually speak a bit on this. As it stands, babel fontification operates by: - sending the src block to a dedicated buffer - turning on the relevant major mode for the language - forcing font-lock of the entire buffer - cloning the font-lock information for the entire buffer back to Org In essence, every keystroke in a babel block causes font-lock lag equal to opening the file for the first time. As you can imagine, this adds up. I think ideally Org would monitor which block you're executing keystrokes in, and if there are a few in a row in the same org-src block keep the font-lock buffer alive and incrementally update the content and intelligently copy back font-lock changes. This has the potential to be much lower latency, but is also going to be hard to get right. In the meantime, you could try executing something like this in your Org buffer ([source]): ,---- | (setq-local jit-lock-defer-time 0.05 | jit-lock-stealth-time 1) `---- This sacrifices immediate font-lock accuracy for responsiveness. All the best, *Timothy* [source] <https://tecosaur.github.io/emacs-config/config.html#font-display,code--7> [-- Attachment #1.2: Type: text/html, Size: 6717 bytes --] ^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2021-06-28 10:46 UTC | newest] Thread overview: 21+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2021-06-21 18:27 Large source block causes org-mode to be unusable Léo Ackermann 2021-06-21 18:43 ` John Hendy 2021-06-21 18:57 ` Sébastien Miquel 2021-06-21 19:22 ` John Kitchin 2021-06-21 19:36 ` John Hendy 2021-06-21 20:41 ` Tom Gillespie 2021-06-22 4:48 ` Tim Cross 2021-06-22 7:54 ` Eric S Fraga 2021-06-22 11:20 ` Léo Ackermann 2021-06-22 12:13 ` Eric S Fraga 2021-06-22 12:32 ` Léo Ackermann 2021-06-22 13:03 ` Eric S Fraga 2021-06-22 13:32 ` Léo Ackermann 2021-06-23 16:40 ` Maxim Nikulin 2021-06-23 19:42 ` Gennady Uraltsev 2021-06-24 7:54 ` Eric S Fraga 2021-06-26 14:10 ` Léo Ackermann 2021-06-28 8:28 ` Sébastien Miquel 2021-06-28 10:42 ` Eric S Fraga 2021-06-28 10:40 ` Eric S Fraga 2021-06-22 6:10 ` Timothy
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).
![[-- Attachment #2: image.png --] [-- Type: image/png, Size: 115679 bytes --]](../../CA+M2ft-YYD7tqbWsb0FsQvOgP6MZwr2n7sKS6mG0hS3xwwQkmg@mail.gmail.com/2-image.png){kind=link}
![[-- Attachment #2: image.png --] [-- Type: image/png, Size: 91453 bytes --]](../../CAFhsWEgwa5wnKmZAKB_kWH-N9Qg6hvp5XhW=j18ucOqhUL9bEg@mail.gmail.com/2-image.png){kind=link}