[DISCUSSION] Refactoring fontification system

[DISCUSSION] Refactoring fontification system

[Ihor Radchenko]

Dear all,

Recently, there have been multiple issues related to incorrect
fontification:
- https://list.orgmode.org/orgmode/23707.20428.546749.44853@frac.u-strasbg.fr/
- https://list.orgmode.org/orgmode/87fsujp7mc.fsf@web.de/
- https://list.orgmode.org/orgmode/87czvqxdn9.fsf@gmail.com/
- https://list.orgmode.org/8735nsv9qo.fsf@nicolasgoaziou.fr/T/#me1c44b6e493dd280cca4f042b833c24749ae4fe0

These issues keep appearing because our current fontification code is
based on regexps and only approximates the actual Org grammar elements.
It is largely a legacy from the times when org-element parser was not a
thing.

Maybe it is a time to upgrade the fontification according to our
state-of-art parser?

Instead of fontifying elements individually via regexps, we can leverage
org-element-map, org-element-parse-buffer, org-element-cache, and
jit-lock-mode. Each type of Org element/object can be assigned with a
fontification function accepting a single argument - the element datum.

Also, the fontification code can be move to a separate library.

WDYT?

Best,
Ihor

Re: [DISCUSSION] Refactoring fontification system

[Bruce D'Arcus]

On Fri, Nov 19, 2021 at 9:11 AM Ihor Radchenko <yantar92@gmail.com> wrote:

> WDYT?

I don't understand all the technical details, but it sounds like a
GREAT idea to me in general!

Bruce

Re: [DISCUSSION] Refactoring fontification system

[Tim Cross]

Ihor Radchenko <yantar92@gmail.com> writes:

> Dear all,
>
> Recently, there have been multiple issues related to incorrect
> fontification:
> - https://list.orgmode.org/orgmode/23707.20428.546749.44853@frac.u-strasbg.fr/
> - https://list.orgmode.org/orgmode/87fsujp7mc.fsf@web.de/
> - https://list.orgmode.org/orgmode/87czvqxdn9.fsf@gmail.com/
> - https://list.orgmode.org/8735nsv9qo.fsf@nicolasgoaziou.fr/T/#me1c44b6e493dd280cca4f042b833c24749ae4fe0
>
> These issues keep appearing because our current fontification code is
> based on regexps and only approximates the actual Org grammar elements.
> It is largely a legacy from the times when org-element parser was not a
> thing.
>
> Maybe it is a time to upgrade the fontification according to our
> state-of-art parser?
>
> Instead of fontifying elements individually via regexps, we can leverage
> org-element-map, org-element-parse-buffer, org-element-cache, and
> jit-lock-mode. Each type of Org element/object can be assigned with a
> fontification function accepting a single argument - the element datum.
>
> Also, the fontification code can be move to a separate library.
>
> WDYT?
>

Sounds like a better approach to me. In addition to being
more accurate, this would mean we don't need to keep 2 separate
definitions in sync or have confusing font locking where the regexp and element
definitions are different. Should gives us increased consistency and
less maintenance. 

Re: [DISCUSSION] Refactoring fontification system

[Nicolas Goaziou]

Hello,

Ihor Radchenko <yantar92@gmail.com> writes:

> Recently, there have been multiple issues related to incorrect
> fontification:
> - https://list.orgmode.org/orgmode/23707.20428.546749.44853@frac.u-strasbg.fr/
> - https://list.orgmode.org/orgmode/87fsujp7mc.fsf@web.de/
> - https://list.orgmode.org/orgmode/87czvqxdn9.fsf@gmail.com/
> - https://list.orgmode.org/8735nsv9qo.fsf@nicolasgoaziou.fr/T/#me1c44b6e493dd280cca4f042b833c24749ae4fe0
>
> These issues keep appearing because our current fontification code is
> based on regexps and only approximates the actual Org grammar elements.
> It is largely a legacy from the times when org-element parser was not a
> thing.
>
> Maybe it is a time to upgrade the fontification according to our
> state-of-art parser?
>
> Instead of fontifying elements individually via regexps, we can leverage
> org-element-map, org-element-parse-buffer, org-element-cache, and
> jit-lock-mode. Each type of Org element/object can be assigned with a
> fontification function accepting a single argument - the element datum.
>
> Also, the fontification code can be move to a separate library.
>
> WDYT?

I wholeheartedly agree with all these points.

Regards,
-- 
Nicolas Goaziou

Re: [DISCUSSION] Refactoring fontification system

[Ihor Radchenko]

Ihor Radchenko <yantar92@gmail.com> writes:

> Instead of fontifying elements individually via regexps, we can leverage
> org-element-map, org-element-parse-buffer, org-element-cache, and
> jit-lock-mode. Each type of Org element/object can be assigned with a
> fontification function accepting a single argument - the element datum.

I have been working on the new fontification library for the last
several months and I have noticed several confusing things in the
current fontification settings.

I'd like to hear if anyone has any idea on how to interpret the
following:

1. org-protecting-blocks is an internal auxiliary variable used to
   determine which blocks should be fontified using different major
   mode.
   It's value is ("src" "example" "export")
   So, #+begin_src lang and #+begin_export lang are fontified according
   to LANG. Makes sense.
   However, what about #+begin_example?
   org-element-example-block-parser does not appear to expect language
   specification in the example blocks. Only switches seems to be
   allowed. Am I missing something and Org actually allows example
   blocks to specify language? Or was it the case in the distant past
   versions of Org?

2. org-script-display is an internal auxiliary variable used to display
   sub/superscripts. Note that it's default value holds 4 possibilities.
   Two for each type of script. For example, for superscripts we have
   two options:
   ((raise 0.3) (height 0.7)) and ((raise 0.5))
   The first one looks more compact (does not change the line height)
   and reduces size of the superscript.
   The second one makes superscript have the same size with the main
   text an increases the line height x1.5.

   org-raise-scripts fontifies sub/superscripts differently inside and
   outside the tables:
   From org-raise-scripts:
   (nth (if table-p 3 1) org-script-display)

   However, it currently uses x1.5 line height for tables creating empty
   space between vertical | separators. It looks like a typo for me. It
   would make more sense to make table lines compact, not vice versa. Am
   I missing something?

3. org-fontify-meta-lines-and-blocks-1 creates a special face for
   ("+title:" "+subtitle:" "+author:" "+email:" "+date:")
   The face name is org-document-info.
   But what about, say, +description: or +language:?
   Would it make more sense to fontify all the keywords from
   org-options-keywords instead?
   
Best,
Ihor

Re: [DISCUSSION] Refactoring fontification system

[Ted Reed via General discussions about Org-mode.]

Ihor Radchenko <yantar92@gmail.com> writes:

> 1. org-protecting-blocks is an internal auxiliary variable used to
>    determine which blocks should be fontified using different major
>    mode.
>    It's value is ("src" "example" "export")
>    So, #+begin_src lang and #+begin_export lang are fontified according
>    to LANG. Makes sense.
>    However, what about #+begin_example?
>    org-element-example-block-parser does not appear to expect language
>    specification in the example blocks. Only switches seems to be
>    allowed. Am I missing something and Org actually allows example
>    blocks to specify language? Or was it the case in the distant past
>    versions of Org?
>

Speaking as a user, I've used fontified example blocks for json
responses when using ob-http:

#+begin_src http :pretty :wrap example json
GET http://es-master.service.consul:9200/_cluster/settings?pretty
#+end_src

#+RESULTS:
#+begin_example json
{
  "persistent": {
    "cluster": {
      "routing": {
        "allocation": {
          "node_concurrent_incoming_recoveries": "20",
          "enable": "all",
          "node_concurrent_outgoing_recoveries": "20"
        }
      }
    },
    "xpack": {
      "monitoring": {
        "collection": {
          "enabled": "true"
        }
      }
    }
  },
  "transient": {}
}
#+end_example

In this case, ob-http seems to default to generating example blocks. I
realized I could add a bit there to get it to fontify as JSON.

I'm unsure if this is correct usage or merely a side effect, or if
there's a better block to be using for this purpose, but I figured I
could at least contribute a user report about this feature.

I'm on org-mode 9.5.3 if that matters.

-- Ted Reed

Re: [DISCUSSION] Refactoring fontification system

[Tim Cross]

Ihor Radchenko <yantar92@gmail.com> writes:

> Ihor Radchenko <yantar92@gmail.com> writes:
>
>> Instead of fontifying elements individually via regexps, we can leverage
>> org-element-map, org-element-parse-buffer, org-element-cache, and
>> jit-lock-mode. Each type of Org element/object can be assigned with a
>> fontification function accepting a single argument - the element datum.
>
> I have been working on the new fontification library for the last
> several months and I have noticed several confusing things in the
> current fontification settings.
>

Very pleased your working on this. I suspect this will, once combined
with the other fine work you have done, generate some very rewarding
results. 

> I'd like to hear if anyone has any idea on how to interpret the
> following:
>
> 1. org-protecting-blocks is an internal auxiliary variable used to
>    determine which blocks should be fontified using different major
>    mode.
>    It's value is ("src" "example" "export")
>    So, #+begin_src lang and #+begin_export lang are fontified according
>    to LANG. Makes sense.
>    However, what about #+begin_example?
>    org-element-example-block-parser does not appear to expect language
>    specification in the example blocks. Only switches seems to be
>    allowed. Am I missing something and Org actually allows example
>    blocks to specify language? Or was it the case in the distant past
>    versions of Org?
>

I don't recall ever being able to specify language type with source
blocks. The only benefit I can see for being able to would be to have
blocks which are not 'executed' (tangled), but you can already achieve
that just using source blocks. So, from my perspective, I would want
example blocks (which I view mainly as a type of verbatim block)
fontified in a distinct manner, but no need for language specific
font-locking - if I want that, I would just use a src block. 

> 2. org-script-display is an internal auxiliary variable used to display
>    sub/superscripts. Note that it's default value holds 4 possibilities.
>    Two for each type of script. For example, for superscripts we have
>    two options:
>    ((raise 0.3) (height 0.7)) and ((raise 0.5))
>    The first one looks more compact (does not change the line height)
>    and reduces size of the superscript.
>    The second one makes superscript have the same size with the main
>    text an increases the line height x1.5.
>
>    org-raise-scripts fontifies sub/superscripts differently inside and
>    outside the tables:
>    From org-raise-scripts:
>    (nth (if table-p 3 1) org-script-display)
>
>    However, it currently uses x1.5 line height for tables creating empty
>    space between vertical | separators. It looks like a typo for me. It
>    would make more sense to make table lines compact, not vice versa. Am
>    I missing something?
>

I need superscript/subscript so rarely I really don't have much of a
position. However, I do find it annoying when a superscript/subscript
character alters the line spacing for the line it happens to display in. 

> 3. org-fontify-meta-lines-and-blocks-1 creates a special face for
>    ("+title:" "+subtitle:" "+author:" "+email:" "+date:")
>    The face name is org-document-info.
>    But what about, say, +description: or +language:?
>    Would it make more sense to fontify all the keywords from
>    org-options-keywords instead?
> 

Yes, I think that would be the right approach. I suspect most people
won't really worry too much. Absent font locking is less of an issue
than incorrect font locking. Soemthing too often seen with regexp based
font-locking which I'm hoping will be avoided with a version based more
on parsed elements etc. 
   

Re: [DISCUSSION] Refactoring fontification system

[Ihor Radchenko]

Ted Reed <treed@zenithia.net> writes:

> #+begin_src http :pretty :wrap example json
> ...
> In this case, ob-http seems to default to generating example blocks. I
> realized I could add a bit there to get it to fontify as JSON.

Have you tried
#+begin_src http :pretty :wrap "src json :eval no"
?

It should give the same effect.

Best,
Ihor

Re: [DISCUSSION] Refactoring fontification system

[Ted Reed via General discussions about Org-mode.]

Ihor Radchenko <yantar92@gmail.com> writes:

> Have you tried
> #+begin_src http :pretty :wrap "src json :eval no"
> ?

It does work the same, thanks!

-- Ted Reed

Re: [DISCUSSION] Refactoring fontification system

[Max Nikulin]

On 03/06/2022 16:45, Ihor Radchenko wrote:
> 
> 1. org-protecting-blocks is an internal auxiliary variable used to
>     determine which blocks should be fontified using different major
>     mode.
>     It's value is ("src" "example" "export")
>     So, #+begin_src lang and #+begin_export lang are fontified according
>     to LANG. Makes sense.
>     However, what about #+begin_example?

I have a lot of #+begin_example with language in my notes. In the 
beginning I accidentally discovered that code is highlighted accordingly 
to language syntax, so I continued to use this feature for snippets that 
are not intended to be evaluated. Later it was a bad surprise that 
export does not preserves highlights. So for me #+begin_example is a 
shortcut for #+begin_src with :eval never (that can be specified as a 
property).

 From my point of view exporters may reuse code block formatters for 
examples. It would allow to pass options to e.g. LaTeX verbatim environment:
Pedro Andres Aranda Gutierrez. Re: A question/bug report(?) Wed, 30 Mar 
2022 07:14:54 +0200 
https://list.orgmode.org/orgmode/CAO48Bk_dJs1=5zgpcZwODaTsRqyRsKq1ALj6WpaXCc4bDjW3FQ@mail.gmail.com/

> 2. org-script-display is an internal auxiliary variable used to display
>     sub/superscripts. Note that it's default value holds 4 possibilities.
>     Two for each type of script. For example, for superscripts we have
>     two options:
>     ((raise 0.3) (height 0.7)) and ((raise 0.5))
>     The first one looks more compact (does not change the line height)

Notice that subscript still increases inter-line space a bit.

>     However, it currently uses x1.5 line height for tables creating empty
>     space between vertical | separators. It looks like a typo for me. It
>     would make more sense to make table lines compact, not vice versa. Am
>     I missing something?

git blame gives 0618aeafb39dbf78e753348eaeaddbb7f8104cd0
It seems smaller font breaks horizontal alignment in tables.

Re: [DISCUSSION] Refactoring fontification system

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

> On 03/06/2022 16:45, Ihor Radchenko wrote:
>> 
>> 1. org-protecting-blocks is an internal auxiliary variable used to
>>     determine which blocks should be fontified using different major
>>     mode.
>>     It's value is ("src" "example" "export")
>>     So, #+begin_src lang and #+begin_export lang are fontified according
>>     to LANG. Makes sense.
>>     However, what about #+begin_example?
>
> I have a lot of #+begin_example with language in my notes. In the 
> beginning I accidentally discovered that code is highlighted accordingly 
> to language syntax, so I continued to use this feature for snippets that 
> are not intended to be evaluated. Later it was a bad surprise that 
> export does not preserves highlights. So for me #+begin_example is a 
> shortcut for #+begin_src with :eval never (that can be specified as a 
> property).
>
>  From my point of view exporters may reuse code block formatters for 
> examples. It would allow to pass options to e.g. LaTeX verbatim environment:
> Pedro Andres Aranda Gutierrez. Re: A question/bug report(?) Wed, 30 Mar 
> 2022 07:14:54 +0200 
> https://list.orgmode.org/orgmode/CAO48Bk_dJs1=5zgpcZwODaTsRqyRsKq1ALj6WpaXCc4bDjW3FQ@mail.gmail.com/

Among other goals, the new fontification is aiming to avoid such bad
surprises.

As for lang parameter support in example blocks, would you mind creating
a separate feature request thread? Extending export blocks export will
require changing in parser syntax and thus should be discussed carefully
in a separate thread.

>> 2. org-script-display is an internal auxiliary variable used to display
>>     sub/superscripts. Note that it's default value holds 4 possibilities.
>>     Two for each type of script. For example, for superscripts we have
>>     two options:
>>     ((raise 0.3) (height 0.7)) and ((raise 0.5))
>>     The first one looks more compact (does not change the line height)
>
> Notice that subscript still increases inter-line space a bit.

Can you (and other interested users) try the following value of
org-script-display and let me know if it looks acceptable:

(defconst org-script-display  '(;; The values are tweaked to not change
                                ;; the line height.
                                ((raise -0.1)  (height 0.7))
				((raise 0.25)  (height 0.7))
                                ;; Alternative properties for tables.
                                ;; FIXME: We cannot change the text
                                ;; height because it will alter the
                                ;; symbol width and thus break the
                                ;; table alignment (at least, until
                                ;; org table are aligned via pixel
                                ;; width).
				((raise -0.5))
				((raise 0.5)))
  "Display properties for showing superscripts and subscripts.")

>>     However, it currently uses x1.5 line height for tables creating empty
>>     space between vertical | separators. It looks like a typo for me. It
>>     would make more sense to make table lines compact, not vice versa. Am
>>     I missing something?
>
> git blame gives 0618aeafb39dbf78e753348eaeaddbb7f8104cd0
> It seems smaller font breaks horizontal alignment in tables.

Thanks! Now it is crystal clear what was the reason behind the different
fontification inside/outside the tables. I will add the explanation to
comments.

Best,
Ihor

Re: [DISCUSSION] Refactoring fontification system

[Tom Gillespie]

> As for lang parameter support in example blocks, would you mind creating
> a separate feature request thread? Extending export blocks export will
> require changing in parser syntax and thus should be discussed carefully
> in a separate thread.

I would strongly caution against allowing an optional #+begin_example lang
syntax. It will lead to extreme confusion, even when users know to use org-lint.
The reason for this is that example blocks do not have (and frankly should not
have) full org-babel support. Babel is already complex enough as is without
having to explain to a user that yes they can noweb an example block into
a src block, but that they cannot noweb a source block into an example block.

One of the most powerful features of src blocks is that they can go from being
dumb examples all the way up to fully executable programs. Example blocks
cannot do that, and adding features that overlap with code blocks is inviting
duplicated effort and will confuse and frustrate users if they have
the misfortune
to start with an example block an then have to change mid way through to a
code block.

I also think that adding a parameter #+begin_example :lang bash to example
blocks will also lead to confusion because now there are two different ways
to specify what lang a block is. To me the answer should be to just use source
blocks if you need highlighting, example blocks should not highlight at all in
order to make the distinction clear.

Best,
Tom

Re: [DISCUSSION] Refactoring fontification system

[Tim Cross]

Tom Gillespie <tgbugs@gmail.com> writes:

>> As for lang parameter support in example blocks, would you mind creating
>> a separate feature request thread? Extending export blocks export will
>> require changing in parser syntax and thus should be discussed carefully
>> in a separate thread.
>
> I would strongly caution against allowing an optional #+begin_example lang
> syntax. It will lead to extreme confusion, even when users know to use org-lint.
> The reason for this is that example blocks do not have (and frankly should not
> have) full org-babel support. Babel is already complex enough as is without
> having to explain to a user that yes they can noweb an example block into
> a src block, but that they cannot noweb a source block into an example block.
>
> One of the most powerful features of src blocks is that they can go from being
> dumb examples all the way up to fully executable programs. Example blocks
> cannot do that, and adding features that overlap with code blocks is inviting
> duplicated effort and will confuse and frustrate users if they have
> the misfortune
> to start with an example block an then have to change mid way through to a
> code block.
>
> I also think that adding a parameter #+begin_example :lang bash to example
> blocks will also lead to confusion because now there are two different ways
> to specify what lang a block is. To me the answer should be to just use source
> blocks if you need highlighting, example blocks should not highlight at all in
> order to make the distinction clear.
>

+1. I hold the same view. 

I'm happy if example blocks have a highlighting which distinguishes them
as a 'block of something' i.e. slightly different background, smaller or different
font etc. However, they don't need font-locking style highlighting or
highlighting which is determined by a language setting. If you want that
level of highlighting, just use a src block, possibly disabling eval
when warranted.

Re: [DISCUSSION] Refactoring fontification system

[Phil Estival]

[2022-06-03 Wed 11:45] Ihor Radchenko <yantar92@gmail.com>:
 >
 > I'd like to hear if anyone has any idea on how to interpret the
 > following:
 >
 > 1. org-protecting-blocks is an internal auxiliary variable used to
 >    determine which blocks should be fontified using different major
 >    mode.
 >    It's value is ("src" "example" "export")
 >    So, #+begin_src lang and #+begin_export lang are fontified according
 >    to LANG. Makes sense.
 >    However, what about #+begin_example?
 >    org-element-example-block-parser does not appear to expect language
 >    specification in the example blocks. Only switches seems to be
 >    allowed. Am I missing something and Org actually allows example
 >    blocks to specify language? Or was it the case in the distant past
 >    versions of Org?


  - org-fontify-meta-lines-and-blocks-1
    is looking for begin_
    what comes after (src) is optional and can be anything

    Next it looks for "language" (match-string 5 to 7
    — it could be helpful to have comments indicating
    the number matching of the groups next to them).
    What gets fontified like a source block turns out to be:
    ,#+begin_{\w} <language> [<switches> <header arguments>]

    So this is fontified:

    #+begin_quote python
    def sss(): pass
    #+end_quote

    and this too:

    #+begin_fly awk
    BEGIN { woosh }
    #+end_fly

    Which is nice, but not interpreted like so
    by any export backend.


 >
 > 3. org-fontify-meta-lines-and-blocks-1 creates a special face for
 >    ("+title:" "+subtitle:" "+author:" "+email:" "+date:")
 >    The face name is org-document-info.
 >    But what about, say, +description: or +language:?
 >    Would it make more sense to fontify all the keywords from
 >    org-options-keywords instead?
 >

Makes more sense, yes.
I would have named them "directives"
rather than "keywords", but it's too late now.


Regards,
Phil

Re: [DISCUSSION] Refactoring fontification system

[Max Nikulin]

On 08/06/2022 11:23, Tom Gillespie wrote:
>> As for lang parameter support in example blocks, would you mind creating
>> a separate feature request thread? Extending export blocks export will
>> require changing in parser syntax and thus should be discussed carefully
>> in a separate thread.
> 
> I would strongly caution against allowing an optional #+begin_example lang
> syntax. It will lead to extreme confusion, even when users know to use org-lint.
> The reason for this is that example blocks do not have (and frankly should not
> have) full org-babel support. Babel is already complex enough as is without
> having to explain to a user that yes they can noweb an example block into
> a src block, but that they cannot noweb a source block into an example block.

My expectation is that treating #+begin_example almost as #+begin_src 
will simplify code. E.g. `org-latex-src-block' has a fallback to plain 
\begin{verbatim} if language is unknown or is not specified, so it may 
handle examples as well. The difference is that for #+begin_example 
evaluation and noweb are disabled, either it is hardcoded (my 
preference) or default values for :noweb and :eval header argument are 
adjusted.

On 09/06/2022 12:37, Pedro Andres Aranda Gutierrez wrote:
> OK, just to add to the discussion. The original intent of my message was
> to give _me_ control over what I want to colour, not delegating that to
> the fontifying engine in Emacs.
> 
> I need limited and controlled colouring for text-books and lab manuals
> where colouring elements of a listing goes against the publishing
> standards. I only need custom colouring to "emulate" terminal output,
> not for the code as such. And I could do that with the #+ATTR: if supported.

Fontification and an attribute to specify an option for environment are 
independent features. My point is that if example and src blocks reused 
the same code than it would be enough to implement :options attribute 
(similar to #+begin_quote) once and both types of blocks would be more 
flexible.

If #+begin_example becomes more dumb, unsure if it is reasonable to 
allow to generate "\begin{verbatim}[commandchars=\\\{\}]" for example 
block but not for source blocks.

Side note: I can not say that #+begin_example with language is widely 
used, but one case may be found in org-manual.org and a dozen in worg.

[PATCH] #+begin_example lang used in manual and worg (was: [DISCUSSION] Refactoring fontification system)

[Ihor Radchenko]

[-- Attachment #1: Type: text/plain, Size: 331 bytes --]

Max Nikulin <manikulin@gmail.com> writes:

> Side note: I can not say that #+begin_example with language is widely 
> used, but one case may be found in org-manual.org and a dozen in worg.

Which should probably be fixed. LANG parameter is ignored on export
anyway. See the attached patches for WORG and emacs-manual.

Best,
Ihor


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-org-manual-Fix-example-block-with-unsupported-langua.patch --]
[-- Type: text/x-patch, Size: 940 bytes --]

From 1f5419fb4c249d6e53940ff16e27e16da55c5dbc Mon Sep 17 00:00:00 2001
Message-Id: <1f5419fb4c249d6e53940ff16e27e16da55c5dbc.1654826607.git.yantar92@gmail.com>
From: Ihor Radchenko <yantar92@gmail.com>
Date: Fri, 10 Jun 2022 10:02:51 +0800
Subject: [PATCH] org-manual: Fix example block with unsupported language name
 parameter

* doc/org-manual.org (Breaking Down Tasks into Subtasks): Remove
unsupported LANG parameter from #+begin_example LANG.
---
 doc/org-manual.org | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 32a45f884..51aa4d77e 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -4488,7 +4488,7 @@ ** Breaking Down Tasks into Subtasks
 include the word =recursive= into the value of the =COOKIE_DATA=
 property.
 
-#+begin_example org
+#+begin_example
 ,* Parent capturing statistics [2/20]
   :PROPERTIES:
   :COOKIE_DATA: todo recursive
-- 
2.35.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0001-Fix-example-blocks-with-unsupported-language-name-pa.patch --]
[-- Type: text/x-patch, Size: 4254 bytes --]

From 4a43d570b5424dadfda74996bddc4982aadafe37 Mon Sep 17 00:00:00 2001
Message-Id: <4a43d570b5424dadfda74996bddc4982aadafe37.1654826554.git.yantar92@gmail.com>
From: Ihor Radchenko <yantar92@gmail.com>
Date: Fri, 10 Jun 2022 10:01:10 +0800
Subject: [PATCH] Fix example blocks with unsupported language name parameter

* org-contrib/babel/languages/ob-doc-shell.org (Requirements and Setup):
* org-hacks.org (Reschedule agenda items to today with a single command):
* org-tutorials/org-jekyll.org (Creating an org File to be Published with Jekyll):
(Showing Blog Posts on the Front Page):
(Creating Archive Pages):
(Inserting Image):
(Using Text Markup in Front Matte): Remove LANG in #+begin_example LANG.
---
 org-contrib/babel/languages/ob-doc-shell.org |  2 +-
 org-hacks.org                                |  2 +-
 org-tutorials/org-jekyll.org                 | 16 ++++++++--------
 3 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/org-contrib/babel/languages/ob-doc-shell.org b/org-contrib/babel/languages/ob-doc-shell.org
index 730f63d7..e7d728b3 100644
--- a/org-contrib/babel/languages/ob-doc-shell.org
+++ b/org-contrib/babel/languages/ob-doc-shell.org
@@ -179,7 +179,7 @@ * Requirements and Setup
 2. The Org Babel language facility must be set to load shell
    functionality[fn:5]:
 
-    #+begin_example emacs-lisp
+    #+begin_example
     ;; active Babel languages
     (org-babel-do-load-languages
     'org-babel-load-languages
diff --git a/org-hacks.org b/org-hacks.org
index 968e511d..d446de9b 100644
--- a/org-hacks.org
+++ b/org-hacks.org
@@ -2196,7 +2196,7 @@ *** Reschedule agenda items to today with a single command
 #+index: Agenda!Reschedule
 This was suggested by Carsten in reply to David Abrahams:
 
-#+begin_example emacs-lisp
+#+begin_example
 (defun org-agenda-reschedule-to-today ()
   (interactive)
   (flet ((org-read-date (&rest rest) (current-time)))
diff --git a/org-tutorials/org-jekyll.org b/org-tutorials/org-jekyll.org
index 4da2d5ad..a0c57991 100644
--- a/org-tutorials/org-jekyll.org
+++ b/org-tutorials/org-jekyll.org
@@ -172,7 +172,7 @@ * Creating an org File to be Published with Jekyll
 
 Below is a short extract from one of my org files showing my setup:
 
-#+BEGIN_EXAMPLE org
+#+BEGIN_EXAMPLE
 #+STARTUP: showall indent
 #+STARTUP: hidestars
 #+BEGIN_EXPORT html
@@ -254,7 +254,7 @@ ** Showing Blog Posts on the Front Page
 Most blogs show the latest posts on their front page. The example
 below shows the title and an excerpt for the five latest posts:
 
-#+BEGIN_EXAMPLE html
+#+BEGIN_EXAMPLE
 <ul class="posts">
 {% for post in site.posts limit: 5 %}
   <div class="post_info">
@@ -274,7 +274,7 @@ ** Creating Archive Pages
 available. You can create a simple list of all blog posts using the
 following markup:
 
-#+begin_example html
+#+begin_example
 <ul>
   {% for post in site.posts %}
   <li>
@@ -295,7 +295,7 @@ * Inserting Image
 You will probably want to insert some images into your blog posts. I
 use the following method:
 
-#+BEGIN_EXAMPLE html
+#+BEGIN_EXAMPLE
 <img src ="/images/skiddaw.jpg"
 alt="John and Ella on Skiddaw" align="left" width="300" height="250"
 title="John and Ella on Skiddaw" class="img"</img>
@@ -304,7 +304,7 @@ * Inserting Image
 Note that the class attribute refers to the class used to style the
 image tag in your css. My css contains:
 
-#+BEGIN_EXAMPLE css
+#+BEGIN_EXAMPLE
 img {
     margin: 15px;
     border: 1px solid blue;
@@ -323,7 +323,7 @@ * Inserting Image
 
 In your =.org= file use the following html to embed the picture:
 
-#+BEGIN_EXAMPLE html
+#+BEGIN_EXAMPLE
 <div class="photofloatr">
   <p><img src="myphoto.jpg" width="300"
     height="150" alt="My Mug Shot"></p>
@@ -333,7 +333,7 @@ * Inserting Image
 
 Now you need to add some information to your style sheet:
 
-#+BEGIN_EXAMPLE css
+#+BEGIN_EXAMPLE
 div.photofloatr {
     float: right;
     border: thin silver solid;
@@ -362,7 +362,7 @@ * Using Text Markup in Front Matte
 I use this to format my page excerpts, which I include in my org files
 Front Matter markup. So in my sites index.html I have:
 
-#+begin_example html
+#+begin_example
 <li>
   <a href="{{ post.url }}">{{ post.title }}</a>
   <span>({{ post.date | date:"%Y-%m-%d" }})</span>
-- 
2.35.1

Re: [PATCH] #+begin_example lang used in manual and worg (was: [DISCUSSION] Refactoring fontification system)

[Max Nikulin]

On 10/06/2022 09:06, Ihor Radchenko wrote:
> Max Nikulin writes:
> 
>> Side note: I can not say that #+begin_example with language is widely
>> used, but one case may be found in org-manual.org and a dozen in worg.
> 
> Which should probably be fixed. LANG parameter is ignored on export
> anyway. See the attached patches for WORG and emacs-manual.

Thank you for your intent to fix it.

> diff --git a/doc/org-manual.org b/doc/org-manual.org
> index 32a45f884..51aa4d77e 100644
> --- a/doc/org-manual.org
> +++ b/doc/org-manual.org
> @@ -4488,7 +4488,7 @@ ** Breaking Down Tasks into Subtasks
>  include the word =recursive= into the value of the =COOKIE_DATA=
>  property.
>  
> -#+begin_example org
> +#+begin_example
>  ,* Parent capturing statistics [2/20]
>    :PROPERTIES:
>    :COOKIE_DATA: todo recursive

It is consistent with other examples in the manual. Only one snippet is 
wrapped into "#+begin_src org" and it is a recent addition caused a long 
discussion on Shakespeare's poetry. I am curious why #+begin_src is used 
for elisp examples, but not for org markup.

For worg pages #+begin_example to #+begin_src substitution may be a 
better option than dropping language.

> --- a/org-tutorials/org-jekyll.org
> +++ b/org-tutorials/org-jekyll.org
> @@ -172,7 +172,7 @@ * Creating an org File to be Published with Jekyll
>  
>  Below is a short extract from one of my org files showing my setup:
>  
> -#+BEGIN_EXAMPLE org
> +#+BEGIN_EXAMPLE
>  #+STARTUP: showall indent
>  #+STARTUP: hidestars
>  #+BEGIN_EXPORT html

It is not the scope of this patch but looks like missed commas to escape 
leading "#".

Re: [PATCH] #+begin_example lang used in manual and worg (was: [DISCUSSION] Refactoring fontification system)

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

>> -#+begin_example org
>> +#+begin_example
>>  ,* Parent capturing statistics [2/20]
>>    :PROPERTIES:
>>    :COOKIE_DATA: todo recursive
>
> It is consistent with other examples in the manual. Only one snippet is 
> wrapped into "#+begin_src org" and it is a recent addition caused a long 
> discussion on Shakespeare's poetry. I am curious why #+begin_src is used 
> for elisp examples, but not for org markup.

I am not sure. In theory, the only significant difference could be that
#+begin_src org could be fontified natively on export (some time in
future; they are not now), which might be slightly confusing.

In any case, I applied the Org manual patch onto main via 0cc4f492f.

> For worg pages #+begin_example to #+begin_src substitution may be a 
> better option than dropping language.

You are right. Would you mind checking if there are any #+begin_example
cases in worg with no language specified that are also worth converting
into #+begin_src?

Best,
Ihor

[BUG] Unescaped #+ lines in WORG example blocks (was: [PATCH] #+begin_example lang used in manual and worg (was: [DISCUSSION] Refactoring fontification system))

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

>> --- a/org-tutorials/org-jekyll.org
>> +++ b/org-tutorials/org-jekyll.org
>> @@ -172,7 +172,7 @@ * Creating an org File to be Published with Jekyll
>>  
>>  Below is a short extract from one of my org files showing my setup:
>>  
>> -#+BEGIN_EXAMPLE org
>> +#+BEGIN_EXAMPLE
>>  #+STARTUP: showall indent
>>  #+STARTUP: hidestars
>>  #+BEGIN_EXPORT html
>
> It is not the scope of this patch but looks like missed commas to escape 
> leading "#".

You are right. We need someone to look through worg pages and spot all
such instances.
(Help appreciated)

Best,
Ihor

Re: [DISCUSSION] Refactoring fontification system

[Max Nikulin]

On 08/06/2022 09:02, Ihor Radchenko wrote:
> Max Nikulin writes:
> 
> Can you (and other interested users) try the following value of
> org-script-display and let me know if it looks acceptable:
> 
> (defconst org-script-display  '(;; The values are tweaked to not change
>                                  ;; the line height.
>                                  ((raise -0.1)  (height 0.7))
> 				((raise 0.25)  (height 0.7))

I do not mind. High symbols like "(|)" cross middle line a bit, but 
since it is not TeX and superscript can not be above subscript it should 
not cause any problem with overlapping.

>                                  ;; Alternative properties for tables.
>                                  ;; FIXME: We cannot change the text
>                                  ;; height because it will alter the
>                                  ;; symbol width and thus break the
>                                  ;; table alignment (at least, until
>                                  ;; org table are aligned via pixel
>                                  ;; width).
> 				((raise -0.5))
> 				((raise 0.5)))

The issue with increased vertical space between vertical bars might be 
alleviated a bit by using e.g. 0.35 instead of 0.5. I hope, subscripts 
and superscripts are still distinguishable.

>>>      However, it currently uses x1.5 line height for tables creating empty
>>>      space between vertical | separators. It looks like a typo for me. It
>>>      would make more sense to make table lines compact, not vice versa. Am
>>>      I missing something?
>>
>> git blame gives 0618aeafb39dbf78e753348eaeaddbb7f8104cd0
>> It seems smaller font breaks horizontal alignment in tables.
> 
> Thanks! Now it is crystal clear what was the reason behind the different
> fontification inside/outside the tables. I will add the explanation to
> comments.

Re: [BUG] Unescaped #+ lines in WORG example blocks (was: [PATCH] #+begin_example lang used in manual and worg (was: [DISCUSSION] Refactoring fontification system))

[Tim Cross]

Ihor Radchenko <yantar92@gmail.com> writes:

> Max Nikulin <manikulin@gmail.com> writes:
>
>>> --- a/org-tutorials/org-jekyll.org
>>> +++ b/org-tutorials/org-jekyll.org
>>> @@ -172,7 +172,7 @@ * Creating an org File to be Published with Jekyll
>>>  
>>>  Below is a short extract from one of my org files showing my setup:
>>>  
>>> -#+BEGIN_EXAMPLE org
>>> +#+BEGIN_EXAMPLE
>>>  #+STARTUP: showall indent
>>>  #+STARTUP: hidestars
>>>  #+BEGIN_EXPORT html
>>
>> It is not the scope of this patch but looks like missed commas to escape 
>> leading "#".
>
> You are right. We need someone to look through worg pages and spot all
> such instances.
> (Help appreciated)
>

Just FYI I plan to go through all the worg files and run org-lint over
them as well as fix up the many problems I've found so far with broken
links, missing images, missing macros, examples not in example blocks
etc. So while I'd appreciate any help on offer, don't worry too much
about fixing some of these as I will get to them eventually. 

One area I would definitely like some help with though are those org
files not in english. For example, there is an org tutorial written in
Spanish which is generating bad link errors. Unfortunately, I'm limited
to only one language, so not 100% sure on the right fix (it looks like
examples of  how to do links in org, but they are not in example blocks,
so org throws an error when doing the export to html. .