* [PATCH] ANSI color on example blocks and fixed width elements
@ 2023-04-05 12:03 Nathaniel Nicandro
2023-04-05 13:43 ` Ihor Radchenko
0 siblings, 1 reply; 9+ messages in thread
From: Nathaniel Nicandro @ 2023-04-05 12:03 UTC (permalink / raw)
To: emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 719 bytes --]
Hello,
Attached is the patch. Without this patch, ANSI escape sequences
generated by the output of a source block will be left in the buffer
without any fontification. With this patch, the escaped text is nicely
colored and escape sequences hidden using overlays.
It works for Emacs versions which have the `PRESERVE-SEQUENCES` argument
to the `ansi-color-apply-on-region` function. It's a bit slow due to
the use of overlays. My implementation of this feature in Emacs-Jupyter
supports older versions of Emacs without that argument, it relies on a
custom version of that function though and uses text properties instead
of overlays.
Let me know what else could be done on my end to get this patch in.
Thanks.
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: ANSI color patch --]
[-- Type: text/x-patch, Size: 1571 bytes --]
diff --git a/lisp/org.el b/lisp/org.el
index 4d12084..24617ad 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -81,6 +81,7 @@ (eval-when-compile (require 'gnus-sum))
(require 'calendar)
(require 'find-func)
(require 'format-spec)
+(require 'ansi-color)
(condition-case nil
(load (concat (file-name-directory load-file-name)
@@ -5326,6 +5327,10 @@ (defsubst org-activate-links (limit)
(defun org-activate-code (limit)
(when (re-search-forward "^[ \t]*\\(:\\(?: .*\\|$\\)\n?\\)" limit t)
(org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0))
+ (let ((ansi-color-apply-face-function
+ (lambda (beg end face)
+ (font-lock-prepend-text-property beg end 'face face))))
+ (ansi-color-apply-on-region (match-beginning 0) (match-end 0) t))
(remove-text-properties (match-beginning 0) (match-end 0)
'(display t invisible t intangible t))
t))
@@ -5421,7 +5426,12 @@ (defun org-fontify-meta-lines-and-blocks-1 (limit)
(let ((face-name
(intern (format "org-block-%s" lang))))
(append (and (facep face-name) (list face-name))
- '(org-block)))))))
+ '(org-block))))))
+ (let ((ansi-color-apply-face-function
+ (lambda (beg end face)
+ (font-lock-prepend-text-property beg end 'face face))))
+ (ansi-color-apply-on-region
+ bol-after-beginline beg-of-endline t)))
((not org-fontify-quote-and-verse-blocks))
((string= block-type "quote")
(add-face-text-property
[-- Attachment #3: Type: text/plain, Size: 16 bytes --]
--
Nathaniel
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH] ANSI color on example blocks and fixed width elements
2023-04-05 12:03 [PATCH] ANSI color on example blocks and fixed width elements Nathaniel Nicandro
@ 2023-04-05 13:43 ` Ihor Radchenko
2023-04-13 20:18 ` [PATCH] Highlight ANSI sequences in the whole buffer (was [PATCH] ANSI color on example blocks and fixed width elements) Nathaniel Nicandro
0 siblings, 1 reply; 9+ messages in thread
From: Ihor Radchenko @ 2023-04-05 13:43 UTC (permalink / raw)
To: Nathaniel Nicandro; +Cc: emacs-orgmode
Nathaniel Nicandro <nathanielnicandro@gmail.com> writes:
> Attached is the patch. Without this patch, ANSI escape sequences
> generated by the output of a source block will be left in the buffer
> without any fontification. With this patch, the escaped text is nicely
> colored and escape sequences hidden using overlays.
>
> It works for Emacs versions which have the `PRESERVE-SEQUENCES` argument
> to the `ansi-color-apply-on-region` function. It's a bit slow due to
> the use of overlays. My implementation of this feature in Emacs-Jupyter
> supports older versions of Emacs without that argument, it relies on a
> custom version of that function though and uses text properties instead
> of overlays.
>
> Let me know what else could be done on my end to get this patch in.
> Thanks.
Thanks for the patch!
This is an interesting idea, but I am not sure if we want to use this
colouring by default. At least, it should be a minor mode. Probably
enabled by default. Because not every possible user may want to have the
escape sequences hidden away.
Further, your patch only allows fontifying ANSI sequences in fixed-width
elements, example blocks, export blocks, and src blocks without known
major mode that does the fontification. I doubt that fontifying ANSI
sequences in this specific subset of elements always makes sense -
example blocks are not always used as src block output; bash code blocks
may purposely contain escape sequences, but your patch will not handle
them; inline src block output is not covered at all.
Ideally, fontifying ANSI sequences should be fully controlled by users:
1. We may not want to touch src blocks by default, when
`org-src-fontify-natively' is set to t. Only, maybe, provide an
option. Or you may better publish a minor mode that does this for
shell scripts.
2. We may allow all the ANSI sequences to be fontified in the whole
buffer.
3. We may limit ANSI sequence fontification to results and only results.
Or just certain types of results.
The easiest will be implementing fontification in the whole buffer,
early during fontification (and early in org-font-lock-keywords; see
org-font-lock-set-keywords-hook).
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH] Highlight ANSI sequences in the whole buffer (was [PATCH] ANSI color on example blocks and fixed width elements)
2023-04-05 13:43 ` Ihor Radchenko
@ 2023-04-13 20:18 ` Nathaniel Nicandro
2023-04-14 8:49 ` Ihor Radchenko
0 siblings, 1 reply; 9+ messages in thread
From: Nathaniel Nicandro @ 2023-04-13 20:18 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: Nathaniel Nicandro, emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 2861 bytes --]
Ihor Radchenko <yantar92@posteo.net> writes:
> Nathaniel Nicandro <nathanielnicandro@gmail.com> writes:
>
>> Attached is the patch. Without this patch, ANSI escape sequences
>> generated by the output of a source block will be left in the buffer
>> without any fontification. With this patch, the escaped text is nicely
>> colored and escape sequences hidden using overlays.
>>
>> It works for Emacs versions which have the `PRESERVE-SEQUENCES` argument
>> to the `ansi-color-apply-on-region` function. It's a bit slow due to
>> the use of overlays. My implementation of this feature in Emacs-Jupyter
>> supports older versions of Emacs without that argument, it relies on a
>> custom version of that function though and uses text properties instead
>> of overlays.
>>
>> Let me know what else could be done on my end to get this patch in.
>> Thanks.
>
> Thanks for the patch!
>
> This is an interesting idea, but I am not sure if we want to use this
> colouring by default. At least, it should be a minor mode. Probably
> enabled by default. Because not every possible user may want to have the
> escape sequences hidden away.
>
> Further, your patch only allows fontifying ANSI sequences in fixed-width
> elements, example blocks, export blocks, and src blocks without known
> major mode that does the fontification. I doubt that fontifying ANSI
> sequences in this specific subset of elements always makes sense -
> example blocks are not always used as src block output; bash code blocks
> may purposely contain escape sequences, but your patch will not handle
> them; inline src block output is not covered at all.
>
> Ideally, fontifying ANSI sequences should be fully controlled by users:
> 1. We may not want to touch src blocks by default, when
> `org-src-fontify-natively' is set to t. Only, maybe, provide an
> option. Or you may better publish a minor mode that does this for
> shell scripts.
> 2. We may allow all the ANSI sequences to be fontified in the whole
> buffer.
I've updated my patch to be a combination of (1) and (2), see the
attached patch. Essentially every sequence is fontified except those in
source blocks and a minor mode has been created to allow users to
disable or enable fontification whenever they want.
I've also attached an example Org file with some ANSI sequences in it
for testing purposes that you can try out.
One issue that remains is how to handle sequences within inline source
blocks. Those don't have a src-block property so any sequences within
an inline source block are currently fontified.
> 3. We may limit ANSI sequence fontification to results and only results.
> Or just certain types of results.
>
> The easiest will be implementing fontification in the whole buffer,
> early during fontification (and early in org-font-lock-keywords; see
> org-font-lock-set-keywords-hook).
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: test-ansi.org --]
[-- Type: text/x-org, Size: 608 bytes --]
* This is a ^[[42mtest^[[0m
Of ^[[31mANSI^[[0m ^[[33mcolor^[[0m sequences
#+begin_src python
for x in y:
print(x + "this is a ^[[43mtest^[[0m")
#+end_src
: this ^[[42mis a^[[0m td
=testing=
In paragraph a ~color ^[[44msequ^[[0mence~ is ^[[41mhere^[[0m.
^[[43mThis is a sequence that covers a block
#+begin_example
should be colored
#+end_example
there should be an end here^[[0m there is the end.
begin ^[[43m
sequence
without end
#+begin_src python
1 + 1
#+end_src
Inline source blocks will have sequences highlighted because we only
look for a src-block text property.
src_python{return "t^[[43mest^[[0ming"}
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0001-Highlight-ANSI-escape-sequences.patch --]
[-- Type: text/x-patch, Size: 7561 bytes --]
From c9b505d022410a481210928ecc4cce1f199ec53b Mon Sep 17 00:00:00 2001
From: Nathaniel Nicandro <nathanielnicandro@gmail.com>
Date: Thu, 13 Apr 2023 15:06:35 -0500
Subject: [PATCH] Highlight ANSI escape sequences
* etc/ORG-NEWS: Describe the new feature.
* org.el (org-fontify-ansi-sequences): New customization variable and
function which does the work of fontifying the sequences.
(org-set-font-lock-defaults): Add the `org-fontify-ansi-sequences`
function to the font-lock keywords.
(org-ansi-mode): New minor mode to enable/disable highlighting of the
sequences. Enabled in Org buffers by default.
---
etc/ORG-NEWS | 12 ++++++
lisp/org.el | 112 +++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 124 insertions(+)
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index b7c88fd..8690540 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -169,6 +169,18 @@ official [[https://clojure.org/guides/deps_and_cli][Clojure CLI tools]].
The command can be customized with ~ob-clojure-cli-command~.
** New features
+*** ANSI escape sequences are now highlighted in the whole buffer
+
+A new customization ~org-fontify-ansi-sequences~ is available which
+tells Org to highlight all ANSI sequences in the buffer if non-nil and
+the new minor mode ~org-ansi-mode~ is enabled.
+
+To disable highlighting of the sequences you can either
+disable ~org-ansi-mode~ or set ~org-fontify-ansi-sequences~ to ~nil~
+and =M-x revert-buffer RET=. Doing the latter will disable
+highlighting of sequences in all newly opened Org buffers whereas
+doing the former disables highlighting locally to the current buffer.
+
*** Add support for ~logind~ idle time in ~org-user-idle-seconds~
When Emacs is built with =dbus= support and
diff --git a/lisp/org.el b/lisp/org.el
index 26d2a86..62a5134 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -81,6 +81,7 @@ (eval-when-compile (require 'gnus-sum))
(require 'calendar)
(require 'find-func)
(require 'format-spec)
+(require 'ansi-color)
(condition-case nil
(load (concat (file-name-directory load-file-name)
@@ -3582,6 +3583,12 @@ (defcustom org-fontify-whole-block-delimiter-line t
:group 'org-appearance
:type 'boolean)
+(defcustom org-fontify-ansi-sequences t
+ "Non-nil means to highlight ANSI escape sequences."
+ :group 'org-appearance
+ :type 'boolean
+ :package-version '(Org . "9.7"))
+
(defcustom org-highlight-latex-and-related nil
"Non-nil means highlight LaTeX related syntax in the buffer.
When non-nil, the value should be a list containing any of the
@@ -5543,6 +5550,72 @@ (defun org-fontify-extend-region (beg end _old-len)
(cons beg (or (funcall extend "end" "]" 1) end)))
(t (cons beg end))))))
+(defvar org-ansi-mode)
+
+(defun org-fontify-ansi-sequences (limit)
+ "Fontify ANSI sequences."
+ (when (and org-fontify-ansi-sequences org-ansi-mode)
+ (let (end)
+ (while (/= (point) limit)
+ (cond
+ ((get-text-property (point) 'src-block)
+ ;; If point is on a src block, skip over it
+ (goto-char (next-single-property-change (point) 'src-block nil limit))
+ (save-restriction
+ ;; Prevent moving past limit
+ (narrow-to-region (point) limit)
+ (forward-line)))
+ (t
+ (setq end (next-single-property-change (point) 'src-block nil limit))
+ (let ((src-block-beg (and (get-text-property end 'src-block) end)))
+ (when src-block-beg
+ ;; Set the end of the region to be fontified to be the
+ ;; beginning of the src block when end is not limit
+ (save-excursion
+ (goto-char src-block-beg)
+ (forward-line -1)
+ (org-skip-whitespace)
+ (setq end (point))))
+ (ansi-color-apply-on-region (point) end t)
+ ;; Reset the context before every fontification cycle. This
+ ;; avoids issues where `ansi-color-apply-on-region' attempts to
+ ;; use an old starting point that may be from a different part
+ ;; of the buffer, leading to "wrong side of point" errors.
+ (setq ansi-color-context-region nil)
+ (goto-char (or src-block-beg end)))))))))
+
+(defvar org-ansi-colors
+ '(black red green yellow blue purple cyan white))
+
+(defun org-ansi-highlight (beg end seq)
+ (save-excursion
+ (goto-char end)
+ (insert "\e")
+ (insert "[0m")
+ (goto-char beg)
+ (insert "\e")
+ (insert (format "[%sm" seq))))
+
+(defun org-ansi-highlight-foreground (beg end color)
+ "Highlight the foreground between BEG and END with COLOR."
+ (interactive
+ (let ((bounds (car (region-bounds))))
+ (list (car bounds) (cdr bounds)
+ (completing-read "Color: " org-ansi-colors nil t))))
+ (let ((n (- (length org-ansi-colors)
+ (length (memq color org-ansi-colors)))))
+ (org-ansi-highlight beg end (+ 30 n))))
+
+(defun org-ansi-highlight-background (beg end color)
+ "Highlight the background between BEG and END with COLOR."
+ (interactive
+ (let ((bounds (car (region-bounds))))
+ (list (car bounds) (cdr bounds)
+ (completing-read "Color: " org-ansi-colors nil t))))
+ (let ((n (- (length org-ansi-colors)
+ (length (memq color org-ansi-colors)))))
+ (org-ansi-highlight beg end (+ 40 n))))
+
(defun org-activate-footnote-links (limit)
"Add text properties for footnotes."
(let ((fn (org-footnote-next-reference-or-definition limit)))
@@ -5861,6 +5934,7 @@ (defun org-set-font-lock-defaults ()
;; Blocks and meta lines
'(org-fontify-meta-lines-and-blocks)
'(org-fontify-inline-src-blocks)
+ '(org-fontify-ansi-sequences)
;; Citations. When an activate processor is specified, if
;; specified, try loading it beforehand.
(progn
@@ -15455,6 +15529,44 @@ (defun org-agenda-prepare-buffers (files)
(when org-agenda-file-menu-enabled
(org-install-agenda-files-menu))))
+\f
+;;;; ANSI sequences minor mode
+
+(defvar org-ansi-mode-map (make-sparse-keymap)
+ "Keymap for the minor `org-ansi-mode'.")
+
+(org-defkey org-ansi-mode-map (kbd "C-c hf") #'org-ansi-highlight-foreground)
+(org-defkey org-ansi-mode-map (kbd "C-c hb") #'org-ansi-highlight-background)
+
+(define-minor-mode org-ansi-mode
+ "Toggle the minor `org-ansi-mode'.
+This mode adds support to highlight ANSI sequences in Org mode.
+The sequences are highlighted only if the customization
+`org-fontify-ansi-sequences' is non-nil when the mode is enabled.
+\\{org-ansi-mode-map}"
+ :lighter " OANSI"
+ (cond
+ ((and org-fontify-ansi-sequences org-ansi-mode)
+ (remove-text-properties (point-min) (point-max) '(fontified t))
+ (font-lock-ensure))
+ (t
+ (dolist (ov (overlays-in (point-min) (point-max)))
+ ;; Attempt to find ANSI specific overlays. See
+ ;; `ansi-color-make-extent'.
+ (when (eq (car-safe (overlay-get ov 'insert-behind-hooks))
+ 'ansi-color-freeze-overlay)
+ ;; Delete the invisible overlays over the escape sequences
+ (dolist (ov (overlays-at (1- (overlay-start ov))))
+ (when (overlay-get ov 'invisible)
+ (delete-overlay ov)))
+ (dolist (ov (overlays-at (1+ (overlay-end ov))))
+ (when (overlay-get ov 'invisible)
+ (delete-overlay ov)))
+ ;; Delete the overlay over the highlighted text
+ (delete-overlay ov))))))
+
+(add-hook 'org-mode-hook #'org-ansi-mode)
+
\f
;;;; CDLaTeX minor mode
--
2.39.1
[-- Attachment #4: Type: text/plain, Size: 15 bytes --]
--
Nathaniel
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH] Highlight ANSI sequences in the whole buffer (was [PATCH] ANSI color on example blocks and fixed width elements)
2023-04-13 20:18 ` [PATCH] Highlight ANSI sequences in the whole buffer (was [PATCH] ANSI color on example blocks and fixed width elements) Nathaniel Nicandro
@ 2023-04-14 8:49 ` Ihor Radchenko
2023-04-25 20:33 ` Nathaniel Nicandro
0 siblings, 1 reply; 9+ messages in thread
From: Ihor Radchenko @ 2023-04-14 8:49 UTC (permalink / raw)
To: Nathaniel Nicandro; +Cc: emacs-orgmode
Nathaniel Nicandro <nathanielnicandro@gmail.com> writes:
>> Ideally, fontifying ANSI sequences should be fully controlled by users:
>> 1. We may not want to touch src blocks by default, when
>> `org-src-fontify-natively' is set to t. Only, maybe, provide an
>> option. Or you may better publish a minor mode that does this for
>> shell scripts.
>> 2. We may allow all the ANSI sequences to be fontified in the whole
>> buffer.
>
> I've updated my patch to be a combination of (1) and (2), see the
> attached patch. Essentially every sequence is fontified except those in
> source blocks and a minor mode has been created to allow users to
> disable or enable fontification whenever they want.
>
> I've also attached an example Org file with some ANSI sequences in it
> for testing purposes that you can try out.
Thanks!
> One issue that remains is how to handle sequences within inline source
> blocks. Those don't have a src-block property so any sequences within
> an inline source block are currently fontified.
You should not use 'src-block property at all. There are scenarios when
jit-lock defers source block fontification (in particular, when source
block spans beyond the screen) and 'src-block property is not yet
applied.
Instead, you should query `org-element-at-point' or
`org-element-context'.
> +*** ANSI escape sequences are now highlighted in the whole buffer
> +
> +A new customization ~org-fontify-ansi-sequences~ is available which
> +tells Org to highlight all ANSI sequences in the buffer if non-nil and
> +the new minor mode ~org-ansi-mode~ is enabled.
> +
> +To disable highlighting of the sequences you can either
> +disable ~org-ansi-mode~ or set ~org-fontify-ansi-sequences~ to ~nil~
> +and =M-x revert-buffer RET=. Doing the latter will disable
> +highlighting of sequences in all newly opened Org buffers whereas
> +doing the former disables highlighting locally to the current buffer.
Rather than asking to use revert-buffer, we usually suggest M-x
org-mode-restart.
> +(defun org-fontify-ansi-sequences (limit)
> + "Fontify ANSI sequences."
> + (when (and org-fontify-ansi-sequences org-ansi-mode)
> + (let (end)
> + (while (/= (point) limit)
Instead of this strict condition and later juggle with
`narrow-to-region', just use the usual (while (< (point) limit) ...).
> + (cond
> + ((get-text-property (point) 'src-block)
As I mentioned above, please use `org-element-at-point'. This function
will also give you information about the block boundaries.
> + (ansi-color-apply-on-region (point) end t)
We should probably limit ANSI colour pairs to a single Org element. It
does not make much sense to have text in-between the quotes below
coloured:
#+begin_quote
... <opening ANSI def> ...
#+end_quote
....
#+begin_quote
... <closing ANSI def> ...
#+end_quote
> + ;; Reset the context before every fontification cycle. This
> + ;; avoids issues where `ansi-color-apply-on-region' attempts to
> + ;; use an old starting point that may be from a different part
> + ;; of the buffer, leading to "wrong side of point" errors.
> + (setq ansi-color-context-region nil)
This looks fragile. AFAIU, `ansi-color-context-region' is used to track
currently active ANSI colour settings. Since your fontification function
may be called with various LIMITs, depending on what is displayed on the
user screen, the fontification results might be unpredictable for ANSI
defs spanning across multiple screens.
> +(defvar org-ansi-colors
> + '(black red green yellow blue purple cyan white))
> +
> +(defun org-ansi-highlight (beg end seq)
> + (save-excursion
> + (goto-char end)
> + (insert "\e")
> + (insert "[0m")
> + (goto-char beg)
> + (insert "\e")
> + (insert (format "[%sm" seq))))
> +
> +(defun org-ansi-highlight-foreground (beg end color)
> + "Highlight the foreground between BEG and END with COLOR."
> + (interactive
> + (let ((bounds (car (region-bounds))))
> + (list (car bounds) (cdr bounds)
> + (completing-read "Color: " org-ansi-colors nil t))))
> + (let ((n (- (length org-ansi-colors)
> + (length (memq color org-ansi-colors)))))
> + (org-ansi-highlight beg end (+ 30 n))))
> +
> +(defun org-ansi-highlight-background (beg end color)
> + "Highlight the background between BEG and END with COLOR."
> + (interactive
> + (let ((bounds (car (region-bounds))))
> + (list (car bounds) (cdr bounds)
> + (completing-read "Color: " org-ansi-colors nil t))))
> + (let ((n (- (length org-ansi-colors)
> + (length (memq color org-ansi-colors)))))
> + (org-ansi-highlight beg end (+ 40 n))))
The above has no relation to fontification and does not belong to Org in
general. Org syntax has no notion of ANSI escapes. We may support them
as a useful feature, but no more. Editing ANSI escapes would make more
sense in shell-script-mode or similar.
> + :lighter " OANSI"
> + (cond
> + ((and org-fontify-ansi-sequences org-ansi-mode)
> + (remove-text-properties (point-min) (point-max) '(fontified t))
> + (font-lock-ensure))
Just use `org-restart-font-lock'.
> + (t
> + (dolist (ov (overlays-in (point-min) (point-max)))
> + ;; Attempt to find ANSI specific overlays. See
> + ;; `ansi-color-make-extent'.
> + (when (eq (car-safe (overlay-get ov 'insert-behind-hooks))
> + 'ansi-color-freeze-overlay)
This is extremely awkward and relies on internal implementation details
of ansi-color. Moreover, we must avoid overlays, if possible - they do
not scale well. I recommend re-defining `ansi-color-apply-face-function'
to something that uses text properties. Using text properties will also
make restarting font-lock sufficient to clear the fontification.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] Highlight ANSI sequences in the whole buffer (was [PATCH] ANSI color on example blocks and fixed width elements)
2023-04-14 8:49 ` Ihor Radchenko
@ 2023-04-25 20:33 ` Nathaniel Nicandro
2023-05-10 10:27 ` Ihor Radchenko
0 siblings, 1 reply; 9+ messages in thread
From: Nathaniel Nicandro @ 2023-04-25 20:33 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 6999 bytes --]
Ihor Radchenko <yantar92@posteo.net> writes:
> Nathaniel Nicandro <nathanielnicandro@gmail.com> writes:
>
>>> Ideally, fontifying ANSI sequences should be fully controlled by users:
>>> 1. We may not want to touch src blocks by default, when
>>> `org-src-fontify-natively' is set to t. Only, maybe, provide an
>>> option. Or you may better publish a minor mode that does this for
>>> shell scripts.
>>> 2. We may allow all the ANSI sequences to be fontified in the whole
>>> buffer.
>>
>> I've updated my patch to be a combination of (1) and (2), see the
>> attached patch. Essentially every sequence is fontified except those in
>> source blocks and a minor mode has been created to allow users to
>> disable or enable fontification whenever they want.
>>
>> I've also attached an example Org file with some ANSI sequences in it
>> for testing purposes that you can try out.
>
> Thanks!
>
>> One issue that remains is how to handle sequences within inline source
>> blocks. Those don't have a src-block property so any sequences within
>> an inline source block are currently fontified.
>
> You should not use 'src-block property at all. There are scenarios when
> jit-lock defers source block fontification (in particular, when source
> block spans beyond the screen) and 'src-block property is not yet
> applied.
>
> Instead, you should query `org-element-at-point' or
> `org-element-context'.
The attached patch now uses `org-element-at-point' and
`org-element-context' to query for the bounds of elements.
Note, I've also attached an updated example file which shows that the
escape sequences in inline source blocks are now handled similarly to
regular source blocks, i.e. they are not fontified.
>
>> +*** ANSI escape sequences are now highlighted in the whole buffer
>> +
>> +A new customization ~org-fontify-ansi-sequences~ is available which
>> +tells Org to highlight all ANSI sequences in the buffer if non-nil and
>> +the new minor mode ~org-ansi-mode~ is enabled.
>> +
>> +To disable highlighting of the sequences you can either
>> +disable ~org-ansi-mode~ or set ~org-fontify-ansi-sequences~ to ~nil~
>> +and =M-x revert-buffer RET=. Doing the latter will disable
>> +highlighting of sequences in all newly opened Org buffers whereas
>> +doing the former disables highlighting locally to the current buffer.
>
> Rather than asking to use revert-buffer, we usually suggest M-x
> org-mode-restart.
Done.
>
>> +(defun org-fontify-ansi-sequences (limit)
>> + "Fontify ANSI sequences."
>> + (when (and org-fontify-ansi-sequences org-ansi-mode)
>> + (let (end)
>> + (while (/= (point) limit)
>
> Instead of this strict condition and later juggle with
> `narrow-to-region', just use the usual (while (< (point) limit) ...).
>
Done.
>> + (cond
>> + ((get-text-property (point) 'src-block)
>
> As I mentioned above, please use `org-element-at-point'. This function
> will also give you information about the block boundaries.
>
>> + (ansi-color-apply-on-region (point) end t)
>
> We should probably limit ANSI colour pairs to a single Org element. It
> does not make much sense to have text in-between the quotes below
> coloured:
>
> #+begin_quote
> ... <opening ANSI def> ...
> #+end_quote
>
>
> ....
>
> #+begin_quote
> ... <closing ANSI def> ...
> #+end_quote
>
Makes sense. Done.
>> + ;; Reset the context before every fontification cycle. This
>> + ;; avoids issues where `ansi-color-apply-on-region' attempts to
>> + ;; use an old starting point that may be from a different part
>> + ;; of the buffer, leading to "wrong side of point" errors.
>> + (setq ansi-color-context-region nil)
>
> This looks fragile. AFAIU, `ansi-color-context-region' is used to track
> currently active ANSI colour settings. Since your fontification function
> may be called with various LIMITs, depending on what is displayed on the
> user screen, the fontification results might be unpredictable for ANSI
> defs spanning across multiple screens.
>
It seems to be safe to reset `ansi-color-context-region' now given that
org-element is used to find the bounds of the element at
`point'. Although the fontification limits are dependent on screen size,
the org-element functions are not and so the bounds used when applying
the fontification for the ANSI sequences won't depend on screen size
either.
Also, re-setting `ansi-color-context-region' has the effect of not
propagating previously applied color settings to other Org elements.
>> +(defvar org-ansi-colors
>> + '(black red green yellow blue purple cyan white))
>> +
>> +(defun org-ansi-highlight (beg end seq)
>> + (save-excursion
>> + (goto-char end)
>> + (insert "\e")
>> + (insert "[0m")
>> + (goto-char beg)
>> + (insert "\e")
>> + (insert (format "[%sm" seq))))
>> +
>> +(defun org-ansi-highlight-foreground (beg end color)
>> + "Highlight the foreground between BEG and END with COLOR."
>> + (interactive
>> + (let ((bounds (car (region-bounds))))
>> + (list (car bounds) (cdr bounds)
>> + (completing-read "Color: " org-ansi-colors nil t))))
>> + (let ((n (- (length org-ansi-colors)
>> + (length (memq color org-ansi-colors)))))
>> + (org-ansi-highlight beg end (+ 30 n))))
>> +
>> +(defun org-ansi-highlight-background (beg end color)
>> + "Highlight the background between BEG and END with COLOR."
>> + (interactive
>> + (let ((bounds (car (region-bounds))))
>> + (list (car bounds) (cdr bounds)
>> + (completing-read "Color: " org-ansi-colors nil t))))
>> + (let ((n (- (length org-ansi-colors)
>> + (length (memq color org-ansi-colors)))))
>> + (org-ansi-highlight beg end (+ 40 n))))
>
> The above has no relation to fontification and does not belong to Org in
> general. Org syntax has no notion of ANSI escapes. We may support them
> as a useful feature, but no more. Editing ANSI escapes would make more
> sense in shell-script-mode or similar.
Removed.
>
>> + :lighter " OANSI"
>> + (cond
>> + ((and org-fontify-ansi-sequences org-ansi-mode)
>> + (remove-text-properties (point-min) (point-max) '(fontified t))
>> + (font-lock-ensure))
>
> Just use `org-restart-font-lock'.
>
Thanks. Done.
>> + (t
>> + (dolist (ov (overlays-in (point-min) (point-max)))
>> + ;; Attempt to find ANSI specific overlays. See
>> + ;; `ansi-color-make-extent'.
>> + (when (eq (car-safe (overlay-get ov 'insert-behind-hooks))
>> + 'ansi-color-freeze-overlay)
>
> This is extremely awkward and relies on internal implementation details
> of ansi-color. Moreover, we must avoid overlays, if possible - they do
> not scale well. I recommend re-defining `ansi-color-apply-face-function'
> to something that uses text properties. Using text properties will also
> make restarting font-lock sufficient to clear the fontification.
I've re-defined `ansi-color-apply-face-function' as you've
said.
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: test-ansi.org --]
[-- Type: text/x-org, Size: 879 bytes --]
* This is a ^[[42mtest^[[0m
:PROPERTIES:
:CUSTOM_ID: 123
:END:
Of ^[[31mANSI^[[0m ^[[33mcolor^[[0m sequences
#+begin_src python
for x in y:
print(x + "this is a ^[[43mtest^[[0m")
#+end_src
: this ^[[42mis a^[[0m td
=testing=
In paragraph a ~color ^[[44msequ^[[0mence~ is ^[[41mhere^[[0m.
^[[43mThis is a sequence that covers a block
#+begin_example
shouldn't be colored
#+end_example
there should be an end here^[[0m there is the end.
begin ^[[43m
sequence
without end
#+begin_src python
1 + 1
#+end_src
#+begin_quote
open ^[[43m
#+end_quote
should not be highlighted
#+begin_quote
close ^[[0m
#+end_quote
This is a paragraph src_python{return "t^[[43mest^[[0ming"} {{{results(=t^[[43mest^[[0ming=)}}} with
multiple inline src_python{return 5*4} {{{results(=20=)}}} source blocks.
An inline source block src_python{return 1+ 1 without an
end. src_python{return "t^[[43mest^[[0ming"}.
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: Patch --]
[-- Type: text/x-patch, Size: 6527 bytes --]
From c59d39d76266670200f9cfe70a1e1c2dad04c8bc Mon Sep 17 00:00:00 2001
From: Nathaniel Nicandro <nathanielnicandro@gmail.com>
Date: Tue, 9 May 2023 19:58:11 -0500
Subject: [PATCH] Highlight ANSI escape sequences
* etc/ORG-NEWS: Describe the new feature.
* org.el (org-fontify-ansi-sequences): New customization variable and
function which does the work of fontifying the sequences.
(org-set-font-lock-defaults): Add the `org-fontify-ansi-sequences`
function to the font-lock keywords.
(org-ansi-mode): New minor mode to enable/disable highlighting of the
sequences. Enabled in Org buffers by default.
---
etc/ORG-NEWS | 12 ++++++++
lisp/org.el | 82 ++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 94 insertions(+)
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index b7c88fd..2c28785 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -169,6 +169,18 @@ official [[https://clojure.org/guides/deps_and_cli][Clojure CLI tools]].
The command can be customized with ~ob-clojure-cli-command~.
** New features
+*** ANSI escape sequences are now highlighted in the whole buffer
+
+A new customization ~org-fontify-ansi-sequences~ is available which
+tells Org to highlight all ANSI sequences in the buffer if non-nil and
+the new minor mode ~org-ansi-mode~ is enabled.
+
+To disable highlighting of the sequences you can either
+disable ~org-ansi-mode~ or set ~org-fontify-ansi-sequences~ to ~nil~
+and =M-x org-mode-restart RET=. Doing the latter will disable
+highlighting of sequences in all newly opened Org buffers whereas
+doing the former disables highlighting locally to the current buffer.
+
*** Add support for ~logind~ idle time in ~org-user-idle-seconds~
When Emacs is built with =dbus= support and
diff --git a/lisp/org.el b/lisp/org.el
index 26d2a86..6742449 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -81,6 +81,7 @@ (eval-when-compile (require 'gnus-sum))
(require 'calendar)
(require 'find-func)
(require 'format-spec)
+(require 'ansi-color)
(condition-case nil
(load (concat (file-name-directory load-file-name)
@@ -3582,6 +3583,12 @@ (defcustom org-fontify-whole-block-delimiter-line t
:group 'org-appearance
:type 'boolean)
+(defcustom org-fontify-ansi-sequences t
+ "Non-nil means to highlight ANSI escape sequences."
+ :group 'org-appearance
+ :type 'boolean
+ :package-version '(Org . "9.7"))
+
(defcustom org-highlight-latex-and-related nil
"Non-nil means highlight LaTeX related syntax in the buffer.
When non-nil, the value should be a list containing any of the
@@ -5543,6 +5550,66 @@ (defun org-fontify-extend-region (beg end _old-len)
(cons beg (or (funcall extend "end" "]" 1) end)))
(t (cons beg end))))))
+(defvar org-ansi-mode)
+
+(defun org-fontify-ansi-sequences (limit)
+ "Fontify ANSI sequences."
+ (when (and org-fontify-ansi-sequences org-ansi-mode)
+ (while (< (point) limit)
+ (let ((el (org-element-at-point)) beg end next)
+ (pcase (org-element-type el)
+ (`src-block
+ (setq beg (org-element-property :end el)
+ end beg
+ next end))
+ (`headline
+ (setq beg (org-element-property :begin el)
+ end (org-element-property :contents-begin el)
+ next end))
+ (`paragraph
+ ;; Compute the regions of the paragraph excluding inline
+ ;; source blocks.
+ (setq beg nil end nil)
+ (let ((pbeg (org-element-property :begin el))
+ (pend (org-element-property :end el)))
+ (goto-char pbeg)
+ (push pbeg beg)
+ (while (re-search-forward
+ "\\<src_\\([^ \t\n[{]+\\)[{[]" pend t)
+ (let ((el (org-element-context)))
+ (when (eq (org-element-type el) 'inline-src-block)
+ (push (org-element-property :begin el) end)
+ (goto-char (org-element-property :end el))
+ (push (point) beg))))
+ (push pend end)
+ (setq beg (nreverse beg)
+ end (nreverse end)
+ next pend)))
+ (_
+ (setq beg (or (org-element-property :contents-begin el)
+ (org-element-property :begin el))
+ end (or (org-element-property :contents-end el)
+ (org-element-property :end el))
+ next (org-element-property :end el))))
+ (cl-letf (((symbol-function #'delete-region)
+ (lambda (beg end)
+ (add-text-properties beg end '(invisible t))))
+ (ansi-color-apply-face-function
+ (lambda (beg end face)
+ (font-lock-prepend-text-property beg end 'face face))))
+ (if (consp beg)
+ (while (consp beg)
+ (ansi-color-apply-on-region (pop beg) (pop end)))
+ (ansi-color-apply-on-region beg end)))
+ ;; Reset the context after applying the color to prevent color
+ ;; settings from propagating to other elements. This also
+ ;; avoids issues where `ansi-color-apply-on-region' attempts
+ ;; to use an old starting point that may be from a different
+ ;; part of the buffer, leading to "wrong side of point"
+ ;; errors.
+ (setq ansi-color-context-region nil)
+ (goto-char next)))))
+
(defun org-activate-footnote-links (limit)
"Add text properties for footnotes."
(let ((fn (org-footnote-next-reference-or-definition limit)))
@@ -5861,6 +5928,7 @@ (defun org-set-font-lock-defaults ()
;; Blocks and meta lines
'(org-fontify-meta-lines-and-blocks)
'(org-fontify-inline-src-blocks)
+ '(org-fontify-ansi-sequences)
;; Citations. When an activate processor is specified, if
;; specified, try loading it beforehand.
(progn
@@ -15455,6 +15523,20 @@ (defun org-agenda-prepare-buffers (files)
(when org-agenda-file-menu-enabled
(org-install-agenda-files-menu))))
+\f
+;;;; ANSI minor mode
+
+(define-minor-mode org-ansi-mode
+ "Toggle the minor `org-ansi-mode'.
+This mode adds support to highlight ANSI sequences in Org mode.
+The sequences are highlighted only if the customization
+`org-fontify-ansi-sequences' is non-nil when the mode is enabled.
+\\{org-ansi-mode-map}"
+ :lighter " OANSI"
+ (org-restart-font-lock))
+
+(add-hook 'org-mode-hook #'org-ansi-mode)
+
\f
;;;; CDLaTeX minor mode
--
2.39.1
[-- Attachment #4: Type: text/plain, Size: 15 bytes --]
--
Nathaniel
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH] Highlight ANSI sequences in the whole buffer (was [PATCH] ANSI color on example blocks and fixed width elements)
2023-04-25 20:33 ` Nathaniel Nicandro
@ 2023-05-10 10:27 ` Ihor Radchenko
2023-05-15 0:18 ` Nathaniel Nicandro
0 siblings, 1 reply; 9+ messages in thread
From: Ihor Radchenko @ 2023-05-10 10:27 UTC (permalink / raw)
To: Nathaniel Nicandro; +Cc: emacs-orgmode
Nathaniel Nicandro <nathanielnicandro@gmail.com> writes:
> The attached patch now uses `org-element-at-point' and
> `org-element-context' to query for the bounds of elements.
Thanks!
> Note, I've also attached an updated example file which shows that the
> escape sequences in inline source blocks are now handled similarly to
> regular source blocks, i.e. they are not fontified.
I do not think that a single exception - source blocks is good enough.
When having something like
ANSI opening term is =<ANSI>=, and closing term is =<ANSI>=
it will be not expected to get things fontified.
A better approach will be:
1. Do not allow ANSI sequences to intersect markup boundaries of the
same AST depth:
*bold <ANSI>* plain text <ANSI> should not trigger fontification
*bold <ANSI> /italic/ <ANSI>* should trigger
plain text <ANSI> *bold* plain text <ANSI> also should
2. Disallow fontification is certain contexts - 'inline-src-block
Further, your current code will do something weird when encountering
greater element:
:DRAWER:
Paragraph <ANSI>
Another paragraph <ANSI>
:END:
You should not consider greater elements when fontifying.
> + (cl-letf (((symbol-function #'delete-region)
> + (lambda (beg end)
> + (add-text-properties beg end '(invisible t))))
This is fragile and relies on internal implementation details of
ansi-color.el. Is there another way?
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] Highlight ANSI sequences in the whole buffer (was [PATCH] ANSI color on example blocks and fixed width elements)
2023-05-10 10:27 ` Ihor Radchenko
@ 2023-05-15 0:18 ` Nathaniel Nicandro
2023-05-18 19:45 ` Ihor Radchenko
0 siblings, 1 reply; 9+ messages in thread
From: Nathaniel Nicandro @ 2023-05-15 0:18 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: emacs-orgmode
Ihor Radchenko <yantar92@posteo.net> writes:
> Nathaniel Nicandro <nathanielnicandro@gmail.com> writes:
>
>> The attached patch now uses `org-element-at-point' and
>> `org-element-context' to query for the bounds of elements.
>
> Thanks!
>
>> Note, I've also attached an updated example file which shows that the
>> escape sequences in inline source blocks are now handled similarly to
>> regular source blocks, i.e. they are not fontified.
>
> I do not think that a single exception - source blocks is good enough.
> When having something like
> ANSI opening term is =<ANSI>=, and closing term is =<ANSI>=
> it will be not expected to get things fontified.
>
> A better approach will be:
> 1. Do not allow ANSI sequences to intersect markup boundaries of the
> same AST depth:
> *bold <ANSI>* plain text <ANSI> should not trigger fontification
> *bold <ANSI> /italic/ <ANSI>* should trigger
> plain text <ANSI> *bold* plain text <ANSI> also should
Just to make sure I'm getting you right. You're saying that
fontification should trigger if the sequences live in the same
org-element-context?
What about cases like:
*<ANSI>bold* plain text <ANSI>
plain <ANSI>text *bold <ANSI>* paragraph end
In the first case, should only "bold" be fontified? Since the sequence
lives in the bold context.
In the second, should only "text"? Since the sequence lives at a higher
depth (the paragraph context, compared to the bold context). Or should
it be that the fontification should extend to the end of the paragraph
because the sequence lives at a higher depth?
> 2. Disallow fontification is certain contexts - 'inline-src-block
What I will do then is not consider sequences in inline-src-block, code,
or verbatim contexts. Are there any other elements or objects that I
should not consider (other than the greater elements as you mention
below)?
For verbatim (and code) contexts, if there are regions like
<ANSIx> plain =<ANSIy>= text <ANSIz>
ANSIy will not get considered and the region between ANSIx and ANSIz
will get highlighted using ANSIx's settings. So the verbatim object
gets highlighted as well.
For inline source blocks, I'll do what I did in the last patch and
decompose a paragraph into regions that exclude inline source blocks and
only consider those regions when processing the sequences. That way the
highlighting doesn't spill over into the inline source blocks (and not
interfere with the syntax highlighting of them).
>
> Further, your current code will do something weird when encountering
> greater element:
>
> :DRAWER:
> Paragraph <ANSI>
>
> Another paragraph <ANSI>
> :END:
>
> You should not consider greater elements when fontifying.
>
Thanks. In the case of greater elements, then, I will only consider
their contents.
For plain-lists and tables I will:
1. (for plain-lists) only consider the contents of the list items
2. (for tables) only consider the table-cells of each table-row
>> + (cl-letf (((symbol-function #'delete-region)
>> + (lambda (beg end)
>> + (add-text-properties beg end '(invisible t))))
>
> This is fragile and relies on internal implementation details of
> ansi-color.el. Is there another way?
Since the context in which the sequences live in need to be considered,
it doesn't look like ansi-color-apply-on-region can be used any more
since it isn't aware of Org objects.
I've come up with a function that calculates the highlightable regions
(considering contexts) and fontifies them, but it requires the use of
private functions from ansi-color. Specifically
ansi-color--face-vec-face, ansi-color--update-face-vec, and
ansi-color--code-as-hex (used internally by ansi-color--face-vec-face).
Does it make sense to copy over these functions into Org for the
purposes of handling ANSI escapes? There would be some backward
compatibility issues, e.g. ansi-color only started using faces as colors
in Emacs 28.
--
Nathaniel
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] Highlight ANSI sequences in the whole buffer (was [PATCH] ANSI color on example blocks and fixed width elements)
2023-05-15 0:18 ` Nathaniel Nicandro
@ 2023-05-18 19:45 ` Ihor Radchenko
2023-05-23 0:55 ` Nathaniel Nicandro
0 siblings, 1 reply; 9+ messages in thread
From: Ihor Radchenko @ 2023-05-18 19:45 UTC (permalink / raw)
To: Nathaniel Nicandro; +Cc: emacs-orgmode
Nathaniel Nicandro <nathanielnicandro@gmail.com> writes:
>> 1. Do not allow ANSI sequences to intersect markup boundaries of the
>> same AST depth:
>> *bold <ANSI>* plain text <ANSI> should not trigger fontification
>> *bold <ANSI> /italic/ <ANSI>* should trigger
>> plain text <ANSI> *bold* plain text <ANSI> also should
>
> Just to make sure I'm getting you right. You're saying that
> fontification should trigger if the sequences live in the same
> org-element-context?
> What about cases like:
>
> *<ANSI>bold* plain text <ANSI>
> plain <ANSI>text *bold <ANSI>* paragraph end
>
> In the first case, should only "bold" be fontified? Since the sequence
> lives in the bold context.
> In the second, should only "text"? Since the sequence lives at a higher
> depth (the paragraph context, compared to the bold context). Or should
> it be that the fontification should extend to the end of the paragraph
> because the sequence lives at a higher depth?
I completely missed the point that <ANSI> codes are not <open ... close>
pairs, but switches; this is completely different from Org syntax.
So, let me re-consider where <ANSI> codes are likely to be used in
practice:
1. Inside shell code blocks (src-block element)
2. Inside results of evaluation, which are usually fixed-width element,
but might also be example-block, export-block, drawer, table, or
other element.
3. Inside shell inline code blocks (inline-src-block object)
4. Inside results of evaluation of an inline code block - usually
code/verbatim markup.
I think that the most reasonable approach to fontify ANSI sequences will
be the following:
1. We will consider ANSI within (a) all greater elements and lesser
elements that have RESULTS affiliated keyword (indicating that they
are result of code block evaluation); (b) otherwise, just lesser
elements, like paragraph, src block, example block, export block,
etc., but _not_ tables (c) otherwise, within verbatim-like objects,
like code, export-snippet, inline-src-block, table-cell, verbatim.
The three groups above should be declared via variables, so that
users can tweak them as necessary.
2. If ANSI sequence is encountered inside a verbatim-like object and we
did not see any ANSI sequences within parent element or greater
element, limit ANSI triggers to the current object.
Example:
#+RESULTS:
Lorem upsum =<ANSI>valor=. Some more text.
(only "valor" will be affected)
3. If the first ANSI sequence is encountered inside element and outside
verbatim-like object, the rest of the element is affected, including
all the objects.
Example:
#+RESULTS:
<ANSI>Lorem upsum =<ANSI>valor=. Some more text.
(the first ANSI affects everything, including verbatim; the second
ANSI also affects everything)
4. If the first ANSI sequence is encountered inside greater element with
RESULTS affiliated keyword, all the lesser elements inside will be
affected.
Example:
#+RESULTS:
:drawer:
<ANSI>Lorem upsum =valor=. Some more text.
Another paragraph inside drawer.
:end:
(everything down to :end: is affected)
or
#+RESULTS:
- <ANSI>list
- one
- two
- three
(everything is affected down to the end of the list)
Does it make sense?
>>> + (cl-letf (((symbol-function #'delete-region)
>>> + (lambda (beg end)
>>> + (add-text-properties beg end '(invisible t))))
>>
>> This is fragile and relies on internal implementation details of
>> ansi-color.el. Is there another way?
>
> Since the context in which the sequences live in need to be considered,
> it doesn't look like ansi-color-apply-on-region can be used any more
> since it isn't aware of Org objects.
>
> I've come up with a function that calculates the highlightable regions
> (considering contexts) and fontifies them, but it requires the use of
> private functions from ansi-color. Specifically
> ansi-color--face-vec-face, ansi-color--update-face-vec, and
> ansi-color--code-as-hex (used internally by ansi-color--face-vec-face).
> Does it make sense to copy over these functions into Org for the
> purposes of handling ANSI escapes? There would be some backward
> compatibility issues, e.g. ansi-color only started using faces as colors
> in Emacs 28.
If we really need to, we can propose an extension of
ansi-color-apply-on-region upstream for Emacs itself.
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] Highlight ANSI sequences in the whole buffer (was [PATCH] ANSI color on example blocks and fixed width elements)
2023-05-18 19:45 ` Ihor Radchenko
@ 2023-05-23 0:55 ` Nathaniel Nicandro
0 siblings, 0 replies; 9+ messages in thread
From: Nathaniel Nicandro @ 2023-05-23 0:55 UTC (permalink / raw)
To: Ihor Radchenko; +Cc: emacs-orgmode
Ihor Radchenko <yantar92@posteo.net> writes:
> Nathaniel Nicandro <nathanielnicandro@gmail.com> writes:
>
>>> 1. Do not allow ANSI sequences to intersect markup boundaries of the
>>> same AST depth:
>>> *bold <ANSI>* plain text <ANSI> should not trigger fontification
>>> *bold <ANSI> /italic/ <ANSI>* should trigger
>>> plain text <ANSI> *bold* plain text <ANSI> also should
>>
>> Just to make sure I'm getting you right. You're saying that
>> fontification should trigger if the sequences live in the same
>> org-element-context?
>
>> What about cases like:
>>
>> *<ANSI>bold* plain text <ANSI>
>> plain <ANSI>text *bold <ANSI>* paragraph end
>>
>> In the first case, should only "bold" be fontified? Since the sequence
>> lives in the bold context.
>
>> In the second, should only "text"? Since the sequence lives at a higher
>> depth (the paragraph context, compared to the bold context). Or should
>> it be that the fontification should extend to the end of the paragraph
>> because the sequence lives at a higher depth?
>
> I completely missed the point that <ANSI> codes are not <open ... close>
> pairs, but switches; this is completely different from Org syntax.
>
> So, let me re-consider where <ANSI> codes are likely to be used in
> practice:
>
> 1. Inside shell code blocks (src-block element)
> 2. Inside results of evaluation, which are usually fixed-width element,
> but might also be example-block, export-block, drawer, table, or
> other element.
> 3. Inside shell inline code blocks (inline-src-block object)
> 4. Inside results of evaluation of an inline code block - usually
> code/verbatim markup.
>
> I think that the most reasonable approach to fontify ANSI sequences will
> be the following:
>
> 1. We will consider ANSI within (a) all greater elements and lesser
> elements that have RESULTS affiliated keyword (indicating that they
> are result of code block evaluation); (b) otherwise, just lesser
> elements, like paragraph, src block, example block, export block,
> etc., but _not_ tables (c) otherwise, within verbatim-like objects,
> like code, export-snippet, inline-src-block, table-cell, verbatim.
>
> The three groups above should be declared via variables, so that
> users can tweak them as necessary.
>
> 2. If ANSI sequence is encountered inside a verbatim-like object and we
> did not see any ANSI sequences within parent element or greater
> element, limit ANSI triggers to the current object.
>
> Example:
>
> #+RESULTS:
> Lorem upsum =<ANSI>valor=. Some more text.
>
> (only "valor" will be affected)
>
> 3. If the first ANSI sequence is encountered inside element and outside
> verbatim-like object, the rest of the element is affected, including
> all the objects.
>
> Example:
>
> #+RESULTS:
> <ANSI>Lorem upsum =<ANSI>valor=. Some more text.
>
> (the first ANSI affects everything, including verbatim; the second
> ANSI also affects everything)
>
> 4. If the first ANSI sequence is encountered inside greater element with
> RESULTS affiliated keyword, all the lesser elements inside will be
> affected.
>
> Example:
>
> #+RESULTS:
> :drawer:
> <ANSI>Lorem upsum =valor=. Some more text.
>
> Another paragraph inside drawer.
> :end:
>
> (everything down to :end: is affected)
>
> or
>
> #+RESULTS:
> - <ANSI>list
> - one
> - two
> - three
>
> (everything is affected down to the end of the list)
>
> Does it make sense?
>
Sounds good to me.
>>>> + (cl-letf (((symbol-function #'delete-region)
>>>> + (lambda (beg end)
>>>> + (add-text-properties beg end '(invisible t))))
>>>
>>> This is fragile and relies on internal implementation details of
>>> ansi-color.el. Is there another way?
>>
>> Since the context in which the sequences live in need to be considered,
>> it doesn't look like ansi-color-apply-on-region can be used any more
>> since it isn't aware of Org objects.
>>
>> I've come up with a function that calculates the highlightable regions
>> (considering contexts) and fontifies them, but it requires the use of
>> private functions from ansi-color. Specifically
>> ansi-color--face-vec-face, ansi-color--update-face-vec, and
>> ansi-color--code-as-hex (used internally by ansi-color--face-vec-face).
>> Does it make sense to copy over these functions into Org for the
>> purposes of handling ANSI escapes? There would be some backward
>> compatibility issues, e.g. ansi-color only started using faces as colors
>> in Emacs 28.
>
> If we really need to, we can propose an extension of
> ansi-color-apply-on-region upstream for Emacs itself.
--
Nathaniel
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2023-05-23 1:05 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-04-05 12:03 [PATCH] ANSI color on example blocks and fixed width elements Nathaniel Nicandro
2023-04-05 13:43 ` Ihor Radchenko
2023-04-13 20:18 ` [PATCH] Highlight ANSI sequences in the whole buffer (was [PATCH] ANSI color on example blocks and fixed width elements) Nathaniel Nicandro
2023-04-14 8:49 ` Ihor Radchenko
2023-04-25 20:33 ` Nathaniel Nicandro
2023-05-10 10:27 ` Ihor Radchenko
2023-05-15 0:18 ` Nathaniel Nicandro
2023-05-18 19:45 ` Ihor Radchenko
2023-05-23 0:55 ` Nathaniel Nicandro
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).