From: Nathaniel Nicandro <nathanielnicandro@gmail.com>
To: Ihor Radchenko <yantar92@posteo.net>
Cc: Nathaniel Nicandro <nathanielnicandro@gmail.com>,
emacs-orgmode <emacs-orgmode@gnu.org>
Subject: Re: [PATCH] Highlight ANSI sequences in the whole buffer (was [PATCH] ANSI color on example blocks and fixed width elements)
Date: Tue, 16 Jan 2024 18:02:14 -0600 [thread overview]
Message-ID: <8734uwhlhj.fsf@gmail.com> (raw)
In-Reply-To: <87le9wq2dg.fsf@localhost>
[-- Attachment #1: Type: text/plain, Size: 4945 bytes --]
Ihor Radchenko <yantar92@posteo.net> writes:
Hello, attached is another updated patch with the following changes:
- Made it possible to add headlines or inline tasks
to `org-ansi-highlightable-elements', these are added by default now.
- To tackle the issue discussed previously about highlights spanning
multiple lines (or elements) being removed when a line is modified I
went ahead and used the font-lock-multiline property (see
font-lock-extend-region-multiline and
font-lock-extend-region-functions) across those regions so that on
any edit of one of the lines, the region including all of the ANSI
sequences that affect that line will be re-fontified. This was the
easier solution, but the downside is that it can cause large regions
to be re-fontified when really all we want to do is apply the
highlighting face to a small line change, for example. An
alternative solution would, when no ANSI sequences are being edited
in the region being fontified and assuming a previous fontification
cycle has applied highlights due to ANSI sequences already, only
apply the highlighting face to the edited region instead of
expanding the region before fontification. The expansion
unnecessarily wastes the fontification cycle on a region larger than
what it needs to be since the information needed for highlighting
the region according to ANSI sequences has already been computed on
a previous fontification cycle. In practice I don't think this
inefficiency will matter much since I would assume most of these
ANSI sequences will be inserted due to the results of code block
execution or inserted by users who want to highlight small regions
of the document so I would consider this problem solved by using
font-lock-multiline for the time being. WDYT?
- To tackle the issue of editing around the invisible ANSI sequences I
left it up to the font-lock process to catch the invisible edits.
Whenever an edit deletes a character of the sequence that renders
the sequence invalid, the font-lock process will reveal the partial
sequence. But I had to limit what was considered a valid ANSI
sequence to get it working in a somewhat acceptable way.
The problem that I found was that if the buffer contains something
like
^[[43mfoo
(where ^[ is the ESC character and can be inserted with "C-q ESC" and
the whole sequence ^[[43m is the ANSI sequence) what was happening was
that deleting into the hidden sequence would leave the region in the
state
^[[43foo
and because the end byte of the ANSI sequence can be any character
in the ASCII range [@A-Z[\]^_`a–z{|}~], ^[[43f would still be a
valid ANSI sequence and would be hidden during the fontification
process after the edit. Since `ansi-color-apply-on-region' only
really handles the sequences that end in an m byte, just rendering
all other ones invisible, I limited the ANSI sequences handled by
this patch to be only those sequences that end in m. This way,
after deleting into the sequence like in the above example the
fontification process would not recognize the region as containing
any sequence. The downside to this solution is that sequences that
end in any other end byte won't get conveniently hidden and the
problem still persists if you have text that starts with an m and
you delete into a hidden sequence.
An alternative solution that doesn't constrain the end byte could be
to add in some extra invisible character like a zero width space and
then use something like the `modification-hooks' text property on
the character to signify that a deletion at the boundary between the
sequence and the text should really delete part of the sequence
instead of the zero width space. I haven't really worked out the
details of this, for example how would it be detected which
direction a deletion is coming from, the front or behind, but I'm
throwing it out there to see if there are any other solutions other
people might be aware of for a similar problem.
- Finally, code has been added to delete the overlays on the hidden
sequences in `org-unfontify-region' so that multiple overlays are not
created on re-fontifying regions containing those sequences.
Other than that, the code is the same as the last patch.
> P.S. I am not yet commenting on the details in the code.
Please let me know what you think of this patch and where I should be
focusing my efforts moving forward to get this submitted to Org.
One thing I would like to start doing is writing some tests for this
feature. It would be great if someone could point me to some tests
that I can peruse so that I can get an idea of how I can go about
writing some of my own. Also, are there any procedures or things I
should be aware of when trying to write my own tests?
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Patch --]
[-- Type: text/x-patch, Size: 15586 bytes --]
From 506e8c1e5a177b797a541b1541ea98c95668d5e1 Mon Sep 17 00:00:00 2001
From: Nathaniel Nicandro <nathanielnicandro@gmail.com>
Date: Tue, 9 May 2023 19:58:11 -0500
Subject: [PATCH] Highlight ANSI escape sequences
* etc/ORG-NEWS: Describe the new feature.
* org.el (org-fontify-ansi-sequences): New customization variable and
function which does the work of fontifying the sequences.
(org-ansi-highlightable-elements)
(org-ansi-highlightable-objects): New customization variables.
(org-ansi-new-context, org-ansi-process-region)
(org-ansi-process-block, org-ansi-process-paragraph)
(org-ansi-process-fixed-width)
(org-fontify-ansi-sequences-1): New functions.
(org-ansi--control-seq-regexp): New variable.
(org-set-font-lock-defaults): Add the `org-fontify-ansi-sequences`
function to the font-lock keywords.
(org-unfontify-region): Delete ANSI specific overlays.
(org-ansi-mode): New minor mode to enable/disable highlighting of the
sequences. Enabled in Org buffers by default.
---
etc/ORG-NEWS | 12 +++
lisp/org.el | 269 +++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 281 insertions(+)
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 1207d6f..76a81e3 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -492,6 +492,18 @@ Currently implemented options are:
iCalendar programs support this usage.
** New features
+*** ANSI escape sequences are now highlighted in the whole buffer
+
+A new customization ~org-fontify-ansi-sequences~ is available which
+tells Org to highlight all ANSI sequences in the buffer if non-nil and
+the new minor mode ~org-ansi-mode~ is enabled.
+
+To disable highlighting of the sequences you can either
+disable ~org-ansi-mode~ or set ~org-fontify-ansi-sequences~ to ~nil~
+and =M-x org-mode-restart RET=. Doing the latter will disable
+highlighting of sequences in all newly opened Org buffers whereas
+doing the former disables highlighting locally to the current buffer.
+
*** =ob-plantuml.el=: Support tikz file format output
=ob-plantuml.el= now output =tikz= :file format via
diff --git a/lisp/org.el b/lisp/org.el
index d2cd0b9..6e4744e 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -81,6 +81,7 @@ (eval-when-compile (require 'gnus-sum))
(require 'calendar)
(require 'find-func)
(require 'format-spec)
+(require 'ansi-color)
(condition-case nil
(load (concat (file-name-directory load-file-name)
@@ -3608,6 +3609,12 @@ (defcustom org-fontify-whole-block-delimiter-line t
:group 'org-appearance
:type 'boolean)
+(defcustom org-fontify-ansi-sequences t
+ "Non-nil means to highlight ANSI escape sequences."
+ :group 'org-appearance
+ :type 'boolean
+ :package-version '(Org . "9.7"))
+
(defcustom org-highlight-latex-and-related nil
"Non-nil means highlight LaTeX related syntax in the buffer.
When non-nil, the value should be a list containing any of the
@@ -5598,6 +5605,243 @@ (defun org-fontify-extend-region (beg end _old-len)
(cons beg (or (funcall extend "end" "]" 1) end)))
(t (cons beg end))))))
+(defcustom org-ansi-highlightable-elements
+ '(plain-list drawer headline inlinetask
+ example-block export-block fixed-width paragraph)
+ "A list of element types that will have ANSI sequences processed."
+ :type '(list (symbol :tag "Element Type"))
+ :version "9.7"
+ :group 'org-appearance)
+
+(defcustom org-ansi-highlightable-objects
+ '(bold code export-snippet italic macro
+ strike-through table-cell underline verbatim)
+ "A list of object types that will have ANSI sequences processed."
+ :type '(list (symbol :tag "Object Type"))
+ :version "9.7"
+ :group 'org-appearance)
+
+(defun org-ansi-new-context (pos)
+ (list (list (make-bool-vector 8 nil)
+ nil nil)
+ (copy-marker pos)))
+
+;; Only match color sequences (escape codes ending with an m).
+;;
+;; This effectively means that other control sequences won't get
+;; conveniently hidden.
+(defvar org-ansi--control-seq-regexp "\e\\[[\x30-\x3F]*[\x20-\x2F]*m")
+
+(defun org-ansi-process-region (beg end &optional context)
+ (or context (setq context (org-ansi-new-context beg)))
+ (move-marker (cadr context) beg)
+ (let ((ansi-color-context-region context)
+ (ansi-color-control-seq-regexp org-ansi--control-seq-regexp)
+ (ansi-color-apply-face-function
+ (lambda (beg end face)
+ (when face
+ (font-lock-prepend-text-property beg end 'face face)
+ (add-text-properties beg end '(font-lock-multiline t))))))
+ (ansi-color-apply-on-region beg end t))
+ (save-excursion
+ (goto-char beg)
+ (while (re-search-forward org-ansi--control-seq-regexp end t)
+ (let ((beg (match-beginning 0))
+ (end (point)))
+ (dolist (ov (overlays-at beg))
+ (when (and (= beg (overlay-start ov))
+ (= end (overlay-end ov))
+ (overlay-get ov 'invisible))
+ ;; Assume this is the overlay added by
+ ;; `ansi-color-apply-on-region'
+ (overlay-put ov 'org-ansi t)))))))
+
+(defun org-ansi-process-block (el &optional context)
+ (let ((beg (org-element-property :begin el))
+ (end (org-element-property :end el)))
+ (save-excursion
+ (goto-char beg)
+ (while (org-at-keyword-p)
+ (forward-line))
+ (setq beg (line-beginning-position 2)))
+ (save-excursion
+ (goto-char end)
+ (skip-chars-backward " \t\n")
+ (setq end (line-beginning-position)))
+ (org-ansi-process-region beg end context)))
+
+(defun org-ansi-process-paragraph (el &optional context)
+ ;; Compute the regions of the paragraph excluding inline
+ ;; source blocks.
+ (let ((pend (org-element-property :contents-end el)) beg end)
+ (push (point) beg)
+ (while (re-search-forward
+ "\\<src_\\([^ \t\n[{]+\\)[{[]" pend t)
+ (let ((el (org-element-context)))
+ (when (eq (org-element-type el) 'inline-src-block)
+ (push (org-element-property :begin el) end)
+ (goto-char (org-element-property :end el))
+ (push (point) beg))))
+ (push pend end)
+ (let ((ansi-context (or context (org-ansi-new-context (point)))))
+ (while beg
+ (org-ansi-process-region (pop beg) (pop end) ansi-context)))))
+
+(defun org-ansi-process-fixed-width (el &optional context)
+ (org-ansi-process-region
+ (org-element-property :begin el)
+ (save-excursion
+ (goto-char (org-element-property :end el))
+ (skip-chars-backward " \t\n")
+ (point))
+ context))
+
+(defun org-fontify-ansi-sequences-1 (limit &optional ansi-context)
+ (let ((skip-to-end-p
+ (lambda (el)
+ (or (null (org-element-property :contents-begin el))
+ (<= (org-element-property :contents-end el)
+ (point)
+ (org-element-property :end el))))))
+ (while (< (point) limit)
+ (let* ((el (org-element-at-point))
+ (type (org-element-type el)))
+ (pcase type
+ ;; Greater elements
+ ((or `headline `inlinetask `item
+ `center-block `quote-block `special-block
+ `drawer)
+ (if (funcall skip-to-end-p el)
+ (goto-char (org-element-property :end el))
+ (goto-char (org-element-property :contents-begin el))))
+ ((or `dynamic-block `footnote-definition `property-drawer)
+ (goto-char (org-element-property :end el)))
+ (`plain-list
+ (let ((end (org-element-property :end el)))
+ (goto-char (org-element-property :contents-begin el))
+ (while (< (point) end)
+ ;; Move to within the first item of a list.
+ (forward-char)
+ (let* ((item (org-element-at-point))
+ (cbeg (org-element-property :contents-begin item)))
+ (when cbeg
+ (goto-char cbeg)
+ (org-fontify-ansi-sequences-1
+ (org-element-property :contents-end item)
+ ansi-context))
+ (goto-char (org-element-property :end item))
+ (skip-chars-forward " \t\n")))))
+ (`table
+ (if (funcall skip-to-end-p el)
+ (goto-char (org-element-property :end el))
+ (goto-char (org-element-property :contents-begin el))
+ ;; Move to within the table-row of a table to continue
+ ;; processing it.
+ (forward-char)))
+ ;; Lesser elements
+ (`table-row
+ (if (eq (org-element-property :type el) 'rule)
+ (goto-char (org-element-property :end el))
+ (let ((end-1 (1- (org-element-property :end el))))
+ (goto-char (org-element-property :contents-begin el))
+ (while (< (point) end-1)
+ (let ((cell (org-element-context)))
+ (org-ansi-process-region
+ (org-element-property :contents-begin cell)
+ (org-element-property :contents-end cell)
+ ansi-context)
+ (goto-char (org-element-property :end cell))))
+ (forward-char))))
+ ((or `example-block `export-block)
+ (org-ansi-process-block el ansi-context)
+ (goto-char (org-element-property :end el)))
+ (`fixed-width
+ (org-ansi-process-fixed-width el ansi-context)
+ (goto-char (org-element-property :end el)))
+ (`paragraph
+ (org-ansi-process-paragraph el ansi-context)
+ (goto-char (org-element-property :end el)))
+ (_
+ (goto-char (org-element-property :end el))))))))
+
+(defvar org-ansi-mode)
+
+(defun org-fontify-ansi-sequences (limit)
+ "Fontify ANSI sequences."
+ (when (and org-fontify-ansi-sequences org-ansi-mode)
+ (while (< (point) limit)
+ (if (re-search-forward org-ansi--control-seq-regexp limit t)
+ (let* ((ctx (progn
+ (goto-char (match-beginning 0))
+ (org-element-context)))
+ (type (org-element-type ctx)))
+ (cond
+ ((memq type org-ansi-highlightable-objects)
+ ;; If the element-context is an object then there has not
+ ;; been a sequence at the element level so limit the
+ ;; effect of the sequence to the object.
+ (org-ansi-process-region
+ (point)
+ (or (org-element-property :contents-end ctx)
+ (- (org-element-property :end ctx)
+ (org-element-property :post-blank ctx)
+ 1))
+ (org-ansi-new-context (point)))
+ (goto-char (org-element-property :end ctx)))
+ ((memq type org-ansi-highlightable-elements)
+ (let ((el ctx))
+ (while (and el (not (org-element-property :results el)))
+ (setq el (org-element-property :parent el)))
+ (if (and el (not (eq el ctx)))
+ ;; If the element-context is a highlightable element
+ ;; that has an ancestor with a RESULTS affiliated
+ ;; keyword, process the full greater element with
+ ;; that keyword.
+ (if (not (memq (org-element-type el)
+ org-ansi-highlightable-elements))
+ ;; Skip over the greater element if not
+ ;; highlightable.
+ (goto-char (org-element-property :end el))
+ (goto-char (org-element-property :begin el))
+ (add-text-properties
+ (point) (org-element-property :end el)
+ '(font-lock-multiline t))
+ (org-fontify-ansi-sequences-1
+ (or (org-element-property :contents-end el)
+ (org-element-property :end el))
+ (org-ansi-new-context (point)))
+ (goto-char (org-element-property :end el)))
+ ;; If the element-context is not a part of a greater
+ ;; element with a RESULTS affiliated keyword, then it
+ ;; is just a highlightable lesser element. Process
+ ;; the element.
+ (pcase type
+ ((or `headline `inlinetask)
+ (org-ansi-process-region
+ (org-element-property :begin ctx)
+ (org-element-property :contents-begin ctx))
+ (goto-char (or (org-element-property :contents-begin ctx)
+ (org-element-property :end ctx))))
+ ((or `example-block `export-block)
+ (org-ansi-process-block ctx)
+ (goto-char (org-element-property :end ctx)))
+ (`fixed-width
+ (org-ansi-process-fixed-width ctx)
+ (goto-char (org-element-property :end ctx)))
+ (`paragraph
+ (org-ansi-process-paragraph ctx)
+ (goto-char (org-element-property :end ctx)))
+ (_
+ (goto-char (org-element-property :end ctx)))))))
+ (t
+ (pcase type
+ ((or `headline `inlinetask)
+ (goto-char (or (org-element-property :contents-begin ctx)
+ (org-element-property :end ctx))))
+ (_
+ (goto-char (org-element-property :end ctx)))))))
+ (goto-char limit)))))
+
(defun org-activate-footnote-links (limit)
"Add text properties for footnotes."
(let ((fn (org-footnote-next-reference-or-definition limit)))
@@ -5915,6 +6159,7 @@ (defun org-set-font-lock-defaults ()
;; Blocks and meta lines
'(org-fontify-meta-lines-and-blocks)
'(org-fontify-inline-src-blocks)
+ '(org-fontify-ansi-sequences)
;; Citations. When an activate processor is specified, if
;; specified, try loading it beforehand.
(progn
@@ -6094,6 +6339,9 @@ (defun org-unfontify-region (beg end &optional _maybe_loudly)
'(mouse-face t keymap t org-linked-text t
invisible t intangible t
org-emphasis t))
+ (dolist (ov (overlays-in beg end))
+ (when (overlay-get ov 'org-ansi)
+ (delete-overlay ov)))
(org-fold-region beg end nil 'org-link)
(org-fold-region beg end nil 'org-link-description)
(org-fold-core-update-optimisation beg end)
@@ -15582,6 +15830,27 @@ (defun org-agenda-prepare-buffers (files)
(when org-agenda-file-menu-enabled
(org-install-agenda-files-menu))))
+\f
+;;;; ANSI minor mode
+
+(define-minor-mode org-ansi-mode
+ "Toggle the minor `org-ansi-mode'.
+This mode adds support to highlight ANSI sequences in Org mode.
+The sequences are highlighted only if the customization
+`org-fontify-ansi-sequences' is non-nil when the mode is enabled.
+\\{org-ansi-mode-map}"
+ :lighter " OANSI"
+ (org-restart-font-lock)
+ (unless org-ansi-mode
+ (org-with-wide-buffer
+ (goto-char (point-min))
+ (while (re-search-forward org-ansi--control-seq-regexp nil t)
+ (dolist (ov (overlays-at (match-beginning 0)))
+ (when (overlay-get ov 'org-ansi)
+ (delete-overlay ov)))))))
+
+(add-hook 'org-mode-hook #'org-ansi-mode)
+
\f
;;;; CDLaTeX minor mode
--
2.39.1
[-- Attachment #3: Type: text/plain, Size: 15 bytes --]
--
Nathaniel
next prev parent reply other threads:[~2024-01-17 0:07 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-04-05 12:03 [PATCH] ANSI color on example blocks and fixed width elements Nathaniel Nicandro
2023-04-05 13:43 ` Ihor Radchenko
2023-04-13 20:18 ` [PATCH] Highlight ANSI sequences in the whole buffer (was [PATCH] ANSI color on example blocks and fixed width elements) Nathaniel Nicandro
2023-04-14 8:49 ` Ihor Radchenko
2023-04-25 20:33 ` Nathaniel Nicandro
2023-05-10 10:27 ` Ihor Radchenko
2023-05-15 0:18 ` Nathaniel Nicandro
2023-05-18 19:45 ` Ihor Radchenko
2023-05-23 0:55 ` Nathaniel Nicandro
2023-08-08 11:02 ` Ihor Radchenko
2023-11-08 9:56 ` Ihor Radchenko
2023-11-08 15:35 ` Nathaniel Nicandro
2023-11-10 10:25 ` Ihor Radchenko
2023-11-17 21:18 ` Nathaniel Nicandro
2023-12-14 14:34 ` Ihor Radchenko
2023-12-24 12:49 ` Nathaniel Nicandro
2024-01-17 0:02 ` Nathaniel Nicandro [this message]
2024-01-17 12:36 ` Ihor Radchenko
2024-03-26 14:02 ` Nathaniel Nicandro
2024-03-28 8:52 ` Ihor Radchenko
2023-12-14 14:37 ` Ihor Radchenko
2023-12-15 12:50 ` Matt
2023-12-25 2:20 ` Nathaniel Nicandro
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.orgmode.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=8734uwhlhj.fsf@gmail.com \
--to=nathanielnicandro@gmail.com \
--cc=emacs-orgmode@gnu.org \
--cc=yantar92@posteo.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).