From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id aNw5DudnOGSIfgEASxT56A (envelope-from ) for ; Thu, 13 Apr 2023 22:36:55 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id KPoaDudnOGQ1AgAAauVa8A (envelope-from ) for ; Thu, 13 Apr 2023 22:36:55 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 268FBCB3A for ; Thu, 13 Apr 2023 22:36:54 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pn3dT-0004nn-HJ; Thu, 13 Apr 2023 16:33:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pn3dR-0004e6-2U for emacs-orgmode@gnu.org; Thu, 13 Apr 2023 16:33:37 -0400 Received: from mail-qt1-x830.google.com ([2607:f8b0:4864:20::830]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pn3dB-0003wi-02 for emacs-orgmode@gnu.org; Thu, 13 Apr 2023 16:33:35 -0400 Received: by mail-qt1-x830.google.com with SMTP id d75a77b69052e-3e9b0bc9e06so1966951cf.0 for ; Thu, 13 Apr 2023 13:33:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681417997; x=1684009997; h=mime-version:message-id:in-reply-to:date:subject:cc:to:from :user-agent:references:from:to:cc:subject:date:message-id:reply-to; bh=RvRV1VzIlEf3UASmBQpOaBthSkScbO9Hs6AdpnTfYVc=; b=YCva8KR72U02TMRCAVKAPoUrhBFka5LHBGpaO6KBFtm9ruV2dUfyMzlgeb1gDVNfYI uRfPHQDVPCnacyOzA0Q/pZ7g0q2+1vBgDLkXKN1urNVETHej91lEtsOqr1y5an25UzH7 Kz5tt07vOmkiEmHIYt4y5wl2ri052SN9SQc3U6bTzQpeIEWXWRbnQA5NzUfVbY60/Omc 5ArV1ugttKmHMwxUKhJwHmIsHXDHp/Eq2Y4fE4sjsnBtb/7eShySkvhngv2q4XCr2zAC oR6JtHuMyuxzGEqQ8LCfSneJNHosipNR9k8XA8+tRt+0drlKJCri3y3oSQ7BeLC5mEH0 HJ2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681417997; x=1684009997; h=mime-version:message-id:in-reply-to:date:subject:cc:to:from :user-agent:references:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=RvRV1VzIlEf3UASmBQpOaBthSkScbO9Hs6AdpnTfYVc=; b=L+7WolW7wUvexmGe/DtL718mXmSYzhzDxRbF/A6qPpNkpNz4GXW9Uh3rp1dfE6yXiH DPVmSWzVv7V5r5OL+cqgv/q50psMlQTd21mKetUpi5xq6K+PpFSoxZQ+2x291xZSaKrN 3Sg/pqxreg7I5O0L8K5LFLuNVhaOPR9FsfqfpTyZstHNqJbdHjZIBgR1sqJ9Al5oEllc McjtHhgLyHaxaHSwyxE6eTopdR+xXeYMp75xXFUQYdMKKo3Awwajhly+6xVFsfdppaUM E9TzeHYLytIB7zAARcd44trsqp5EpXRbeSmFtlTRqFSVm4uYE1YkTBxNIeQOkAmY4T32 gi2g== X-Gm-Message-State: AAQBX9cxdPWmh+eN5sQuqpmpzUNWFg9DDXaRnMExUoT07Tnuoc+k4pTK zcb4silZXHZA42CCP0eJ94Z3Ootr6b8= X-Google-Smtp-Source: AKy350bZhK2GUnoZvL17jOvLnaw4zwbeMnnEUXealJTng/gUtUiCd9Wdh8+e6rc5LWMpqfeWvgBwsw== X-Received: by 2002:a05:6214:5195:b0:5de:5da:b873 with SMTP id kl21-20020a056214519500b005de05dab873mr5464834qvb.3.1681417997263; Thu, 13 Apr 2023 13:33:17 -0700 (PDT) Received: from entropy ([2601:241:8c80:25a0::bb6a]) by smtp.gmail.com with ESMTPSA id ol2-20020a0562143d0200b005e750d07153sm658549qvb.135.2023.04.13.13.33.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Apr 2023 13:33:16 -0700 (PDT) References: <874jpuijpc.fsf@gmail.com> <87y1n6igvo.fsf@localhost> User-agent: mu4e 1.8.13; emacs 30.0.50 From: Nathaniel Nicandro To: Ihor Radchenko Cc: Nathaniel Nicandro , emacs-orgmode Subject: [PATCH] Highlight ANSI sequences in the whole buffer (was [PATCH] ANSI color on example blocks and fixed width elements) Date: Thu, 13 Apr 2023 15:18:17 -0500 In-reply-to: <87y1n6igvo.fsf@localhost> Message-ID: <878rev1q0k.fsf@gmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=2607:f8b0:4864:20::830; envelope-from=nathanielnicandro@gmail.com; helo=mail-qt1-x830.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Seal: i=1; s=key1; d=yhetil.org; t=1681418214; a=rsa-sha256; cv=none; b=LqD0SVg1rQuacXEoPFb8JlyV34NNJ46V57T44Mq/Jka++UqDholWciywcW70GpQU4KfyM/ BweCrRTS/5EY4xuexkKIJfEiT6Yx1qzhgPU5XBCpkoaRcqf79uqmIZ0M2HH62dtVJ+zZzL yarfRY2qG8cAjDje2iGtzzjrlqjJsTHR4x1RAei6R0vyqzqXbeRLC0awYrigO8HmBhgLlm e7B1akiemgzizDHh0PnNzSJrgYzsnGB/tLj2V/OVh6vEQRBeNWQEnRq3s7UtDdLOWH/3iE OwJqx7KRUdK57mLGrJpeOcTgG4i0jGMb8uihvrOFxl1clOHPrJbdKGJUQbjp1w== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=gmail.com header.s=20221208 header.b=YCva8KR7; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1681418214; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=k0NBi6rclZ+P3Cr77X3Ta8E9O9f7nc2xx1uSxYl3oKY=; b=lrrIE8dfql3IMEkvM+pn+PQZApB2vednJMue1j2hKG2bHcNevaOSyIhvAL04fm+MqhAnUm bVSgr/O8tBFtSTJ2qZzg7flMoidGvtmawaz9UWhBEguOx+tC4voxpTQZz2YL0aD8IpfysU HQqJPrrMCN99Jb1u+RsyPXJk3hn4OaPtEuB/6bTT5dCltGFwAtjB96KDofBzv+KM53craQ iKxqFCCjTs+B7Ed2jZgr34QbS2UWk/mzi89GWAIUHoOn2jSXnOdgniQ1J5TZR+R6yJWrpP 2Gl3Ds9tgONLHjOubgnpEOjfzj4yyyGNZQd1LwlARvvxsTKCuP7cmAVb7t7d5w== X-Migadu-Scanner: scn1.migadu.com X-Migadu-Spam-Score: -1.82 X-Spam-Score: -1.82 X-Migadu-Queue-Id: 268FBCB3A Authentication-Results: aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=gmail.com header.s=20221208 header.b=YCva8KR7; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none) X-TUID: XAoT7AFKNHoo --=-=-= Content-Type: text/plain Ihor Radchenko writes: > Nathaniel Nicandro writes: > >> Attached is the patch. Without this patch, ANSI escape sequences >> generated by the output of a source block will be left in the buffer >> without any fontification. With this patch, the escaped text is nicely >> colored and escape sequences hidden using overlays. >> >> It works for Emacs versions which have the `PRESERVE-SEQUENCES` argument >> to the `ansi-color-apply-on-region` function. It's a bit slow due to >> the use of overlays. My implementation of this feature in Emacs-Jupyter >> supports older versions of Emacs without that argument, it relies on a >> custom version of that function though and uses text properties instead >> of overlays. >> >> Let me know what else could be done on my end to get this patch in. >> Thanks. > > Thanks for the patch! > > This is an interesting idea, but I am not sure if we want to use this > colouring by default. At least, it should be a minor mode. Probably > enabled by default. Because not every possible user may want to have the > escape sequences hidden away. > > Further, your patch only allows fontifying ANSI sequences in fixed-width > elements, example blocks, export blocks, and src blocks without known > major mode that does the fontification. I doubt that fontifying ANSI > sequences in this specific subset of elements always makes sense - > example blocks are not always used as src block output; bash code blocks > may purposely contain escape sequences, but your patch will not handle > them; inline src block output is not covered at all. > > Ideally, fontifying ANSI sequences should be fully controlled by users: > 1. We may not want to touch src blocks by default, when > `org-src-fontify-natively' is set to t. Only, maybe, provide an > option. Or you may better publish a minor mode that does this for > shell scripts. > 2. We may allow all the ANSI sequences to be fontified in the whole > buffer. I've updated my patch to be a combination of (1) and (2), see the attached patch. Essentially every sequence is fontified except those in source blocks and a minor mode has been created to allow users to disable or enable fontification whenever they want. I've also attached an example Org file with some ANSI sequences in it for testing purposes that you can try out. One issue that remains is how to handle sequences within inline source blocks. Those don't have a src-block property so any sequences within an inline source block are currently fontified. > 3. We may limit ANSI sequence fontification to results and only results. > Or just certain types of results. > > The easiest will be implementing fontification in the whole buffer, > early during fontification (and early in org-font-lock-keywords; see > org-font-lock-set-keywords-hook). --=-=-= Content-Type: text/x-org Content-Disposition: attachment; filename=test-ansi.org Content-Transfer-Encoding: 8bit * This is a test Of ANSI color sequences #+begin_src python for x in y: print(x + "this is a test") #+end_src : this is a td =testing= In paragraph a ~color sequence~ is here. This is a sequence that covers a block #+begin_example should be colored #+end_example there should be an end here there is the end. begin  sequence without end #+begin_src python 1 + 1 #+end_src Inline source blocks will have sequences highlighted because we only look for a src-block text property. src_python{return "testing"} --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Highlight-ANSI-escape-sequences.patch >From c9b505d022410a481210928ecc4cce1f199ec53b Mon Sep 17 00:00:00 2001 From: Nathaniel Nicandro Date: Thu, 13 Apr 2023 15:06:35 -0500 Subject: [PATCH] Highlight ANSI escape sequences * etc/ORG-NEWS: Describe the new feature. * org.el (org-fontify-ansi-sequences): New customization variable and function which does the work of fontifying the sequences. (org-set-font-lock-defaults): Add the `org-fontify-ansi-sequences` function to the font-lock keywords. (org-ansi-mode): New minor mode to enable/disable highlighting of the sequences. Enabled in Org buffers by default. --- etc/ORG-NEWS | 12 ++++++ lisp/org.el | 112 +++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 124 insertions(+) diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS index b7c88fd..8690540 100644 --- a/etc/ORG-NEWS +++ b/etc/ORG-NEWS @@ -169,6 +169,18 @@ official [[https://clojure.org/guides/deps_and_cli][Clojure CLI tools]]. The command can be customized with ~ob-clojure-cli-command~. ** New features +*** ANSI escape sequences are now highlighted in the whole buffer + +A new customization ~org-fontify-ansi-sequences~ is available which +tells Org to highlight all ANSI sequences in the buffer if non-nil and +the new minor mode ~org-ansi-mode~ is enabled. + +To disable highlighting of the sequences you can either +disable ~org-ansi-mode~ or set ~org-fontify-ansi-sequences~ to ~nil~ +and =M-x revert-buffer RET=. Doing the latter will disable +highlighting of sequences in all newly opened Org buffers whereas +doing the former disables highlighting locally to the current buffer. + *** Add support for ~logind~ idle time in ~org-user-idle-seconds~ When Emacs is built with =dbus= support and diff --git a/lisp/org.el b/lisp/org.el index 26d2a86..62a5134 100644 --- a/lisp/org.el +++ b/lisp/org.el @@ -81,6 +81,7 @@ (eval-when-compile (require 'gnus-sum)) (require 'calendar) (require 'find-func) (require 'format-spec) +(require 'ansi-color) (condition-case nil (load (concat (file-name-directory load-file-name) @@ -3582,6 +3583,12 @@ (defcustom org-fontify-whole-block-delimiter-line t :group 'org-appearance :type 'boolean) +(defcustom org-fontify-ansi-sequences t + "Non-nil means to highlight ANSI escape sequences." + :group 'org-appearance + :type 'boolean + :package-version '(Org . "9.7")) + (defcustom org-highlight-latex-and-related nil "Non-nil means highlight LaTeX related syntax in the buffer. When non-nil, the value should be a list containing any of the @@ -5543,6 +5550,72 @@ (defun org-fontify-extend-region (beg end _old-len) (cons beg (or (funcall extend "end" "]" 1) end))) (t (cons beg end)))))) +(defvar org-ansi-mode) + +(defun org-fontify-ansi-sequences (limit) + "Fontify ANSI sequences." + (when (and org-fontify-ansi-sequences org-ansi-mode) + (let (end) + (while (/= (point) limit) + (cond + ((get-text-property (point) 'src-block) + ;; If point is on a src block, skip over it + (goto-char (next-single-property-change (point) 'src-block nil limit)) + (save-restriction + ;; Prevent moving past limit + (narrow-to-region (point) limit) + (forward-line))) + (t + (setq end (next-single-property-change (point) 'src-block nil limit)) + (let ((src-block-beg (and (get-text-property end 'src-block) end))) + (when src-block-beg + ;; Set the end of the region to be fontified to be the + ;; beginning of the src block when end is not limit + (save-excursion + (goto-char src-block-beg) + (forward-line -1) + (org-skip-whitespace) + (setq end (point)))) + (ansi-color-apply-on-region (point) end t) + ;; Reset the context before every fontification cycle. This + ;; avoids issues where `ansi-color-apply-on-region' attempts to + ;; use an old starting point that may be from a different part + ;; of the buffer, leading to "wrong side of point" errors. + (setq ansi-color-context-region nil) + (goto-char (or src-block-beg end))))))))) + +(defvar org-ansi-colors + '(black red green yellow blue purple cyan white)) + +(defun org-ansi-highlight (beg end seq) + (save-excursion + (goto-char end) + (insert "\e") + (insert "[0m") + (goto-char beg) + (insert "\e") + (insert (format "[%sm" seq)))) + +(defun org-ansi-highlight-foreground (beg end color) + "Highlight the foreground between BEG and END with COLOR." + (interactive + (let ((bounds (car (region-bounds)))) + (list (car bounds) (cdr bounds) + (completing-read "Color: " org-ansi-colors nil t)))) + (let ((n (- (length org-ansi-colors) + (length (memq color org-ansi-colors))))) + (org-ansi-highlight beg end (+ 30 n)))) + +(defun org-ansi-highlight-background (beg end color) + "Highlight the background between BEG and END with COLOR." + (interactive + (let ((bounds (car (region-bounds)))) + (list (car bounds) (cdr bounds) + (completing-read "Color: " org-ansi-colors nil t)))) + (let ((n (- (length org-ansi-colors) + (length (memq color org-ansi-colors))))) + (org-ansi-highlight beg end (+ 40 n)))) + (defun org-activate-footnote-links (limit) "Add text properties for footnotes." (let ((fn (org-footnote-next-reference-or-definition limit))) @@ -5861,6 +5934,7 @@ (defun org-set-font-lock-defaults () ;; Blocks and meta lines '(org-fontify-meta-lines-and-blocks) '(org-fontify-inline-src-blocks) + '(org-fontify-ansi-sequences) ;; Citations. When an activate processor is specified, if ;; specified, try loading it beforehand. (progn @@ -15455,6 +15529,44 @@ (defun org-agenda-prepare-buffers (files) (when org-agenda-file-menu-enabled (org-install-agenda-files-menu)))) + +;;;; ANSI sequences minor mode + +(defvar org-ansi-mode-map (make-sparse-keymap) + "Keymap for the minor `org-ansi-mode'.") + +(org-defkey org-ansi-mode-map (kbd "C-c hf") #'org-ansi-highlight-foreground) +(org-defkey org-ansi-mode-map (kbd "C-c hb") #'org-ansi-highlight-background) + +(define-minor-mode org-ansi-mode + "Toggle the minor `org-ansi-mode'. +This mode adds support to highlight ANSI sequences in Org mode. +The sequences are highlighted only if the customization +`org-fontify-ansi-sequences' is non-nil when the mode is enabled. +\\{org-ansi-mode-map}" + :lighter " OANSI" + (cond + ((and org-fontify-ansi-sequences org-ansi-mode) + (remove-text-properties (point-min) (point-max) '(fontified t)) + (font-lock-ensure)) + (t + (dolist (ov (overlays-in (point-min) (point-max))) + ;; Attempt to find ANSI specific overlays. See + ;; `ansi-color-make-extent'. + (when (eq (car-safe (overlay-get ov 'insert-behind-hooks)) + 'ansi-color-freeze-overlay) + ;; Delete the invisible overlays over the escape sequences + (dolist (ov (overlays-at (1- (overlay-start ov)))) + (when (overlay-get ov 'invisible) + (delete-overlay ov))) + (dolist (ov (overlays-at (1+ (overlay-end ov)))) + (when (overlay-get ov 'invisible) + (delete-overlay ov))) + ;; Delete the overlay over the highlighted text + (delete-overlay ov)))))) + +(add-hook 'org-mode-hook #'org-ansi-mode) + ;;;; CDLaTeX minor mode -- 2.39.1 --=-=-= Content-Type: text/plain -- Nathaniel --=-=-=--