[BUG] Warning (emacs): org-element--cache: Unregistered buffer modifications detected. Resetting. [9.5 (release_9.5-223-g876e81 @ /usr/share/emacs/site-lisp/org/)]

[BUG] Warning (emacs): org-element--cache: Unregistered buffer modifications detected. Resetting. [9.5 (release_9.5-223-g876e81 @ /usr/share/emacs/site-lisp/org/)]

[Vladimir Lomov]

Remember to cover the basics, that is, what you expected to happen and
what in fact did happen.  You don't know how to make a good report?  See

     https://orgmode.org/manual/Feedback.html#Feedback

Your bug report will be posted to the Org mailing list.
------------------------------------------------------------------------

Hello.

A few weeks ago I started seeing these warnings from Emacs:

Warning (emacs): org-element--cache: Unregistered buffer modifications detected. Resetting.
If this warning appears regularly, please report it to Org mode mailing list (M-x org-submit-bug-report).
The buffer is: charges.org
 Current command: org-self-insert-command
 Backtrace:
nil Disable showing Disable logging
Warning (emacs): org-element--cache: Unregistered buffer modifications detected. Resetting.
If this warning appears regularly, please report it to Org mode mailing list (M-x org-submit-bug-report).
The buffer is: charges.org
 Current command: org-self-insert-command
 Backtrace:
nil Disable showing Disable logging

This happens when I edit a table. Even if I disable the warning
("showing" and "logging"), editing is still very slow compared to what it
was.

Emacs  : GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.2430, cairo version 1.17.4)
 of 2021-11-10
Package: Org mode version 9.5 (release_9.5-223-g876e81 @ /usr/share/emacs/site-lisp/org/)

current state:
==============
(setq
 org-archive-location "~/works/org/archive/%s::* Завершённые задания\n\n"
 org-link-elisp-confirm-function 'yes-or-no-p
 org-format-latex-header "\\documentclass{article}\n\\usepackage[conf=org-preview]{isu-th3}\n[NO-PACKAGES]\n[NO-DEFAULT-PACKAGES]\n[EXTRA]"
 org-bibtex-headline-format-function #[257 "\300\236A\207" [:title] 3 "\n\n(fn ENTRY)"]
 org-log-done 'note
 org-log-into-drawer t
 org-agenda-files "/home/vladimir/works/org/agenda.txt"
 org-capture-templates '(("t" "Список заданий" entry (file+headline (format "%s/refile.org" x-org-directory) "Задания")
                          (format "%%[%s/emacs/org/capture-todo.txt]" x-data-dir))
                         ("j" "Журнал событий" entry (file+olp+datetree (format "%s/journal.org" x-org-directory))
                          (format "%%[%s/emacs/org/capture-journal.txt]" x-data-dir))
                         ("n" "Идеи, короткие заметки" entry (file+olp+datetree (format "%s/refile.org" x-org-directory))
                          (format "%%[%s/emacs/org/capture-note-idea.txt]" x-data-dir))
                         )
 org-persist-after-read-hook '(org-element--cache-persist-after-read)
 org-export-before-parsing-hook '(org-attach-expand-links)
 org-tag-alist '(("срочно" . 117) ("позвонить" . 99) ("задания" . 101) ("daily" . 100) ("schedule" . 115) ("работа" . 119)
                 ("встреча" . 109) ("личное" . 112) (:startgroup) ("@универ" . 110) ("@политех" . 111) ("@институт" . 105)
                 ("@дом" . 104) (:endgroup))
 org-default-notes-file "/home/vladimir/works/org/refile.org"
 org-archive-hook '(org-attach-archive-delete-maybe)
 org-file-apps '(("\\.pdf\\'" . "zathura %s") ("\\.pdf::\\([[:digit:]]+\\)\\'" . "zathura --page=%1 %s") (auto-mode . emacs)
                 (system . system))
 org-cycle-hook '(org-cycle-hide-archived-subtrees org-cycle-hide-drawers org-cycle-show-empty-lines
                  org-optimize-window-after-visibility-change)
 org-persist-before-read-hook '(org-element--cache-persist-before-read)
 org-modules '(ol-bbdb ol-bibtex ol-docview org-habit org-id ol-info org-protocol)
 org-mode-hook '((lambda nil (electric-indent-local-mode -1))
                 #[0 "\300\301\302\303\304$\207" [add-hook change-major-mode-hook org-show-all append local] 5]
                 #[0 "\300\301\302\303\304$\207" [add-hook change-major-mode-hook org-babel-show-result-all append local] 5]
                 org-babel-result-hide-spec org-babel-hide-all-hashes
                 #[0 "\301\211\207" [imenu-create-index-function org-imenu-get-tree] 2])
 org-babel-load-languages '((emacs-lisp . t) (gnuplot . t) (asymptote . t) (perl . t) (R . t) (shell . t))
 org-src-window-setup 'split-window-right
 org-id-locations-file "/home/vladimir/.cache/emacs/org-id-locations"
 org-link-file-path-type 'relative
 org-confirm-shell-link-function 'yes-or-no-p
 outline-isearch-open-invisible-function 'outline-isearch-open-invisible
 org-use-sub-superscripts '{}
 org-highlight-latex-and-related '(latex scripts entities)
 org-edit-src-auto-save-idle-delay 3
 org-agenda-before-write-hook '(org-agenda-add-entry-text)
 org-babel-tangle-lang-exts '(("perl" . "pl") ("asymptote" . "asy") ("emacs-lisp" . "el") ("elisp" . "el"))
 org-src-mode-hook '(org-src-babel-configure-edit-buffer org-src-mode-configure-edit-buffer
                     (lambda nil (define-key org-src-mode-map [f2] 'org-edit-src-save)))
 org-confirm-elisp-link-function 'yes-or-no-p
 org-todo-keywords '((sequence "TODO(t)" "|" "DONE(d)") (types "УНИВЕР(u)" "ИНСТИТУТ(i)" "ПОЛИТЕХ(p)" "ДОМ(h)" "|" "СДЕЛАНО(c)"))
 org-speed-command-hook '(org-speed-command-activate org-babel-speed-command-activate)
 org-export-backends '(ascii html latex odt org)
 org-persist-before-write-hook '(org-element--cache-persist-before-write)
 org-tab-first-hook '(org-babel-hide-result-toggle-maybe org-babel-header-arg-expand)
 org-link-shell-confirm-function 'yes-or-no-p
 org-babel-pre-tangle-hook '(save-buffer)
 org-agenda-loop-over-headlines-in-active-region nil
 org-occur-hook '(org-first-headline-recenter)
 org-metadown-hook '(org-babel-pop-to-session-maybe)
 org-link-parameters '(("attachment" :follow org-attach-follow :complete org-attach-complete-link)
                       ("info" :follow org-info-open :export org-info-export :store org-info-store-link)
                       ("docview" :follow org-docview-open :export org-docview-export :store org-docview-store-link)
                       ("bibtex" :follow org-bibtex-open :store org-bibtex-store-link)
                       ("bbdb" :follow org-bbdb-open :export org-bbdb-export :complete org-bbdb-complete-link :store
                        org-bbdb-store-link)
                       ("id" :follow org-id-open) ("file+sys") ("file+emacs") ("doi" :follow org-link--open-doi)
                       ("elisp" :follow org-link--open-elisp) ("file" :complete org-link-complete-file)
                       ("ftp" :follow #[514 "\301\300\302Q\"\207" ["ftp" browse-url ":"] 6 "\n\n(fn URL ARG)"])
                       ("help" :follow org-link--open-help :store org-link--store-help)
                       ("http" :follow #[514 "\301\300\302Q\"\207" ["http" browse-url ":"] 6 "\n\n(fn URL ARG)"])
                       ("https" :follow #[514 "\301\300\302Q\"\207" ["https" browse-url ":"] 6 "\n\n(fn URL ARG)"])
                       ("mailto" :follow #[514 "\301\300\302Q\"\207" ["mailto" browse-url ":"] 6 "\n\n(fn URL ARG)"])
                       ("message" :follow (lambda (path) (browse-url (concat "message:" path))))
                       ("news" :follow #[514 "\301\300\302Q\"\207" ["news" browse-url ":"] 6 "\n\n(fn URL ARG)"])
                       ("shell" :follow org-link--open-shell))
 org-mark-ring-length 64
 org-metaup-hook '(org-babel-load-in-session-maybe)
 org-return-follows-link t
 org-src-lang-modes '(("elisp" . emacs-lisp) ("asymptote" . asy) ("dot" . fundamental) ("sqlite" . sql) ("calc" . fundamental)
                      ("C" . c) ("cpp" . c++) ("C++" . c++) ("screen" . shell-script) ("shell" . sh) ("bash" . sh) ("latex" . LaTeX))
 org-attach-store-link-p 'attached
 )

--
WBR, Vladimir Lomov

God save us from a bad neighbor and a beginner on the fiddle.

Critical Infrastructure & Key Resources Fedayeen SHA PGP white noise
advisors Water borne FKS Venezuela CBOT PSAC NSA Tamiflu Maple Vaccine

-- 

Re: [BUG] Warning (emacs): org-element--cache: Unregistered buffer modifications detected. Resetting. [9.5 (release_9.5-223-g876e81 @ /usr/share/emacs/site-lisp/org/)]

[Ihor Radchenko]

Vladimir Lomov <lomov.vl@yandex.ru> writes:

> A few weeks ago I started seeing these warnings from Emacs:
>
> Warning (emacs): org-element--cache: Unregistered buffer modifications detected. Resetting.
> ...
> This happens when I edit a table. Even if I disable the warning
> ("showing" and "logging"), editing is still very slow compared to what it
> was.

Thanks for reporting!

Are you able to reproduce with emacs -Q? Are you using any third-party
packages that have anything to do with tables?

Best,
Ihor

Re: [BUG] Warning (emacs): org-element--cache: Unregistered buffer modifications detected. Resetting. [9.5 (release_9.5-223-g876e81 @ /usr/share/emacs/site-lisp/org/)]

[Vladimir Lomov]

[-- Attachment #1: Type: text/plain, Size: 1665 bytes --]

Hello,
** Ihor Radchenko <yantar92@gmail.com> [2021-11-11 13:54:05 +0800]:

> Vladimir Lomov <lomov.vl@yandex.ru> writes:

>> A few weeks ago I started seeing these warnings from Emacs:
>>
>> Warning (emacs): org-element--cache: Unregistered buffer modifications detected. Resetting.
>> ...
>> This happens when I edit a table. Even if I disable the warning
>> ("showing" and "logging"), editing is still very slow compared to what it
>> was.

> Thanks for reporting!

> Are you able to reproduce with emacs -Q?

No, editing table 'charges.org' in Emacs running as 'emacs -Q' works fine, but
it uses org mode shipped with Emacs. If I launch emacs as 'emacs -Q -l
org-min.el' then I see the same messages.

An interesting observation (I didn't check that before): if I use English
language (latin keyboard) all goes fine, but when I switch input method (C-\,
Russian language) I see these warnings and editing slows down.

> Are you using any third-party packages that have anything to do with tables?

I use several emacs packages, but they don't interact with Org buffers (except
CDLaTeX but I don't use it in particular file).

This is 'org-min.el' file:

--------------------------------- 8< -----------------------------------------
(add-to-list 'load-path "/usr/share/emacs/site-lisp/org")
; (add-to-list 'load-path "/usr/share/emacs/site-lisp/org_contrib")
--------------------------------- 8< -----------------------------------------

> Best,
> Ihor

---
WBR, Vladimir Lomov

-- 
Q:	What's the difference between USL and the Graf Zeppelin?
A:	The Graf Zeppelin represented cutting edge technology for its time.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

table and Cyrillic characters: org-element--cache: Unregistered buffer modifications detected. Resetting

[Max Nikulin]

On 11/11/2021 13:50, Vladimir Lomov wrote:
> ** Ihor Radchenko:
> 
>> Vladimir Lomov writes:
>>> Warning (emacs): org-element--cache: Unregistered buffer modifications detected. Resetting.
> 
>> Are you able to reproduce with emacs -Q?

I can confirm it starting with a simple file

---- >8 ----
| 1 |
---- 8< ----

LANG=en_US.UTF-8 emacs -Q -L ~/src/org-mode/lisp/ cyrtable.org

C-\ russian-computer RET to switch input method
TAB to create a new cell
Any letter

Warning (emacs): org-element--cache: Unregistered buffer modifications 
detected. Resetting.
If this warning appears regularly, please report it to Org mode mailing 
list (M-x org-submit-bug-report).
The buffer is: cyrtable.org
  Current command: org-self-insert-command
  Backtrace:
nil

Org main, Emacs-26.3

There is no such problem if Russian keyboard layout is switched at XOrg 
level using xkb.

Re: table and Cyrillic characters: org-element--cache: Unregistered buffer modifications detected. Resetting

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

> On 11/11/2021 13:50, Vladimir Lomov wrote:
>> ** Ihor Radchenko:
>> 
>>> Vladimir Lomov writes:
>>>> Warning (emacs): org-element--cache: Unregistered buffer modifications detected. Resetting.
>> 
>>> Are you able to reproduce with emacs -Q?
>
> I can confirm it starting with a simple file
>
> ---- >8 ----
> | 1 |
> ---- 8< ----
>
> LANG=en_US.UTF-8 emacs -Q -L ~/src/org-mode/lisp/ cyrtable.org
>
> C-\ russian-computer RET to switch input method
> TAB to create a new cell
> Any letter

Confirmed

I can also reproduce with russian-computer and at least arabic. Seems to
be an issue with non-latin input methods.

The warning is triggered because return value of
buffer-chars-modified-tick with non-latin input method changes _before_
text is inserted. If I add debug-on-entry for self-insert-command or
org-self-insert-command, buffer-chars-modified-tick changes twice: (1)
some time after pressing the keyboard key but before entering
self-insert function (symbol is not inserted into buffer); (2) after
actual insertion. The problem happens in non-Org buffers as well and
with emacs -Q.

org-element-cache relies on the return value of
buffer-chars-modified-tick to control if all the changes in buffer are
reflected in the cache. The docstring says:

>> By comparing the values returned by two individual calls of
>> buffer-chars-modified-tick, you can tell whether a character change
>> occurred in that buffer in between these calls

So, what we observe looks like a Emacs bug.

On Org side, this bug is very bad news. We cannot wait for the Emacs
fixing the bug - older Emacs versions will still be affected.
Alternative ways to control buffer modifications are buffer-hash and
secure-hash. In the past, I had some random failures when buffer-hash
did not reliably reflect buffer updates. The only alternative is
secure-hash, but I am not sure about it's performance. buffer-hash
docstring says that "It should be somewhat more efficient on larger
buffers than secure-hash is, and should not allocate more memory.". So,
I need to test the actual performance on large buffers before switching
from buffer-chars-modified-tick to secure-hash.

Also, I am somehow unable to reproduce the problem in my private Org
branch. Maybe there is some alternative fix without getting rid of
buffer-chars-modified-tick.

Vladimir, if the issue is affecting your workflows, you can disable
org-element-cache until we fix the bug. Just set org-element-use-cache
to nil in your config before loading Org.

Best,
Ihor

Re: table and Cyrillic characters: org-element--cache: Unregistered buffer modifications detected. Resetting

[Ihor Radchenko]

Ihor Radchenko <yantar92@gmail.com> writes:

> Confirmed

Fixed

I tried to change buffer-chars-modified-tick to buffer-hash, but it is
too slow. After discussion with Emacs devs [1], we found some heuristics
that can filter away this particular case. Fixed on main via d8606c316.

[1] https://lists.gnu.org/archive/html/bug-gnu-emacs/2021-11/msg00894.html

Best,
Ihor

Re: table and Cyrillic characters: org-element--cache: Unregistered buffer modifications detected. Resetting

[Max Nikulin]

On 14/11/2021 14:56, Ihor Radchenko wrote:
> Ihor Radchenko writes:
> 
>> Confirmed

Ihor, sorry, I removed "[BUG]" from the subject, so the magic did not work.

> Fixed

Unintentionally I pressed some keys and it appeared again

Warning (emacs): org-element--cache: Unregistered buffer modifications 
detected. Resetting.
If this warning appears regularly, please report it to Org mode mailing 
list (M-x org-submit-bug-report).
The buffer is: cyr.org
  Current command: quail-self-insert-command
  Backtrace:
nil

LANG=en_US.UTF-8 emacs -Q -L ~/src/org-mode/lisp/ cyr.org

C-\ russian-computer RET
||

By "|" I mean Shift+\ that inserts "/" with russian-computer input method.

Re: table and Cyrillic characters: org-element--cache: Unregistered buffer modifications detected. Resetting

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

> Unintentionally I pressed some keys and it appeared again
> ...
> C-\ russian-computer RET
> ||
>
> By "|" I mean Shift+\ that inserts "/" with russian-computer input method.

I pushed yet another workaround.

That part of code is turning into one giant FIXME and I do not see
anything better than patching Emacs itself. Hopefully, not many more
built-in places in Emacs are doing similar silent modifications.

Best,
Ihor

Re: table and Cyrillic characters: org-element--cache: Unregistered buffer modifications detected. Resetting

[Max Nikulin]

On 17/11/2021 19:15, Ihor Radchenko wrote:
> Max Nikulin writes:
> 
>> Unintentionally I pressed some keys and it appeared again
>> ...
>> C-\ russian-computer RET
>> ||
>>
>> By "|" I mean Shift+\ that inserts "/" with russian-computer input method.
> 
> I pushed yet another workaround.
> 
> That part of code is turning into one giant FIXME and I do not see
> anything better than patching Emacs itself. Hopefully, not many more
> built-in places in Emacs are doing similar silent modifications.

Unfortunately the issue persists in Emacs-26.3

Re: table and Cyrillic characters: org-element--cache: Unregistered buffer modifications detected. Resetting

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

>>> By "|" I mean Shift+\ that inserts "/" with russian-computer input method.
>> 
>> I pushed yet another workaround.
>> 
>> That part of code is turning into one giant FIXME and I do not see
>> anything better than patching Emacs itself. Hopefully, not many more
>> built-in places in Emacs are doing similar silent modifications.
>
> Unfortunately the issue persists in Emacs-26.3

Fixed for me by 28d47cfb8. See my reply to
https://orgmode.org/list/sn5pgh$8f0$1@ciao.gmane.io

Best,
Ihor