bug#45091: 27.1; M-x org-table-paste-rectangle

bug#45091: 27.1; M-x org-table-paste-rectangle

[João Távora]

> I apologize that the file I posted is NG.  Please try the following file
> as shown below.

Thanks, now I did reproduce... something.  I get the "Marker does not
point anywhere" error you describe, but also mentions to other broken
functionality, not only the "antiblink":

Error in post-command-hook (jit-lock--antiblink-post-command): (error
"Marker does not point anywhere")
primitive-undo: Marker does not point anywhereError during redisplay:
(jit-lock-function 772) signaled (error "Marker does not point
anywhere")
Error during redisplay: (jit-lock-function 1272) signaled (error
"Marker does not point anywhere")
Error during redisplay: (jit-lock-function 1772) signaled (error
"Marker does not point anywhere")
Error during redisplay: (jit-lock-function 2272) signaled (error
"Marker does not point anywhere")
Error during redisplay: (jit-lock-function 2772) signaled (error
"Marker does not point anywhere")

If I turn on debug-on-error, I also get the stacktrace below
on RET, which runs newline.
It seems the problem comes from "syntax-ppss", which is also
called from "antiblink". I cannot always reproduce this, and
often I can reproduce without needing the "undo".

I'll keep investigating: I still don't know which marker the message
is referring to.  It could be antiblink's marker, but it could be
anything else.

João

Debugger entered--Lisp error: (error "Marker does not point anywhere")
  syntax-ppss()
  electric-indent-post-self-insert-function()
  self-insert-command(1)
  newline(nil 1)
  funcall-interactively(newline nil 1)
  call-interactively(newline)
  (let (orgtbl-mode) (call-interactively (or (key-binding "\15")
(key-binding [(return)]) 'orgtbl-error)))
  (if (org-at-table-p) (call-interactively 'orgtbl-ret) (let
(orgtbl-mode) (call-interactively (or (key-binding "\15") (key-binding
[(return)]) 'orgtbl-error))))
  orgtbl-hijacker-command-101(1)
  funcall-interactively(orgtbl-hijacker-command-101 1)
  call-interactively(orgtbl-hijacker-command-101 nil nil)
  command-execute(orgtbl-hijacker-command-101)

bug#45091: 27.1; M-x org-table-paste-rectangle

[João Távora]

João Távora <joaotavora@gmail.com> writes:

> I'll keep investigating: I still don't know which marker the message
> is referring to.  It could be antiblink's marker, but it could be
> anything else.
>
> João

I've now reproduced after setting

  (setq jit-lock-antiblink-grace nil)

Which turns off the antiblink feature _almost_ entirely.  I've also been
able to reproduce it from the command line via:

    ~/Source/Emacs/emacs-27/src/emacs -Q ~/tmp/taks-file.latex                         \
      --eval "(progn                                                                   \
               (require 'org)                                                          \
               (require 'org-table)                                                    \
               (orgtbl-mode)                                                           \
               (remove-hook 'post-command-hook                                         \
                            'jit-lock--antiblink-post-command t))"

Which _completely_ turns off the antiblink feature.

It doesn't need a window system, -nw is fine to reproduce.

This was with a fresh build of the Emacs 27.1 tag.

It's hard to reproduce sometimes, but iterating the undo/paste rectangle
cycle eventually gets one there.  It has to be done on the very same
file that Tak Kunihiro posted.  Sometimes I mess the file slightly (in
ways that I can't tell) and it doesn't happen again.

Tak Kunihiro, what is in your opinion the deciding character of these
files? I.e. what did you change from the first file (where I couldn't
reproduce) to the second?

I'm quite lost as to why this happens, of course, but it seems it's
always coming from syntax-ppss.  When I evaluate that definition (rather
than compiling), I get more clues:

Debugger entered--Lisp error: (error "Marker does not point anywhere")
  >(#<marker in no buffer> 2919)
  (and old-pos (> old-pos pos))
  (if (and old-pos (> old-pos pos)) (setq old-pos nil))
  (let* ((cell (syntax-ppss--data)) (ppss-last (car cell)) (ppss-cache (cdr cell)) (old-ppss (cdr ppss-last)) (old-pos (car pps$
  (progn (set-syntax-table (or syntax-ppss-table (syntax-table))) (let* ((cell (syntax-ppss--data)) (ppss-last (car cell)) (pps$
  (unwind-protect (progn (set-syntax-table (or syntax-ppss-table (syntax-table))) (let* ((cell (syntax-ppss--data)) (ppss-last $
  (let ((table (syntax-table)) (buffer (current-buffer))) (unwind-protect (progn (set-syntax-table (or syntax-ppss-table (synta$
  syntax-ppss()

I'll keep looking a bit, but at this point it doesn't seem to have
anything to do with the antiblink feature.  I might be wrong, but I
think that only shows up first in the messages buffer because it's
unlucky enough to be one of the first users of syntax-ppss after a
command.

João

bug#45091: 27.1; M-x org-table-paste-rectangle

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  45091@debbugs.gnu.org,
>   tkk@misasa.okayama-u.ac.jp
> Date: Tue, 08 Dec 2020 17:02:13 +0000
> 
> I'm quite lost as to why this happens, of course, but it seems it's
> always coming from syntax-ppss.  When I evaluate that definition (rather
> than compiling), I get more clues:
> 
> Debugger entered--Lisp error: (error "Marker does not point anywhere")
>   >(#<marker in no buffer> 2919)
>   (and old-pos (> old-pos pos))
>   (if (and old-pos (> old-pos pos)) (setq old-pos nil))
>   (let* ((cell (syntax-ppss--data)) (ppss-last (car cell)) (ppss-cache (cdr cell)) (old-ppss (cdr ppss-last)) (old-pos (car pps$
>   (progn (set-syntax-table (or syntax-ppss-table (syntax-table))) (let* ((cell (syntax-ppss--data)) (ppss-last (car cell)) (pps$
>   (unwind-protect (progn (set-syntax-table (or syntax-ppss-table (syntax-table))) (let* ((cell (syntax-ppss--data)) (ppss-last $
>   (let ((table (syntax-table)) (buffer (current-buffer))) (unwind-protect (progn (set-syntax-table (or syntax-ppss-table (synta$
>   syntax-ppss()
> 
> I'll keep looking a bit, but at this point it doesn't seem to have
> anything to do with the antiblink feature.  I might be wrong, but I
> think that only shows up first in the messages buffer because it's
> unlucky enough to be one of the first users of syntax-ppss after a
> command.

Stefan, any ideas?

bug#45091: 27.1; M-x org-table-paste-rectangle

[Tak Kunihiro]

> Tak Kunihiro, what is in your opinion the deciding character of these
> files? I.e. what did you change from the first file (where I couldn't
> reproduce) to the second?

The difference between the first and the second files is, the existence
of plain sentences after the second org-table.

I see the problem randomly when I make a LaTeX table using org-mode
radio-table.

I was writing a recipe with the two org-tables.  I thought that
org-tables are the essence of the problem; however, it seems that
org-tables followed by plain sentences seems to be the source of the
problem.

Again, I do not see problem on 26.3.

bug#45091: 27.1; M-x org-table-paste-rectangle

[Tak Kunihiro]

> I'll keep looking a bit, but at this point it doesn't seem to have
> anything to do with the antiblink feature.  I might be wrong, but I
> think that only shows up first in the messages buffer because it's
> unlucky enough to be one of the first users of syntax-ppss after a
> command.

I confirmed that I can reproduce the problem with 27.1.90.  Although the
recipe seems complicated, I see the problem quite often when I work with
LaTeX-mode.  I still have to stick on 26.3.

I appreciate if the problem would be fixed.

bug#45091: 27.1; M-x org-table-paste-rectangle

[João Távora]

On Sun, Dec 20, 2020 at 12:15 AM Tak Kunihiro <homeros.misasa@gmail.com> wrote:
>
> > I'll keep looking a bit, but at this point it doesn't seem to have
> > anything to do with the antiblink feature.  I might be wrong, but I
> > think that only shows up first in the messages buffer because it's
> > unlucky enough to be one of the first users of syntax-ppss after a
> > command.
>
> I confirmed that I can reproduce the problem with 27.1.90.  Although the
> recipe seems complicated, I see the problem quite often when I work with
> LaTeX-mode.  I still have to stick on 26.3.
>
> I appreciate if the problem would be fixed.

Are you familiar with `git bisect`?  It's a bit of work, but it usually points
to the culprit commit.  Emacs history is very good.  I don't have
time for this right now, but maybe you have?  It's a question
of compiling Emacs methodically and then testing your recipe
each time.

João Távora

bug#45091: 27.1; M-x org-table-paste-rectangle

[Tak Kunihiro]

>> > I'll keep looking a bit, but at this point it doesn't seem to have
>> > anything to do with the antiblink feature.  I might be wrong, but I
>> > think that only shows up first in the messages buffer because it's
>> > unlucky enough to be one of the first users of syntax-ppss after a
>> > command.
>>
>> I confirmed that I can reproduce the problem with 27.1.90.  Although the
>> recipe seems complicated, I see the problem quite often when I work with
>> LaTeX-mode.  I still have to stick on 26.3.
>>
>> I appreciate if the problem would be fixed.
> 
> Are you familiar with `git bisect`?  It's a bit of work, but it usually points
> to the culprit commit.  Emacs history is very good.  I don't have
> time for this right now, but maybe you have?  It's a question
> of compiling Emacs methodically and then testing your recipe
> each time.

I am not familiar with `git bisect` but I understand how to detect the
source of the problem.  I will ... try.

bug#45091: 27.1; M-x org-table-paste-rectangle

[Tak Kunihiro]

Naoya showed me workaround.

#+begin_src emacs-lisp
(when (equal emacs-major-version 27)
  (with-eval-after-load "org-compat"
    (defalias 'org-font-lock-ensure 'ignore)))
#+end_src

Now I switched from 26.3 to 27.1.

bug#45091: 27.1; M-x org-table-paste-rectangle

[Naoya Yamashita]

Hi, I'm new on this thread, but I can reproduce tak's issue and I
tried to do bisect.

Finally, I got 5b45c269cda09fe46e110adb6f6767040f4ade59 is a
commit has a some problem.

I'm happy if this information could help you.

```
5b45c269cda09fe46e110adb6f6767040f4ade59
New jit-lock-antiblink-grace feature

* lisp/jit-lock.el (jit-lock-antiblink-grace): New defcustom.
(jit-lock--antiblink-line-beginning-position)
(jit-lock--antiblink-string-or-comment): New variables
(jit-lock--antiblink-post-command): New helper.
(jit-lock-mode): Tweak post-command-hook and
jit-lock-context-timer.

* etc/NEWS: Mention jit-lock-antiblink-grace

2 files changed, 81 insertions(+), 1 deletion(-)
etc/NEWS         |  7 ++++++
lisp/jit-lock.el | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-
```

---

Here is my bash script and do `C-c l` (org-table-copy-region) on
first table and `C-c ;` (org-table-paste-rectangle) on second table.
(~/dev/tmp/taks-file.tex have first tak's mention contents.)

```sh
make clean
./autogen.sh
./configure --prefix=$HOME/.local/head
make -j6
./src/emacs -Q ~/dev/tmp/taks-file.tex --eval "             \
  (progn                                                    \
    (setq truncate-lines t)                                 \
    (setq debug-on-error t)                                 \
    (require 'org)                                          \
    (require 'org-table)                                    \
    (orgtbl-mode)                                           \
    (global-set-key (kbd \"C-c l\")                         \
       (lambda ()                                           \
         (interactive)                                      \
         (org-table-copy-region (1+ (org-table-begin)) (- (org-table-end) 2)))) \
    (global-set-key (kbd \"C-c ;\")                         \
       (lambda ()                                           \
         (interactive)                                      \
         (org-table-paste-rectangle)                        \
         (set-buffer-modified-p nil))))"                    \
2>/dev/null
```

And here is my bisect log.
```
Bisect Log (14)
git bisect start 'emacs-27.1' 'emacs-26.3'
git bisect good 38564f8a664347c42f7614d9c91e0d49e4a073e8
git bisect good 825fd4d34165e5edc718a9df93838a410d25ea81
git bisect good 5c5c1b559313d06aab6516ff1b1acf1da3a01c7d
git bisect bad d7cd4ab7d948427104003f1d35cf52a4eac45d0a
git bisect bad 4b2c2faab83fe3b13430b837be7d450b5cd47caf
git bisect good 16ce6dbef279bda70b4d60b4d2d0aff008bfa5f7
git bisect bad 5063e38921de8cb872965abda32bcc6fd8894532
git bisect bad 4c933077157ba409d645f4649c8a3a8e534d53d5
git bisect good 096be9c4541329af259273fe604dc4f8669fbd8a
git bisect bad bd95d66ae97f3dcfe73b972de935e27e434e49be
git bisect bad 5b45c269cda09fe46e110adb6f6767040f4ade59
git bisect good 0e774d4f355b4f12a625da5ca9602d1ba876bcc1
5b45c269cda09fe46e110adb6f6767040f4ade59 is the first bad commit
```

bug#45091: 27.1; M-x org-table-paste-rectangle

[João Távora]

Okay, your bisect scripts points to one of my commits.

But I wish it was more automated.  Because I tried quite
hard and sometimes I _could_ reproduce the problem and
sometimes I couldn't.  So you could be seeing noise here.
Maybe you can make your script  do those copy-paste
operations automatically and reliably?  Then we'd be sure.

Anyway, I think is might be enough to point to a bug
that was introduced by my commit, or rather exposed
by it.  I'll have another look.

João

On Fri, Jan 15, 2021 at 12:11 PM Naoya Yamashita <conao3@gmail.com> wrote:
>
>
> Hi, I'm new on this thread, but I can reproduce tak's issue and I
> tried to do bisect.
>
> Finally, I got 5b45c269cda09fe46e110adb6f6767040f4ade59 is a
> commit has a some problem.
>
> I'm happy if this information could help you.
>
> ```
> 5b45c269cda09fe46e110adb6f6767040f4ade59
> New jit-lock-antiblink-grace feature
>
> * lisp/jit-lock.el (jit-lock-antiblink-grace): New defcustom.
> (jit-lock--antiblink-line-beginning-position)
> (jit-lock--antiblink-string-or-comment): New variables
> (jit-lock--antiblink-post-command): New helper.
> (jit-lock-mode): Tweak post-command-hook and
> jit-lock-context-timer.
>
> * etc/NEWS: Mention jit-lock-antiblink-grace
>
> 2 files changed, 81 insertions(+), 1 deletion(-)
> etc/NEWS         |  7 ++++++
> lisp/jit-lock.el | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-
> ```
>
> ---
>
> Here is my bash script and do `C-c l` (org-table-copy-region) on
> first table and `C-c ;` (org-table-paste-rectangle) on second table.
> (~/dev/tmp/taks-file.tex have first tak's mention contents.)
>
> ```sh
> make clean
> ./autogen.sh
> ./configure --prefix=$HOME/.local/head
> make -j6
> ./src/emacs -Q ~/dev/tmp/taks-file.tex --eval "             \
>   (progn                                                    \
>     (setq truncate-lines t)                                 \
>     (setq debug-on-error t)                                 \
>     (require 'org)                                          \
>     (require 'org-table)                                    \
>     (orgtbl-mode)                                           \
>     (global-set-key (kbd \"C-c l\")                         \
>        (lambda ()                                           \
>          (interactive)                                      \
>          (org-table-copy-region (1+ (org-table-begin)) (- (org-table-end) 2)))) \
>     (global-set-key (kbd \"C-c ;\")                         \
>        (lambda ()                                           \
>          (interactive)                                      \
>          (org-table-paste-rectangle)                        \
>          (set-buffer-modified-p nil))))"                    \
> 2>/dev/null
> ```
>
> And here is my bisect log.
> ```
> Bisect Log (14)
> git bisect start 'emacs-27.1' 'emacs-26.3'
> git bisect good 38564f8a664347c42f7614d9c91e0d49e4a073e8
> git bisect good 825fd4d34165e5edc718a9df93838a410d25ea81
> git bisect good 5c5c1b559313d06aab6516ff1b1acf1da3a01c7d
> git bisect bad d7cd4ab7d948427104003f1d35cf52a4eac45d0a
> git bisect bad 4b2c2faab83fe3b13430b837be7d450b5cd47caf
> git bisect good 16ce6dbef279bda70b4d60b4d2d0aff008bfa5f7
> git bisect bad 5063e38921de8cb872965abda32bcc6fd8894532
> git bisect bad 4c933077157ba409d645f4649c8a3a8e534d53d5
> git bisect good 096be9c4541329af259273fe604dc4f8669fbd8a
> git bisect bad bd95d66ae97f3dcfe73b972de935e27e434e49be
> git bisect bad 5b45c269cda09fe46e110adb6f6767040f4ade59
> git bisect good 0e774d4f355b4f12a625da5ca9602d1ba876bcc1
> 5b45c269cda09fe46e110adb6f6767040f4ade59 is the first bad commit
> ```
>
>
>


-- 
João Távora

bug#45091: 27.1; M-x org-table-paste-rectangle

[Naoya Yamashita]

> Okay, your bisect scripts points to one of my commits.
> 
> But I wish it was more automated.  Because I tried quite
> hard and sometimes I _could_ reproduce the problem and
> sometimes I couldn't.  So you could be seeing noise here.
> Maybe you can make your script  do those copy-paste
> operations automatically and reliably?  Then we'd be sure.
> 
> Anyway, I think is might be enough to point to a bug
> that was introduced by my commit, or rather exposed
> by it.  I'll have another look.

Sorry, you're true.  I try to build (semi-)automatically
reproduce step and do bisect with it, I got different commit.

It is 165f7383822086d465519ebe6e4283723923f097 (Update Org to 9.3).

(I wanted to reproduce the problem completely in auto, but the
problem is related to Emacs font-lock and post-command-hook, and
I could not reproduce it with --batch.  If you know how to
partially pass control to Emacs main-loop while running Elisp,
please tell me that.)


## Repro step

1. Prepare below contents.

```latex <~/dev/tmp/taks-file.tex>
-*- mode: latex -*-

| rownames                    |  acq |
| ref-coal-krt-2000-4-s1@4104 | 4104 |
| ref-coal-krt-2000-4-s1@4105 | 4105 |

|------------------------+--------+--------+---------------+---------+---------+-----+------+------|
|                        | 12C12C | 12C14N | 12C14Nz12C12C | d13Cion | d15Nion | N/C | d13C | d15N |
|------------------------+--------+--------+---------------+---------+---------+-----+------+------|
| !cniso-mtx-ogl1-2@4112 |        |        |               |         |         |     |      |      |
| cniso-mtx-ogl1-2@4113  |        |        |               |         |         |     |      |      |
| cniso-mtx-ogl1-2@4114  |        |        |               |         |         |     |      |      |
| cniso-mtx-ogl1-2@4115  |        |        |               |         |         |     |      |      |
| cniso-mtx-ogl1-2@4116  |        |        |               |         |         |     |      |      |
| cniso-mtx-ogl1-2@4117  |        |        |               |         |         |     |      |      |
| cniso-mtx-ogl1-2@4118  |        |        |               |         |         |     |      |      |
| cniso-mtx-ogl1-2@4119  |        |        |               |         |         |     |      |      |
| cniso-mtx-ogl1-2@4124  |        |        |               |         |         |     |      |      |
| cniso-mtx-ogl1-2@4125  |        |        |               |         |         |     |      |      |
| cniso-mtx-ogl1-2@4126  |        |        |               |         |         |     |      |      |
| cniso-mtx-ogl1-2@4127  |        |        |               |         |         |     |      |      |
| cniso-mtx-ogl1-2@4128  |        |        |               |         |         |     |      |      |
| cniso-mtx-ogl1-2@4129  |        |        |               |         |         |     |      |      |
| cniso-mtx-ogl1-2@4130  |        |        |               |         |         |     |      |      |
| cniso-mtx-ogl1-2@4131  |        |        |               |         |         |     |      |      |
| cniso-mtx-ogl1-2@4132  |        |        |               |         |         |     |      |      |
| cniso-mtx-ogl1-2@4133  |        |        |               |         |         |     |      |      |
| cniso-mtx-ogl1-2@4134  |        |        |               |         |         |     |      |      |
| cniso-mtx-ogl1-2@4135  |        |        |               |         |         |     |      |      |
| cniso-mtx-ogl1-2@4136  |        |        |               |         |         |     |      |      |
|------------------------+--------+--------+---------------+---------+---------+-----+------+------|
```

```sh <~/dev/tmp/bisect-script.sh>
git clean -fdx
make clean
./autogen.sh
./configure --prefix=$HOME/.local/head
make -j6
./src/emacs -Q ~/dev/tmp/taks-file.tex --eval "
  (progn
    (setq truncate-lines t)
    (setq debug-on-error t)
    (require 'org)
    (require 'org-table)
    (orgtbl-mode)
    
    (search-forward \"|\")
    (org-table-copy-region (1+ (org-table-begin)) (- (org-table-end) 2))
    
    (search-forward \"!\")
    
    (global-set-key (kbd \"<f5>\")
      (lambda ()
        (interactive)
        (org-table-paste-rectangle)
        (set-buffer-modified-p nil)
        (condition-case e
            (insert \"text\")
          (error
           (kill-emacs 1)))
        (kill-emacs 0)))
    )" \
2>/dev/null
```

2. Do below command in terminal
```sh
git bisect start emacs-27.1 emacs-26.3
git bisect run ~/dev/tmp/bisect-script.sh
```

3. Wait for build and wake Emacs, then press <f5>.

4. Got commit it has some problem.
(I got bellow bisect log in Magit)
```
Bisect Log (15)
git bisect start 'emacs-27.1' 'emacs-26.3'
git bisect good 38564f8a664347c42f7614d9c91e0d49e4a073e8
git bisect good 825fd4d34165e5edc718a9df93838a410d25ea81
git bisect good 5c5c1b559313d06aab6516ff1b1acf1da3a01c7d
git bisect bad d7cd4ab7d948427104003f1d35cf52a4eac45d0a
git bisect bad 4b2c2faab83fe3b13430b837be7d450b5cd47caf
git bisect good 16ce6dbef279bda70b4d60b4d2d0aff008bfa5f7
git bisect bad 5063e38921de8cb872965abda32bcc6fd8894532
git bisect bad 4c933077157ba409d645f4649c8a3a8e534d53d5
git bisect bad 096be9c4541329af259273fe604dc4f8669fbd8a
git bisect bad 8f2267c87bc96d6301aadeda66b870a61a0433e8
git bisect good 8bea7e9ab4453da71d9766d582089154f31de907
git bisect bad 165f7383822086d465519ebe6e4283723923f097
git bisect good 821de968434d2096bdea67dd24301bf6b517aef1
165f7383822086d465519ebe6e4283723923f097 is the first bad commit
```