Stack overflow in regexp matcher

Stack overflow in regexp matcher

[Michael Brand]

Hi all

Here is an issue for the quite common work flow of copy pasting some
table columns in a whole from a spreadsheet application into Org
table. For this situation I suggest to provide a better user
information than only the current error message "Stack overflow in
regexp matcher". Reproduce like this:

in a spreadsheet application like OpenOffice.org Calc or Excel:
- create a 1x2 table with two columns A and B
- fill out the fields A1 and B1 with some simple content
- select the columns A and B and copy to clipboard

in a new Org buffer:
- paste from clipboard
- select the line with A1 and B1
- convert the selection to Org table: C-c |
- change A1 to the assignment `=1'
- try to update Org table: C-c C-c

This will produce the above mentioned error message. The reason are
the 65535 lines with a TAB following the Org table that have been
copied when selecting the two first columns A and B in the spreadsheet
application. It can be quite annoying to find this out with only the
current error message and with TABs not made visible so I thought it
would be helpful to point this out in the hope of someone finding a
better user guidance. I'd like to provide a patch by myself but in
this case I don't understand more than to find out that the many TABs
are the problem and have to be removed or to be not copied.

Michael

Re: Stack overflow in regexp matcher

[Matt Lundin]

Michael Brand <michael.ch.brand@gmail.com> writes:

> Here is an issue for the quite common work flow of copy pasting some
> table columns in a whole from a spreadsheet application into Org
> table. For this situation I suggest to provide a better user
> information than only the current error message "Stack overflow in
> regexp matcher". Reproduce like this:
>
> in a spreadsheet application like OpenOffice.org Calc or Excel:
> - create a 1x2 table with two columns A and B
> - fill out the fields A1 and B1 with some simple content
> - select the columns A and B and copy to clipboard
>
> in a new Org buffer:
> - paste from clipboard
> - select the line with A1 and B1
> - convert the selection to Org table: C-c |
> - change A1 to the assignment `=1'
> - try to update Org table: C-c C-c
>
> This will produce the above mentioned error message. The reason are
> the 65535 lines with a TAB following the Org table that have been
> copied when selecting the two first columns A and B in the spreadsheet
> application. It can be quite annoying to find this out with only the
> current error message and with TABs not made visible so I thought it
> would be helpful to point this out in the hope of someone finding a
> better user guidance. I'd like to provide a patch by myself but in
> this case I don't understand more than to find out that the many TABs
> are the problem and have to be removed or to be not copied.
>

I can reproduce this with OpenOffice. Here's the backtrace:

Debugger entered--Lisp error: (error "Stack overflow in regexp matcher")
  looking-at("\\([ 	]*\n\\)*[ 	]*#\\+TBLFM: *\\(.*\\)")
  org-table-get-stored-formulas()
  org-table-get-formula(#("1" 0 1 (face org-formula fontified t)) nil)
  org-table-eval-formula(nil #("1" 0 1 (face org-formula fontified t)))
  org-table-maybe-eval-formula()
  org-ctrl-c-ctrl-c(nil)
  call-interactively(org-ctrl-c-ctrl-c nil nil)

Best,
Matt

Stack overflow in regexp matcher

[Alan Schmitt]

[-- Attachment #1: Type: text/plain, Size: 2966 bytes --]

Hello,

I've just got the following backtrace when opening an org
file. Unfortunately I cannot reproduce it reliably, but I was wondering
if others had seen something similar.

Debugger entered--Lisp error: (error "Stack overflow in regexp matcher")
  re-search-forward("^[^%]*\\\\usepackage.*{biblatex}" nil t)
  reftex-using-biblatex-p()
  reftex-locate-bibliography-files("/Users/schmitta/Documents/Org/")
  byte-code("…" [file master-dir file-found buf docstruct reftex-keep-temporary-buffers reftex-locate-file "tex" reftex-get-buffer-visiting buffer-file-name file-error throw exit nil message "Scanning file %s" reftex-get-file-buffer-force t bof syntax-table ((set-syntax-table saved-syntax)) set-syntax-table 1 re-search-forward reftex-match-string 0 7 "label{" fboundp TeX-current-macro LaTeX-current-environment match-data ((byte-code "…" [save-match-data-internal set-match-data evaporate] 3)) ((byte-code "…" [save-match-data-internal set-match-data evaporate] 3)) reftex-label-info 3 92 -1 5 "Scanning %s %s ..." rassoc 6 delq mapcar #[(x) "…" [x include-file string-match] 3] reftex-parse-from-file 9 reftex-init-section-numbers appendix 10 ...] 7)
  reftex-parse-from-file("/Users/schmitta/Documents/Org/boardgames-runx.org" nil "/Users/schmitta/Documents/Org/")
  reftex-do-parse(1 nil)
  reftex-access-scan-info((16))
  reftex-parse-all()
  (progn (reftex-parse-all) (reftex-set-cite-format (quote ((98 . "[[bib:%l][%l-bib]]") (99 . "\\cite{%l}") (110 . "[[note:%l][%l-note]]") (108 . "%t (%a)\n[[bibdesk:/Users/schmitta/projets/bib/alan.bib::%l][%l-BibDesk]]") (116 . "%t") (104 . "** %t (%a)\n:PROPERTIES:\n:Custom_ID: %l\n:END:\n[[bib:%l][%l-bib]]\n[[bibdesk:/Users/schmitta/projets/bib/alan.bib::%l][%l-BibDesk]]")))))
  (and (buffer-file-name) (file-exists-p (buffer-file-name)) (progn (reftex-parse-all) (reftex-set-cite-format (quote ((98 . "[[bib:%l][%l-bib]]") (99 . "\\cite{%l}") (110 . "[[note:%l][%l-note]]") (108 . "%t (%a)\n[[bibdesk:/Users/schmitta/projets/bib/alan.bib::%l][%l-BibDesk]]") (116 . "%t") (104 . "** %t (%a)\n:PROPERTIES:\n:Custom_ID: %l\n:END:\n[[bib:%l][%l-bib]]\n[[bibdesk:/Users/schmitta/projets/bib/alan.bib::%l][%l-BibDesk]]"))))))
  org-mode-reftex-setup()
  run-hooks(change-major-mode-after-body-hook text-mode-hook outline-mode-hook org-mode-hook)
  apply(run-hooks (change-major-mode-after-body-hook text-mode-hook outline-mode-hook org-mode-hook))
  run-mode-hooks(org-mode-hook)
  org-mode()
  set-auto-mode-0(org-mode nil)
  set-auto-mode()
  normal-mode(t)
  after-find-file(nil t)
  find-file-noselect-1(#<buffer boardgames-runx.org> "~/Documents/Org/boardgames-runx.org" nil nil "~/Documents/Org/boardgames-runx.org" (1095881 16777220))
  find-file-noselect("/Users/schmitta/Documents/Org/boardgames-runx.org" nil nil WILDCARDS)
  find-file("/Users/schmitta/Documents/Org/boardgames-runx.org" WILDCARDS)

Thanks,

Alan

-- 
OpenPGP Key ID : 040D0A3B4ED2E5C7

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 494 bytes --]

Re: Stack overflow in regexp matcher

[Charles Berry]

Alan Schmitt <alan.schmitt <at> polytechnique.org> writes:

> 
> Hello,
> 
> I've just got the following backtrace when opening an org
> file. Unfortunately I cannot reproduce it reliably, but I was wondering
> if others had seen something similar.
> 

I do. Try this. Start emacs 24.4.1.

1) Copy this line:

    CLOCK: [2013-07-22 Mon 12:29]--[2013-07-22 Mon 15:26] =>  2:57

into a new *.org file.

2) Save and close the file and reopen. 

3) If no such error, kill the text in the buffer, and yank it twice.

4) Repeat 2 and 3 till the message

   File mode specification error: (error "Stack overflow in regexp matcher")

shows up. 

5) prune lines from the file and try step 2 till the error disappears.


For me somewhere between 400 and 500 lines is the difference between opening
with no error message and opening with the message.

Emacs 24.3.1 does not show this behavior - no error message. 

HTH,

Chuck

Re: Stack overflow in regexp matcher

[Gregor Zattler]

Hi Charles, org-mode developers
* Charles Berry <ccberry@ucsd.edu> [24. Oct. 2014]:
> I do. Try this. Start emacs 24.4.1.
> 
> 1) Copy this line:
> 
>     CLOCK: [2013-07-22 Mon 12:29]--[2013-07-22 Mon 15:26] =>  2:57
> 
> into a new *.org file.
> 
> 2) Save and close the file and reopen. 
> 
> 3) If no such error, kill the text in the buffer, and yank it twice.
> 
> 4) Repeat 2 and 3 till the message
> 
>    File mode specification error: (error "Stack overflow in regexp matcher")
> 
> shows up. 
> 
> 5) prune lines from the file and try step 2 till the error disappears.
> 
> 
> For me somewhere between 400 and 500 lines is the difference between opening
> with no error message and opening with the message.
> 
> Emacs 24.3.1 does not show this behavior - no error message. 

I cannot reproduce this.  I even tried with a file with 8264216
such lines.  It took less than 4 minutes to open the file, less
than 24 minutes to recalculate the second to last line.  Then
there was a message in the echo area: "Buffer test.org has shrunk
a lot; auto save disabled in that buffer until next real save",
but the buffer was actually as large as before (520645608 bytes).

Mine is
GNU Emacs 24.4.51.1 (i686-pc-linux-gnu, GTK+ Version 3.14.3) of 2014-10-24
and
Org-mode version 8.2.10 (release_8.2.10 @ /usr/local/stow/emacs-snapshot/share/emacs/24.4.51/lisp/org/)

Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

Re: Stack overflow in regexp matcher

[Alan Schmitt]

[-- Attachment #1: Type: text/plain, Size: 1314 bytes --]

On 2014-10-24 19:02, Charles Berry <ccberry@ucsd.edu> writes:

> Alan Schmitt <alan.schmitt <at> polytechnique.org> writes:
>
>> 
>> Hello,
>> 
>> I've just got the following backtrace when opening an org
>> file. Unfortunately I cannot reproduce it reliably, but I was wondering
>> if others had seen something similar.
>> 
>
> I do. Try this. Start emacs 24.4.1.
>
> 1) Copy this line:
>
>     CLOCK: [2013-07-22 Mon 12:29]--[2013-07-22 Mon 15:26] =>  2:57
>
> into a new *.org file.
>
> 2) Save and close the file and reopen. 
>
> 3) If no such error, kill the text in the buffer, and yank it twice.
>
> 4) Repeat 2 and 3 till the message
>
>    File mode specification error: (error "Stack overflow in regexp matcher")
>
> shows up. 
>
> 5) prune lines from the file and try step 2 till the error disappears.
>
>
> For me somewhere between 400 and 500 lines is the difference between opening
> with no error message and opening with the message.

Interesting. I don't use clocking, but the files in question use tags
and logbooks. I can actually open the file when I try a second time, so
to bisect this I would need to restart emacs each time. I'll try to see
if I can get a reproduceable recipe on my side.

Thanks,

Alan

-- 
OpenPGP Key ID : 040D0A3B4ED2E5C7

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 494 bytes --]

Re: Stack overflow in regexp matcher

[Charles C. Berry]

On Fri, 24 Oct 2014, Gregor Zattler wrote:

> Hi Charles, org-mode developers
> * Charles Berry <ccberry@ucsd.edu> [24. Oct. 2014]:
>> I do. Try this. Start emacs 24.4.1.
>>
>> 1) Copy this line:
>>
>>     CLOCK: [2013-07-22 Mon 12:29]--[2013-07-22 Mon 15:26] =>  2:57
>>
>> into a new *.org file.
>>
>> 2) Save and close the file and reopen.
>>
>> 3) If no such error, kill the text in the buffer, and yank it twice.
>>
>> 4) Repeat 2 and 3 till the message
>>
>>    File mode specification error: (error "Stack overflow in regexp matcher")
>>
>> shows up.
>>
>> 5) prune lines from the file and try step 2 till the error disappears.
>>
>>
>> For me somewhere between 400 and 500 lines is the difference between opening
>> with no error message and opening with the message.
>>
>> Emacs 24.3.1 does not show this behavior - no error message.
>
> I cannot reproduce this.  I even tried with a file with 8264216

[rest deleted]

OK, thanks. It had to be something in my init.

I have (defun org-mode-reftex-setup () ... ) from

http://www.mfasold.net/blog/2009/02/using-emacs-org-mode-to-draft-papers/

in my init.

It turns out that

   (re-search-forward "^[^%]*\\\\usepackage.*{biblatex}" nil t))

in reftex-using-biblatex-p is the culprit.

So, the ECM is to save a file like that decribed above without the *.org 
extension.

Open it (in fundamental mode) and issue

M-: (re-search-forward "^[^%]*\\\\usepackage.*{biblatex}" nil t))

and get

eval: Stack overflow in regexp matcher

So, maybe this is not an org-mode issue. ??


HTH,

Chuck

Re: Stack overflow in regexp matcher

[Charles C. Berry]

On Sat, 25 Oct 2014, Charles C. Berry wrote:

> On Fri, 24 Oct 2014, Gregor Zattler wrote:
>
>> Hi Charles, org-mode developers
>> * Charles Berry <ccberry@ucsd.edu> [24. Oct. 2014]:
>>> I do. Try this. Start emacs 24.4.1.
>>> 
>>> 1) Copy this line:
>>>
>>>     CLOCK: [2013-07-22 Mon 12:29]--[2013-07-22 Mon 15:26] =>  2:57
>>> 
>>> into a new *.org file.
>>> 
>>> 2) Save and close the file and reopen.
>>> 
>>> 3) If no such error, kill the text in the buffer, and yank it twice.
>>> 
>>> 4) Repeat 2 and 3 till the message
>>>
>>>    File mode specification error: (error "Stack overflow in regexp 
>>> matcher")
>>> 
>>> shows up.
>>> 
>>> 5) prune lines from the file and try step 2 till the error disappears.
>>> 
>>>

[snip]

>
> It turns out that
>
>  (re-search-forward "^[^%]*\\\\usepackage.*{biblatex}" nil t))
>
> in reftex-using-biblatex-p is the culprit.
>
> So, the ECM is to save a file like that decribed above without the *.org 
> extension.
>
> Open it (in fundamental mode) and issue
>
> M-: (re-search-forward "^[^%]*\\\\usepackage.*{biblatex}" nil t))
>
> and get
>
> eval: Stack overflow in regexp matcher
>
> So, maybe this is not an org-mode issue. ??


Further, changing that regexp to

  "^[^%\n]*\\\\usepackage.*{biblatex}"

and byte-compiling reftex-parse.el.gz seems to make the problem go away.

This seems to be well-known problem:

    http://www.emacswiki.org/emacs/MultilineRegexp

Still, I am not sure why this happens in 24.4.1 and not previously.

HTH,

Chuck

Re: Stack overflow in regexp matcher

[Alan Schmitt]

[-- Attachment #1: Type: text/plain, Size: 1613 bytes --]

On 2014-10-25 11:00, "Charles C. Berry" <ccberry@ucsd.edu> writes:

> OK, thanks. It had to be something in my init.
>
> I have (defun org-mode-reftex-setup () ... ) from
>
> http://www.mfasold.net/blog/2009/02/using-emacs-org-mode-to-draft-papers/
>
> in my init.

Interesting, I have this

#+begin_src emacs-lisp
;; from http://orgmode.org/worg/org-faq.html#using-reftex-in-org-mode

;; Changed it to follow http://tincman.wordpress.com/2011/01/04/research-paper-management-with-emacs-org-mode-and-reftex/

  (defun org-mode-reftex-setup ()
    (load-library "reftex")
    (and (buffer-file-name) (file-exists-p (buffer-file-name))
         (progn 
           (reftex-parse-all)
           ;add a custom reftex cite format to insert links
           (reftex-set-cite-format
            '((?b . "[[bib:%l][%l-bib]]")
              (?c . "\\cite{%l}")
              (?n . "[[note:%l][%l-note]]")
              (?l . "%t (%a)\n[[bibdesk:/Users/schmitta/projets/bib/alan.bib::%l][%l-BibDesk]]")
              (?t . "%t")
              (?h . "** %t (%a)\n:PROPERTIES:\n:Custom_ID: %l\n:END:\n[[bib:%l][%l-bib]]\n[[bibdesk:/Users/schmitta/projets/bib/alan.bib::%l][%l-BibDesk]]"))))))
#+end_src

> So, the ECM is to save a file like that decribed above without the
> *.org extension.
>
> Open it (in fundamental mode) and issue
>
> M-: (re-search-forward "^[^%]*\\\\usepackage.*{biblatex}" nil t))
>
> and get
>
> eval: Stack overflow in regexp matcher
>
> So, maybe this is not an org-mode issue. ??

Indeed.

Thanks,

Alan

-- 
OpenPGP Key ID : 040D0A3B4ED2E5C7

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 494 bytes --]

Re: Stack overflow in regexp matcher

[Alan Schmitt]

[-- Attachment #1: Type: text/plain, Size: 910 bytes --]

On 2014-10-25 12:34, "Charles C. Berry" <ccberry@ucsd.edu> writes:

> Further, changing that regexp to
>
>  "^[^%\n]*\\\\usepackage.*{biblatex}"
>
> and byte-compiling reftex-parse.el.gz seems to make the problem go away.

The auctex developers have fixed the issue, which will be available in
the next version of emacs (since reftex is bundled).

In the meantime, they suggested to add the following in a configuration
file:

#+begin_src emacs-lisp
(with-eval-after-load 'reftex-parse
  (defun reftex-using-biblatex-p ()
    "Return non-nil if we are using biblatex rather than bibtex."
    (if (boundp 'TeX-active-styles)
	;; the sophisticated AUCTeX way
	(member "biblatex" TeX-active-styles)
      ;; poor-man's check...
      (save-excursion
	(re-search-forward "^[^%\n]*?\\\\usepackage.*{biblatex}" nil t)))))
#+end_src

Best,

Alan

-- 
OpenPGP Key ID : 040D0A3B4ED2E5C7

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 494 bytes --]