Re: [ANN] faster org-table-to-lisp

[tbanelwebmin <tbanelwebmin@free.fr>]

Le 30/04/2020 à 10:09, Nicolas Goaziou a écrit :

> Hello,
>
> tbanelwebmin <tbanelwebmin@free.fr> writes:
>
>> Here is an alternative, faster version of org-table-to-lisp. It can be
>> more than 100 times faster.
> Great! Thank you!
>
>> #+BEGIN_SRC elisp
>> (defun org-table-to-lisp-faster (&optional org-table-at-p-done)
>>   "Convert the table at point to a Lisp structure.
>> The structure will be a list.  Each item is either the symbol `hline'
>> for a horizontal separator line, or a list of field values as strings.
>> The table is taken from the buffer at point.
>> When the optional ORG-TABLE-AT-P-DONE parameter is not nil, it is
>> assumed that (org-at-table-p) was already called."
> Since you're changing the signature, I suggest to provide the table
> element instead of ORG-AT-TABLE-P. AFAICT, `org-babel-read-element',
> through `org-babel-read-table', would greatly benefit from this.
>
> Or, to be backward compatible, I suggest
>
>   &optional TEXT TABLE
>
>>   (or org-table-at-p-done (org-at-table-p) (user-error "No table at point"))
>>   (save-excursion
>>     (goto-char (org-table-begin))
>>     (let ((end (org-table-end))
>>           (row)
>>           (table))
> Nitpick:
>
>     (row nil)
>     (table nil)
>
>>       (while (< (point) end)
>>         (setq row nil)
>>         (search-forward "|" end)
>>         (if (looking-at "-")
>>             (progn
>>               (search-forward "\n" end)
> (forward-line)
>
>>               (push 'hline table))
>>           (while (not (search-forward-regexp "\\=\n" end t))
> (unless (eolp)
>   ...)
>
>>             (unless (search-forward-regexp "\\=\\s-*\\([^|]*\\)" end t)
>>               (user-error "Malformed table at char %s" (point)))
> A row may not be properly ended. It doesn't warrant an error. Could you
> make it more tolerant?
>
> Also `search-forward-regexp' -> `re-search-forward', i.e., use the
> original.
>
>>             (let ((b (match-beginning 1))
>>           (e (match-end       1)))
> Nitpick: spurious spaces.
>
>>               (and (search-backward-regexp "[^ \t]" b t)
>>                (forward-char 1))
>   (skip-chars-backward " \t")
>
>> It is faster because it operates directly on the buffer with
>> (search-forward-regexp). Whereas the standard function splits a string
>> extracted from the buffer.
> You are right. I guess the initial implementation didn't have these
> monster tables in mind.
>
>> This function is a drop-in replacement for the standard one. It can
>> benefit to Babel and Gnuplot.
>>
>> Would it make sense to upgrade Org Mode code base?
> Certainly. Could you add an entry in ORG-NEWS, in "Miscellaneous"?
>
> Regards,
>
Thanks Nicolas for your nice suggestions. I've taken them into
account. Particularly, the use of (skip-chars-backward " \t") gave a
small additional speedup, and simplified the code.

I found a way to ensure full backward compatibility. I keep the same
signature. When a table is given as a string parameter, it is inserted
into a temporary buffer, which is then parsed. Overall, the resulting
speed is quite satisfactory.

I also made the function more tolerant to ill-formed tables: missing
"|" or excess of spaces at the end of a row are now gracefully
accepted.

Regards
Thierry

#+BEGIN_SRC elisp
(defun org-table-to-lisp (&optional txt)
  "Convert the table at point to a Lisp structure.
The structure will be a list.  Each item is either the symbol `hline'
for a horizontal separator line, or a list of field values as strings.
The table is taken from the parameter TXT, or from the buffer at point."
  (if txt
      (with-temp-buffer
        (insert txt)
        (goto-char (point-min))
        (org-table-to-lisp))
    (unless (org-at-table-p) (user-error "No table at point"))
    (save-excursion
      (goto-char (org-table-begin))
      (let ((end (org-table-end))
            (row nil)
            (table nil))
        (while (< (point) end)
          (setq row nil)
          (search-forward "|" end)
          (if (looking-at "-")
              (progn
                (forward-line)
                (push 'hline table))
            (while (not (re-search-forward "\\=\\s-*\n" end t))
              (unless (re-search-forward "\\=\\s-*\\([^|\n]*\\)\\(|?\\)" end t)
                (user-error "Malformed table at char %s" (point)))
              (goto-char (match-end 1))
              (skip-chars-backward " \t" (match-beginning 1))
              (push
               (buffer-substring-no-properties (match-beginning 1) (point))
               row)
              (goto-char (match-end 2)))
            (push (nreverse row) table)))
        (nreverse table)))))
#+END_SRC


* Version 9.4 (not yet released)
** Miscellaneous
*** Faster org-table-to-lisp

The new implementation can be more than 100 times faster. This enhances
responsiveness of Babel or Gnuplot blocks handling thousands long tables.