Re: speeding up Babel Gnuplot

[Thierry Banel <tbanelwebmin@free.fr>]

Babel Gnuplot can be further accelerated. A table is converted to a
temporary file readable by Gnuplot. I found two issues in this process:
1. the temporary file is generated twice,
2. the generation is quadratic.

I have not provided a committable patch because I am not happy with myfixes.

Of course there is no hurry in fixing that because large tables do not
happen so often.

------------------------------

1. Temporary generated twice
Because org-babel-gnuplot-process-vars is called twice.

There is no obvious fix. Here is a dirty patch. It caches the name of
the temporary file in the 'param' list.

#+BEGIN_SRC elisp
  (defun org-babel-gnuplot-process-vars (params)
    "Extract variables from PARAMS and process the variables.
  Dumps all vectors into files and returns an association list
  of variable names and the related value to be used in the gnuplot
  code."
    (let ((*org-babel-gnuplot-missing* (cdr (assq :missing params))))
      (mapcar
       (lambda (pair)
     (cons
      (car pair) ;; variable name
      (let ((val (cdr pair))) ;; variable value
        (if (not (listp val))
        val
          (let ((temp-file (org-babel-temp-file "gnuplot-"))
            (first  (car val)))
        (setcdr pair temp-file)  ;; <------ caching here
        (org-babel-gnuplot-table-to-data
         (if (or (listp first) (symbolp first))
             val
           (mapcar 'list val))
         temp-file params))))))
       (org-babel--get-vars params))))
#+END_SRC

------------------------------

2. Quadratic behavior
The spot is at ox.el::5119(the lambda in org-export-table-row-number).

This lambda is called a number of times equal to the square of thesize
of the table being plotted. For a 2000 rows table, this is
2000x2000 = four millions times. The cache a few lines before does
nothelp because each row is visited only once.

I don't know how to fix that. Hereafter is a quick-and-dirty patch to
turn the behavior into a linear one (lambda called 2000 timesinstead).
The cache is pre-filled upon creation with a single call to
org-element-map. So instead of calling org-element-map for each row
(forcing it to go through all previous rows over and over), it iscalled
only once.

#+BEGIN_SRC elisp
(defun prefill-cache (table cache info)
  (let ((number 0))
    (org-element-map   ;; <--- called once here
          table
      'table-row
      (lambda (row)
        (if (eq (org-element-property :type row) 'standard)
           (progn
          (puthash row number cache)
          (cl-incf number))))
      info )))

(defun org-export-table-row-number (table-row info)
  "Return TABLE-ROW number.
INFO is a plist used as a communication channel.  Return value is
zero-based and ignores separators.  The function returns nil for
special columns and separators."
  (let* ((cache (or (plist-get info :table-row-number-cache)
                    (let ((table (make-hash-table :test #'eq)))
                      (plist-put info :table-row-number-cache table)
                      (prefill-cache
                        (org-export-get-parent-table table-row)
                        table
                        info)
                     table)))
         (cached (gethash table-row cache 'no-cache)))
    (if (not (eq cached 'no-cache)) cached
      (puthash table-row
       (and (eq (org-element-property :type table-row) 'standard)
            (not (org-export-table-row-is-special-p table-row info))
            (let ((number 0))
              (org-element-map   ;; <--- called many times here
                (org-export-get-parent-table table-row)
                'table-row
                (lambda (row)
                  (cond ((eq row table-row) number)
                    ((eq (org-element-property :type row) 'standard)
                     (cl-incf number) nil)))
                info 'first-match)))
           cache))))
#+END_SRC

------------------------------

3. Test case

First generate a 2000 rows table:

#+BEGIN_SRC elisp :results none
  (goto-char (point-max))
  (insert "#+name: data\n")
  (let ((i 2000))
    (while (> i 0)
      (goto-char (point-max))
      (insert (format "| %4s |\n" i))
      (setq i (1- i))))
#+END_SRC

Then run Babel Gnuplot:

#+BEGIN_SRC gnuplot :var data=data :file x.svg :session none :term svg
plot data
#+END_SRC