From mboxrd@z Thu Jan 1 00:00:00 1970 From: Thierry Banel Subject: Re: speeding up Babel Gnuplot Date: Sun, 01 Jan 2017 21:17:15 +0100 Message-ID: <586963CB.1000006@free.fr> References: <5864217C.7060001@free.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:57372) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cNmZI-0005Yu-8J for emacs-orgmode@gnu.org; Sun, 01 Jan 2017 15:17:25 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cNmZE-0001f4-8O for emacs-orgmode@gnu.org; Sun, 01 Jan 2017 15:17:24 -0500 Received: from smtp2-g21.free.fr ([212.27.42.2]:14945) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cNmZD-0001eo-Q6 for emacs-orgmode@gnu.org; Sun, 01 Jan 2017 15:17:20 -0500 Received: from [IPv6:2a01:e35:2e21:def0:3446:5ae8:8c86:63ac] (unknown [IPv6:2a01:e35:2e21:def0:3446:5ae8:8c86:63ac]) by smtp2-g21.free.fr (Postfix) with ESMTP id 86DB92003D9 for ; Sun, 1 Jan 2017 21:17:15 +0100 (CET) In-Reply-To: <5864217C.7060001@free.fr> List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: "Emacs-orgmode" To: emacs-orgmode@gnu.org Babel Gnuplot can be further accelerated. A table is converted to a temporary file readable by Gnuplot. I found two issues in this process: 1. the temporary file is generated twice, 2. the generation is quadratic. I have not provided a committable patch because I am not happy with myfixes. Of course there is no hurry in fixing that because large tables do not happen so often. ------------------------------ 1. Temporary generated twice Because org-babel-gnuplot-process-vars is called twice. There is no obvious fix. Here is a dirty patch. It caches the name of the temporary file in the 'param' list. #+BEGIN_SRC elisp (defun org-babel-gnuplot-process-vars (params) "Extract variables from PARAMS and process the variables. Dumps all vectors into files and returns an association list of variable names and the related value to be used in the gnuplot code." (let ((*org-babel-gnuplot-missing* (cdr (assq :missing params)))) (mapcar (lambda (pair) (cons (car pair) ;; variable name (let ((val (cdr pair))) ;; variable value (if (not (listp val)) val (let ((temp-file (org-babel-temp-file "gnuplot-")) (first (car val))) (setcdr pair temp-file) ;; <------ caching here (org-babel-gnuplot-table-to-data (if (or (listp first) (symbolp first)) val (mapcar 'list val)) temp-file params)))))) (org-babel--get-vars params)))) #+END_SRC ------------------------------ 2. Quadratic behavior The spot is at ox.el::5119(the lambda in org-export-table-row-number). This lambda is called a number of times equal to the square of thesize of the table being plotted. For a 2000 rows table, this is 2000x2000 = four millions times. The cache a few lines before does nothelp because each row is visited only once. I don't know how to fix that. Hereafter is a quick-and-dirty patch to turn the behavior into a linear one (lambda called 2000 timesinstead). The cache is pre-filled upon creation with a single call to org-element-map. So instead of calling org-element-map for each row (forcing it to go through all previous rows over and over), it iscalled only once. #+BEGIN_SRC elisp (defun prefill-cache (table cache info) (let ((number 0)) (org-element-map ;; <--- called once here table 'table-row (lambda (row) (if (eq (org-element-property :type row) 'standard) (progn (puthash row number cache) (cl-incf number)))) info ))) (defun org-export-table-row-number (table-row info) "Return TABLE-ROW number. INFO is a plist used as a communication channel. Return value is zero-based and ignores separators. The function returns nil for special columns and separators." (let* ((cache (or (plist-get info :table-row-number-cache) (let ((table (make-hash-table :test #'eq))) (plist-put info :table-row-number-cache table) (prefill-cache (org-export-get-parent-table table-row) table info) table))) (cached (gethash table-row cache 'no-cache))) (if (not (eq cached 'no-cache)) cached (puthash table-row (and (eq (org-element-property :type table-row) 'standard) (not (org-export-table-row-is-special-p table-row info)) (let ((number 0)) (org-element-map ;; <--- called many times here (org-export-get-parent-table table-row) 'table-row (lambda (row) (cond ((eq row table-row) number) ((eq (org-element-property :type row) 'standard) (cl-incf number) nil))) info 'first-match))) cache)))) #+END_SRC ------------------------------ 3. Test case First generate a 2000 rows table: #+BEGIN_SRC elisp :results none (goto-char (point-max)) (insert "#+name: data\n") (let ((i 2000)) (while (> i 0) (goto-char (point-max)) (insert (format "| %4s |\n" i)) (setq i (1- i)))) #+END_SRC Then run Babel Gnuplot: #+BEGIN_SRC gnuplot :var data=data :file x.svg :session none :term svg plot data #+END_SRC