From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id +BV9C+pwlF+tZAAA0tVLHw (envelope-from ) for ; Sat, 24 Oct 2020 18:22:34 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id qNhUB+pwlF+xAQAAB5/wlQ (envelope-from ) for ; Sat, 24 Oct 2020 18:22:34 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 1D1F49402A2 for ; Sat, 24 Oct 2020 18:22:32 +0000 (UTC) Received: from localhost ([::1]:53834 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWOBS-0004Dx-GU for larch@yhetil.org; Sat, 24 Oct 2020 14:22:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45492) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWOAp-0004Dj-7w for emacs-orgmode@gnu.org; Sat, 24 Oct 2020 14:21:51 -0400 Received: from mail-pf1-x441.google.com ([2607:f8b0:4864:20::441]:40301) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kWOAk-0005vl-4c; Sat, 24 Oct 2020 14:21:50 -0400 Received: by mail-pf1-x441.google.com with SMTP id w21so3719863pfc.7; Sat, 24 Oct 2020 11:21:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=references:user-agent:from:to:cc:subject:date:in-reply-to :message-id:mime-version; bh=B2eL8fmAeXlxyr5jUVaPx+Sa2YUzywO8fbgMAxa6gg0=; b=QpqwheEQVhLJUnAVtX79ojGRqfQza5rFSGpsrBF03hiNJX389Dn8kJxPjw4q7BRq6k yY61fqSP+Fvs0E+umSrLZGpyXf9c3Dz1UXjLAc+lub/y9e6fPKRZtUcF1lA+pBevtEJC nwxJAmKWZV8O5HtI3+ImXQ4lwhEJmS4QC3Hg2flCeln+xctxjs9zyxxjqmYU4cwzVQxw xBa1Wig0LL65HHQbniVf6yPR0+zcH5ARNlJj9c/gArQi21n3/4GF0wh1tmDsyDZQMfwj mqmzKyDQPYwRkzlV45HEIYRLals5ZDV6CCYZJuVWSuImscn0PElhNcgKU+WkzHnnRK3N bcxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:references:user-agent:from:to:cc:subject:date :in-reply-to:message-id:mime-version; bh=B2eL8fmAeXlxyr5jUVaPx+Sa2YUzywO8fbgMAxa6gg0=; b=phtybCTnQmrU0MVIZ7qHGzTUifD4nT837DkrHoL3n3Ap1//MGqKAVHAN/6ZeDZdF4R iSA09BC1ogCMPHwqlmRzjcdV3NzMbyywg4v60fAYQIhwA/589gExKOKbClcsowtvJ0vY d/9DQNnWFOUVsybwRVJP3MtOTHe4vF9gxIpeDrCHfB1EgykJR2oSKVus+kjJsWflDuTI lyX1DQBh3UnehVhwWZJvOpQiJwlZyPNAD8VSOa/3SqQZswx60SxQDt8naQChxuo9UZYG /xTxpsRRj7nTGTk5bYMVfhoHxerNQAN0e36ea/XbhtDmBNnZcuohFcW7z8bHbxOOHbcC lOuQ== X-Gm-Message-State: AOAM531+Gl+isuaS7GP0KNkAPc4I0vsctEHpfor+s5DuPmPDx1in+tfX HYHtnPBnBnlNb0a1O/SbSVoJoum8adQ= X-Google-Smtp-Source: ABdhPJzdPaNoBTrefc6Sc4l9DbBUbN5J+YaEc9yrlNnhJZQRqyQsvvM9b4Pj3r+4SvwwZZ8id72OdQ== X-Received: by 2002:a65:5c43:: with SMTP id v3mr8374982pgr.271.1603563704098; Sat, 24 Oct 2020 11:21:44 -0700 (PDT) Received: from localhost (180-150-91-8.b4965b.per.nbn.aussiebb.net. [180.150.91.8]) by smtp.gmail.com with ESMTPSA id e2sm6072409pgd.27.2020.10.24.11.21.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 24 Oct 2020 11:21:42 -0700 (PDT) References: <87imcrfntf.fsf@gmail.com> <87blijmnv9.fsf@gnu.org> <87lfhbhfhe.fsf@gmail.com> <87r1qp3fu1.fsf@gmail.com> <871rhxd2ib.fsf@gmail.com> <87r1pnoo5y.fsf@bzg.fr> User-agent: mu4e 1.4.13; emacs 27.1 From: TEC To: Bastien Subject: Re: [PATCH] org-plot abstractions and extension Date: Sun, 25 Oct 2020 02:16:08 +0800 In-reply-to: <87r1pnoo5y.fsf@bzg.fr> Message-ID: <87lffv8oy5.fsf@gmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=2607:f8b0:4864:20::441; envelope-from=tecosaur@gmail.com; helo=mail-pf1-x441.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: org-mode-email Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=QpqwheEQ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Spam-Score: -1.71 X-TUID: ihCjqp4fz3/8 --=-=-= Content-Type: text/plain; format=flowed Bastien writes: > I'm not an org-plot.el user so I cannot test, but by reading the > patches, they look okay. > > Let's go in "optimistic merging" mode and commit your patches? Sounds good then. I don't expect the changes to compromise any existing functionality. > Is https://orgmode.org/list/87lfhbhfhe.fsf@gmail.com/ the latest > version I should use? I've smoothed a rough edge or two, and added a documentation entry. I'll attach all the patches to this email, so there's no ambiguity. (crosses fingers for attachments working as expected) -- Timothy --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-org-plot.el-make-indentation-method-consistent.patch >From 3743e507775b446f5f8188958c20f65861fac3fb Mon Sep 17 00:00:00 2001 From: TEC Date: Wed, 8 Jul 2020 18:34:46 +0800 Subject: [PATCH 01/15] org-plot.el: make indentation method consistent * lisp/org-plot.el (org-plot/gnuplot): Make indentation consistent, by replacing a few spaces with tabs. Only 6 of 347 lines used spaces instead of tabs. --- lisp/org-plot.el | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/lisp/org-plot.el b/lisp/org-plot.el index 0ff96af67..c08bc144e 100644 --- a/lisp/org-plot.el +++ b/lisp/org-plot.el @@ -325,12 +325,12 @@ line directly before or after the table." (with-temp-buffer (if (plist-get params :script) ; user script (progn (insert - (org-plot/gnuplot-script data-file num-cols params t)) - (insert "\n") - (insert-file-contents (plist-get params :script)) - (goto-char (point-min)) - (while (re-search-forward "\\$datafile" nil t) - (replace-match data-file nil nil))) + (org-plot/gnuplot-script data-file num-cols params t)) + (insert "\n") + (insert-file-contents (plist-get params :script)) + (goto-char (point-min)) + (while (re-search-forward "\\$datafile" nil t) + (replace-match data-file nil nil))) (insert (org-plot/gnuplot-script data-file num-cols params))) ;; Graph table. (gnuplot-mode) -- 2.28.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0002-org-plot.el-add-new-option-transpose.patch >From c62e817b04dfbe624ee8b2090ebcde257bbd3f23 Mon Sep 17 00:00:00 2001 From: TEC Date: Wed, 8 Jul 2020 19:26:07 +0800 Subject: [PATCH 02/15] org-plot.el: add new option :transpose * lisp/org-plot.el (org-plot/add-options-to-plist, org-plot/add-options-to-plist): Add a new option :transpose, and a shorter alias :trans. Transposition is performed if the argument is yes, y, or t. This treats the table as a matrix and performs matrix transposition on it. If an hline is present, it is assumed that it is a marks a separation from a first header row. The first row is then treated as the new header by inserting a hline in the transposed data. This is quite useful for some plots, where across multiple categories, there are a large number of data points. Without this, the data points would be columns and the table can spread irritatingly wide. --- lisp/org-plot.el | 44 +++++++++++++++++++++++++++++--------------- 1 file changed, 29 insertions(+), 15 deletions(-) diff --git a/lisp/org-plot.el b/lisp/org-plot.el index c08bc144e..6ff633130 100644 --- a/lisp/org-plot.el +++ b/lisp/org-plot.el @@ -50,19 +50,21 @@ "Parse an OPTIONS line and set values in the property list P. Returns the resulting property list." (when options - (let ((op '(("type" . :plot-type) - ("script" . :script) - ("line" . :line) - ("set" . :set) - ("title" . :title) - ("ind" . :ind) - ("deps" . :deps) - ("with" . :with) - ("file" . :file) - ("labels" . :labels) - ("map" . :map) - ("timeind" . :timeind) - ("timefmt" . :timefmt))) + (let ((op '(("type" . :plot-type) + ("script" . :script) + ("line" . :line) + ("set" . :set) + ("title" . :title) + ("ind" . :ind) + ("deps" . :deps) + ("with" . :with) + ("file" . :file) + ("labels" . :labels) + ("map" . :map) + ("timeind" . :timeind) + ("timefmt" . :timefmt) + ("trans" . :transpose) + ("transpose" . :transpose))) (multiples '("set" "line")) (regexp ":\\([\"][^\"]+?[\"]\\|[(][^)]+?[)]\\|[^ \t\n\r;,.]*\\)") (start 0)) @@ -289,8 +291,20 @@ line directly before or after the table." (setf params (plist-put params (car pair) (cdr pair))))) ;; collect table and table information (let* ((data-file (make-temp-file "org-plot")) - (table (org-table-collapse-header (org-table-to-lisp))) - (num-cols (length (car table)))) + (table (let ((tbl (org-table-to-lisp))) + (when (pcase (plist-get params :transpose) + ('y t) + ('yes t) + ('t t)) + (if (memq 'hline tbl) + (setq tbl (apply #'cl-mapcar #'list tbl)) + ;; When present, remove hlines as they can't (currentily) be easily transposed. + (setq tbl (apply #'cl-mapcar #'list + (remove 'hline tbl))) + (push 'hline (cdr tbl)))) + tbl)) + (num-cols (length (if (eq (nth 0 table) 'hline) (nth 1 table) + (nth 0 table))))) (run-with-idle-timer 0.1 nil #'delete-file data-file) (when (eq (cadr table) 'hline) (setf params -- 2.28.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0003-org-plot.el-add-new-custom-gnuplot-preamble.patch >From fc7f4015c726e4a685002e8d69fad1eb1d605790 Mon Sep 17 00:00:00 2001 From: TEC Date: Wed, 8 Jul 2020 22:26:21 +0800 Subject: [PATCH 03/15] org-plot.el: add new custom gnuplot preamble * lisp/org-plot.el: Define new custom variable `org-plot/gnuplot-script-preamble' which can be either a string or a function. The value of this (when executed, in the case of the function) is inserted near the top of the generated gnuplot script. (org-plot/gnuplot-script): Use the new variable `org-plot/gnuplot-script-preamble' in the manner described. This allows for the user to set the font/colour-scheme, default precision, and much more. --- lisp/org-plot.el | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/lisp/org-plot.el b/lisp/org-plot.el index 6ff633130..f8db45273 100644 --- a/lisp/org-plot.el +++ b/lisp/org-plot.el @@ -181,6 +181,13 @@ and dependent variables." (setf back-edge "") (setf front-edge "")))) row-vals)) +(defcustom org-plot/gnuplot-script-preamble "" + "String or function which provides content to be inserted into the GNUPlot +script before the plot command. Not that this is in addition to, not instead of +other content generated in `org-plot/gnuplot-script'." + :group 'org-plot + :type '(choice string function)) + (defun org-plot/gnuplot-script (data-file num-cols params &optional preface) "Write a gnuplot script to DATA-FILE respecting the options set in PARAMS. NUM-COLS controls the number of columns plotted in a 2-d plot. @@ -213,6 +220,12 @@ manner suitable for prepending to a user-specified script." (when file ; output file (funcall ats (format "set term %s" (file-name-extension file))) (funcall ats (format "set output '%s'" file))) + + (funcall ats + (if (stringp org-plot/gnuplot-script-preamble) + org-plot/gnuplot-script-preamble + (org-plot/gnuplot-script-preamble))) + (pcase type ; type (`2d ()) (`3d (when map (funcall ats "set map"))) -- 2.28.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0004-org-plot.el-abstract-plot-types-into-custom-var.patch >From fc1ecf42cd8d0d27cda98ced2c2be365ad305df7 Mon Sep 17 00:00:00 2001 From: TEC Date: Thu, 9 Jul 2020 04:27:18 +0800 Subject: [PATCH 04/15] org-plot.el: abstract plot types into custom var * lisp/org-plot.el (org-plot/gnuplot-script): Abstract the generation of gnuplot commands from the three hardcoded types: 2d, 3d, and grid. A new custom variable `org-plot/preset-plot-types' is defined to declare plot types and provide a lambda which is called with a fixed signature to generate associated gnuplot code. The previously hardcoded types are implemented as the default value. By extracting these types to a custom variable, users are able to create their own presets for frequently used setups. Note that while this moves the most significant hardcoding of the 2d, 3d, and grid types in `org-plot/gnuplot-script', there are still a few minor fragments that I am not sure how to best address --- yet. --- lisp/org-plot.el | 71 ++++++++++++++++++++++++++++++++---------------- 1 file changed, 48 insertions(+), 23 deletions(-) diff --git a/lisp/org-plot.el b/lisp/org-plot.el index f8db45273..207f5d4af 100644 --- a/lisp/org-plot.el +++ b/lisp/org-plot.el @@ -188,6 +188,49 @@ other content generated in `org-plot/gnuplot-script'." :group 'org-plot :type '(choice string function)) +(defcustom org-plot/preset-plot-types + '((2d (lambda (data-file num-cols params plot-str) + (let* ((type (plist-get params :plot-type)) + (with (if (eq type 'grid) 'pm3d (plist-get params :with))) + (ind (plist-get params :ind)) + (deps (if (plist-member params :deps) (plist-get params :deps))) + (text-ind (plist-get params :textind)) + (col-labels (plist-get params :labels)) + res) + (dotimes (col num-cols res) + (unless (and (eq type '2d) + (or (and ind (equal (1+ col) ind)) + (and deps (not (member (1+ col) deps))))) + (setf res + (cons + (format plot-str data-file + (or (and ind (> ind 0) + (not text-ind) + (format "%d:" ind)) "") + (1+ col) + (if text-ind (format ":xticlabel(%d)" ind) "") + with + (or (nth col col-labels) + (format "%d" (1+ col)))) + res))))))) + (3d (lambda (data-file num-cols params plot-str) + (let* ((type (plist-get params :plot-type)) + (with (if (eq type 'grid) 'pm3d (plist-get params :with)))) + (list (format "'%s' matrix with %s title ''" + data-file with))))) + (grid (lambda (data-file num-cols params plot-str) + (let* ((type (plist-get params :plot-type)) + (with (if (eq type 'grid) 'pm3d (plist-get params :with)))) + (list (format "'%s' with %s title ''" + data-file with)))))) + "List of plot presets with the type name as the car, and a function +which yeilds plot-lines (a list of strings) as the cdr. +The parameters of `org-plot/gnuplot-script' and PLOT-STR are passed to +that function. i.e. it is called with the following arguments: + DATA-FILE NUM-COLS PARAMS PLOT-STR" + :group 'org-plot + :type '(alist :value-type (symbol group))) + (defun org-plot/gnuplot-script (data-file num-cols params &optional preface) "Write a gnuplot script to DATA-FILE respecting the options set in PARAMS. NUM-COLS controls the number of columns plotted in a 2-d plot. @@ -254,29 +297,11 @@ manner suitable for prepending to a user-specified script." (or timefmt ; timefmt passed to gnuplot "%Y-%m-%d-%H:%M:%S") "\""))) (unless preface - (pcase type ; plot command - (`2d (dotimes (col num-cols) - (unless (and (eq type '2d) - (or (and ind (equal (1+ col) ind)) - (and deps (not (member (1+ col) deps))))) - (setf plot-lines - (cons - (format plot-str data-file - (or (and ind (> ind 0) - (not text-ind) - (format "%d:" ind)) "") - (1+ col) - (if text-ind (format ":xticlabel(%d)" ind) "") - with - (or (nth col col-labels) - (format "%d" (1+ col)))) - plot-lines))))) - (`3d - (setq plot-lines (list (format "'%s' matrix with %s title ''" - data-file with)))) - (`grid - (setq plot-lines (list (format "'%s' with %s title ''" - data-file with))))) + (let ((type-func (cadr (assoc type org-plot/preset-plot-types)))) + (when type-func + (setq plot-lines + (funcall type-func data-file num-cols params plot-str)))) + (funcall ats (concat plot-cmd " " (mapconcat #'identity (reverse plot-lines) -- 2.28.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0005-org-plot.el-add-utility-functions-for-range-ticks.patch >From d99a61170bb0ff10b9fc7b99cdc957ec574c1e51 Mon Sep 17 00:00:00 2001 From: TEC Date: Thu, 9 Jul 2020 04:47:40 +0800 Subject: [PATCH 05/15] org-plot.el: add utility functions for range,ticks * lisp/org-plot.el (org-plot/add-options-to-plist): Add the options :ymin :ymax :xmin :xmax, as well as :min and :max as aliases to the y{min,max} options. The :ticks option is also added, for specifying how many ticks should be used. (org--plot/values-stats, org--plot/sensible-tick-num, org--plot/nice-frequency-pick, org--plot/merge-alists, org--plot/item-frequencies, org--plot/prime-factors): New utility functions added to allow for somewhat sensible determination of a :ticks value when none is provided. This turns out to be harder than expected, and so a number of functions are used to attempt to do so. The essence of the method used, is to round values and find their prime decompositions. From this we try to select the most common components to give a reasonable step size. We also add a 'ticks' parameter for manually setting the number of ticks, and (y)min/max parameters similarly. --- lisp/org-plot.el | 100 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 100 insertions(+) diff --git a/lisp/org-plot.el b/lisp/org-plot.el index 207f5d4af..2a9c0f5bd 100644 --- a/lisp/org-plot.el +++ b/lisp/org-plot.el @@ -63,6 +63,11 @@ Returns the resulting property list." ("map" . :map) ("timeind" . :timeind) ("timefmt" . :timefmt) + ("min" . :ymin) + ("max" . :ymax) + ("ymin" . :ymin) + ("xmax" . :xmax) + ("ticks" . :ticks) ("trans" . :transpose) ("transpose" . :transpose))) (multiples '("set" "line")) @@ -181,6 +186,101 @@ and dependent variables." (setf back-edge "") (setf front-edge "")))) row-vals)) +(defun org--plot/values-stats (nums &optional hard-min hard-max) + "From a list of NUMS return a plist containing some rudamentry statistics on the +values, namely regarding the range." + (let* ((minimum (or hard-min (apply #'min nums))) + (maximum (or hard-max (apply #'max nums))) + (range (- maximum minimum)) + (rangeOrder (ceiling (- 1 (log10 range)))) + (range-factor (expt 10 rangeOrder)) + (nice-min (/ (float (floor (* minimum range-factor))) range-factor)) + (nice-max (/ (float (ceiling (* maximum range-factor))) range-factor))) + `(:min ,minimum :max ,maximum :range ,range + :range-factor ,range-factor + :nice-min ,nice-min :nice-max ,nice-max :nice-range ,(- nice-max nice-min)))) + +(defun org--plot/sensible-tick-num (table &optional hard-min hard-max) + "From a the values in a TABLE of data, attempt to guess an appropriate number of ticks." + (let* ((row-data + (mapcar (lambda (row) (org--plot/values-stats + (mapcar #'string-to-number (cdr row)) + hard-min + hard-max)) table)) + (row-normalised-ranges (mapcar (lambda (r-data) + (let ((val (round (* + (plist-get r-data :range-factor) + (plist-get r-data :nice-range))))) + (if (= (% val 10) 0) (/ val 10) val))) + row-data)) + (range-prime-decomposition (mapcar #'org--plot/prime-factors row-normalised-ranges)) + (weighted-factors (sort (apply #'org--plot/merge-alists #'+ 0 + (mapcar (lambda (factors) (org--plot/item-frequencies factors t)) + range-prime-decomposition)) + (lambda (a b) (> (cdr a) (cdr b)))))) + (apply #'* (org--plot/nice-frequency-pick weighted-factors)))) + +(defun org--plot/nice-frequency-pick (frequencies) + "From a list of frequences, try to sensibly pick a sample of the most frequent." + ;; TODO this mosly works decently, but counld do with some tweaking to work more consistently. + (case (length frequencies) + (1 (list (car (nth 0 frequencies)))) + (2 (if (<= 3 (/ (cdr (nth 0 frequencies)) + (cdr (nth 1 frequencies)))) + (make-list 2 + (car (nth 0 frequencies))) + (list (car (nth 0 frequencies)) + (car (nth 1 frequencies))))) + (t + (let* ((total-count (apply #'+ (mapcar #'cdr frequencies))) + (n-freq (mapcar (lambda (freq) `(,(car freq) . ,(/ (float (cdr freq)) total-count))) frequencies)) + (f-pick (list (car (car n-freq)))) + (1-2-ratio (/ (cdr (nth 0 n-freq)) + (cdr (nth 1 n-freq)))) + (2-3-ratio (/ (cdr (nth 1 n-freq)) + (cdr (nth 2 n-freq)))) + (1-3-ratio (* 1-2-ratio 2-3-ratio)) + (1-val (car (nth 0 n-freq))) + (2-val (car (nth 1 n-freq))) + (3-val (car (nth 2 n-freq)))) + (when (> 1-2-ratio 4) (push 1-val f-pick)) + (when (and (< 1-2-ratio 2-val) + (< (* (apply #'* f-pick) 2-val) 30)) + (push 2-val f-pick)) + (when (and (< 1-3-ratio 3-val) + (< (* (apply #'* f-pick) 3-val) 30)) + (push 3-val f-pick)) + f-pick)))) + +(defun org--plot/merge-alists (function default alist1 alist2 &rest alists) + "Using FUNCTION, combine the elements of all given ALISTS. When an element is +only present in one alist, DEFAULT is used as the second argument for the FUNCTION." + (when (> (length alists) 0) + (setq alist2 (apply #'org--plot/merge-alists function default alist2 alists))) + (flet ((keys (alist) (mapcar #'car alist)) + (lookup (key alist) (or (cdr (assoc key alist)) default))) + (loop with keys = (union (keys alist1) (keys alist2) :test 'equal) + for k in keys collect + (cons k (funcall function (lookup k alist1) (lookup k alist2)))))) + +(defun org--plot/item-frequencies (values &optional normalise) + "Return an alist indicating the frequency of values in VALUES list." + (let ((normaliser (if normalise (float (length values)) 1))) + (cl-loop for (n . m) in (seq-group-by #'identity values) + collect (cons n (/ (length m) normaliser))))) + +(defun org--plot/prime-factors (value) + "Return the prime decomposition of VALUE, e.g. for 12, '(3 2 2)" + (let ((factors '(1)) (i 1)) + (while (/= 1 value) + (setq i (1+ i)) + (when (eq 0 (% value i)) + (push i factors) + (setq value (/ value i)) + (setq i (1- i)) + )) + (subseq factors 0 -1))) + (defcustom org-plot/gnuplot-script-preamble "" "String or function which provides content to be inserted into the GNUPlot script before the plot command. Not that this is in addition to, not instead of -- 2.28.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0006-org-plot.el-add-custom-var-for-affecting-the-term.patch >From 26c09d431030bacfc5a4ce84103b2eca186b2229 Mon Sep 17 00:00:00 2001 From: TEC Date: Thu, 9 Jul 2020 05:00:03 +0800 Subject: [PATCH 06/15] org-plot.el: add custom var for affecting the term * lisp/org-plot.el (org-plot/gnuplot-script): Allow for customisation of org-plot's term by adding a custom variable `org-plot/gnuplot-term-extra' which allows the user to tweak the gnuplot term settings. This allows for setting characteristics such as default size, or background colour. --- lisp/org-plot.el | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/lisp/org-plot.el b/lisp/org-plot.el index 2a9c0f5bd..ed4cea195 100644 --- a/lisp/org-plot.el +++ b/lisp/org-plot.el @@ -331,6 +331,13 @@ that function. i.e. it is called with the following arguments: :group 'org-plot :type '(alist :value-type (symbol group))) +(defcustom org-plot/gnuplot-term-extra "" + "String or function which provides the extra term options. +E.g. a value of \"size 1050,650\" would cause +\"set term ... size 1050,650\" to be used." + :group 'org-plot + :type '(choice string function)) + (defun org-plot/gnuplot-script (data-file num-cols params &optional preface) "Write a gnuplot script to DATA-FILE respecting the options set in PARAMS. NUM-COLS controls the number of columns plotted in a 2-d plot. @@ -360,8 +367,15 @@ manner suitable for prepending to a user-specified script." ;; ats = add-to-script (ats (lambda (line) (setf script (concat script "\n" line)))) plot-lines) - (when file ; output file - (funcall ats (format "set term %s" (file-name-extension file))) + + + ;; handle output file, background, and size + (funcall ats (format "set term %s %s" + (if file (file-name-extension file) "GNUTERM") + (if (stringp org-plot/gnuplot-term-extra) + org-plot/gnuplot-term-extra + (org-plot/gnuplot-term-extra)))) + (when file ; output file (funcall ats (format "set output '%s'" file))) (funcall ats -- 2.28.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0007-org-plot.el-tweak-term-preamble-custom-vars.patch >From 5a1b9ff8f3ba5be565828137460023cd39194b6c Mon Sep 17 00:00:00 2001 From: TEC Date: Thu, 9 Jul 2020 05:05:20 +0800 Subject: [PATCH 07/15] org-plot.el: tweak term, preamble custom vars * lisp/org-plot.el (org-plot/gnuplot-script): Call the term and preamble functions (mentioned below) with the plot type as the argument. (org-plot/gnuplot-script-preamble, org-plot/gnuplot-term-extra): update docstring. --- lisp/org-plot.el | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/lisp/org-plot.el b/lisp/org-plot.el index ed4cea195..52422ea2f 100644 --- a/lisp/org-plot.el +++ b/lisp/org-plot.el @@ -284,7 +284,8 @@ only present in one alist, DEFAULT is used as the second argument for the FUNCTI (defcustom org-plot/gnuplot-script-preamble "" "String or function which provides content to be inserted into the GNUPlot script before the plot command. Not that this is in addition to, not instead of -other content generated in `org-plot/gnuplot-script'." +other content generated in `org-plot/gnuplot-script'. +If a function, it is called with the plot type as the argument." :group 'org-plot :type '(choice string function)) @@ -334,7 +335,8 @@ that function. i.e. it is called with the following arguments: (defcustom org-plot/gnuplot-term-extra "" "String or function which provides the extra term options. E.g. a value of \"size 1050,650\" would cause -\"set term ... size 1050,650\" to be used." +\"set term ... size 1050,650\" to be used. +If a function, it is called with the plot type as the argument." :group 'org-plot :type '(choice string function)) @@ -374,14 +376,14 @@ manner suitable for prepending to a user-specified script." (if file (file-name-extension file) "GNUTERM") (if (stringp org-plot/gnuplot-term-extra) org-plot/gnuplot-term-extra - (org-plot/gnuplot-term-extra)))) + (org-plot/gnuplot-term-extra type)))) (when file ; output file (funcall ats (format "set output '%s'" file))) (funcall ats (if (stringp org-plot/gnuplot-script-preamble) org-plot/gnuplot-script-preamble - (org-plot/gnuplot-script-preamble))) + (org-plot/gnuplot-script-preamble type))) (pcase type ; type (`2d ()) -- 2.28.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0008-org-plot.el-add-radar-plot-type.patch >From 3e9338962a4af033bd56e8ab7a1abe5e636d71c5 Mon Sep 17 00:00:00 2001 From: TEC Date: Thu, 9 Jul 2020 05:21:44 +0800 Subject: [PATCH 08/15] org-plot.el: add radar plot type * lisp/org-plot.el (org--plot/radar): Implement a new plot type "radar". (org--plot/radar-template): A huge template sting for `org-plot/radar'. (org--plot/radar-ticks, org--plot/radar-setup-template): Smaller template strings for use in `org-plot/radar'. (org-plot/preset-plot-types): Add the new "radar" type to the list of default types. The radar type has a long and complex implementation, but that's exactly what makes it perfect for something like this. A complex plot can be produced with a simple keyword in the #+PLOT options. There are still a few kinks that would benefit from being ironed out, but the current state is fully-functional. --- lisp/org-plot.el | 138 ++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 137 insertions(+), 1 deletion(-) diff --git a/lisp/org-plot.el b/lisp/org-plot.el index 52422ea2f..fd92a12a1 100644 --- a/lisp/org-plot.el +++ b/lisp/org-plot.el @@ -323,7 +323,9 @@ If a function, it is called with the plot type as the argument." (let* ((type (plist-get params :plot-type)) (with (if (eq type 'grid) 'pm3d (plist-get params :with)))) (list (format "'%s' with %s title ''" - data-file with)))))) + data-file with))))) + (radar (lambda (data-file num-cols params plot-str) + (list (org--plot/radar table params))))) "List of plot presets with the type name as the car, and a function which yeilds plot-lines (a list of strings) as the cdr. The parameters of `org-plot/gnuplot-script' and PLOT-STR are passed to @@ -332,6 +334,140 @@ that function. i.e. it is called with the following arguments: :group 'org-plot :type '(alist :value-type (symbol group))) +(defvar org--plot/radar-template + "### spider plot/chart with gnuplot +# also known as: radar chart, web chart, star chart, cobweb chart, +# radar plot, web plot, star plot, cobweb plot, etc. ... +set datafile separator ' ' +set size square +unset tics +set angles degree +set key bmargin center horizontal +unset border + +# Load data and settup +load \"%s\" + +# General settings +DataColCount = words($Data[1])-1 +AxesCount = |$Data|-HeaderLines +AngleOffset = 90 +Max = 1 +d=0.1*Max +Direction = -1 # counterclockwise=1, clockwise = -1 + +# Tic settings +TicCount = %s +TicOffset = 0.1 +TicValue(axis,i) = real(i)*(word($Settings[axis],3)-word($Settings[axis],2)) \\ + / word($Settings[axis],4)+word($Settings[axis],2) +TicLabelPosX(axis,i) = PosX(axis,i/TicCount) + PosY(axis, TicOffset) +TicLabelPosY(axis,i) = PosY(axis,i/TicCount) - PosX(axis, TicOffset) +TicLen = 0.03 +TicdX(axis,i) = 0.5*TicLen*cos(alpha(axis)-90) +TicdY(axis,i) = 0.5*TicLen*sin(alpha(axis)-90) + +# Label +LabOffset = 0.10 +LabX(axis) = PosX(axis+1,Max+2*d) + PosY(axis, LabOffset) +LabY(axis) = PosY($0+1,Max+2*d) + +# Functions +alpha(axis) = (axis-1)*Direction*360.0/AxesCount+AngleOffset +PosX(axis,R) = R*cos(alpha(axis)) +PosY(axis,R) = R*sin(alpha(axis)) +Scale(axis,value) = real(value-word($Settings[axis],2))/(word($Settings[axis],3)-word($Settings[axis],2)) + +# Spider settings +set style arrow 1 dt 1 lw 1.0 @fgal head filled size 0.06,25 # style for axes +set style arrow 2 dt 2 lw 0.5 @fgal nohead # style for weblines +set style arrow 3 dt 1 lw 1 @fgal nohead # style for axis tics +set samples AxesCount +set isosamples TicCount +set urange[1:AxesCount] +set vrange[1:TicCount] +set style fill transparent solid 0.2 + +set xrange[-Max-4*d:Max+4*d] +set yrange[-Max-4*d:Max+4*d] +plot \\ + '+' u (0):(0):(PosX($0,Max+d)):(PosY($0,Max+d)) w vec as 1 not, \\ + $Data u (LabX($0)): \\ + (LabY($0)):1 every ::HeaderLines w labels center enhanced @fgt not, \\ + for [i=1:DataColCount] $Data u (PosX($0+1,Scale($0+1,column(i+1)))): \\ + (PosY($0+1,Scale($0+1,column(i+1)))) every ::HeaderLines w filledcurves lt i title word($Data[1],i+1), \\ +%s +# '++' u (PosX($1,$2/TicCount)-TicdX($1,$2/TicCount)): \\ +# (PosY($1,$2/TicCount)-TicdY($1,$2/TicCount)): \\ +# (2*TicdX($1,$2/TicCount)):(2*TicdY($1,$2/TicCount)) \\ +# w vec as 3 not, \\ +### end of code +") + +(defvar org--plot/radar-ticks + " '++' u (PosX($1,$2/TicCount)):(PosY($1,$2/TicCount)): \\ + (PosX($1+1,$2/TicCount)-PosX($1,$2/TicCount)): \\ + (PosY($1+1,$2/TicCount)-PosY($1,$2/TicCount)) w vec as 2 not, \\ + '++' u (TicLabelPosX(%s,$2)):(TicLabelPosY(%s,$2)): \\ + (sprintf('%%g',TicValue(%s,$2))) w labels font ',8' @fgat not") + +(defvar org--plot/radar-setup-template + "# Data +$Data <From 8b1ed7fb3cc418bb90fe48d3c4c8cb711decfded Mon Sep 17 00:00:00 2001 From: TEC Date: Thu, 30 Jul 2020 18:25:19 +0800 Subject: [PATCH 09/15] org-plot.el: fix logic error in transposition * lisp/org-plot.el (org-plot/gnuplot): If statement in transposition treated condition as its negative, to fix this the condition was inverted. It was also noticed that the code could not operate as expected as the user-supplied #+plot options were not fetched. Resolved by re-inserting relevant code from an older version of org-plot. --- lisp/org-plot.el | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/lisp/org-plot.el b/lisp/org-plot.el index fd92a12a1..1b227d698 100644 --- a/lisp/org-plot.el +++ b/lisp/org-plot.el @@ -579,6 +579,10 @@ line directly before or after the table." (dolist (pair org-plot/gnuplot-default-options) (unless (plist-member params (car pair)) (setf params (plist-put params (car pair) (cdr pair))))) + ;; Collect options. + (save-excursion (while (and (equal 0 (forward-line -1)) + (looking-at "[[:space:]]*#\\+")) + (setf params (org-plot/collect-options params)))) ;; collect table and table information (let* ((data-file (make-temp-file "org-plot")) (table (let ((tbl (org-table-to-lisp))) @@ -586,7 +590,7 @@ line directly before or after the table." ('y t) ('yes t) ('t t)) - (if (memq 'hline tbl) + (if (not (memq 'hline tbl)) (setq tbl (apply #'cl-mapcar #'list tbl)) ;; When present, remove hlines as they can't (currentily) be easily transposed. (setq tbl (apply #'cl-mapcar #'list -- 2.28.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0010-org-plot.el-complete-transition-to-softcoded-type.patch >From bcd45c562966a893449d2412b363f31368ee5983 Mon Sep 17 00:00:00 2001 From: TEC Date: Thu, 30 Jul 2020 18:36:11 +0800 Subject: [PATCH 10/15] org-plot.el: complete transition to softcoded type * lisp/org-plot.el (org-plot/preset-plot-types): Adapt structure to cover all type-specific logic within org-plot. (org-plot/gnuplot-script, org-plot/gnuplot): Replace type-specific logic with references to properties of the type from `org-plot/preset-plot-types'. --- lisp/org-plot.el | 242 ++++++++++++++++++++++++++++------------------- 1 file changed, 143 insertions(+), 99 deletions(-) diff --git a/lisp/org-plot.el b/lisp/org-plot.el index 1b227d698..53186bb75 100644 --- a/lisp/org-plot.el +++ b/lisp/org-plot.el @@ -290,7 +290,10 @@ If a function, it is called with the plot type as the argument." :type '(choice string function)) (defcustom org-plot/preset-plot-types - '((2d (lambda (data-file num-cols params plot-str) + '((2d :plot-cmd "plot" + :check-ind-type t + :plot-func + (lambda (_table data-file num-cols params plot-str) (let* ((type (plist-get params :plot-type)) (with (if (eq type 'grid) 'pm3d (plist-get params :with))) (ind (plist-get params :ind)) @@ -314,23 +317,60 @@ If a function, it is called with the plot type as the argument." (or (nth col col-labels) (format "%d" (1+ col)))) res))))))) - (3d (lambda (data-file num-cols params plot-str) + (3d :plot-cmd "splot" + :plot-pre (lambda (_table _data-file _num-cols params _plot-str) + (if (plist-get params :map) "set map")) + :plot-func + (lambda (_table data-file _num-cols params _plot-str) (let* ((type (plist-get params :plot-type)) (with (if (eq type 'grid) 'pm3d (plist-get params :with)))) (list (format "'%s' matrix with %s title ''" data-file with))))) - (grid (lambda (data-file num-cols params plot-str) + (grid :plot-cmd "splot" + :plot-pre (lambda (_table _data-file _num-cols params _plot-str) + (if (plist-get params :map) "set pm3d map" "set map")) + :data-dump (lambda (table data-file params _num-cols) + (let ((y-labels (org-plot/gnuplot-to-grid-data + table data-file params))) + (when y-labels (plist-put params :ylabels y-labels)))) + :plot-func + (lambda (table data-file _num-cols params _plot-str) (let* ((type (plist-get params :plot-type)) (with (if (eq type 'grid) 'pm3d (plist-get params :with)))) - (list (format "'%s' with %s title ''" - data-file with))))) - (radar (lambda (data-file num-cols params plot-str) + (list (format "'%s' with %s title ''" + data-file with))))) + (radar :plot-func + (lambda (table _data-file _num-cols params plot-str) (list (org--plot/radar table params))))) - "List of plot presets with the type name as the car, and a function -which yeilds plot-lines (a list of strings) as the cdr. -The parameters of `org-plot/gnuplot-script' and PLOT-STR are passed to -that function. i.e. it is called with the following arguments: - DATA-FILE NUM-COLS PARAMS PLOT-STR" + "List of plists describing the avalible plot types. +The car is the type name, and the property :plot-func must be set. +The value of :plot-func is a lambda which yields plot-lines +(a list of strings) as the cdr. + +All lambda functions have the parameters of `org-plot/gnuplot-script' and PLOT-STR passed to them. +i.e. they are called with the following signature: (TABLE DATA-FILE NUM-COLS PARAMS PLOT-STR) + +Potentially useful parameters in PARAMS include: + :set :line :map :title :file :ind :timeind :timefmt :textind + :deps :labels :xlabels :ylabels :xmin :xmax :ymin :ymax :ticks + +In addition to :plot-func, the following optional properties may be set. + +- :plot-cmd - A gnuplot command appended to each plot-line. + Accepts string or nil. Default value: nil. + +- :check-ind-type - Whether the types of ind values should be checked. + Accepts boolean. + +- :plot-str - the formula string passed to :plot-func as PLOT-STR + Accepts string. Default value: \"'%s' using %s%d%s with %s title '%s'\" + +- :data-dump - Function to dump the table to a datafile for ease of use. + Accepts lambda function. Default lambda body: (org-plot/gnuplot-to-data table data-file params) + +- :plot-pre - Gnuplot code to be inserted early into the script, just after term and output have been set. + Accepts string, nil, or lambda function which returns string or nil. Defaults to nil. +" :group 'org-plot :type '(alist :value-type (symbol group))) @@ -476,89 +516,90 @@ If a function, it is called with the plot type as the argument." :group 'org-plot :type '(choice string function)) -(defun org-plot/gnuplot-script (data-file num-cols params &optional preface) +(defun org-plot/gnuplot-script (table data-file num-cols params &optional preface) "Write a gnuplot script to DATA-FILE respecting the options set in PARAMS. NUM-COLS controls the number of columns plotted in a 2-d plot. Optional argument PREFACE returns only option parameters in a manner suitable for prepending to a user-specified script." - (let* ((type (plist-get params :plot-type)) - (with (if (eq type 'grid) 'pm3d (plist-get params :with))) - (sets (plist-get params :set)) - (lines (plist-get params :line)) - (map (plist-get params :map)) - (title (plist-get params :title)) - (file (plist-get params :file)) - (ind (plist-get params :ind)) - (time-ind (plist-get params :timeind)) - (timefmt (plist-get params :timefmt)) - (text-ind (plist-get params :textind)) - (deps (if (plist-member params :deps) (plist-get params :deps))) - (col-labels (plist-get params :labels)) - (x-labels (plist-get params :xlabels)) - (y-labels (plist-get params :ylabels)) - (plot-str "'%s' using %s%d%s with %s title '%s'") - (plot-cmd (pcase type - (`2d "plot") - (`3d "splot") - (`grid "splot"))) - (script "reset") - ;; ats = add-to-script - (ats (lambda (line) (setf script (concat script "\n" line)))) - plot-lines) - - - ;; handle output file, background, and size - (funcall ats (format "set term %s %s" - (if file (file-name-extension file) "GNUTERM") - (if (stringp org-plot/gnuplot-term-extra) - org-plot/gnuplot-term-extra - (org-plot/gnuplot-term-extra type)))) - (when file ; output file - (funcall ats (format "set output '%s'" file))) - - (funcall ats - (if (stringp org-plot/gnuplot-script-preamble) - org-plot/gnuplot-script-preamble - (org-plot/gnuplot-script-preamble type))) - - (pcase type ; type - (`2d ()) - (`3d (when map (funcall ats "set map"))) - (`grid (funcall ats (if map "set pm3d map" "set pm3d")))) - (when title (funcall ats (format "set title '%s'" title))) ; title - (mapc ats lines) ; line - (dolist (el sets) (funcall ats (format "set %s" el))) ; set - ;; Unless specified otherwise, values are TAB separated. - (unless (string-match-p "^set datafile separator" script) - (funcall ats "set datafile separator \"\\t\"")) - (when x-labels ; x labels (xtics) - (funcall ats - (format "set xtics (%s)" - (mapconcat (lambda (pair) - (format "\"%s\" %d" (cdr pair) (car pair))) - x-labels ", ")))) - (when y-labels ; y labels (ytics) - (funcall ats - (format "set ytics (%s)" - (mapconcat (lambda (pair) - (format "\"%s\" %d" (cdr pair) (car pair))) - y-labels ", ")))) - (when time-ind ; timestamp index - (funcall ats "set xdata time") - (funcall ats (concat "set timefmt \"" - (or timefmt ; timefmt passed to gnuplot - "%Y-%m-%d-%H:%M:%S") "\""))) - (unless preface - (let ((type-func (cadr (assoc type org-plot/preset-plot-types)))) - (when type-func - (setq plot-lines - (funcall type-func data-file num-cols params plot-str)))) + (let* ((type-name (plist-get params :plot-type)) + (type (cdr (assoc type-name org-plot/preset-plot-types)))) + (unless type + (user-error "Org-plot type `%s' is undefined." type-name)) + (let* ((sets (plist-get params :set)) + (lines (plist-get params :line)) + (map (plist-get params :map)) + (title (plist-get params :title)) + (file (plist-get params :file)) + (ind (plist-get params :ind)) + (time-ind (plist-get params :timeind)) + (timefmt (plist-get params :timefmt)) + (text-ind (plist-get params :textind)) + (deps (if (plist-member params :deps) (plist-get params :deps))) + (col-labels (plist-get params :labels)) + (x-labels (plist-get params :xlabels)) + (y-labels (plist-get params :ylabels)) + (plot-str (or (plist-get type :plot-str) + "'%s' using %s%d%s with %s title '%s'")) + (plot-cmd (plist-get type :plot-cmd)) + (plot-pre (plist-get type :plot-pre)) + (script "reset") + ;; ats = add-to-script + (ats (lambda (line) (when line (setf script (concat script "\n" line))))) + plot-lines) + + + ;; handle output file, background, and size + (funcall ats (format "set term %s %s" + (if file (file-name-extension file) "GNUTERM") + (if (stringp org-plot/gnuplot-term-extra) + org-plot/gnuplot-term-extra + (funcall org-plot/gnuplot-term-extra type)))) + (when file ; output file + (funcall ats (format "set output '%s'" file))) + + (when plot-pre + (funcall ats (funcall plot-pre table data-file num-cols params plot-str))) (funcall ats - (concat plot-cmd " " (mapconcat #'identity - (reverse plot-lines) - ",\\\n ")))) - script)) + (if (stringp org-plot/gnuplot-script-preamble) + org-plot/gnuplot-script-preamble + (funcall org-plot/gnuplot-script-preamble type))) + + (when title (funcall ats (format "set title '%s'" title))) ; title + (mapc ats lines) ; line + (dolist (el sets) (funcall ats (format "set %s" el))) ; set + ;; Unless specified otherwise, values are TAB separated. + (unless (string-match-p "^set datafile separator" script) + (funcall ats "set datafile separator \"\\t\"")) + (when x-labels ; x labels (xtics) + (funcall ats + (format "set xtics (%s)" + (mapconcat (lambda (pair) + (format "\"%s\" %d" (cdr pair) (car pair))) + x-labels ", ")))) + (when y-labels ; y labels (ytics) + (funcall ats + (format "set ytics (%s)" + (mapconcat (lambda (pair) + (format "\"%s\" %d" (cdr pair) (car pair))) + y-labels ", ")))) + (when time-ind ; timestamp index + (funcall ats "set xdata time") + (funcall ats (concat "set timefmt \"" + (or timefmt ; timefmt passed to gnuplot + "%Y-%m-%d-%H:%M:%S") "\""))) + (unless preface + (let ((type-func (plist-get type :plot-func))) + (when type-func + (setq plot-lines + (funcall type-func table data-file num-cols params plot-str)))) + (funcall ats + (concat plot-cmd + (when plot-cmd " ") + (mapconcat #'identity + (reverse plot-lines) + ",\\\n ")))) + script))) ;;----------------------------------------------------------------------------- ;; facade functions @@ -598,7 +639,13 @@ line directly before or after the table." (push 'hline (cdr tbl)))) tbl)) (num-cols (length (if (eq (nth 0 table) 'hline) (nth 1 table) - (nth 0 table))))) + (nth 0 table)))) + (type (assoc (plist-get params :plot-type) + org-plot/preset-plot-types))) + + (unless type + (user-error "Org-plot type `%s' is undefined." type-name)) + (run-with-idle-timer 0.1 nil #'delete-file data-file) (when (eq (cadr table) 'hline) (setf params @@ -608,15 +655,12 @@ line directly before or after the table." (save-excursion (while (and (equal 0 (forward-line -1)) (looking-at "[[:space:]]*#\\+")) (setf params (org-plot/collect-options params)))) - ;; Dump table to datafile (very different for grid). - (pcase (plist-get params :plot-type) - (`2d (org-plot/gnuplot-to-data table data-file params)) - (`3d (org-plot/gnuplot-to-data table data-file params)) - (`grid (let ((y-labels (org-plot/gnuplot-to-grid-data - table data-file params))) - (when y-labels (plist-put params :ylabels y-labels))))) + ;; Dump table to datafile + (if-let ((dump-func (plist-get type :data-dump))) + (funcall dump-func table data-file num-cols params) + (org-plot/gnuplot-to-data table data-file params)) ;; Check type of ind column (timestamp? text?) - (when (eq `2d (plist-get params :plot-type)) + (when (plist-get params :check-ind-type) (let* ((ind (1- (plist-get params :ind))) (ind-column (mapcar (lambda (row) (nth ind row)) table))) (cond ((< ind 0) nil) ; ind is implicit @@ -633,13 +677,13 @@ line directly before or after the table." (with-temp-buffer (if (plist-get params :script) ; user script (progn (insert - (org-plot/gnuplot-script data-file num-cols params t)) + (org-plot/gnuplot-script table data-file num-cols params t)) (insert "\n") (insert-file-contents (plist-get params :script)) (goto-char (point-min)) (while (re-search-forward "\\$datafile" nil t) (replace-match data-file nil nil))) - (insert (org-plot/gnuplot-script data-file num-cols params))) + (insert (org-plot/gnuplot-script table data-file num-cols params))) ;; Graph table. (gnuplot-mode) (gnuplot-send-buffer-to-gnuplot)) -- 2.28.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0011-org-plot.el-avoid-arithmetic-overflow-error.patch >From a299ec63c91260b68237da3a6c19e8ed8523fd6d Mon Sep 17 00:00:00 2001 From: TEC Date: Sat, 5 Sep 2020 21:05:36 +0800 Subject: [PATCH 11/15] org-plot.el: avoid arithmetic overflow error * lisp/org-plot.el (org--plot/values-stats): A set of numbers with the same value (i.e. 0 range) should not produce an arithmetic overflow error. This error was caused by taking the log of 0 (when the range is 0). This is mitigated by explicit checking against this case. --- lisp/org-plot.el | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/lisp/org-plot.el b/lisp/org-plot.el index 53186bb75..0a9694263 100644 --- a/lisp/org-plot.el +++ b/lisp/org-plot.el @@ -192,10 +192,13 @@ values, namely regarding the range." (let* ((minimum (or hard-min (apply #'min nums))) (maximum (or hard-max (apply #'max nums))) (range (- maximum minimum)) - (rangeOrder (ceiling (- 1 (log10 range)))) + (rangeOrder (if (= range 0) 0 + (ceiling (- 1 (log10 range))))) (range-factor (expt 10 rangeOrder)) - (nice-min (/ (float (floor (* minimum range-factor))) range-factor)) - (nice-max (/ (float (ceiling (* maximum range-factor))) range-factor))) + (nice-min (if (= range 0) (car nums) + (/ (float (floor (* minimum range-factor))) range-factor))) + (nice-max (if (= range 0) (car nums) + (/ (float (ceiling (* maximum range-factor))) range-factor)))) `(:min ,minimum :max ,maximum :range ,range :range-factor ,range-factor :nice-min ,nice-min :nice-max ,nice-max :nice-range ,(- nice-max nice-min)))) -- 2.28.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0012-org-plot.el-add-missing-cl-prefixes.patch >From 2076b9ea3c7d5459b6b7eca1d1c876518b75cba0 Mon Sep 17 00:00:00 2001 From: TEC Date: Sun, 25 Oct 2020 01:43:01 +0800 Subject: [PATCH 12/15] org-plot.el: add missing cl- prefixes * lisp/org-plot.el ( org--plot/merge-alists, org--plot/item-frequencies, org--plot/prime-factors): Add missing cl- prefic to cl-lib functions called. (org--plot/radar): refactor f function to remove dependency. (org--plot/values-stats, org--plot/nice-frequency-pick): autoformatting. --- lisp/org-plot.el | 72 ++++++++++++++++++++++++------------------------ 1 file changed, 36 insertions(+), 36 deletions(-) diff --git a/lisp/org-plot.el b/lisp/org-plot.el index 0a9694263..449edff20 100644 --- a/lisp/org-plot.el +++ b/lisp/org-plot.el @@ -193,7 +193,7 @@ values, namely regarding the range." (maximum (or hard-max (apply #'max nums))) (range (- maximum minimum)) (rangeOrder (if (= range 0) 0 - (ceiling (- 1 (log10 range))))) + (ceiling (- 1 (log10 range))))) (range-factor (expt 10 rangeOrder)) (nice-min (if (= range 0) (car nums) (/ (float (floor (* minimum range-factor))) range-factor))) @@ -227,44 +227,44 @@ values, namely regarding the range." "From a list of frequences, try to sensibly pick a sample of the most frequent." ;; TODO this mosly works decently, but counld do with some tweaking to work more consistently. (case (length frequencies) - (1 (list (car (nth 0 frequencies)))) - (2 (if (<= 3 (/ (cdr (nth 0 frequencies)) - (cdr (nth 1 frequencies)))) - (make-list 2 - (car (nth 0 frequencies))) - (list (car (nth 0 frequencies)) - (car (nth 1 frequencies))))) - (t - (let* ((total-count (apply #'+ (mapcar #'cdr frequencies))) - (n-freq (mapcar (lambda (freq) `(,(car freq) . ,(/ (float (cdr freq)) total-count))) frequencies)) - (f-pick (list (car (car n-freq)))) - (1-2-ratio (/ (cdr (nth 0 n-freq)) - (cdr (nth 1 n-freq)))) - (2-3-ratio (/ (cdr (nth 1 n-freq)) - (cdr (nth 2 n-freq)))) - (1-3-ratio (* 1-2-ratio 2-3-ratio)) - (1-val (car (nth 0 n-freq))) - (2-val (car (nth 1 n-freq))) - (3-val (car (nth 2 n-freq)))) - (when (> 1-2-ratio 4) (push 1-val f-pick)) - (when (and (< 1-2-ratio 2-val) - (< (* (apply #'* f-pick) 2-val) 30)) - (push 2-val f-pick)) - (when (and (< 1-3-ratio 3-val) - (< (* (apply #'* f-pick) 3-val) 30)) - (push 3-val f-pick)) - f-pick)))) + (1 (list (car (nth 0 frequencies)))) + (2 (if (<= 3 (/ (cdr (nth 0 frequencies)) + (cdr (nth 1 frequencies)))) + (make-list 2 + (car (nth 0 frequencies))) + (list (car (nth 0 frequencies)) + (car (nth 1 frequencies))))) + (t + (let* ((total-count (apply #'+ (mapcar #'cdr frequencies))) + (n-freq (mapcar (lambda (freq) `(,(car freq) . ,(/ (float (cdr freq)) total-count))) frequencies)) + (f-pick (list (car (car n-freq)))) + (1-2-ratio (/ (cdr (nth 0 n-freq)) + (cdr (nth 1 n-freq)))) + (2-3-ratio (/ (cdr (nth 1 n-freq)) + (cdr (nth 2 n-freq)))) + (1-3-ratio (* 1-2-ratio 2-3-ratio)) + (1-val (car (nth 0 n-freq))) + (2-val (car (nth 1 n-freq))) + (3-val (car (nth 2 n-freq)))) + (when (> 1-2-ratio 4) (push 1-val f-pick)) + (when (and (< 1-2-ratio 2-val) + (< (* (apply #'* f-pick) 2-val) 30)) + (push 2-val f-pick)) + (when (and (< 1-3-ratio 3-val) + (< (* (apply #'* f-pick) 3-val) 30)) + (push 3-val f-pick)) + f-pick)))) (defun org--plot/merge-alists (function default alist1 alist2 &rest alists) "Using FUNCTION, combine the elements of all given ALISTS. When an element is only present in one alist, DEFAULT is used as the second argument for the FUNCTION." (when (> (length alists) 0) (setq alist2 (apply #'org--plot/merge-alists function default alist2 alists))) - (flet ((keys (alist) (mapcar #'car alist)) - (lookup (key alist) (or (cdr (assoc key alist)) default))) - (loop with keys = (union (keys alist1) (keys alist2) :test 'equal) - for k in keys collect - (cons k (funcall function (lookup k alist1) (lookup k alist2)))))) + (cl-flet ((keys (alist) (mapcar #'car alist)) + (lookup (key alist) (or (cdr (assoc key alist)) default))) + (cl-loop with keys = (cl-union (keys alist1) (keys alist2) :test 'equal) + for k in keys collect + (cons k (funcall function (lookup k alist1) (lookup k alist2)))))) (defun org--plot/item-frequencies (values &optional normalise) "Return an alist indicating the frequency of values in VALUES list." @@ -282,7 +282,7 @@ only present in one alist, DEFAULT is used as the second argument for the FUNCTI (setq value (/ value i)) (setq i (1- i)) )) - (subseq factors 0 -1))) + (cl-subseq factors 0 -1))) (defcustom org-plot/gnuplot-script-preamble "" "String or function which provides content to be inserted into the GNUPlot @@ -499,8 +499,8 @@ EOD ))) table))) (setup-file (make-temp-file "org-plot-setup"))) - (f-write-text (format org--plot/radar-setup-template data settings) - 'utf-8 setup-file) + (let ((coding-system-for-write 'utf-8)) + (write-region (format org--plot/radar-setup-template data settings) nil setup-file nil :silent)) (format org--plot/radar-template setup-file (if (eq ticks 0) 2 ticks) -- 2.28.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0013-org-plot.el-radar-plot-join-last-points-to-first.patch >From 4ee4089848816fd0afedfe3f030912041bffece3 Mon Sep 17 00:00:00 2001 From: TEC Date: Sun, 25 Oct 2020 01:47:57 +0800 Subject: [PATCH 13/15] org-plot.el: radar plot, join last points to first * lisp/org-plot.el (org--plot/radar-template, org--plot/radar): Duplicate first points at the end so that a line is drawn between them. --- lisp/org-plot.el | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/lisp/org-plot.el b/lisp/org-plot.el index 449edff20..da9f6ea74 100644 --- a/lisp/org-plot.el +++ b/lisp/org-plot.el @@ -393,7 +393,7 @@ load \"%s\" # General settings DataColCount = words($Data[1])-1 -AxesCount = |$Data|-HeaderLines +AxesCount = |$Data|-HeaderLines-1 AngleOffset = 90 Max = 1 d=0.1*Max @@ -478,7 +478,7 @@ EOD "\"%s\" %s" (car row) (s-join " " (cdr row)))) - table)))) + (append table (list (car table))))))) (ticks (or (plist-get params :ticks) (org--plot/sensible-tick-num table (plist-get params :ymin) @@ -497,7 +497,7 @@ EOD (plist-get data :nice-max)) (if (eq ticks 0) 2 ticks) ))) - table))) + (append table (list (car table)))))) (setup-file (make-temp-file "org-plot-setup"))) (let ((coding-system-for-write 'utf-8)) (write-region (format org--plot/radar-setup-template data settings) nil setup-file nil :silent)) -- 2.28.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0014-org-plot.el-Make-min-max-keywords-consistent.patch >From 66540fd5f4f2c7e11d70f572814daa90ec22f92b Mon Sep 17 00:00:00 2001 From: TEC Date: Sun, 25 Oct 2020 02:10:53 +0800 Subject: [PATCH 14/15] org-plot.el: Make min/max keywords consistent * lisp/org-plot.el: (org-plot/add-options-to-plist): Have both x/y min/max, but have min/max aliased to the y-axis options. --- lisp/org-plot.el | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/lisp/org-plot.el b/lisp/org-plot.el index da9f6ea74..f6348907f 100644 --- a/lisp/org-plot.el +++ b/lisp/org-plot.el @@ -64,8 +64,10 @@ Returns the resulting property list." ("timeind" . :timeind) ("timefmt" . :timefmt) ("min" . :ymin) - ("max" . :ymax) ("ymin" . :ymin) + ("max" . :ymax) + ("ymax" . :ymax) + ("xmin" . :xmin) ("xmax" . :xmax) ("ticks" . :ticks) ("trans" . :transpose) -- 2.28.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0015-org-manual.org-document-org-plot-changes.patch >From 373bf71d6b108a638c6ba2ec6937d055c1d0f7dd Mon Sep 17 00:00:00 2001 From: TEC Date: Sun, 25 Oct 2020 02:14:37 +0800 Subject: [PATCH 15/15] org-manual.org: document org-plot changes * doc/org-manual.org: Document changes to org-plot.el. --- doc/org-manual.org | 39 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 38 insertions(+), 1 deletion(-) diff --git a/doc/org-manual.org b/doc/org-manual.org index 066092238..7eff42aee 100644 --- a/doc/org-manual.org +++ b/doc/org-manual.org @@ -2809,6 +2809,19 @@ following table. | Morelia | 257.56 | 17.67 | #+end_example +Org Plot supports a range of plot types, and provides the ability to add more. +For example, a radar plot can be generated like so: +#+begin_example +,#+PLOT: title:"An evaluation of plaintext document formats" transpose:yes type:radar min:0 max:4 +| Format | Fine-grained-control | Initial Effort | Syntax simplicity | Editor Support | Integrations | Ease-of-referencing | Versatility | +|-------------------+----------------------+----------------+-------------------+----------------+--------------+---------------------+-------------| +| Word | 2 | 4 | 4 | 2 | 3 | 2 | 2 | +| LaTeX | 4 | 1 | 1 | 3 | 2 | 4 | 3 | +| Org Mode | 4 | 2 | 3.5 | 1 | 4 | 4 | 4 | +| Markdown | 1 | 3 | 3 | 4 | 3 | 3 | 1 | +| Markdown + Pandoc | 2.5 | 2.5 | 2.5 | 3 | 3 | 3 | 2 | +#+end_example + Notice that Org Plot is smart enough to apply the table's headers as labels. Further control over the labels, type, content, and appearance of plots can be exercised through the =PLOT= keyword @@ -2839,9 +2852,15 @@ For more information and examples see the [[https://orgmode.org/worg/org-tutoria the third and fourth columns. Defaults to graphing all other columns aside from the =ind= column. +- transpose :: + + When =y=, =yes=, or =t= attempt to transpose the table data before plotting. Also + recognises the shorthand option =trans=. + - =type= :: - Specify whether the plot is =2d=, =3d=, or =grid=. + Specify the type of the plot, by default one of =2d=, =3d=, =radar=, or =grid=. + Available types can be customised with ~org-plot/preset-plot-types~. - =with= :: @@ -2868,6 +2887,24 @@ For more information and examples see the [[https://orgmode.org/worg/org-tutoria When plotting =3d= or =grid= types, set this to =t= to graph a flat mapping rather than a =3d= slope. +- min :: + + Provides a minimum axis value that may be used by a plot type. Implicitly + assumes the =y= axis is being referred to. Can explicitly provide a value for a + either the =x= or =y= axis with =xmin= and =ymin=. + +- max :: + + Provides a maximum axis value that may be used by a plot type. Implicitly + assumes the =y= axis is being referred to. Can explicitly provide a value for a + either the =x= or =y= axis with =xmax= and =ymax=. + +- ticks :: + + Provides a desired number of axis ticks to display, that may be used by a plot + type. If none is given a plot type that requires ticks will use + ~org--plot/sensible-tick-num~ to try to determine a good value. + - =timefmt= :: Specify format of Org mode timestamps as they will be parsed by -- 2.28.0 --=-=-=--