[PATCH] allow for multiline headers

[PATCH] allow for multiline headers

[Mario Frasca]

[-- Attachment #1: Type: text/plain, Size: 1315 bytes --]

it misses unit tests, I need to make the collapse optional, and I need 
to hear from users what they think of it.  this patch allows me to write 
a table like the following, having a three-lines header:

|  n | data | prove, |  casi, | ratio |   prove, |    casi, |
|    |      | valore | valore |       |    media | media |
|    |      | diario | diario |       | corrente | corrente |
|----+------+--------+--------+-------+----------+----------|

and this is seen as a single header line.

|  n | data | prove, valore diario | casi, valore diario | ratio | 
prove, media corrente | casi, media corrente |
|----+------+----------------------+---------------------+-------+-----------------------+----------------------|

I find it helpful, because this way my columns stay narrower.

as it is, ¡it fails on headerless tables!

I have not yet clear what's the best approach to handling headerless 
tables, was thinking of a org-table-max-collapse-header variable, which 
you would set to the number of lines which you are maximally expecting 
to collapse into the leading header line.  Or maybe an extra option to 
org-plot, where you would state if headers need be collapsed.

or we already had ways to achieve this same thing, and I missed them all?

best regards,

MF


[-- Attachment #2: 0001-lisp-org-table.el-Allow-collapsing-header-into-singl.patch --]
[-- Type: text/x-patch, Size: 4565 bytes --]

From 649f46a591474afb6cef8b9d5151ff6b0bae38aa Mon Sep 17 00:00:00 2001
From: mfrasca <mario@anche.no>
Date: Fri, 12 Jun 2020 11:42:34 -0500
Subject: [PATCH] lisp/org-table.el: Allow collapsing header into single line

* lisp/org-table.el (org-table-collapse-header): new function
that collapses multiple header lines into one list.
(org-table-to-lisp): simplify code, changing a `while' to a
`cl-loop', remove leading `hline' symbols from result, edit
documentation to reflect change.

* lisp/org-plot.el (org-plot/gnuplot): use
org-table-collapse-header and trust there's no leading `hline'
symbols in lisp table.
---
 lisp/org-plot.el  |  6 ++---
 lisp/org-table.el | 59 ++++++++++++++++++++++++++++++++---------------
 2 files changed, 43 insertions(+), 22 deletions(-)

diff --git a/lisp/org-plot.el b/lisp/org-plot.el
index a23195d2a..662d38e54 100644
--- a/lisp/org-plot.el
+++ b/lisp/org-plot.el
@@ -289,11 +289,9 @@ line directly before or after the table."
 	(setf params (plist-put params (car pair) (cdr pair)))))
     ;; collect table and table information
     (let* ((data-file (make-temp-file "org-plot"))
-	   (table (org-table-to-lisp))
-	   (num-cols (length (if (eq (nth 0 table) 'hline) (nth 1 table)
-			       (nth 0 table)))))
+	   (table (org-table-collapse-header (org-table-to-lisp)))
+	   (num-cols (length (nth 0 table))))
       (run-with-idle-timer 0.1 nil #'delete-file data-file)
-      (while (eq 'hline (car table)) (setf table (cdr table)))
       (when (eq (cadr table) 'hline)
 	(setf params
 	      (plist-put params :labels (nth 0 table))) ; headers to labels
diff --git a/lisp/org-table.el b/lisp/org-table.el
index 6462b99c4..6549e178a 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -5433,30 +5433,53 @@ a radio table."
 (defun org-table-to-lisp (&optional txt)
   "Convert the table at point to a Lisp structure.
 
-The structure will be a list.  Each item is either the symbol `hline'
-for a horizontal separator line, or a list of field values as strings.
-The table is taken from the parameter TXT, or from the buffer at point."
+The returned structure is a list, where each item is either the
+symbol `hline', for a horizontal separator line, or a list of
+field values as strings.  The table is taken from the parameter
+TXT, or from the buffer at point.  Leading `hline' symbols are
+trimmed, so the first item in the result is a list"
   (if txt
       (with-temp-buffer
         (insert txt)
-        (goto-char (point-min))
         (org-table-to-lisp))
     (save-excursion
       (goto-char (org-table-begin))
-      (let ((table nil))
-        (while (re-search-forward "\\=[ \t]*|" nil t)
-	  (let ((row nil))
-	    (if (looking-at "-")
-		(push 'hline table)
-	      (while (not (progn (skip-chars-forward " \t") (eolp)))
-		(push (buffer-substring
-		       (point)
-		       (progn (re-search-forward "[ \t]*\\(|\\|$\\)")
-			      (match-beginning 0)))
-		      row))
-	      (push (nreverse row) table)))
-	  (forward-line))
-        (nreverse table)))))
+      (let ((table (cl-loop
+		    until (not (re-search-forward "\\=[ \t]*|" nil t))
+		    collect (if (looking-at "-")
+				'hline
+			      (cl-loop
+                               do (skip-chars-forward " \t")
+		               collect (buffer-substring
+					(point)
+					(progn (re-search-forward "[ \t]*\\(|\\|$\\)")
+			                       (match-beginning 0)))
+                               until (looking-at "$")))
+		    do (forward-line))))
+	(while (equal 'hline (car table))
+          (setq table (cdr table)))
+	table))))
+
+(defun org-table-collapse-header (table &optional glue)
+  "Collapse the lines before 'hline into a single header.
+
+The given TABLE is a list of lists as returned by `org-table-to-lisp'.
+The leading lines before the first `hline' symbol are considered
+forming the table header.  This function collapses all leading header
+lines into a single header line, followed by the `hline' symbol, and
+the rest of the TABLE.  Header cells are GLUEd together with a space,
+or the given character."
+  (setq glue (or glue " "))
+  (let* ((header (cl-loop for i in table
+                          until (equal i 'hline)
+                          collect i))
+         (position (length header)))
+    (cons (apply #'cl-mapcar
+		 #'(lambda (&rest x)
+		     (string-trim
+		      (mapconcat #'identity x glue)))
+		 header)
+          (nthcdr position table))))
 
 (defun orgtbl-send-table (&optional maybe)
   "Send a transformed version of table at point to the receiver position.
-- 
2.20.1

Re: [PATCH] allow for multiline headers

[Nicolas Goaziou]

Hello,

Mario Frasca <mario@anche.no> writes:

> (org-table-to-lisp): simplify code, changing a `while' to a
> `cl-loop', remove leading `hline' symbols from result, edit
> documentation to reflect change.

Please don't. Even if it has its uses, cl-loop is a complicated
non-Lispy beast, and introducing it somewhere is seldom
a simplification.

If you think cl-loop is more efficient in this function, please provide
benchmarks. But I really hope cl-loop (or cl-do, for that matter) is not
going to sprout everywhere.

Also, the first hline is used to determine where to end the header.
A table starting with a hline has no header. Therefore, I suggest to
avoid removing hlines at the beginning of a table, we would lose
information.


Regards,

-- 
Nicolas Goaziou

Re: [PATCH] allow for multiline headers

[Mario Frasca]

hi Nicolas,

is there an agreement on cl-lib usage within the project?  I was hinted 
at cl-loop in this mailing list, and I liked it, in particular the 
`collect' clause, the destructuring feature, and less parentheses.  I 
had no exposure to cl-loop before the hint received here.

I can leave existing loops in peace, or edit them keeping them 
cl-loop-free.  as for myself, I find it practical and readable. let's 
say I'll try to limit usage.

On 12/06/2020 17:44, Nicolas Goaziou wrote:
> Also, the first hline is used to determine where to end the header.
> A table starting with a hline has no header. Therefore, I suggest to
> avoid removing hlines at the beginning of a table, we would lose
> information.

this is not a correct description of the current status, at least not 
within org-plot.

(let ((table (org-table-to-lisp)) …

in org-plot we expect either the first or the second element of `table' 
to be a list;

any leading `hline' is removed;

if the second element of `table' is a hline, the first element of 
`table' is considered to be the header (in this case -and only in this 
case- any subsequent `hline' is then removed from `table');

the presence of hline elements in `table' will crash the routine.

----------------------

a table looking like hline-header-hline-data is treated as having a 
header, while your description says it's all data, no header, because of 
the leading hline.

but in my opinion if there's any problem or misunderstanding here, it's 
because there's no unit tests that describe the correct behaviour.  can 
we work at that?

in fact, there's not even a org-table-headers function, that would 
return the table headers.

anyhow, what do you think of the multiple-lines-header option?

MF

Re: [PATCH] allow for multiline headers

[Mario Frasca]

[-- Attachment #1: Type: text/plain, Size: 73 bytes --]

what about these two groups of tests, and the header collapse function?


[-- Attachment #2: 0001-lisp-org-table.el-Allow-collapsing-header-into-singl.patch --]
[-- Type: text/x-patch, Size: 6052 bytes --]

From ceb21024159a75dbdb9fef32eebe1fc8c7076d2f Mon Sep 17 00:00:00 2001
From: mfrasca <mario@anche.no>
Date: Fri, 12 Jun 2020 11:42:34 -0500
Subject: [PATCH] lisp/org-table.el: Allow collapsing header into single line

* lisp/org-table.el (org-table-collapse-header): new function that
collapses multiple header lines into one list.

* lisp/org-plot.el (org-plot/gnuplot): use org-table-collapse-header
and trust there will be no more leading `hline' symbols in lisp table.

* testing/lisp/test-org-table.el (test-org-table/to-lisp):
adding tests to already existing to-lisp function.
(test-org-table/collapse-header): adding tests to new
collapse-header function.
---
 lisp/org-plot.el               |  6 ++--
 lisp/org-table.el              | 27 ++++++++++++++++
 testing/lisp/test-org-table.el | 58 ++++++++++++++++++++++++++++++++++
 3 files changed, 87 insertions(+), 4 deletions(-)

diff --git a/lisp/org-plot.el b/lisp/org-plot.el
index a23195d2a..662d38e54 100644
--- a/lisp/org-plot.el
+++ b/lisp/org-plot.el
@@ -289,11 +289,9 @@ line directly before or after the table."
 	(setf params (plist-put params (car pair) (cdr pair)))))
     ;; collect table and table information
     (let* ((data-file (make-temp-file "org-plot"))
-	   (table (org-table-to-lisp))
-	   (num-cols (length (if (eq (nth 0 table) 'hline) (nth 1 table)
-			       (nth 0 table)))))
+	   (table (org-table-collapse-header (org-table-to-lisp)))
+	   (num-cols (length (nth 0 table))))
       (run-with-idle-timer 0.1 nil #'delete-file data-file)
-      (while (eq 'hline (car table)) (setf table (cdr table)))
       (when (eq (cadr table) 'hline)
 	(setf params
 	      (plist-put params :labels (nth 0 table))) ; headers to labels
diff --git a/lisp/org-table.el b/lisp/org-table.el
index 6462b99c4..c40ad5bea 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -5458,6 +5458,33 @@ The table is taken from the parameter TXT, or from the buffer at point."
 	  (forward-line))
         (nreverse table)))))
 
+(defun org-table-collapse-header (table &optional glue max-header-lines)
+  "Collapse the lines before 'hline into a single header.
+
+The given TABLE is a list of lists as returned by `org-table-to-lisp'.
+The leading lines before the first `hline' symbol are considered
+forming the table header.  This function collapses all leading header
+lines into a single header line, followed by the `hline' symbol, and
+the rest of the TABLE.  Header cells are GLUEd together with a space,
+or the given character."
+  (setq glue (or glue " "))
+  (setq max-header-lines (or max-header-lines 4))
+  (while (equal 'hline (car table))
+    (setq table (cdr table)))
+  (let* ((trailer table)
+	 (header-lines (cl-loop for line in table
+				until (equal line 'hline)
+				collect line
+				do (setq trailer (cdr trailer)))))
+    (if (and trailer (<= (length header-lines) max-header-lines))
+	(cons (apply #'cl-mapcar
+		     #'(lambda (&rest x)
+			 (org-trim
+			  (mapconcat #'identity x glue)))
+		     header-lines)
+	      trailer)
+      table)))
+
 (defun orgtbl-send-table (&optional maybe)
   "Send a transformed version of table at point to the receiver position.
 With argument MAYBE, fail quietly if no transformation is defined
diff --git a/testing/lisp/test-org-table.el b/testing/lisp/test-org-table.el
index 64a1b4b16..5d54f4999 100644
--- a/testing/lisp/test-org-table.el
+++ b/testing/lisp/test-org-table.el
@@ -1304,6 +1304,64 @@ See also `test-org-table/copy-field'."
       (should (string= got
 		       expect)))))
 
+;;; the initial to lisp converter
+
+(ert-deftest test-org-table/to-lisp ()
+  "Test `orgtbl-to-lisp' specifications."
+  ;; 2x2 no header
+  (should
+   (equal '(("a" "b") ("c" "d"))
+	  (org-table-to-lisp "|a|b|\n|c|d|")))
+  ;; 2x2 with 1-line header
+  (should
+   (equal '(("a" "b") hline ("c" "d"))
+	  (org-table-to-lisp "|a|b|\n|-\n|c|d|")))
+  ;; 2x4 with 2-line header
+  (should
+   (equal '(("a" "b") ("A" "B") hline ("c" "d") ("aa" "bb"))
+	  (org-table-to-lisp "|a|b|\n|A|B|\n|-\n|c|d|\n|aa|bb|")))
+  ;; leading hlines do not get stripped
+  (should
+   (equal '(hline ("a" "b") hline ("c" "d"))
+	  (org-table-to-lisp "|-\n|a|b|\n|-\n|c|d|")))
+  (should
+   (equal '(hline ("a" "b") ("c" "d"))
+	  (org-table-to-lisp "|-\n|a|b|\n|c|d|")))
+  (should
+   (equal '(hline hline hline hline ("a" "b") ("c" "d"))
+	  (org-table-to-lisp "|-\n|-\n|-\n|-\n|a|b|\n|c|d|"))))
+
+(ert-deftest test-org-table/collapse-header ()
+  "Test `orgtbl-to-lisp' specifications."
+  ;; 2x2 no header - no collapsing
+  (should
+   (equal '(("a" "b") ("c" "d"))
+	  (org-table-collapse-header (org-table-to-lisp "|a|b|\n|c|d|"))))
+  ;; 2x2 with 1-line header - no collapsing
+  (should
+   (equal '(("a" "b") hline ("c" "d"))
+	  (org-table-collapse-header (org-table-to-lisp "|a|b|\n|-\n|c|d|"))))
+  ;; 2x4 with 2-line header - collapsed
+  (should
+   (equal '(("a A" "b B") hline ("c" "d") ("aa" "bb"))
+	  (org-table-collapse-header (org-table-to-lisp "|a|b|\n|A|B|\n|-\n|c|d|\n|aa|bb|"))))
+  ;; 2x4 with 2-line header, custom glue - collapsed
+  (should
+   (equal '(("a.A" "b.B") hline ("c" "d") ("aa" "bb"))
+	  (org-table-collapse-header (org-table-to-lisp "|a|b|\n|A|B|\n|-\n|c|d|\n|aa|bb|") ".")))
+  ;; 2x4 with 2-line header, threshold 1 - not collapsed
+  (should
+   (equal '(("a" "b") ("A" "B") hline ("c" "d") ("aa" "bb"))
+	  (org-table-collapse-header (org-table-to-lisp "|a|b|\n|A|B|\n|-\n|c|d|\n|aa|bb|") nil 1)))
+  ;; 2x4 with 2-line header, threshold 2 - collapsed
+  (should
+   (equal '(("a A" "b B") hline ("c" "d") ("aa" "bb"))
+	  (org-table-collapse-header (org-table-to-lisp "|a|b|\n|A|B|\n|-\n|c|d|\n|aa|bb|") nil 2)))
+  ;; 2x8 with 6-line header, default threshold 5 - not collapsed
+  (should
+   (equal '(("a" "b") ("A" "B") ("a" "b") ("A" "B") ("a" "b") ("A" "B") hline ("c" "d") ("aa" "bb"))
+	  (org-table-collapse-header (org-table-to-lisp "|a|b|\n|A|B|\n|a|b|\n|A|B|\n|a|b|\n|A|B|\n|-\n|c|d|\n|aa|bb|")))))
+
 ;;; Radio Tables
 
 (ert-deftest test-org-table/to-generic ()
-- 
2.20.1

Re: [PATCH] allow for multiline headers

[Nicolas Goaziou]

Hello,

Mario Frasca <mario@anche.no> writes:

> is there an agreement on cl-lib usage within the project?

Using cl-lib is OK, even though I wish we could use "seq.el" instead.

> I was hinted at cl-loop in this mailing list, and I liked it, in
> particular the `collect' clause, the destructuring feature, and less
> parentheses.

That's exactly my point. It does way too much, and it is very far from
Lisp. Since it does so much stuff, it is tempting to use it all over the
place, for every single iteration. And then, your code doesn't look like
Lisp anymore. See how they defaced that poor "org-contacts.el" =/

> I can leave existing loops in peace, or edit them keeping them
> cl-loop-free.  as for myself, I find it practical and readable.

Then you'll enjoy reading, e.g., `org-contacts-try-completion-prefix'.
FWIW, I don't.

> this is not a correct description of the current status, at least not
> within org-plot.
>
> (let ((table (org-table-to-lisp)) …
>
> in org-plot we expect either the first or the second element of
> `table' to be a list;
>
> any leading `hline' is removed;

I don't know Org Plot enough, but these expectations should be located
in "org-plot.el". I expect `org-table-to-lisp' to be as faithful as
possible. In particular, this function is used in `org-table-align'; as
such, it should not do anything fancy.

> a table looking like hline-header-hline-data is treated as having
> a header, while your description says it's all data, no header,
> because of the leading hline.

The manual states:

    Rows before the first horizontal rule are header lines.

But you are right, this is ambiguous. 

Moreover, I mis-remembered how I defined header lines in the export
framework. Per "ox.el", the header is the first row group. A row group
is a group of one or more data rows located between row separators or
table boundaries. This is more accurate than the above.

> but in my opinion if there's any problem or misunderstanding here,
> it's because there's no unit tests that describe the correct
> behaviour.  can we work at that?

Unit tests are not worth a formal definition. However, "test-ox.el"
contains unit tests.

> anyhow, what do you think of the multiple-lines-header option?

Multiple lines header are already a thing, at least in the export
framework. Try to export, e.g.,

    | a |
    | b |
    |---|
    | c |

in HTML. They are also assumed to be valid in the manual, per above.

So, if your question is: "should Org support multiple lines headers?",
I'd say that it already does, but it should definitely be made more
consistent across the various libraries (e.g., Org Plot).

Regards,

-- 
Nicolas Goaziou

Re: [PATCH] allow for multiline headers

[Mario Frasca]

[-- Attachment #1: Type: text/plain, Size: 2029 bytes --]

Hi Nicolas,

On 13/06/2020 17:18, Nicolas Goaziou wrote:
> Hello,
>
> Mario Frasca <mario@anche.no> writes:
>
>> I can leave existing loops in peace, or edit them keeping them
>> cl-loop-free.  as for myself, I find it practical and readable.
> Then you'll enjoy reading, e.g., `org-contacts-try-completion-prefix'.
> FWIW, I don't.

oops.

I do see your point here.  no, at this level, it's not pleasant any more.

> I don't know Org Plot enough, but these expectations should be located
> in "org-plot.el". I expect `org-table-to-lisp' to be as faithful as
> possible. In particular, this function is used in `org-table-align'; as
> such, it should not do anything fancy.

I have removed the changes, and added some tests that freeze the current 
behaviour.

that was already in the last patch I sent.

> Unit tests are not worth a formal definition. However, "test-ox.el"
> contains unit tests.

I'm not sure what you mean by the first sentence, but I found the header 
tests in the test-ox.el file, and added one where multiple lines header 
is accepted.  is it related enough as to be included in my proposal?

to me, unit tests are readable function definitions, often the best form 
of technical documentation.  I have had colleagues writing tests that 
weren't "unitary" at all, mixing all sort of elements together.  when 
written as an after-thought, it's a serious risk.

> So, if your question is: "should Org support multiple lines headers?",
> I'd say that it already does, but it should definitely be made more
> consistent across the various libraries (e.g., Org Plot).

I can move the org-table-collapse-header function from org-table.el to 
org-plot.el, but to me it makes little sense, relegating a generic 
function to a sub-module: others will look for the functionality in 
org-table, not see it, and duplicate the function somewhere else.

for example, do I understand it correctly, that the concept defined in 
the export functionality, but not in org-table itself?

anyhow, what do we do next?

MF


[-- Attachment #2: 0001-lisp-org-table.el-Allow-collapsing-header-into-singl.patch --]
[-- Type: text/x-patch, Size: 6837 bytes --]

From b407252fc82d29646d267974c626dbc30145f07f Mon Sep 17 00:00:00 2001
From: mfrasca <mario@anche.no>
Date: Fri, 12 Jun 2020 11:42:34 -0500
Subject: [PATCH] lisp/org-table.el: Allow collapsing header into single line

* lisp/org-table.el (org-table-collapse-header): new function that
collapses multiple header lines into one list.

* lisp/org-plot.el (org-plot/gnuplot): use org-table-collapse-header
and trust there will be no more leading `hline' symbols in lisp table.

* testing/lisp/test-org-table.el (test-org-table/to-lisp):
adding tests to already existing to-lisp function.
(test-org-table/collapse-header): adding tests to new
collapse-header function.

* testing/lisp/test-ox.el (test-org-export/has-header-p): testing
exporting table with multi-line header.
---
 lisp/org-plot.el               |  6 ++--
 lisp/org-table.el              | 27 ++++++++++++++++
 testing/lisp/test-org-table.el | 58 ++++++++++++++++++++++++++++++++++
 testing/lisp/test-ox.el        | 10 ++++++
 4 files changed, 97 insertions(+), 4 deletions(-)

diff --git a/lisp/org-plot.el b/lisp/org-plot.el
index a23195d2a..662d38e54 100644
--- a/lisp/org-plot.el
+++ b/lisp/org-plot.el
@@ -289,11 +289,9 @@ line directly before or after the table."
 	(setf params (plist-put params (car pair) (cdr pair)))))
     ;; collect table and table information
     (let* ((data-file (make-temp-file "org-plot"))
-	   (table (org-table-to-lisp))
-	   (num-cols (length (if (eq (nth 0 table) 'hline) (nth 1 table)
-			       (nth 0 table)))))
+	   (table (org-table-collapse-header (org-table-to-lisp)))
+	   (num-cols (length (nth 0 table))))
       (run-with-idle-timer 0.1 nil #'delete-file data-file)
-      (while (eq 'hline (car table)) (setf table (cdr table)))
       (when (eq (cadr table) 'hline)
 	(setf params
 	      (plist-put params :labels (nth 0 table))) ; headers to labels
diff --git a/lisp/org-table.el b/lisp/org-table.el
index 6462b99c4..c40ad5bea 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -5458,6 +5458,33 @@ The table is taken from the parameter TXT, or from the buffer at point."
 	  (forward-line))
         (nreverse table)))))
 
+(defun org-table-collapse-header (table &optional glue max-header-lines)
+  "Collapse the lines before 'hline into a single header.
+
+The given TABLE is a list of lists as returned by `org-table-to-lisp'.
+The leading lines before the first `hline' symbol are considered
+forming the table header.  This function collapses all leading header
+lines into a single header line, followed by the `hline' symbol, and
+the rest of the TABLE.  Header cells are GLUEd together with a space,
+or the given character."
+  (setq glue (or glue " "))
+  (setq max-header-lines (or max-header-lines 4))
+  (while (equal 'hline (car table))
+    (setq table (cdr table)))
+  (let* ((trailer table)
+	 (header-lines (cl-loop for line in table
+				until (equal line 'hline)
+				collect line
+				do (setq trailer (cdr trailer)))))
+    (if (and trailer (<= (length header-lines) max-header-lines))
+	(cons (apply #'cl-mapcar
+		     #'(lambda (&rest x)
+			 (org-trim
+			  (mapconcat #'identity x glue)))
+		     header-lines)
+	      trailer)
+      table)))
+
 (defun orgtbl-send-table (&optional maybe)
   "Send a transformed version of table at point to the receiver position.
 With argument MAYBE, fail quietly if no transformation is defined
diff --git a/testing/lisp/test-org-table.el b/testing/lisp/test-org-table.el
index 64a1b4b16..5d54f4999 100644
--- a/testing/lisp/test-org-table.el
+++ b/testing/lisp/test-org-table.el
@@ -1304,6 +1304,64 @@ See also `test-org-table/copy-field'."
       (should (string= got
 		       expect)))))
 
+;;; the initial to lisp converter
+
+(ert-deftest test-org-table/to-lisp ()
+  "Test `orgtbl-to-lisp' specifications."
+  ;; 2x2 no header
+  (should
+   (equal '(("a" "b") ("c" "d"))
+	  (org-table-to-lisp "|a|b|\n|c|d|")))
+  ;; 2x2 with 1-line header
+  (should
+   (equal '(("a" "b") hline ("c" "d"))
+	  (org-table-to-lisp "|a|b|\n|-\n|c|d|")))
+  ;; 2x4 with 2-line header
+  (should
+   (equal '(("a" "b") ("A" "B") hline ("c" "d") ("aa" "bb"))
+	  (org-table-to-lisp "|a|b|\n|A|B|\n|-\n|c|d|\n|aa|bb|")))
+  ;; leading hlines do not get stripped
+  (should
+   (equal '(hline ("a" "b") hline ("c" "d"))
+	  (org-table-to-lisp "|-\n|a|b|\n|-\n|c|d|")))
+  (should
+   (equal '(hline ("a" "b") ("c" "d"))
+	  (org-table-to-lisp "|-\n|a|b|\n|c|d|")))
+  (should
+   (equal '(hline hline hline hline ("a" "b") ("c" "d"))
+	  (org-table-to-lisp "|-\n|-\n|-\n|-\n|a|b|\n|c|d|"))))
+
+(ert-deftest test-org-table/collapse-header ()
+  "Test `orgtbl-to-lisp' specifications."
+  ;; 2x2 no header - no collapsing
+  (should
+   (equal '(("a" "b") ("c" "d"))
+	  (org-table-collapse-header (org-table-to-lisp "|a|b|\n|c|d|"))))
+  ;; 2x2 with 1-line header - no collapsing
+  (should
+   (equal '(("a" "b") hline ("c" "d"))
+	  (org-table-collapse-header (org-table-to-lisp "|a|b|\n|-\n|c|d|"))))
+  ;; 2x4 with 2-line header - collapsed
+  (should
+   (equal '(("a A" "b B") hline ("c" "d") ("aa" "bb"))
+	  (org-table-collapse-header (org-table-to-lisp "|a|b|\n|A|B|\n|-\n|c|d|\n|aa|bb|"))))
+  ;; 2x4 with 2-line header, custom glue - collapsed
+  (should
+   (equal '(("a.A" "b.B") hline ("c" "d") ("aa" "bb"))
+	  (org-table-collapse-header (org-table-to-lisp "|a|b|\n|A|B|\n|-\n|c|d|\n|aa|bb|") ".")))
+  ;; 2x4 with 2-line header, threshold 1 - not collapsed
+  (should
+   (equal '(("a" "b") ("A" "B") hline ("c" "d") ("aa" "bb"))
+	  (org-table-collapse-header (org-table-to-lisp "|a|b|\n|A|B|\n|-\n|c|d|\n|aa|bb|") nil 1)))
+  ;; 2x4 with 2-line header, threshold 2 - collapsed
+  (should
+   (equal '(("a A" "b B") hline ("c" "d") ("aa" "bb"))
+	  (org-table-collapse-header (org-table-to-lisp "|a|b|\n|A|B|\n|-\n|c|d|\n|aa|bb|") nil 2)))
+  ;; 2x8 with 6-line header, default threshold 5 - not collapsed
+  (should
+   (equal '(("a" "b") ("A" "B") ("a" "b") ("A" "B") ("a" "b") ("A" "B") hline ("c" "d") ("aa" "bb"))
+	  (org-table-collapse-header (org-table-to-lisp "|a|b|\n|A|B|\n|a|b|\n|A|B|\n|a|b|\n|A|B|\n|-\n|c|d|\n|aa|bb|")))))
+
 ;;; Radio Tables
 
 (ert-deftest test-org-table/to-generic ()
diff --git a/testing/lisp/test-ox.el b/testing/lisp/test-ox.el
index 92ccec08e..e3b078a3c 100644
--- a/testing/lisp/test-ox.el
+++ b/testing/lisp/test-ox.el
@@ -4129,6 +4129,16 @@ Another text. (ref:text)
      (org-export-table-has-header-p
       (org-element-map tree 'table 'identity info 'first-match)
       info)))
+  ;; With an multi-line header.
+  (should
+   (org-test-with-parsed-data "
+| a | b |
+| 0 | 1 |
+|---+---|
+| a | w |"
+     (org-export-table-has-header-p
+      (org-element-map tree 'table 'identity info 'first-match)
+      info)))
   ;; Without an header.
   (should-not
    (org-test-with-parsed-data "
-- 
2.20.1

Re: [PATCH] allow for multiline headers

[Nicolas Goaziou]

Hello,

Mario Frasca <mario@anche.no> writes:

> On 13/06/2020 17:18, Nicolas Goaziou wrote:

>> Unit tests are not worth a formal definition. However, "test-ox.el"
>> contains unit tests.
>
> I'm not sure what you mean by the first sentence

I mean that, even though unit tests are great, and certainly welcome, we
first and foremost need a clear definition of a table header. This
concept is suggested in the manual, and more accurately defined in
ox.el. It would be nice to define it properly in the syntax.

> but I found the header tests in the test-ox.el file, and added one
> where multiple lines header is accepted.  is it related enough as to
> be included in my proposal?

Sure.

> I can move the org-table-collapse-header function from org-table.el to
> org-plot.el, but to me it makes little sense, relegating a generic
> function to a sub-module: others will look for the functionality in
> org-table, not see it, and duplicate the function somewhere else.

This doesn't seem to be a generic function, but a very specific one.
More on this below.

> +  (setq glue (or glue " "))
> +  (setq max-header-lines (or max-header-lines 4))

Please use `let' instead of `setq' whenever possible, e.g., here.

> +  (while (equal 'hline (car table))

equal -> eq

> +    (setq table (cdr table)))

(pop table)

> +  (let* ((trailer table)
> +	 (header-lines (cl-loop for line in table
> +				until (equal line 'hline)
> +				collect line
> +				do (setq trailer (cdr trailer)))))

You couldn't resist. 

This could be extracted as an independent function, which would return
the header, or nil. We can also imagine a function returning a cons cell
(HEADER . BODY), both HEADER and BODY being list of rows (possibly
empty).

Note that the function may be more complicated than the above, because,
in the following tables

    | a |
    | b |

or

    |---|
    | a |
    | b |
    |---|

there's a body, but no header. It depends on the definition chosen for
headers.

> +    (if (and trailer (<= (length header-lines) max-header-lines))
> +	(cons (apply #'cl-mapcar
> +		     #'(lambda (&rest x)

The #', aka, `function', is just noise before `lambda'.

   --> (lambda (&rest x))

> +			 (org-trim
> +			  (mapconcat #'identity x glue)))
> +		     header-lines)
> +	      trailer)
> +      table)))

Here you are going further than setting the definition for tables
headers. You imply that:

    | header line 1 |
    | header line 2 |
    |---------------|
    | body          |

is equivalent to

    | header line 1 header line 2 |
    |-----------------------------|
    | body                        |

I'm not so sure this is a good idea. It might be a good idea for Org
Plot, I don't know, but generally speaking, is it?

Regards,
-- 
Nicolas Goaziou

Re: [PATCH] allow for multiline headers

[Mario Frasca]

[-- Attachment #1: Type: text/plain, Size: 1611 bytes --]

Hi Nicolas and the whole world,

On 24/06/2020 02:19, Nicolas Goaziou wrote:
>
>>> This could be extracted as an independent function, which would return
>>> the header, or nil. We can also imagine a function returning a cons cell
>>> (HEADER . BODY), both HEADER and BODY being list of rows (possibly
>>> empty).
>> I was thinking of this myself too.  but, after all, the goal of this
>> function is not only to find the header, but to collapse it into
>> a single line.
> I suggested this because you were saying earlier in this thread IIRC
> that Org has no tooling to handle table headers.
I would like to discuss this in a chat, who's available to join 
#org-mode on freenode?
>> if it was splitting the header from the body, then yes, it would
>> definitely make sense, the cons cell you suggest.
> It _is_ splitting the header from the body. Barring initial `hline'
> symbols, header-lines and trailer variables are exactly HEADER and BODY
> above.
same as above, I wish to hear opinions, collect them, and that we take a 
decision with shorter communication lines.
>
>> +	   (table (org-table-collapse-header (org-table-to-lisp)))
>> +	   (num-cols (length (car table))))
> Note that there is no guarantee that all rows have the same length.
> E.g.,
>
>    | a |
>    | b | c |

many other points in the code assume rows have the same length. I 
haven't checked if the assumption is correct, I just used it as I saw 
the code already does.


I think I have processed most other remarks in the new patch.

and I have signed and received confirmation of reception of my FSF 
paperwork :-)

ciao,

Mario


[-- Attachment #2: 0001-lisp-org-table.el-Allow-collapsing-header-into-singl.patch --]
[-- Type: text/x-patch, Size: 7338 bytes --]

From ca92fb1e4ee66ed39e5b567880faccc513d263d4 Mon Sep 17 00:00:00 2001
From: mfrasca <mario@anche.no>
Date: Fri, 12 Jun 2020 11:42:34 -0500
Subject: [PATCH] lisp/org-table.el: Allow collapsing header into single line

* lisp/org-table.el (org-table-collapse-header): New function.

* lisp/org-plot.el (org-plot/gnuplot): Use org-table-collapse-header
and trust there will be no more leading `hline' symbols in lisp table.

* testing/lisp/test-org-table.el (test-org-table/to-lisp):
Adding tests to already existing to-lisp function.
(test-org-table/collapse-header): Adding tests to new
collapse-header function.

* testing/lisp/test-ox.el (test-org-export/has-header-p): Testing
exporting table with multi-line header.
---
 lisp/org-plot.el               |  8 ++---
 lisp/org-table.el              | 27 +++++++++++++++-
 testing/lisp/test-org-table.el | 58 ++++++++++++++++++++++++++++++++++
 testing/lisp/test-ox.el        | 10 ++++++
 4 files changed, 97 insertions(+), 6 deletions(-)

diff --git a/lisp/org-plot.el b/lisp/org-plot.el
index a23195d2a..35077cfc3 100644
--- a/lisp/org-plot.el
+++ b/lisp/org-plot.el
@@ -289,14 +289,12 @@ line directly before or after the table."
 	(setf params (plist-put params (car pair) (cdr pair)))))
     ;; collect table and table information
     (let* ((data-file (make-temp-file "org-plot"))
-	   (table (org-table-to-lisp))
-	   (num-cols (length (if (eq (nth 0 table) 'hline) (nth 1 table)
-			       (nth 0 table)))))
+	   (table (org-table-collapse-header (org-table-to-lisp)))
+	   (num-cols (length (car table))))
       (run-with-idle-timer 0.1 nil #'delete-file data-file)
-      (while (eq 'hline (car table)) (setf table (cdr table)))
       (when (eq (cadr table) 'hline)
 	(setf params
-	      (plist-put params :labels (nth 0 table))) ; headers to labels
+	      (plist-put params :labels (car table))) ; headers to labels
 	(setf table (delq 'hline (cdr table)))) ; clean non-data from table
       ;; Collect options.
       (save-excursion (while (and (equal 0 (forward-line -1))
diff --git a/lisp/org-table.el b/lisp/org-table.el
index 6462b99c4..248b1ed50 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -5458,6 +5458,31 @@ The table is taken from the parameter TXT, or from the buffer at point."
 	  (forward-line))
         (nreverse table)))))
 
+(defun org-table-collapse-header (table &optional separator max-header-lines)
+  "Collapse the lines before 'hline into a single header.
+
+The given TABLE is a list of lists as returned by `org-table-to-lisp'.
+The leading lines before the first `hline' symbol are considered
+forming the table header.  This function collapses all leading header
+lines into a single header line, followed by the `hline' symbol, and
+the rest of the TABLE.  Header cells are glued together with a space,
+or the given SEPARATOR."
+  (while (eq (car table) 'hline) (pop table))
+  (let* ((separator (or separator " "))
+	 (max-header-lines (or max-header-lines 4))
+	 (trailer table)
+	 (header-lines (cl-loop for line in table
+				until (eq 'hline line)
+				collect (pop trailer))))
+    (if (and trailer (<= (length header-lines) max-header-lines))
+	(cons (apply #'mapcar
+		     (lambda (&rest x)
+		       (org-trim
+			(mapconcat #'identity x separator)))
+		     header-lines)
+	      trailer)
+      table)))
+
 (defun orgtbl-send-table (&optional maybe)
   "Send a transformed version of table at point to the receiver position.
 With argument MAYBE, fail quietly if no transformation is defined
@@ -6139,7 +6164,7 @@ which will prompt for the width."
 	       ((numberp ask) ask)
 	       (t 12))))
     ;; Skip any hline a the top of table.
-    (while (eq (car table) 'hline) (setq table (cdr table)))
+    (while (eq (car table) 'hline) (pop table))
     ;; Skip table header if any.
     (dolist (x (or (cdr (memq 'hline table)) table))
       (when (consp x)
diff --git a/testing/lisp/test-org-table.el b/testing/lisp/test-org-table.el
index 64a1b4b16..5d54f4999 100644
--- a/testing/lisp/test-org-table.el
+++ b/testing/lisp/test-org-table.el
@@ -1304,6 +1304,64 @@ See also `test-org-table/copy-field'."
       (should (string= got
 		       expect)))))
 
+;;; the initial to lisp converter
+
+(ert-deftest test-org-table/to-lisp ()
+  "Test `orgtbl-to-lisp' specifications."
+  ;; 2x2 no header
+  (should
+   (equal '(("a" "b") ("c" "d"))
+	  (org-table-to-lisp "|a|b|\n|c|d|")))
+  ;; 2x2 with 1-line header
+  (should
+   (equal '(("a" "b") hline ("c" "d"))
+	  (org-table-to-lisp "|a|b|\n|-\n|c|d|")))
+  ;; 2x4 with 2-line header
+  (should
+   (equal '(("a" "b") ("A" "B") hline ("c" "d") ("aa" "bb"))
+	  (org-table-to-lisp "|a|b|\n|A|B|\n|-\n|c|d|\n|aa|bb|")))
+  ;; leading hlines do not get stripped
+  (should
+   (equal '(hline ("a" "b") hline ("c" "d"))
+	  (org-table-to-lisp "|-\n|a|b|\n|-\n|c|d|")))
+  (should
+   (equal '(hline ("a" "b") ("c" "d"))
+	  (org-table-to-lisp "|-\n|a|b|\n|c|d|")))
+  (should
+   (equal '(hline hline hline hline ("a" "b") ("c" "d"))
+	  (org-table-to-lisp "|-\n|-\n|-\n|-\n|a|b|\n|c|d|"))))
+
+(ert-deftest test-org-table/collapse-header ()
+  "Test `orgtbl-to-lisp' specifications."
+  ;; 2x2 no header - no collapsing
+  (should
+   (equal '(("a" "b") ("c" "d"))
+	  (org-table-collapse-header (org-table-to-lisp "|a|b|\n|c|d|"))))
+  ;; 2x2 with 1-line header - no collapsing
+  (should
+   (equal '(("a" "b") hline ("c" "d"))
+	  (org-table-collapse-header (org-table-to-lisp "|a|b|\n|-\n|c|d|"))))
+  ;; 2x4 with 2-line header - collapsed
+  (should
+   (equal '(("a A" "b B") hline ("c" "d") ("aa" "bb"))
+	  (org-table-collapse-header (org-table-to-lisp "|a|b|\n|A|B|\n|-\n|c|d|\n|aa|bb|"))))
+  ;; 2x4 with 2-line header, custom glue - collapsed
+  (should
+   (equal '(("a.A" "b.B") hline ("c" "d") ("aa" "bb"))
+	  (org-table-collapse-header (org-table-to-lisp "|a|b|\n|A|B|\n|-\n|c|d|\n|aa|bb|") ".")))
+  ;; 2x4 with 2-line header, threshold 1 - not collapsed
+  (should
+   (equal '(("a" "b") ("A" "B") hline ("c" "d") ("aa" "bb"))
+	  (org-table-collapse-header (org-table-to-lisp "|a|b|\n|A|B|\n|-\n|c|d|\n|aa|bb|") nil 1)))
+  ;; 2x4 with 2-line header, threshold 2 - collapsed
+  (should
+   (equal '(("a A" "b B") hline ("c" "d") ("aa" "bb"))
+	  (org-table-collapse-header (org-table-to-lisp "|a|b|\n|A|B|\n|-\n|c|d|\n|aa|bb|") nil 2)))
+  ;; 2x8 with 6-line header, default threshold 5 - not collapsed
+  (should
+   (equal '(("a" "b") ("A" "B") ("a" "b") ("A" "B") ("a" "b") ("A" "B") hline ("c" "d") ("aa" "bb"))
+	  (org-table-collapse-header (org-table-to-lisp "|a|b|\n|A|B|\n|a|b|\n|A|B|\n|a|b|\n|A|B|\n|-\n|c|d|\n|aa|bb|")))))
+
 ;;; Radio Tables
 
 (ert-deftest test-org-table/to-generic ()
diff --git a/testing/lisp/test-ox.el b/testing/lisp/test-ox.el
index 92ccec08e..a5b3bd770 100644
--- a/testing/lisp/test-ox.el
+++ b/testing/lisp/test-ox.el
@@ -4129,6 +4129,16 @@ Another text. (ref:text)
      (org-export-table-has-header-p
       (org-element-map tree 'table 'identity info 'first-match)
       info)))
+  ;; With a multi-line header.
+  (should
+   (org-test-with-parsed-data "
+| a | b |
+| 0 | 1 |
+|---+---|
+| a | w |"
+     (org-export-table-has-header-p
+      (org-element-map tree 'table 'identity info 'first-match)
+      info)))
   ;; Without an header.
   (should-not
    (org-test-with-parsed-data "
-- 
2.20.1

Re: [PATCH] allow for multiline headers

[Nicolas Goaziou]

Hello,

Mario Frasca <mario@anche.no> writes:

> On 24/06/2020 02:19, Nicolas Goaziou wrote:
>>
>>>> This could be extracted as an independent function, which would return
>>>> the header, or nil. We can also imagine a function returning a cons cell
>>>> (HEADER . BODY), both HEADER and BODY being list of rows (possibly
>>>> empty).
>>> I was thinking of this myself too.  but, after all, the goal of this
>>> function is not only to find the header, but to collapse it into
>>> a single line.
>> I suggested this because you were saying earlier in this thread IIRC
>> that Org has no tooling to handle table headers.
> I would like to discuss this in a chat, who's available to join
> #org-mode on freenode?

Note that, usually, discussions happen on this mailing list, so anyone
can comment, not only those available on the IRC channel at a given
time.

>>> if it was splitting the header from the body, then yes, it would
>>> definitely make sense, the cons cell you suggest.
>> It _is_ splitting the header from the body. Barring initial `hline'
>> symbols, header-lines and trailer variables are exactly HEADER and BODY
>> above.
> same as above, I wish to hear opinions, collect them, and that we take
> a decision with shorter communication lines.

I'm sorry if I'm not fast enough to answer. In any case, the above is
just a suggestion.

>>> +	   (table (org-table-collapse-header (org-table-to-lisp)))
>>> +	   (num-cols (length (car table))))
>> Note that there is no guarantee that all rows have the same length.
>> E.g.,
>>
>>    | a |
>>    | b | c |
>
> many other points in the code assume rows have the same length.

That's true. However, the above is still a correct table, so it is good
to keep it in mind. No worries if you don't want to take care of that
particular point.

> I think I have processed most other remarks in the new patch.

I applied your patch. Thank you.

Regards,
-- 
Nicolas Goaziou

Re: [PATCH] allow for multiline headers

[Mario Frasca]

On 28/06/2020 18:17, Nicolas Goaziou wrote:
> This could be extracted as an independent function, which would return
> the header, or nil. We can also imagine a function returning a cons cell
> (HEADER . BODY), both HEADER and BODY being list of rows (possibly
> empty).

not only this … it could also be like (:header HEADER :body BODY :body 
BODY), that is, repeating block.

I don't know, just doubting how to be more general.

in practice, I'm thinking of the case when I have a header, a body, and 
a summary, and I'm only interested in the header and body, not the summary.

just scattered thoughts … who fills in?

Re: [PATCH] allow for multiline headers

[Nicolas Goaziou]

Hello,

Mario Frasca <mario@anche.no> writes:

> not only this … it could also be like (:header HEADER :body BODY :body
> BODY), that is, repeating block.

I'm not sure about what you mean.

> I don't know, just doubting how to be more general.
>
> in practice, I'm thinking of the case when I have a header, a body,
> and a summary, and I'm only interested in the header and body, not the
> summary.
>
> just scattered thoughts … who fills in?

There is no such thing as a "table summary" in Org syntax.

Regards,
-- 
Nicolas Goaziou

Re: [PATCH] allow for multiline headers

[Mario Frasca]

Hi Nicolas,

sorry for not explaining it clearly.  see, I'm considering from the 
point of view of a user, who organizes a table in terms of logical 
units, like a header, one or more body blocks, and a summary line.  
these concepts are not yet all supported by the org-table code, and 
again, I'm not considering what the code implements, just looking at the 
table from the user's point of view.

again, my idea is that we're describing something new, and I'm wondering 
what lisp construct could be used to describe a table like this one:

| h1 | h2 | h3 |
|----+----+----|
| 11 | 12 | 13 |
| 21 | 22 | 23 |
| 31 | 32 | 33 |
|----+----+----|
| a1 | a2 | a3 |
| b1 | b2 | b3 |
|----+----+----|
|  5 |  8 |  9 |

and my guess in the above case would be '(:header ("h1" "h2" "h3") :body 
((11 12 13)(21 22 23)(31 32 33)) :body ((a1 a2 a3)(b1 b2 b3)) :body ((5 
8 9)))

with a option for collapsing all :body parts into one, like in '(:header 
("h1" "h2" "h3") :body ((11 12 13)(21 22 23)(31 32 33)(a1 a2 a3)(b1 b2 
b3)(5 8 9)))

with a option for separating the last row (if the one-but-last is a 
'hline) as a summary, like in '(:header ("h1" "h2" "h3") :body ((11 12 
13)(21 22 23)(31 32 33)) :body ((a1 a2 a3)(b1 b2 b3)) :summary (5 8 9))

or using both options, like in '(:header ("h1" "h2" "h3") :body ((11 12 
13)(21 22 23)(31 32 33)(a1 a2 a3)(b1 b2 b3)) :summary (5 8 9))

your suggestion as a cons cell would only cover the first case, as in 
'(("h1" "h2" "h3") . ((11 12 13)(21 22 23)(31 32 33)(a1 a2 a3)(b1 b2 
b3)(5 8 9)))

in my very brief experience with org-tables, mostly from the point of 
view of org-plot, I think this (separating the bottom "summary" line) 
would be useful (I had to remove the "country summary" bottom line from 
a demographics histogram table).  for sure, if I follow your hint to use 
gnuplot code and refer to the table, then org-plot can stay as it is 
now.  in fact it would not need any maintenance, would it?

cheers,

Mario

On 29/06/2020 07:50, Nicolas Goaziou wrote:
> Hello,
>
> Mario Frasca <mario@anche.no> writes:
>
>> not only this … it could also be like (:header HEADER :body BODY :body
>> BODY), that is, repeating block.
> I'm not sure about what you mean.
>
>> I don't know, just doubting how to be more general.
>>
>> in practice, I'm thinking of the case when I have a header, a body,
>> and a summary, and I'm only interested in the header and body, not the
>> summary.
>>
>> just scattered thoughts … who fills in?
> There is no such thing as a "table summary" in Org syntax.
>
> Regards,

Re: [PATCH] allow for multiline headers

[Nicolas Goaziou]

Mario Frasca <mario@anche.no> writes:

> again, my idea is that we're describing something new, and I'm
> wondering what lisp construct could be used to describe a table like
> this one:
>
> | h1 | h2 | h3 |
> |----+----+----|
> | 11 | 12 | 13 |
> | 21 | 22 | 23 |
> | 31 | 32 | 33 |
> |----+----+----|
> | a1 | a2 | a3 |
> | b1 | b2 | b3 |
> |----+----+----|
> |  5 |  8 |  9 |
>
> and my guess in the above case would be '(:header ("h1" "h2"
> "h3") :body ((11 12 13)(21 22 23)(31 32 33)) :body ((a1 a2 a3)(b1 b2
> b3)) :body ((5 8 9)))

These are called "row groups" in ox.el, but I don't think we should
parse them like the above. It is easy enough to get this kind of result
just walking through the table.

> with a option for collapsing all :body parts into one, like in
> '(:header ("h1" "h2" "h3") :body ((11 12 13)(21 22 23)(31 32 33)(a1 a2
> a3)(b1 b2 b3)(5 8 9)))
>
> with a option for separating the last row (if the one-but-last is
> a 'hline) as a summary, like in '(:header ("h1" "h2" "h3") :body ((11
> 12 13)(21 22 23)(31 32 33)) :body ((a1 a2 a3)(b1 b2 b3)) :summary (5
> 8 9))
>
> or using both options, like in '(:header ("h1" "h2" "h3") :body ((11
> 12 13)(21 22 23)(31 32 33)(a1 a2 a3)(b1 b2 b3)) :summary (5 8 9))
>
> your suggestion as a cons cell would only cover the first case, as in
> '(("h1" "h2" "h3") . ((11 12 13)(21 22 23)(31 32 33)(a1 a2 a3)(b1 b2
> b3)(5 8 9)))

You're missing `hline' symbols in the body. I didn't suggest to remove
them, except for the separator between the header and the body, and
trailing separators.

> in my very brief experience with org-tables, mostly from the point of
> view of org-plot, I think this (separating the bottom "summary" line)
> would be useful (I had to remove the "country summary" bottom line
> from a demographics histogram table).  for sure, if I follow your hint
> to use gnuplot code and refer to the table, then org-plot can stay as
> it is now.  in fact it would not need any maintenance, would it?

You can indeed extract a specific part of the table as an input of
a gnuplot code block. IMO, Org Plot should be very basic, and complexity
should go into Babel code blocks, which are more capable.

Regards,

Re: [PATCH] allow for multiline headers

[Mario Frasca]

On 29/06/2020 13:36, Nicolas Goaziou wrote:
> IMO, Org Plot should be very basic, and complexity
> should go into Babel code blocks, which are more capable.

IMO, we are keeping org-plot artificially basic.  But you know what?  
I'm just happy with my additions, my point was to share them with the 
wider public.  I can also keep my own branch in my own repository.  I'm 
fine either way.  When solving conflicts will become a burden, I'll 
write again to the list.

I haven't yet delved into Babel code blocks.  I would need to write 
code, isn't it?  Now I just need defining settings.  … sounds easier, so 
I'll keep it this way, here on my PC at least.

anyhow, cheers, Mario.

Re: [PATCH] allow for multiline headers

[Nicolas Goaziou]

Hello,

Mario Frasca <mario@anche.no> writes:

> IMO, we are keeping org-plot artificially basic.

Exactly. 

There are multiple ways to draw a diagram from a table. One could use
gnuplot, or asymptote, or R, or, matplotlib… From a design point of
view, there is no particular reason to favor one way or the other, and
I don't see how Org Plot could fit to handle all of these. OTOH, Babel
source code blocks handle all of them.

I didn't check but I think Org Plot predates Org Babel. It probably
wouldn't exist otherwise.

> But you know what? 
> I'm just happy with my additions, my point was to share them with the
> wider public.  I can also keep my own branch in my own repository. 
> I'm fine either way.  When solving conflicts will become a burden,
> I'll write again to the list.
>
> I haven't yet delved into Babel code blocks.  I would need to write
> code, isn't it?  Now I just need defining settings.  … sounds easier,
> so I'll keep it this way, here on my PC at least.

I don't know if there's much difference between writing multiple PLOT
lines and writing a gnuplot script. It may be interesting to compare use
cases.

Also, note that when you suggest something like PLOT[0] and PLOT[1], you
are effectively suggesting to write code, with conditionals.

Anyhow, I think it is more future-proof to use, and possibly improve
"ob-gnuplot.el" so it is easier to use, than to develop PLOT keywords.

WDYT?

Regards,
-- 
Nicolas Goaziou

Re: [PATCH] allow for multiline headers

[Mario Frasca]

Good morning Nicolas,

On 01/07/2020 05:46, Nicolas Goaziou wrote:
> There are multiple ways to draw a diagram from a table. […]
>
> I didn't check but I think Org Plot predates Org Babel. It probably
> wouldn't exist otherwise.

I'm happy it exists.  it has a very low entrance threshold, just use it 
and tweak it bit by bit.  the other methods, apart from gnuplot itself, 
are out of reach on my system, a Maemo-freemantle N900.  it's not my 
original system, just a recent replacement, and I had some trouble to 
find all software I was used to.


> I don't know if there's much difference between writing multiple PLOT
> lines and writing a gnuplot script. It may be interesting to compare use
> cases.

it's data about the pandemic that I'm collecting, http://ix.io/2qDf.  as 
said, I have never used ob-gnuplot, so I would not know how would the 
equivalent code look like.

I wanted to comment out a couple of lines, then switch them back again, 
and it was easier to do it with the new `use´ keyword.

regards, MF

Re: [PATCH] allow for multiline headers

[Nicolas Goaziou]

Hello,

Mario Frasca <mario@anche.no> writes:

> I'm happy it exists.  it has a very low entrance threshold, just use
> it and tweak it bit by bit.  the other methods, apart from gnuplot
> itself, are out of reach on my system, a Maemo-freemantle N900.  it's
> not my original system, just a recent replacement, and I had some
> trouble to find all software I was used to.
>
>
>> I don't know if there's much difference between writing multiple PLOT
>> lines and writing a gnuplot script. It may be interesting to compare use
>> cases.
>
> it's data about the pandemic that I'm collecting, http://ix.io/2qDf. 
> as said, I have never used ob-gnuplot, so I would not know how would
> the equivalent code look like.
>
> I wanted to comment out a couple of lines, then switch them back
> again, and it was easier to do it with the new `use´ keyword.

I'm sure this PLOT extension is useful. 

My point is that complexity should be moved to Org Babel, if possible,
not into affiliated keywords. IOW, it would be nice if gnuplot source
code blocks were as easy to use as your extension.

It already might be as simple, or there may be some work to do. I'd
appreciate some enlightenment on the topic. For example, could you show
what code your extension generates with the table above?

Regards,
-- 
Nicolas Goaziou

Re: [PATCH] allow for multiline headers

[Mario Frasca]

Hi Nicolas,

On 04/07/2020 03:58, Nicolas Goaziou wrote:
> I'm sure this PLOT extension is useful.
>
> My point is that complexity should be moved to Org Babel, if possible,
> not into affiliated keywords. IOW, it would be nice if gnuplot source
> code blocks were as easy to use as your extension.

I do not use org-babel, and I do not plan to move to it.  I'm happy with 
my extension, but I do see that there's no positive reaction from users, 
only the group of reviewers who doesn't like the idea.  as said, let's 
just keep it this way.


> It already might be as simple, or there may be some work to do. I'd
> appreciate some enlightenment on the topic. For example, could you show
> what code your extension generates with the table above?

your above request suggests me you're missing the point.  there is 
nothing to be seen in the generated gnuplot source, because all is 
solved in lisp.  or it's me missing your point, that can be, too.

anyhow, these are a couple of webpastes for you:

this is the complete gnuplot session, showing the code produced with 
`use:1' and then with `use:2', given the table I already shared.

http://ix.io/2qRq

what will show you the logic is the lisp code, which I thought I already 
shared, but I guess I did not insist up to the latest version.  anyhow, 
here are a couple of snippets, partial or complete rewrites of 
org-plot.el code.

http://ix.io/2qRs/elisp

http://ix.io/2qRt/elisp

http://ix.io/2qRu

best regards,

MF