Quoting formula "cookies" in table?

Quoting formula "cookies" in table?

[Achim Gratz]

I seem to have hit a snag with tables:  I didn't find a way to quote any
of the special characters that are recognized to make formulas.  Tabbing
through such cells will create a malformed #+TBLFM: line at the end of
the table.  I can clean this up no problem, but on export the "formulas"
produce interesting results again.  Depending on the content of other
cells some lines will be moved out in front of the table.  Try HTML
subtree export on this test:

* Table Test

  |-------------+----|
  | unrelated 1 | >= |
  | Test1       | =  |
  | unrelated 2 | <= |
  | Test2       | == |
  | unrelated 3 | -= |
  | Test3       | := |
  | unrelated 4 | >= |
  | Test4       | =  |
  | unrelated 5 | <= |
  | Test5       | == |
  | unrelated 6 | -= |
  | Test6       | := |
  |-------------+----|

  |-------------+----|
  | unrelated 1 | >= |
  | Test1       | =  |
  | unrelated 2 | <= |
  | Test2       | == |
  | unrelated 4 | >= |
  | Test4       | =  |
  | unrelated 5 | <= |
  | Test5       | == |
  | unrelated 6 | -= |
  |-------------+----|

  |-------------+----|
  | unrelated 1 | XX |
  | Test1       | =  |
  | unrelated 2 | XX |
  | Test2       | == |
  | unrelated 3 | >= |
  | Test3       | := |
  | unrelated 4 | XX |
  | Test4       | =  |
  | unrelated 5 | XX |
  | Test5       | == |
  | unrelated 6 | XX |
  | Test6       | := |
  |-------------+----|

So is there a way to quote these special chars in an unobtrusive way
that cleanly exports to all backends?  If not I'd suggest to have a
special table form for "text-only" tables where these characters aren't
special (no, I don't have an idea off-hand what to use - @ in place of
the first | maybe?


Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: Quoting formula "cookies" in table?

[Carsten Dominik]

Hi Achim,

On Oct 4, 2010, at 7:46 PM, Achim Gratz wrote:

>
> I seem to have hit a snag with tables:  I didn't find a way to quote  
> any
> of the special characters that are recognized to make formulas.   
> Tabbing
> through such cells will create a malformed #+TBLFM: line at the end of
> the table.  I can clean this up no problem, but on export the  
> "formulas"
> produce interesting results again.  Depending on the content of other
> cells some lines will be moved out in front of the table.  Try HTML
> subtree export on this test:
>
> * Table Test
>
>  |-------------+----|
>  | unrelated 1 | >= |
>  | Test1       | =  |
>  | unrelated 2 | <= |
>  | Test2       | == |
>  | unrelated 3 | -= |
>  | Test3       | := |
>  | unrelated 4 | >= |
>  | Test4       | =  |
>  | unrelated 5 | <= |
>  | Test5       | == |
>  | unrelated 6 | -= |
>  | Test6       | := |
>  |-------------+----|


The best way to deal with this is:

* Table Test

  |-------------+------|
  | unrelated 1 | ~>=~ |
  | Test1       | ~=~  |
  | unrelated 2 | ~<=~ |
  | Test2       | ~==~ |
  | unrelated 3 | ~-=~ |
  | Test3       | ~:=~ |
  | unrelated 4 | ~>=~ |
  | Test4       | ~=~  |
  | unrelated 5 | ~<=~ |
  | Test5       | ~==~ |
  | unrelated 6 | ~-=~ |
  | Test6       | ~:=~ |
  |-------------+------|


etc

HTH

- Carsten

>
>  |-------------+----|
>  | unrelated 1 | >= |
>  | Test1       | =  |
>  | unrelated 2 | <= |
>  | Test2       | == |
>  | unrelated 4 | >= |
>  | Test4       | =  |
>  | unrelated 5 | <= |
>  | Test5       | == |
>  | unrelated 6 | -= |
>  |-------------+----|
>
>  |-------------+----|
>  | unrelated 1 | XX |
>  | Test1       | =  |
>  | unrelated 2 | XX |
>  | Test2       | == |
>  | unrelated 3 | >= |
>  | Test3       | := |
>  | unrelated 4 | XX |
>  | Test4       | =  |
>  | unrelated 5 | XX |
>  | Test5       | == |
>  | unrelated 6 | XX |
>  | Test6       | := |
>  |-------------+----|
>
> So is there a way to quote these special chars in an unobtrusive way
> that cleanly exports to all backends?  If not I'd suggest to have a
> special table form for "text-only" tables where these characters  
> aren't
> special (no, I don't have an idea off-hand what to use - @ in place of
> the first | maybe?
>
>
> Achim.
> -- 
> +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+
>
> SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
> http://Synth.Stromeko.net/Downloads.html#KorgSDada
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Quoting formula "cookies" in table?

[Achim Gratz]

Carsten Dominik <carsten.dominik@gmail.com> writes:
[...]
> The best way to deal with this is:
>
> * Table Test
>
>  |-------------+------|
>  | unrelated 1 | ~>=~ |
>  | Test1       | ~=~  |
[...]

Ah, thank you -- I've settled for *bold* instead since that is a bit
more pleasing in the HTML output.  Now that I export the file to HTML
nobody gets to see the org file anymore, so while it looks a bit clumsy
in the source I can live with that.

It would still be nice if there was a way to quote without changing the
output or to just disable the formula mechanism entirely for a table,
but I recognize that these things are very deeply embedded into the
table code and not so easy to change...


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: Re: Quoting formula "cookies" in table?

[Carsten Dominik]

On Oct 5, 2010, at 7:10 PM, Achim Gratz wrote:

> Carsten Dominik <carsten.dominik@gmail.com> writes:
> [...]
>> The best way to deal with this is:
>>
>> * Table Test
>>
>> |-------------+------|
>> | unrelated 1 | ~>=~ |
>> | Test1       | ~=~  |
> [...]
>
> Ah, thank you -- I've settled for *bold* instead since that is a bit
> more pleasing in the HTML output.  Now that I export the file to HTML
> nobody gets to see the org file anymore, so while it looks a bit  
> clumsy
> in the source I can live with that.
>
> It would still be nice if there was a way to quote without changing  
> the
> output or to just disable the formula mechanism entirely for a table,
> but I recognize that these things are very deeply embedded into the
> table code and not so easy to change...

Don't forget that this is Org-mode, so pretty much everything can be  
changed.

:-)

     (setq org-table-formula-evaluate-inline nil)

might do this - I am not completely sure....


- Carsten

[Feature Request] Cross headings in tables

[Achim Gratz]

Now that I've started exporting some org files (to HTML mostly), I run
into some limitations here and there.  I've worked around most of them,
but have not been able to do anything about this (other than editing the
resulting HTML):

I have some tables where I used (without thinking much about it)
cross headings like this:

|----------+---------------------+-----------------|
| Header   | some more           | and more        |
| actually | multi-line headings | just work!      |
|----------+---------------------+-----------------|
| item     | stuff               | things          |
| etc.     | pp.                 | ad nauseam      |
|----------+---------------------+-----------------|
| Header   | to keep             | things together |
|----------+---------------------+-----------------|
| ...      |                     |                 |
|          | ...                 |                 |
|          |                     | ...             |
|----------+---------------------+-----------------|

I don't want to rip these apart into separate tables since that causes
the different parts to all have different widths (both in the source and
the export) and I really need these to line up and be just a single
table.

I guess that I could somehow manage to use the "splicing" of tables to
get the HTML export done, but I'd have that problem again for each
export backend.  Also, this would interrupt the table in the source and
make a formatting mess unless I'd define the column width for each of
the table parts by hand (which is equally undesirable).

So I'd like to have first-class cross headings, maybe like this:

|----------+-----------+-----------------|
| Header   | some more | and more        |
|----------+-----------+-----------------|
| item     | stuff     | things          |
| etc.     | pp.       | ad nauseam      |
|----------+-----------+-----------------|
|> Header  | to keep   | things together |
|----------+-----------+-----------------|
|          | ...       |                 |
|----------+-----------+-----------------|

So, the first heading would be determined by the first horizontal
separator inside the table (for backwards compatibility) and any
following heading would need get some special syntax (like the "|>"
above, but anything that doesn't collide with existing syntax will just
be fine I think).  If there's a heading marker before the first
horizontal separation, it should probably take precedence over the
backwards-compatible markup.  That would also enable to have table
headings without a separator, something that's not possible today.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: Quoting formula "cookies" in table?

[Achim Gratz]

Carsten Dominik <carsten.dominik@gmail.com> writes:
>     (setq org-table-formula-evaluate-inline nil)
>
> might do this - I am not completely sure....

Unfortunately it doesn't.  For starters I seem to need to set this
globally (tried to make it buffer-local, but that didn't work).  I can
then tab through the table without getting funny #+TBLFM lines, but any
function that tries to re-calculate formulas still barfs.  Somewhere in
the HTML export function this still happens (not quite sure where), so
the export result is again mangled.

While rummaging around in the source files I've probably found a dead
function: org-export-splice-style is defined in org-html, but I can find
no calls to it.


Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf Blofeld V1.15B11:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [Feature Request] Cross headings in tables

[Achim Gratz]

[-- Attachment #1: Type: text/plain, Size: 4325 bytes --]

Achim Gratz <Stromeko@nexgo.de> writes:
[...]
> So I'd like to have first-class cross headings, maybe like this:
>
> |----------+-----------+-----------------|
> | Header   | some more | and more        |
> |----------+-----------+-----------------|
> | item     | stuff     | things          |
> | etc.     | pp.       | ad nauseam      |
> |----------+-----------+-----------------|
> |> Header  | to keep   | things together |
> |----------+-----------+-----------------|
> |          | ...       |                 |
> |----------+-----------+-----------------|
>
> So, the first heading would be determined by the first horizontal
> separator inside the table (for backwards compatibility) and any
> following heading would need get some special syntax (like the "|>"
> above, but anything that doesn't collide with existing syntax will
> just be fine I think).  If there's a heading marker before the first
> horizontal separation, it should probably take precedence over the
> backwards-compatible markup.  That would also enable to have table
> headings without a separator, something that's not possible today.

I've spent some time on this and have developed a patch that gets
halfway there.  You can have consecutive headers, headers inside the
table and even headers at the end of the table:

--8<---------------cut here---------------start------------->8---
  |-------------+
  | unrelated 1 |
  |~~~~~~~~~~~~~|
  | Test1       |
  |-------------+
  | unrelated 2 |
  | Test2       |
  | unrelated 3 |
  |~~~~~~~~~~~~~|
  | Test3       |
  |-------------+
  | unrelated 4 |
  | Test4       |
  | unrelated 5 |
  | Test5       |
  |~~~~~~~~~~~~~|
  | unrelated 6 |
  | Test6       |
  |-------------+
--8<---------------cut here---------------end--------------->8---

The first header is still determined like it always was.  Headers inside
table need to get a special "hline", the choice of "~" for this was
dictated by most of the other characters already being used for various
markup inside or outside tables.  When I say "halfway there", I mean
that the export is working and the lines are recognized as hlines
everywhere I could find (there may still be some regexpressions floating
around that don't).  However, aligning tables will replace the wigglies
with plain dashes since I have not yet found a way to inject the correct
character for the currently hardcoded "-".  The HTML export for the
above table looks like this:

--8<---------------cut here---------------start------------->8---
<table border="2" cellspacing="0" cellpadding="6" rules="groups" frame="hsides">
<caption></caption>
<colgroup><col class="left" /><col class="left" />
</colgroup>
<thead>
<tr><th scope="col" class="left">unrelated 1</th><th scope="col" class="left">&gt;=</th></tr>
</thead>
<thead>
<tr><th scope="col" class="left">Test1</th><th scope="col" class="left">=</th></tr>
</thead>
<tbody>
<tr><td class="left">unrelated 2</td><td class="left">&lt;=</td></tr>
<tr><td class="left">Test2</td><td class="left">==</td></tr>
<tr><td class="left">unrelated 3</td><td class="left">-=</td></tr>
</tbody>
<thead>
<tr><th scope="col" class="left">Test3</th><th scope="col" class="left">:=</th></tr>
</thead>
<tbody>
<tr><td class="left">unrelated 4</td><td class="left">&gt;=</td></tr>
<tr><td class="left">Test4</td><td class="left">=</td></tr>
<tr><td class="left">unrelated 5</td><td class="left">&lt;=</td></tr>
<tr><td class="left">Test5</td><td class="left">==</td></tr>
</tbody>
<thead>
<tr><th scope="col" class="left">unrelated 6</th><th scope="col" class="left">-=</th></tr>
<tr><th scope="col" class="left">Test6</th><th scope="col" class="left">:=</th></tr>
</thead>
</table>
--8<---------------cut here---------------end--------------->8---


And ASCII:
--8<---------------cut here---------------start------------->8---
    unrelated 1 
   -------------
    Test1       
   -------------
    unrelated 2 
    Test2       
    unrelated 3 
   -------------
    Test3       
   -------------
    unrelated 4 
    Test4       
    unrelated 5 
    Test5       
   -------------
    unrelated 6 
    Test6       
--8<---------------cut here---------------end--------------->8---

If somebody has an idea how to make the table alignment work, please
lend me a hand.  Experimental patch is attached, comments are 
welcome.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Table-Headings --]
[-- Type: text/x-patch, Size: 7699 bytes --]

From 0fd4e39641ab17ae1586747396acbe1e9fa48321 Mon Sep 17 00:00:00 2001
From: Achim Gratz <Stromeko@Stromeko.DE>
Date: Sun, 16 Jan 2011 19:06:13 +0100
Subject: [PATCH] Allow headings inside tables without splicing them.

*EXPERIMENTAL*

This patch is an incomplete implementation, most notably,
table (re-)alignment does not work.
---
 lisp/org-html.el  |   15 +++++++++++++--
 lisp/org-table.el |   30 +++++++++++++++---------------
 lisp/org.el       |    4 ++--
 3 files changed, 30 insertions(+), 19 deletions(-)

diff --git a/lisp/org-html.el b/lisp/org-html.el
index 9a5d225..d69d037 100644
--- a/lisp/org-html.el
+++ b/lisp/org-html.el
@@ -1899,7 +1899,7 @@ for formatting.  This is required for the DocBook exporter."
 			  html-table-tag attributes))
 	 (head (and org-export-highlight-first-table-line
 		    (delq nil (mapcar
-			       (lambda (x) (string-match "^[ \t]*|-" x))
+			       (lambda (x) (string-match "^[ \t]*|[-~]" x))
 			       (cdr lines)))))
 	 (nline 0) fnum nfields i (cnt 0)
 	 tbopen line fields html gr colgropen rowstart rowend
@@ -1913,7 +1913,7 @@ for formatting.  This is required for the DocBook exporter."
     (setq tbopen t)
     (while (setq line (pop lines))
       (catch 'next-line
-	(if (string-match "^[ \t]*|-" line)
+	(if (string-match "^[ \t]*|[-]" line)
 	    (progn
 	      (unless splice
 		(push (if head "</thead>" "</tbody>") html)
@@ -1921,6 +1921,16 @@ for formatting.  This is required for the DocBook exporter."
 	      (setq head nil)   ;; head ends here, first time around
 	      ;; ignore this line
 	      (throw 'next-line t)))
+	; interspersed heading
+	(if (string-match "^[ \t]*|[~]" line)
+	    (progn
+	      (unless splice
+		(push (if head "</thead>" "</tbody>") html)
+		(push "<thead>" html)
+		(setq tbopen nil))
+	      (setq head t) ; head starts again here, until the next |- hline
+	      ;; ignore this line
+	      (throw 'next-line t)))
 	;; Break the line into fields
 	(setq fields (org-split-string line "[ \t]*|[ \t]*"))
 	(unless fnum (setq fnum (make-vector (length fields) 0)
@@ -1955,6 +1965,7 @@ for formatting.  This is required for the DocBook exporter."
 		       fields "")
 		      rowend)
 	      html)))
+    (unless splice (if head (push "</thead>" html)))
     (unless splice (if tbopen (push "</tbody>" html)))
     (unless splice (push "</table>\n" html))
     (setq html (nreverse html))
diff --git a/lisp/org-table.el b/lisp/org-table.el
index 951bff7..9437ae1 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -567,7 +567,7 @@ property, locally or anywhere up in the hierarchy."
 			 (if (string-match org-table-hline-regexp x)
 			     'hline
 			   (org-remove-by-index
-			    (org-split-string (org-trim x) "\\s-*|\\s-*")
+			    (org-split-string (org-trim x) "\\s[-~]*|\\s[-~]*")
 			    skipcols i0)))
 		       lines))
 	       (fun (if (= i0 2) 'cdr 'identity))
@@ -680,7 +680,7 @@ When nil, simply write \"#ERROR\" in corrupted fields.")
     ;; Mark the hlines by setting the corresponding element to nil
     ;; At the same time, we remove trailing space.
     (setq lines (mapcar (lambda (l)
-			  (if (string-match "^ *|-" l)
+			  (if (string-match "^ *|[-~]" l)
 			      nil
 			    (if (string-match "[ \t]+$" l)
 				(substring l 0 (match-beginning 0))
@@ -897,9 +897,9 @@ Before doing so, re-align the table if necessary."
 	      (re-search-forward "|" end))
 	  (if (and (looking-at "-")
 		   org-table-tab-jumps-over-hlines
-		   (re-search-forward "^[ \t]*|\\([^-]\\)" end t))
+		   (re-search-forward "^[ \t]*|\\([^-~]\\)" end t))
 	      (goto-char (match-beginning 1)))
-	  (if (looking-at "-")
+	  (if (looking-at "[-~]")
 	      (progn
 		(beginning-of-line 0)
 		(org-table-insert-row 'below))
@@ -1417,7 +1417,7 @@ With prefix ABOVE, insert above the current line."
     (org-table-maybe-recalculate-line)
     (org-table-insert-hline)
     (end-of-line 2)
-    (if (looking-at "\n[ \t]*|-")
+    (if (looking-at "\n[ \t]*|[-~]")
 	(progn (insert "\n|") (org-table-align))
       (org-table-next-field))
     (if same-column (org-table-goto-column col))))
@@ -1425,7 +1425,7 @@ With prefix ABOVE, insert above the current line."
 (defun org-table-clean-line (s)
   "Convert a table line S into a string with only \"|\" and space.
 In particular, this does handle wide and invisible characters."
-  (if (string-match "^[ \t]*|-" s)
+  (if (string-match "^[ \t]*|[-~]" s)
       ;; It's a hline, just map the characters
       (setq s (mapconcat (lambda (x) (if (member x '(?| ?+)) "|" " ")) s ""))
     (while (string-match "|\\([ \t]*?[^ \t\r\n|][^\r\n|]*\\)|" s)
@@ -1630,7 +1630,7 @@ blindly applies a recipe that works for simple tables."
 	      (end (move-marker (make-marker) (org-table-end))))
 	  ;; first, get rid of all horizontal lines
 	  (goto-char beg)
-	  (while (re-search-forward "^\\([ \t]*\\)|-.*\n" end t)
+	  (while (re-search-forward "^\\([ \t]*\\)|[-~].*\n" end t)
 	    (replace-match ""))
 	  ;; insert a hline before first
 	  (goto-char beg)
@@ -1642,11 +1642,11 @@ blindly applies a recipe that works for simple tables."
 	  (goto-char beg)
 	  (setq end (move-marker end (org-table-end)))
 	  ;; replace "+" at beginning and ending of hlines
-	  (while (re-search-forward "^\\([ \t]*\\)|-" end t)
-	    (replace-match "\\1+-"))
+	  (while (re-search-forward "^\\([ \t]*\\)|\\([-~]\\)" end t)
+	    (replace-match "\\1+\\2"))
 	  (goto-char beg)
-	  (while (re-search-forward "-|[ \t]*$" end t)
-	    (replace-match "-+"))
+	  (while (re-search-forward "\\([-~]\\)|[ \t]*$" end t)
+	    (replace-match "\\1+"))
 	  (goto-char beg)))))
 
 (defun org-table-wrap-region (arg)
@@ -1801,12 +1801,12 @@ If NLAST is a number, only the NLAST fields will actually be summed."
        (t
 	(setq col (org-table-current-column))
 	(goto-char (org-table-begin))
-	(unless (re-search-forward "^[ \t]*|[^-]" nil t)
+	(unless (re-search-forward "^[ \t]*|[^-~]" nil t)
 	  (error "No table data"))
 	(org-table-goto-column col)
 	(setq beg (point))
 	(goto-char (org-table-end))
-	(unless (re-search-backward "^[ \t]*|[^-]" nil t)
+	(unless (re-search-backward "^[ \t]*|[^-~]" nil t)
 	  (error "No table data"))
 	(org-table-goto-column col)
 	(setq end (point))))
@@ -2067,7 +2067,7 @@ For all numbers larger than LIMIT, shift them by DELTA."
       (setq org-table-current-begin-line (org-current-line)
 	    org-table-current-begin-pos (point)
 	    l org-table-current-begin-line)
-      (while (looking-at "[ \t]*|\\(-\\)?")
+      (while (looking-at "[ \t]*|\\([-~]\\)?")
 	(push (if (match-end 1) 'hline 'dline) types)
 	(if (match-end 1) (push l hlines) (push l dlines))
 	(beginning-of-line 2)
@@ -2142,7 +2142,7 @@ of the new mark."
 	 (have-col
 	  (save-excursion
 	    (goto-char beg)
-	    (not (re-search-forward "^[ \t]*|[^-|][^|]*[^#!$*_^| \t][^|]*|" end t))))
+	    (not (re-search-forward "^[ \t]*|[^-~|][^|]*[^#!$*_^| \t][^|]*|" end t))))
 	 (col (org-table-current-column))
 	 (forcenew (car (assoc newchar org-recalc-marks)))
 	 epos new)
diff --git a/lisp/org.el b/lisp/org.el
index 6e8aeac..5721b53 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -3599,9 +3599,9 @@ Normal means, no org-mode-specific context."
   "Detect an org-type or table-type table.")
 (defconst org-table-line-regexp "^[ \t]*|"
   "Detect an org-type table line.")
-(defconst org-table-dataline-regexp "^[ \t]*|[^-]"
+(defconst org-table-dataline-regexp "^[ \t]*|[^-~]"
   "Detect an org-type table line.")
-(defconst org-table-hline-regexp "^[ \t]*|-"
+(defconst org-table-hline-regexp "^[ \t]*|[-~]"
   "Detect an org-type table hline.")
 (defconst org-table1-hline-regexp "^[ \t]*\\+-[-+]"
   "Detect a table-type table hline.")
-- 
1.7.1


[-- Attachment #3: Type: text/plain, Size: 190 bytes --]



Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

[-- Attachment #4: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: [Feature Request] Cross headings in tables

[Lawrence Mitchell]

Achim Gratz wrote:

[...]

> The first header is still determined like it always was.  Headers inside
> table need to get a special "hline", the choice of "~" for this was
> dictated by most of the other characters already being used for various
> markup inside or outside tables.  When I say "halfway there", I mean
> that the export is working and the lines are recognized as hlines
> everywhere I could find (there may still be some regexpressions floating
> around that don't).  However, aligning tables will replace the wigglies
> with plain dashes since I have not yet found a way to inject the correct
> character for the currently hardcoded "-".  The HTML export for the
> above table looks like this:

[...]

> If somebody has an idea how to make the table alignment work, please
> lend me a hand.  Experimental patch is attached, comments are
> welcome.

How about the following two patches on top.  The first fixes
table alignment, the second fixes LaTeX export of these tables.


From c555b7e15b617538490210a041bd4af45e51d752 Mon Sep 17 00:00:00 2001
From: Lawrence Mitchell <wence@gmx.li>
Date: Wed, 2 Feb 2011 12:20:12 +0000
Subject: [PATCH 1/2] Correctly realign tables with internal headers
To: emacs-orgmode@gnu.org

* lisp/org-table.el (org-table-align): Deal with internal headers
(specified by ?~) when realigning.
---
 lisp/org-table.el |   38 ++++++++++++++++++++++++--------------
 1 files changed, 24 insertions(+), 14 deletions(-)

diff --git a/lisp/org-table.el b/lisp/org-table.el
index 9437ae1..498a6fc 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -632,7 +632,7 @@ When nil, simply write \"#ERROR\" in corrupted fields.")
 	 lines (new "") lengths l typenums ty fields maxfields i
 	 column
 	 (indent "") cnt frac
-	 rfmt hfmt
+	 rfmt hfmt tfmt
 	 (spaces '(1 . 1))
 	 (sp1 (car spaces))
 	 (sp2 (cdr spaces))
@@ -640,6 +640,8 @@ When nil, simply write \"#ERROR\" in corrupted fields.")
 		 (make-string sp2 ?\ ) "%%%s%ds" (make-string sp1 ?\ ) "|"))
 	 (hfmt1 (concat
 		 (make-string sp2 ?-) "%s" (make-string sp1 ?-) "+"))
+	 (tfmt1 (concat
+		 (make-string sp2 ?~) "%s" (make-string sp1 ?~) "+"))
 	 emptystrings links dates emph raise narrow
 	 falign falign1 fmax f1 len c e space)
     (untabify beg end)
@@ -680,17 +682,19 @@ When nil, simply write \"#ERROR\" in corrupted fields.")
     ;; Mark the hlines by setting the corresponding element to nil
     ;; At the same time, we remove trailing space.
     (setq lines (mapcar (lambda (l)
-			  (if (string-match "^ *|[-~]" l)
-			      nil
-			    (if (string-match "[ \t]+$" l)
-				(substring l 0 (match-beginning 0))
-			      l)))
+			  (cond ((string-match "^ *|[-]" l)
+				 'dash)
+				((string-match "^ *|[~]" l)
+				 'tilde)
+				((string-match "[ \t]+$" l)
+				 (substring l 0 (match-beginning 0)))
+				(t l)))
 			lines))
     ;; Get the data fields by splitting the lines.
     (setq fields (mapcar
 		  (lambda (l)
 		      (org-split-string l " *| *"))
-		  (delq nil (copy-sequence lines))))
+		  (delq 'dash (delq 'tilde (copy-sequence lines)))))
     ;; How many fields in the longest line?
     (condition-case nil
 	(setq maxfields (apply 'max (mapcar 'length fields)))
@@ -770,19 +774,25 @@ When nil, simply write \"#ERROR\" in corrupted fields.")
 				    (concat (car c) space))))))))
 
     ;; Compute the formats needed for output of the table
-    (setq rfmt (concat indent "|") hfmt (concat indent "|"))
+    (setq rfmt (concat indent "|") hfmt (concat indent "|")
+	  tfmt (concat indent "|"))
     (while (setq l (pop lengths))
       (setq ty (if (pop typenums) "" "-")) ; number types flushright
       (setq rfmt (concat rfmt (format rfmt1 ty l))
-	    hfmt (concat hfmt (format hfmt1 (make-string l ?-)))))
+	    hfmt (concat hfmt (format hfmt1 (make-string l ?-)))
+	    tfmt (concat tfmt (format tfmt1 (make-string l ?~)))))
     (setq rfmt (concat rfmt "\n")
-	  hfmt (concat (substring hfmt 0 -1) "|\n"))
-
+	  hfmt (concat (substring hfmt 0 -1) "|\n")
+	  tfmt (concat (substring tfmt 0 -1) "|\n"))
     (setq new (mapconcat
 	       (lambda (l)
-		 (if l (apply 'format rfmt
-			      (append (pop fields) emptystrings))
-		   hfmt))
+		 (cond ((eq l 'dash)
+			hfmt)
+		       ((eq l 'tilde)
+			tfmt)
+		       (t
+			(apply 'format rfmt
+			       (append (pop fields) emptystrings)))))
 	       lines ""))
     (if (equal (char-before) ?\n)
 	;; This hack is for org-indent, to force redisplay of the


From 306605c2f49710a80d47553cb9bdf318dadf4db7 Mon Sep 17 00:00:00 2001
From: Lawrence Mitchell <wence@gmx.li>
Date: Wed, 2 Feb 2011 12:21:13 +0000
Subject: [PATCH 2/2] Correctly export internal table headers during LaTeX export
To: emacs-orgmode@gnu.org

* lisp/org-latex.el (org-export-latex-tables): Allow for tildes in
table format (indicating internal headers).
(org-export-latex-content): Export tables before special characters to
deal with new tilde header style.
---
 lisp/org-latex.el |   10 +++++-----
 1 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/lisp/org-latex.el b/lisp/org-latex.el
index 9290c35..61dced4 100644
--- a/lisp/org-latex.el
+++ b/lisp/org-latex.el
@@ -1389,6 +1389,9 @@ links, keywords, lists, tables, fixed-width"
    (unless (memq 'emphasis exclude-list)
      (when (plist-get org-export-latex-options-plist :emphasize)
        (org-export-latex-fontify)))
+   (unless (memq 'tables exclude-list)
+     (org-export-latex-tables
+      (plist-get org-export-latex-options-plist :tables)))
    (unless (memq 'sub-superscript exclude-list)
      (org-export-latex-special-chars
       (plist-get org-export-latex-options-plist :sub-superscript)))
@@ -1398,9 +1401,6 @@ links, keywords, lists, tables, fixed-width"
      (org-export-latex-keywords))
    (unless (memq 'lists exclude-list)
      (org-export-latex-lists))
-   (unless (memq 'tables exclude-list)
-     (org-export-latex-tables
-      (plist-get org-export-latex-options-plist :tables)))
    (unless (memq 'fixed-width exclude-list)
      (org-export-latex-fixed-width
       (plist-get org-export-latex-options-plist :fixed-width)))
@@ -1804,7 +1804,7 @@ The conversion is made depending of STRING-BEFORE and STRING-AFTER."
             ;; make a formatting string to reflect alignment
             (setq olines lines)
             (while (and (not line-fmt) (setq line (pop olines)))
-              (unless (string-match "^[ \t]*|-" line)
+              (unless (string-match "^[ \t]*|[-~]" line)
                 (setq fields (org-split-string line "[ \t]*|[ \t]*"))
                 (setq fnum (make-vector (length fields) 0))
                 (setq line-fmt
@@ -1839,7 +1839,7 @@ The conversion is made depending of STRING-BEFORE and STRING-AFTER."
             (setq lines
                   (mapcar
                    (lambda(elem)
-                     (or (and (string-match "[ \t]*|-+" elem) 'hline)
+                     (or (and (string-match "[ \t]*|[-~]+" elem) 'hline)
                          (org-split-string (org-trim elem) "|")))
                    lines))
             (when insert
-- 
1.7.4.rc2.18.gb20e9

Re: [Feature Request] Cross headings in tables

[Achim Gratz]

Lawrence Mitchell <wence@gmx.li> writes:
> How about the following two patches on top.  The first fixes
> table alignment, the second fixes LaTeX export of these tables.

Thank you for this, brilliant idea of replacing the nil with a
symbol... It integrates cleanly with what I have so far, I will need
some more testing (just discovered a boundary case that I fixed).  Also
need to check the other export backends, thank you for taking care of
LaTeX.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves

Re: [Feature Request] Cross headings in tables

[Achim Gratz]

If anybody wants to test the current state of affairs, I've just set up
a fork repository to make it easier.  Assuming you already have
orgmode.git cloned, do a

git remote add -t tableheadings remote-tableheadings git://repo.or.cz/org-mode/org-tableheadings.git
git fetch remote-tableheadings tableheadings:local-tableheadings
git checkout local-tableheadings

to get it (change remote-tableheadings and local-tableheadings to suit
your naming conventions for remotes and local branches, respectively).
I will be _rebasing_ against master during development, so expect
history in this branch to be volatile.

As before, test cases and comments welcome.


Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [Feature Request] Cross headings in tables

[Achim Gratz]

Achim Gratz <Stromeko@nexgo.de> writes:
> git://repo.or.cz/org-mode/org-tableheadings.git

I've just pushed a commit to have the key C-c _ (underscore, since
tilde "~" is already in use for conversion of table.el tables to org
tables) insert table header rows, just like C-c - inserts a hline.  That
would make this feature complete as far as I am concerned.

As always, feedback and suggestions are welcome.


Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf rackAttack V1.04R1:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [Orgmode] Re: [Feature Request] Cross headings in tables

[Carsten Dominik]

On 8.2.2011, at 22:52, Achim Gratz wrote:

> 
> If anybody wants to test the current state of affairs, I've just set up
> a fork repository to make it easier.  Assuming you already have
> orgmode.git cloned, do a
> 
> git remote add -t tableheadings remote-tableheadings git://repo.or.cz/org-mode/org-tableheadings.git
> git fetch remote-tableheadings tableheadings:local-tableheadings
> git checkout local-tableheadings
> 
> to get it (change remote-tableheadings and local-tableheadings to suit
> your naming conventions for remotes and local branches, respectively).
> I will be _rebasing_ against master during development, so expect
> history in this branch to be volatile.
> 
> As before, test cases and comments welcome.

Hi Achim, hi Lawrence,

this looks pretty good.  One thing I found missing is that
header lines should be exempted from column formulas being
applied.  This works for the headlines at the top of the
table, but not in the middle.  Try C-c C-c in the TBLFM
line of:

| aaa | bbb | ccc | ddd |
|-----+-----+-----+-----|
| ddd | eee | fff |  41 |
| ddd | eee | fff |  41 |
| ddd | eee | fff |  41 |
|~~~~~+~~~~~+~~~~~+~~~~~|
| ddd |     | fff | ddd |
|-----+-----+-----+-----|
| ddd | eee | fff |  41 |
| ddd | eee | fff |  41 |
| xxx | yyy | zzz |  41 |
#+TBLFM: $4=41

Also documentation in the manual is missing - one
or two sentences in the right place should be enough.

- Carsten

Re: [Orgmode] Re: [Feature Request] Cross headings in tables

[Achim Gratz]

Hi Carsten,

Carsten Dominik <carsten.dominik@gmail.com> writes:
> this looks pretty good.  One thing I found missing is that
> header lines should be exempted from column formulas being
> applied.  This works for the headlines at the top of the
> table, but not in the middle.

thank you for having a look.  You are right, but this looks like
something that goes a bit deeper than my current understanding, I'm
afraid.

Header lines don't appear to be marked in any way, but simply skipped
over before starting the formula calculation.  Consequently, the column
formula is evaluated for any header line when doing a C-c * directly in
the header, which might be considered a bug.  I won't touched this
aspect of the code since it will probably have farther reaching
consequences if changed.  I quickly hacked in some extra stuff that
looks for cross headers and skips them while the full table is updated —
but I'm not sure this is the right thing to do and the way the program
logic works requires me to check certain pathological cases twice, which
is a bit ugly.  I've just rebased against current master and pushed
everything back to the repo.

> Also documentation in the manual is missing - one
> or two sentences in the right place should be enough.

Sure, but I'd like to get it working correctly first. :-)

I'm using this code at work and things have gone smoothly, but I only
use the HTML backend and certainly don't exercise the full breadth of
the table functionality (obviously no column formulas were used by me,
for instance).

While testing I stumbled upon this: if a buffer has no undo information
associated, org-self-insert-command produces an error while trying to
edit the formula.  This code path may need to be protected against this
(admittedly rare) case, I've been triggering it while testing your
example directly in the gnus article buffer.

Debugger entered--Lisp error: (wrong-type-argument listp t)
  cadr(t)
  (not (cadr buffer-undo-list))
  (and (> org-self-insert-command-undo-counter 0) buffer-undo-list (not (cadr buffer-undo-list)) (setcdr buffer-undo-list (cddr buffer-undo-list)))
  (if (>= org-self-insert-command-undo-counter 20) (setq org-self-insert-command-undo-counter 1) (and (> org-self-insert-command-undo-counter 0) buffer-undo-list (not ...) (setcdr buffer-undo-list ...)) (setq org-self-insert-command-undo-counter (1+ org-self-insert-command-undo-counter)))
  (if (not (eq last-command ...)) (setq org-self-insert-command-undo-counter 1) (if (>= org-self-insert-command-undo-counter 20) (setq org-self-insert-command-undo-counter 1) (and ... buffer-undo-list ... ...) (setq org-self-insert-command-undo-counter ...)))
  (if org-self-insert-cluster-for-undo (if (not ...) (setq org-self-insert-command-undo-counter 1) (if ... ... ... ...)))
  (cond ((and org-use-speed-commands ...) (cond ... ... ... ...)) ((and ... ... ... ...) (let ... ... ... ... ...)) (t (setq org-table-may-need-update t) (self-insert-command N) (org-fix-tags-on-the-fly) (if org-self-insert-cluster-for-undo ...)))
  org-self-insert-command(1)


Regards
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [Orgmode] Re: [Feature Request] Cross headings in tables

[Carsten Dominik]

Hi Achim,

On 30.5.2011, at 23:02, Achim Gratz wrote:

> Hi Carsten,
> 
> Carsten Dominik <carsten.dominik@gmail.com> writes:
>> this looks pretty good.  One thing I found missing is that
>> header lines should be exempted from column formulas being
>> applied.  This works for the headlines at the top of the
>> table, but not in the middle.
> 
> thank you for having a look.  You are right, but this looks like
> something that goes a bit deeper than my current understanding, I'm
> afraid.
> 
> Header lines don't appear to be marked in any way, but simply skipped
> over before starting the formula calculation.  Consequently, the column
> formula is evaluated for any header line when doing a C-c * directly in
> the header, which might be considered a bug.  I won't touched this
> aspect of the code since it will probably have farther reaching
> consequences if changed.  I quickly hacked in some extra stuff that
> looks for cross headers and skips them while the full table is updated —
> but I'm not sure this is the right thing to do and the way the program
> logic works requires me to check certain pathological cases twice, which
> is a bit ugly.  I've just rebased against current master and pushed
> everything back to the repo.

I think a better strategy would be to find these additional
header lines right before this section of the recalculate function:

    ;; Now evaluate the column formulas, but skip fields covered by
    ;; field formulas

and mark those extra header lines with the org-untouchable text property.
Then you can let the column formulas do their game, and fields marked
by this property will automatically exempted......

> 
>> Also documentation in the manual is missing - one
>> or two sentences in the right place should be enough.
> 
> Sure, but I'd like to get it working correctly first. :-)

Fair enough.

> I'm using this code at work and things have gone smoothly, but I only
> use the HTML backend and certainly don't exercise the full breadth of
> the table functionality (obviously no column formulas were used by me,
> for instance).

Another part which might need a look to support this
would be `orgtbl-to-generic'.

> While testing I stumbled upon this: if a buffer has no undo information
> associated, org-self-insert-command produces an error while trying to
> edit the formula.  This code path may need to be protected against this
> (admittedly rare) case, I've been triggering it while testing your
> example directly in the gnus article buffer.

This issue is fixed, thanks for the report.

Finally:  this patch goes clearly beyond the TINYCHANGE
limits.  What is yours, and Lawrence's copyright status with the FSF?

- Carsten

> 
> Debugger entered--Lisp error: (wrong-type-argument listp t)
> cadr(t)
> (not (cadr buffer-undo-list))
> (and (> org-self-insert-command-undo-counter 0) buffer-undo-list (not (cadr buffer-undo-list)) (setcdr buffer-undo-list (cddr buffer-undo-list)))
> (if (>= org-self-insert-command-undo-counter 20) (setq org-self-insert-command-undo-counter 1) (and (> org-self-insert-command-undo-counter 0) buffer-undo-list (not ...) (setcdr buffer-undo-list ...)) (setq org-self-insert-command-undo-counter (1+ org-self-insert-command-undo-counter)))
> (if (not (eq last-command ...)) (setq org-self-insert-command-undo-counter 1) (if (>= org-self-insert-command-undo-counter 20) (setq org-self-insert-command-undo-counter 1) (and ... buffer-undo-list ... ...) (setq org-self-insert-command-undo-counter ...)))
> (if org-self-insert-cluster-for-undo (if (not ...) (setq org-self-insert-command-undo-counter 1) (if ... ... ... ...)))
> (cond ((and org-use-speed-commands ...) (cond ... ... ... ...)) ((and ... ... ... ...) (let ... ... ... ... ...)) (t (setq org-table-may-need-update t) (self-insert-command N) (org-fix-tags-on-the-fly) (if org-self-insert-cluster-for-undo ...)))
> org-self-insert-command(1)
> 
> 
> Regards
> Achim.
> -- 
> +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+
> 
> Factory and User Sound Singles for Waldorf Blofeld:
> http://Synth.Stromeko.net/Downloads.html#WaldorfSounds
> 
> 

Re: [Orgmode] Re: [Feature Request] Cross headings in tables

[Lawrence Mitchell]

On 31/05/2011 08:21, Carsten Dominik wrote:

[...]

> Finally:  this patch goes clearly beyond the TINYCHANGE
> limits.  What is yours, and Lawrence's copyright status with the FSF?

I have papers with the FSF for past and future changes to Emacs, so I
believe my contributions are covered.

Lawrence

Re: [Orgmode] Re: [Feature Request] Cross headings in tables

[Achim Gratz]

Hi Carsten,

Carsten Dominik <carsten.dominik@gmail.com> writes:
> I think a better strategy would be to find these additional
> header lines right before this section of the recalculate function:
[...]

I'll have a look (probably not today), but I'd rather tag them with an
"org-header" property and arrange it so that this can be used in other
places as well.

> Another part which might need a look to support this
> would be `orgtbl-to-generic'.

Noted.

> Finally:  this patch goes clearly beyond the TINYCHANGE
> limits.  What is yours, and Lawrence's copyright status with the FSF?

My status with the FSF is succinctly and fully characterized as
"non-existing".  I've sent that mail form to the FSF and I'll see what
happens next — it seems I'll get snail mail in a few weeks?


Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: [Orgmode] Re: [Feature Request] Cross headings in tables

[Jambunathan K]

> My status with the FSF is succinctly and fully characterized as
> "non-existing".  I've sent that mail form to the FSF and I'll see what
> happens next ― it seems I'll get snail mail in a few weeks?

In my case the process took close to 4 months. 

Jambunathan K.

-- 

Re: [Orgmode] Re: [Feature Request] Cross headings in tables

[Carsten Dominik]

On 31.5.2011, at 20:01, Achim Gratz wrote:

> Hi Carsten,
> 
> Carsten Dominik <carsten.dominik@gmail.com> writes:
>> I think a better strategy would be to find these additional
>> header lines right before this section of the recalculate function:
> [...]
> 
> I'll have a look (probably not today), but I'd rather tag them with an
> "org-header" property and arrange it so that this can be used in other
> places as well.

This property is not attached to these lines for long - it is just 
a temporary way of marking fields that should not be overwritten
by column formulas.  The property is added, and also removed
again (I think) during the recalculation process.

> 
>> Another part which might need a look to support this
>> would be `orgtbl-to-generic'.
> 
> Noted.
> 
>> Finally:  this patch goes clearly beyond the TINYCHANGE
>> limits.  What is yours, and Lawrence's copyright status with the FSF?
> 
> My status with the FSF is succinctly and fully characterized as
> "non-existing".  I've sent that mail form to the FSF and I'll see what
> happens next — it seems I'll get snail mail in a few weeks?

Thank you.  Please let us know when the process gets stuck or completes.

- Carsten

Re: [Orgmode] Re: [Feature Request] Cross headings in tables

[Achim Gratz]

Carsten Dominik <carsten.dominik@gmail.com> writes:
> I think a better strategy would be to find these additional
> header lines right before this section of the recalculate function:
>
>     ;; Now evaluate the column formulas, but skip fields covered by
>     ;; field formulas
>
> and mark those extra header lines with the org-untouchable text property.
> Then you can let the column formulas do their game, and fields marked
> by this property will automatically exempted......

I just did that (and fixing a bug with the :org-untouchable property not
being removed in certain circumstances which you might want to commit to
master upfront).  It looks much cleaner, see if you like it, too.  I've
also cleaned up all the commit messages, I hope they conform to the
standard now.

> Another part which might need a look to support this
> would be `orgtbl-to-generic'.

That I will have to think over a bit.  This support depends on parsing
the table into a list (AFAIK) and that currently treats cross headings
as another sort of hlines.  Changing this has a lot of repercussions
elsewhere in the code as far as I can see.  It might be an opportunity
to re-factor some of the code that still works on the text
representation, but that is certainly not something that I'd take
lightly.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra

Re: [Feature Request] Cross headings in tables

[Achim Gratz]

I've just looked at how to implement this using org-element and the new
exporter.  Much to my dismay I found that table headers are not a
separate row or row group type and the new exporter is still using this
"everything before the first separator is a heading" kludge.  It's easy
enough to introduce a second type of separator in org-element (BTW, I
don't think that 'rule is a particularly good symbol name for this) by
adding another type of table rows, but then these lines get ignored by
the new exporter.  While the logic used there is amendable (not as
easily), it would be more hackish than I hoped.  Also, it appears that
each element can have only one property or did I miss something?  It
seems that having a way to get the (main) property and then asking if
there are sub-properties on that element would allow simpler code in
this particular instance and likely elsewhere.  Maybe I missed how to do
it, though.

The cleanest way to implement this would be if being a header would be a
property of the row group (maybe returnable as the sign of the row group
number).  There'd be two types of separators, "|-" and "|~".  If the
line past the end of a row group is "|~", then it is a header group,
otherwise it is a normal group.  This is incompatible with tradition,
but it would allow to use row groups in formulas without introducing
unwanted headers.

Thoughts?


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Waldorf MIDI Implementation & additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs

Re: [Feature Request] Cross headings in tables

[Nicolas Goaziou]

Hello,

Achim Gratz <Stromeko@nexgo.de> writes:

> I've just looked at how to implement this using org-element and the
> new exporter. Much to my dismay I found that table headers are not
> a separate row or row group type and the new exporter is still using
> this "everything before the first separator is a heading" kludge.

I wouldn't call that a kludge. That's where most of the mere mortals
expect the heading to be anyway. Also, it's somewhat back-end specific.

> It's easy enough to introduce a second type of separator in
> org-element (BTW, I don't think that 'rule is a particularly good
> symbol name for this) by adding another type of table rows, but then
> these lines get ignored by the new exporter. While the logic used
> there is amendable (not as easily), it would be more hackish than
> I hoped.

You only need to modify org-element if Org syntax has been changed.

>  Also, it appears that each element can have only one property or did
> I miss something? It seems that having a way to get the (main)
> property and then asking if there are sub-properties on that element
> would allow simpler code in this particular instance and likely
> elsewhere. Maybe I missed how to do it, though.

I don't understand that part. For example:

--8<---------------cut here---------------start------------->8---
   * TODO Headline                                          :tag:
--8<---------------cut here---------------end--------------->8---

In the following buffer, you can have:

  (org-element-property :priority (org-element-at-point))
  (org-element-property :tag (org-element-at-point))
  (org-element-property :todo-keyword (org-element-at-point))
  (org-element-property :todo-type (org-element-at-point))
  (org-element-property :level: (org-element-at-point))
  ...

So there is more than one property. See:

  http://orgmode.org/worg/dev/org-export-reference.html

for a list of all properties associated to a given element or object
type.
 
> The cleanest way to implement this would be if being a header would be a
> property of the row group (maybe returnable as the sign of the row group
> number).  There'd be two types of separators, "|-" and "|~".  If the
> line past the end of a row group is "|~", then it is a header group,
> otherwise it is a normal group.  This is incompatible with tradition,
> but it would allow to use row groups in formulas without introducing
> unwanted headers.
>
> Thoughts?

I think the cleanest way to implement this would be to _not_ modify Org
syntax, because it is export back-end very specific. Something like:

  #+attr_html: :header-groups (1 3)
  | This      | will      |
  | be        | a header  |
  |-----------+-----------|
  | This      | won't     |
  |-----------+-----------|
  | This will | be too    |
  |-----------+-----------|
  | This      | won't too |

You are talking about formulas, so, perhaps you have plans for the
spreadsheet. Anyway, if Org syntax changes, org-element.el will have to
be updated accordingly, and so will have all the back-ends.


Regards,

-- 
Nicolas Goaziou

Re: [Feature Request] Cross headings in tables

[Achim Gratz]

Nicolas Goaziou writes:
> You only need to modify org-element if Org syntax has been changed.

Yes, thats exactly what I want.

>>  Also, it appears that each element can have only one property or did
>> I miss something? It seems that having a way to get the (main)
>> property and then asking if there are sub-properties on that element
>> would allow simpler code in this particular instance and likely
>> elsewhere. Maybe I missed how to do it, though.
>
> I don't understand that part.

(org-element-property :type row)

This will return a single symbol 'standard or 'rule inside a table.  If
there was a way to ask "which kind of rule" later, this would enable a
much more elegant implementation.

>   http://orgmode.org/worg/dev/org-export-reference.html
>
> for a list of all properties associated to a given element or object
> type.

So you are saying that a table row could get an additional property to
record which rule-type it has and whether it is a header row or not?
  
> I think the cleanest way to implement this would be to _not_ modify Org
> syntax, because it is export back-end very specific.

This is an impressive hack, but is largely unuseable IMHO.  And whether
a table row is a header or not is certainly not back-end specific or at
least it shouldn't be: the back-end gets told if it should format a
table row as a header or as a normal row and that code is all in ox.
The only other part of the code that needs to know which row groups are
headers is when column formulas are involved.

The original hack tried to keep things backwards compatible, but I think
that it would have merit to allow tables with separators that don't
introduce any headers (as several people have asked for in the past) and
that simply isn't in the cards with the current syntax.  So, a row group
would only be a header when the following rule was "|~" and the first
row group wouldn't need to be special any more.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [Feature Request] Cross headings in tables

[Achim Gratz]

Achim Gratz writes:
> Nicolas Goaziou writes:
>> You only need to modify org-element if Org syntax has been changed.
>
> Yes, thats exactly what I want.

A new rule type was easy enough to implement:

http://repo.or.cz/w/org-mode/org-tableheadings.git/commit/17a4e9c599c555f53c2831fe8c473f169ccd8679

It doesn't do anything yet since nothing else checks the extra property,
but that's a start.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [Feature Request] Cross headings in tables

[Carsten Dominik]

On 17 feb. 2013, at 09:35, Nicolas Goaziou <n.goaziou@gmail.com> wrote:

> Hello,
> 
> Achim Gratz <Stromeko@nexgo.de> writes:
> 
>> I've just looked at how to implement this using org-element and the
>> new exporter. Much to my dismay I found that table headers are not
>> a separate row or row group type and the new exporter is still using
>> this "everything before the first separator is a heading" kludge.
> 
> I wouldn't call that a kludge. That's where most of the mere mortals
> expect the heading to be anyway. Also, it's somewhat back-end specific.
> 
>> It's easy enough to introduce a second type of separator in
>> org-element (BTW, I don't think that 'rule is a particularly good
>> symbol name for this) by adding another type of table rows, but then
>> these lines get ignored by the new exporter. While the logic used
>> there is amendable (not as easily), it would be more hackish than
>> I hoped.
> 
> You only need to modify org-element if Org syntax has been changed.
> 
>> Also, it appears that each element can have only one property or did
>> I miss something? It seems that having a way to get the (main)
>> property and then asking if there are sub-properties on that element
>> would allow simpler code in this particular instance and likely
>> elsewhere. Maybe I missed how to do it, though.
> 
> I don't understand that part. For example:
> 
> --8<---------------cut here---------------start------------->8---
>   * TODO Headline                                          :tag:
> --8<---------------cut here---------------end--------------->8---
> 
> In the following buffer, you can have:
> 
>  (org-element-property :priority (org-element-at-point))
>  (org-element-property :tag (org-element-at-point))
>  (org-element-property :todo-keyword (org-element-at-point))
>  (org-element-property :todo-type (org-element-at-point))
>  (org-element-property :level: (org-element-at-point))
>  ...
> 
> So there is more than one property. See:
> 
>  http://orgmode.org/worg/dev/org-export-reference.html
> 
> for a list of all properties associated to a given element or object
> type.
> 
>> The cleanest way to implement this would be if being a header would be a
>> property of the row group (maybe returnable as the sign of the row group
>> number).  There'd be two types of separators, "|-" and "|~".  If the
>> line past the end of a row group is "|~", then it is a header group,
>> otherwise it is a normal group.  This is incompatible with tradition,
>> but it would allow to use row groups in formulas without introducing
>> unwanted headers.
>> 
>> Thoughts?
> 
> I think the cleanest way to implement this would be to _not_ modify Org
> syntax, because it is export back-end very specific. Something like:
> 
>  #+attr_html: :header-groups (1 3)
>  | This      | will      |
>  | be        | a header  |
>  |-----------+-----------|
>  | This      | won't     |
>  |-----------+-----------|
>  | This will | be too    |
>  |-----------+-----------|
>  | This      | won't too |

I really like this approach, to mark the header groups in an attribute - maybe an backend-independent attribute?  The reason why I prefer this approach is that I am weary of new syntax in Org-mode that will take up new characters of character chains.  For the case of tables, if I could go back, I would even remove some of the syntax I introduced, for example for defining the values of constants - that should have been an attribute-link thing as well.  Probably even row and column naming, could have been done in this way.

- Carsten

Re: [Feature Request] Cross headings in tables

[Achim Gratz]

Carsten Dominik writes:
>> I think the cleanest way to implement this would be to _not_ modify Org
>> syntax, because it is export back-end very specific. Something like:
>> 
>>  #+attr_html: :header-groups (1 3)
>>  | This      | will      |
>>  | be        | a header  |
>>  |-----------+-----------|
>>  | This      | won't     |
>>  |-----------+-----------|
>>  | This will | be too    |
>>  |-----------+-----------|
>>  | This      | won't too |
>
> I really like this approach, to mark the header groups in an attribute
> - maybe an backend-independent attribute?

I can see the appeal from a programmers' point of view, but from a user
perspective this is extremely obscure, especially if that table spans a
few pages and has many such extra header rows (hey, why not ask the user
to write the table in XML to start with?).  In any case, the current way
of dealing with headers in org-element and the exporters doesn't admit
either.

>  The reason why I prefer this approach is that I am weary of new
> syntax in Org-mode that will take up new characters of character
> chains.  For the case of tables, if I could go back, I would even
> remove some of the syntax I introduced, for example for defining the
> values of constants - that should have been an attribute-link thing as
> well.  Probably even row and column naming, could have been done in
> this way.

Hindsight is 20/20. :-)

But going this route takes us even further from "Your life in plain
text." towards "Your life in another programming language." because you
will need a special viewer or editor to make sense of it for all but the
most trivial cases.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [Feature Request] Cross headings in tables

[Carsten Dominik]

On 22.2.2013, at 21:33, Achim Gratz <Stromeko@nexgo.de> wrote:

> Carsten Dominik writes:
>>> I think the cleanest way to implement this would be to _not_ modify Org
>>> syntax, because it is export back-end very specific. Something like:
>>> 
>>> #+attr_html: :header-groups (1 3)
>>> | This      | will      |
>>> | be        | a header  |
>>> |-----------+-----------|
>>> | This      | won't     |
>>> |-----------+-----------|
>>> | This will | be too    |
>>> |-----------+-----------|
>>> | This      | won't too |
>> 
>> I really like this approach, to mark the header groups in an attribute
>> - maybe an backend-independent attribute?
> 
> I can see the appeal from a programmers' point of view, but from a user
> perspective this is extremely obscure, especially if that table spans a
> few pages and has many such extra header rows (hey, why not ask the user
> to write the table in XML to start with?).  In any case, the current way
> of dealing with headers in org-element and the exporters doesn't admit
> either.
> 
>> The reason why I prefer this approach is that I am weary of new
>> syntax in Org-mode that will take up new characters of character
>> chains.  For the case of tables, if I could go back, I would even
>> remove some of the syntax I introduced, for example for defining the
>> values of constants - that should have been an attribute-link thing as
>> well.  Probably even row and column naming, could have been done in
>> this way.
> 
> Hindsight is 20/20. :-)
> 
> But going this route takes us even further from "Your life in plain
> text." towards "Your life in another programming language." because you
> will need a special viewer or editor to make sense of it for all but the
> most trivial cases.

Point taken.

- Carsten

> 
> 
> Regards,
> Achim.
> -- 
> +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+
> 
> SD adaptation for Waldorf microQ V2.22R2:
> http://Synth.Stromeko.net/Downloads.html#WaldorfSDada
> 
> 

Re: [Feature Request] Cross headings in tables

[Bastien]

Achim Gratz <Stromeko@nexgo.de> writes:

> But going this route takes us even further from "Your life in plain
> text." towards "Your life in another programming language." because you
> will need a special viewer or editor to make sense of it for all but the
> most trivial cases.

Point taken too -- this is even more relevant considering the fact
that there are .org files parsers outside of Org.  Tables are one of
the "Wow!" features in Org, using |~~~| to separate the header would
be all right I guess (especially considering that the regexp should
be safe enough with no false-positive.)

-- 
 Bastien

Re: [Feature Request] Cross headings in tables

[Achim Gratz]

Nicolas Goaziou writes:
> I think the cleanest way to implement this would be to _not_ modify Org
> syntax, because it is export back-end very specific. Something like:
>
>   #+attr_html: :header-groups (1 3)
>   | This      | will      |
>   | be        | a header  |
>   |-----------+-----------|
>   | This      | won't     |
>   |-----------+-----------|
>   | This will | be too    |
>   |-----------+-----------|
>   | This      | won't too |
>
> You are talking about formulas, so, perhaps you have plans for the
> spreadsheet. Anyway, if Org syntax changes, org-element.el will have to
> be updated accordingly, and so will have all the back-ends.

I've been thinking about this some more.  We've had enough inquiries on
how to import tables from R or other programs and give it headers and
other decorations, maybe even formulas without having to construct the
table as Org syntax in the other language.  If we made such attributes
_table_ attributes (i.e. extend the table model of Org and not just some
decoration in the export) then this would become an easy thing to do.
So if a modification in this direction is in the cards, I certainly
support this proposal.  Defining a more user-friendly syntax for the
main features so that most of that complexity would be out of sight most
of the time shouldn't be too difficult.  It adds an extra step to the
normalization of tables before import/export, but the added
functionality should make up for that many times over.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds