Tables: missing multi-col/row syntax

Tables: missing multi-col/row syntax

[TEC]

Hi all,

This is a pretty major 'feature request', but I think also an 
important
one.

When developing large tables, it can often be /necessary/ to start 
using
multi-column/row cells for clarity, and sensible exporting 
results.

As far as I am aware, in Org does not currently have any 
multi-col/row
syntax. The only viable method seems to be re-implementing the 
table
using export blocks in every backend you may want to export to (in 
my
case, usually TeX + HTML). This is clumsy, difficult to work with, 
and
could be avoided should org gain support for multi-col/row syntax.

I appreciate that this would constitute a major change both the 
Org's
syntax and the codebase, but I believe such a change is warranted 
by the
advantages it would provide.

Both how this can be implemented while minimising/eliminating the 
chance
of breaking well-formed current table elements, and what syntax 
may be
both acceptable and seem sensible to use.

I would anticipate such a feature working by designating two 
characters
to indicate "add row" and "add column". For example "|" and "-". 
These
characters would take affect when /immediately following/ (no 
space) a
cell separator ("|"), and designate the dimensions of the top 
right cell.

Example:
| a | b | c |
|---+---+---|
| a | - | | |
| - | b | . |
| . | | | c |

Would be interpreted just as any current table is.

However,

| hello | there | you  |
|-------+-------+------|
|| two column   | cell |

Contains a 2x1 cell.

| a little | test |
|----------+------|
|- hello   | hi   |
| two row  | you  |

Contains a 1x2 cell. In a more complex example:

| a | b | c |
|---+---+---|
||-- hi | a |
| two x | . |
| three | b |
| c | - | . |

Contains a 2x3 cell.

This is just the first syntax that comes to mind, but hopefully 
the
general form of this idea seems viable.

All the best,

Timothy.

Re: Tables: missing multi-col/row syntax

[Tom Gillespie]

Any support for something like this would need to retain backward
compatibility as well to avoid older versions reformatting the tables
due to e.g. the presence of a double pipe. I also think that extending
the table syntax in ways that makes it more complex than it already
is, will be a non-starter. Thus, an alternate but more likely approach
would be to allow specification of what cells to merge outside the
table as is done for formulas. It is not elegant, but it would be a
layer on top of existing syntax, and it would allow the fundamental
structure of the table to remain the same -- rows of cells. For
example #+TBLCELLMERGE: @2-3$1 or something like that. Thoughts?
Tom

On Mon, Nov 2, 2020 at 1:37 PM TEC <tecosaur@gmail.com> wrote:
>
> Hi all,
>
> This is a pretty major 'feature request', but I think also an
> important
> one.
>
> When developing large tables, it can often be /necessary/ to start
> using
> multi-column/row cells for clarity, and sensible exporting
> results.
>
> As far as I am aware, in Org does not currently have any
> multi-col/row
> syntax. The only viable method seems to be re-implementing the
> table
> using export blocks in every backend you may want to export to (in
> my
> case, usually TeX + HTML). This is clumsy, difficult to work with,
> and
> could be avoided should org gain support for multi-col/row syntax.
>
> I appreciate that this would constitute a major change both the
> Org's
> syntax and the codebase, but I believe such a change is warranted
> by the
> advantages it would provide.
>
> Both how this can be implemented while minimising/eliminating the
> chance
> of breaking well-formed current table elements, and what syntax
> may be
> both acceptable and seem sensible to use.
>
> I would anticipate such a feature working by designating two
> characters
> to indicate "add row" and "add column". For example "|" and "-".
> These
> characters would take affect when /immediately following/ (no
> space) a
> cell separator ("|"), and designate the dimensions of the top
> right cell.
>
> Example:
> | a | b | c |
> |---+---+---|
> | a | - | | |
> | - | b | . |
> | . | | | c |
>
> Would be interpreted just as any current table is.
>
> However,
>
> | hello | there | you  |
> |-------+-------+------|
> || two column   | cell |
>
> Contains a 2x1 cell.
>
> | a little | test |
> |----------+------|
> |- hello   | hi   |
> | two row  | you  |
>
> Contains a 1x2 cell. In a more complex example:
>
> | a | b | c |
> |---+---+---|
> ||-- hi | a |
> | two x | . |
> | three | b |
> | c | - | . |
>
> Contains a 2x3 cell.
>
> This is just the first syntax that comes to mind, but hopefully
> the
> general form of this idea seems viable.
>
> All the best,
>
> Timothy.
>

Re: Tables: missing multi-col/row syntax

[TEC]

Tom Gillespie <tgbugs@gmail.com> writes:

> Any support for something like this would need to retain 
> backward
> compatibility as well to avoid older versions reformatting the 
> tables
> due to e.g. the presence of a double pipe. I also think that 
> extending
> the table syntax in ways that makes it more complex than it 
> already
> is, will be a non-starter.

I am also concerned with avoiding disrupting previous table. I 
think
that non-space double+ pipes should be alright (hence their part 
in my
suggestion) as it still maintains that the number of columns is 
equal to
the number of pipes. E.g. currently
|| a    | b | = 3 columns, and is reformatted to |  | a | b |
proposal
|| a    | b | = 2 cells, 2+1=3 columns (same)

> Thus, an alternate but more likely approach
> would be to allow specification of what cells to merge outside 
> the
> table as is done for formulas. It is not elegant, but it would 
> be a
> layer on top of existing syntax, and it would allow the 
> fundamental
> structure of the table to remain the same -- rows of cells. For
> example #+TBLCELLMERGE: @2-3$1 or something like that. Thoughts?
> Tom

I like how this seems to address the issue while keeping backwards
compatibility, but I have two big concerns:
 1. I think this could make it hard to see which cells are 
 'masked' by a
 merge at a glance, and would make associated errors easy
 2. I think that for say a table with 2-3 large sub-sections this 
 could
 lead to problematic formating.

Regarding 2, E.g. (content and form made up on the spot)

| Powerpoint | Voltage ripple while drawing 2A continuous load 
  over Xs |    |     |     |     | Current ripple while drawing 
  24V continuous load over Xs |    |    |     |    |     |
|            | 
1s | 5s | 10s | 20s | 30s | 
1s | 2s | 5s | 10s | 20 | 30s |
|------------+---------------------------------------------------------+----+-----+-----+-----+----------------------------------------------------------+----+----+-----+----+-----|
| Kitchen    | 
  3% | 2% |  3% |  1% |  2% | 
  1% | 4% | 3% |  5% | 3% |  2% |
| Bedroom    | 
  1% | 2% |  1% |  2% |  1% | 
  2% | 1% | 1% |  1% | 2% |  1% |
#+TBLCELLMERGE: @2-6$1,@7-11$1

vs.

| Powerpoint ||||| Voltage ripple while drawing 2A continuous load 
  over Xs |||||| Current ripple while drawing 24V continuous load 
  over Xs |
|            | 1s | 5s | 10s | 20s | 30s 
               | 1s | 2s | 5s | 10s | 20 | 30s 
               |
|------------+----+----+-----+-----+---------------------------------------+----+----+----+-----+----+-------------------------------------|
| Kitchen    | 3% | 2% |  3% |  1% |  2% 
  | 1% | 4% | 3% |  5% | 3% |  2% 
  |
| Bedroom    | 1% | 2% |  1% |  2% |  1% 
  | 2% | 1% | 1% |  1% | 2% |  1% 
  |

(NB: clearer without line wrapping, more concise examples 
definitely
available if I thought about it a tad more)

> On Mon, Nov 2, 2020 at 1:37 PM TEC <tecosaur@gmail.com> wrote:
>>
>> Hi all,
>>
>> This is a pretty major 'feature request', but I think also an
>> important
>> one.
>>
>> When developing large tables, it can often be /necessary/ to 
>> start
>> using
>> multi-column/row cells for clarity, and sensible exporting
>> results.
>>
>> As far as I am aware, in Org does not currently have any
>> multi-col/row
>> syntax. The only viable method seems to be re-implementing the
>> table
>> using export blocks in every backend you may want to export to 
>> (in
>> my
>> case, usually TeX + HTML). This is clumsy, difficult to work 
>> with,
>> and
>> could be avoided should org gain support for multi-col/row 
>> syntax.
>>
>> I appreciate that this would constitute a major change both the
>> Org's
>> syntax and the codebase, but I believe such a change is 
>> warranted
>> by the
>> advantages it would provide.
>>
>> Both how this can be implemented while minimising/eliminating 
>> the
>> chance
>> of breaking well-formed current table elements, and what syntax
>> may be
>> both acceptable and seem sensible to use.
>>
>> I would anticipate such a feature working by designating two
>> characters
>> to indicate "add row" and "add column". For example "|" and 
>> "-".
>> These
>> characters would take affect when /immediately following/ (no
>> space) a
>> cell separator ("|"), and designate the dimensions of the top
>> right cell.
>>
>> Example:
>> | a | b | c |
>> |---+---+---|
>> | a | - | | |
>> | - | b | . |
>> | . | | | c |
>>
>> Would be interpreted just as any current table is.
>>
>> However,
>>
>> | hello | there | you  |
>> |-------+-------+------|
>> || two column   | cell |
>>
>> Contains a 2x1 cell.
>>
>> | a little | test |
>> |----------+------|
>> |- hello   | hi   |
>> | two row  | you  |
>>
>> Contains a 1x2 cell. In a more complex example:
>>
>> | a | b | c |
>> |---+---+---|
>> ||-- hi | a |
>> | two x | . |
>> | three | b |
>> | c | - | . |
>>
>> Contains a 2x3 cell.
>>
>> This is just the first syntax that comes to mind, but hopefully
>> the
>> general form of this idea seems viable.
>>
>> All the best,
>>
>> Timothy.
>>

Re: Tables: missing multi-col/row syntax

[David Rogers]

TEC <tecosaur@gmail.com> writes:

> Hi all,
>
> This is a pretty major 'feature request', but I think also an 
> important
> one.
>
> When developing large tables, it can often be /necessary/ to 
> start using
> multi-column/row cells for clarity, and sensible exporting 
> results.

IMO this can (and definitely should) be regarded as a purely 
cosmetic problem, to be resolved by purely cosmetic methods. 
I think the idea that each table cell is exactly one unit of 
information (and can’t be a collection or array of units of 
information) is more important than this issue.

In other words, I think it’s better to ridiculously overload 
fontification and output formatting, than to sacrifice the main 
logic of how tables currently work. I just don’t believe that it 
could be worth it.

-- 
David Rogers

Re: Tables: missing multi-col/row syntax

[TEC]

David Rogers <davidandrewrogers@gmail.com> writes:

> IMO this can (and definitely should) be regarded as a purely 
> cosmetic problem,
> to be resolved by purely cosmetic methods. I think the idea that 
> each table cell
> is exactly one unit of information (and can’t be a collection or 
> array of units
> of information) is more important than this issue.
>
> In other words, I think it’s better to ridiculously overload 
> fontification and
> output formatting, than to sacrifice the main logic of how 
> tables currently
> work. I just don’t believe that it could be worth it.

The appearance of the tables is purely cosmetic ... however when 
one
ends up maintaining three copies of the same table, I don't think 
that
dismissing it offhand as a "cosmetic problem" is a productive 
approach.

--
Timothy

Re: Tables: missing multi-col/row syntax

[David Rogers]

TEC <tecosaur@gmail.com> writes:

> David Rogers <davidandrewrogers@gmail.com> writes:
>
>> IMO this can (and definitely should) be regarded as a purely 
>> cosmetic problem,
>> to be resolved by purely cosmetic methods. I think the idea 
>> that each table cell
>> is exactly one unit of information (and can’t be a collection 
>> or array of units
>> of information) is more important than this issue.
>>
>> In other words, I think it’s better to ridiculously overload 
>> fontification and
>> output formatting, than to sacrifice the main logic of how 
>> tables currently
>> work. I just don’t believe that it could be worth it.
>
> The appearance of the tables is purely cosmetic ... however when 
> one
> ends up maintaining three copies of the same table, I don't 
> think that
> dismissing it offhand as a "cosmetic problem" is a productive 
> approach.

I’m not using the word in a dismissive way; I just think fixing 
the problem “head-on“, solving a display problem by display 
solutions, is better than disruptively (and permanently) changing 
the logic in order to improve the display.

-- 
David Rogers

Re: Tables: missing multi-col/row syntax

[Christian Moe]

+1 for enabling table-cell merges in export. I imagine this would be a
tricky job for developers, but it would relieve me as a user of much
repeated fiddling with exported drafts.

+1 for doing it without adding clutter to the table syntax, but
specifying merges on a separate line like formulas, like Tom's

  #+TBLCELLMERGE: @2..3$1

(amended here to use the established '..' rather than hyphen for range)

Though if we do add such a line, we might also think of a more general
solution that could over time be extended with additional formatting
options, e.g. something like

  #+TBLSTYLE: @2..3$1='(:merge t)::@4$1='(:bgcolor yellow :color red)

But obviously that could open a can of worms, aka potentially endless
feature requests requiring different implementations for each backend.

Yours,
Christian



Tom Gillespie writes:

> Any support for something like this would need to retain backward
> compatibility as well to avoid older versions reformatting the tables
> due to e.g. the presence of a double pipe. I also think that extending
> the table syntax in ways that makes it more complex than it already
> is, will be a non-starter. Thus, an alternate but more likely approach
> would be to allow specification of what cells to merge outside the
> table as is done for formulas. It is not elegant, but it would be a
> layer on top of existing syntax, and it would allow the fundamental
> structure of the table to remain the same -- rows of cells. For
> example #+TBLCELLMERGE: @2-3$1 or something like that. Thoughts?
> Tom
>
> On Mon, Nov 2, 2020 at 1:37 PM TEC <tecosaur@gmail.com> wrote:
>>
>> Hi all,
>>
>> This is a pretty major 'feature request', but I think also an
>> important
>> one.
>>
>> When developing large tables, it can often be /necessary/ to start
>> using
>> multi-column/row cells for clarity, and sensible exporting
>> results.
>>
>> As far as I am aware, in Org does not currently have any
>> multi-col/row
>> syntax. The only viable method seems to be re-implementing the
>> table
>> using export blocks in every backend you may want to export to (in
>> my
>> case, usually TeX + HTML). This is clumsy, difficult to work with,
>> and
>> could be avoided should org gain support for multi-col/row syntax.
>>
>> I appreciate that this would constitute a major change both the
>> Org's
>> syntax and the codebase, but I believe such a change is warranted
>> by the
>> advantages it would provide.
>>
>> Both how this can be implemented while minimising/eliminating the
>> chance
>> of breaking well-formed current table elements, and what syntax
>> may be
>> both acceptable and seem sensible to use.
>>
>> I would anticipate such a feature working by designating two
>> characters
>> to indicate "add row" and "add column". For example "|" and "-".
>> These
>> characters would take affect when /immediately following/ (no
>> space) a
>> cell separator ("|"), and designate the dimensions of the top
>> right cell.
>>
>> Example:
>> | a | b | c |
>> |---+---+---|
>> | a | - | | |
>> | - | b | . |
>> | . | | | c |
>>
>> Would be interpreted just as any current table is.
>>
>> However,
>>
>> | hello | there | you  |
>> |-------+-------+------|
>> || two column   | cell |
>>
>> Contains a 2x1 cell.
>>
>> | a little | test |
>> |----------+------|
>> |- hello   | hi   |
>> | two row  | you  |
>>
>> Contains a 1x2 cell. In a more complex example:
>>
>> | a | b | c |
>> |---+---+---|
>> ||-- hi | a |
>> | two x | . |
>> | three | b |
>> | c | - | . |
>>
>> Contains a 2x3 cell.
>>
>> This is just the first syntax that comes to mind, but hopefully
>> the
>> general form of this idea seems viable.
>>
>> All the best,
>>
>> Timothy.
>>

Re: Tables: missing multi-col/row syntax

[TEC]

I think we may be able get something promising by merging your
(Christian + Tom) ideas and David's. What if we have have a
#+TBLCELLMERGE key which acts as you describe, and /just using the
current table syntax/ have something like this (using the example 
from
my first email)

| a | b  | c |
|---+----+---|
| hi    || a |
| two x || . |
| three || b |
|  c | - | . |
#+TBLCELLMERGE: @2$1..@4$2

This is /currently/ a valid Org table, which /currently/ 
autoformats to:
| a     | b | c |
|-------+---+---|
| hi    |   | a |
| two x |   | . |
| three |   | b |
| c     | - | . |

So with an autoformatting change + an overlay, perhaps we can do 
this
nicely without any syntax changes 😃.

Thoughts?

Christian Moe <mail@christianmoe.com> writes:

> +1 for enabling table-cell merges in export. I imagine this 
> would be a
> tricky job for developers, but it would relieve me as a user of 
> much
> repeated fiddling with exported drafts.
>
> +1 for doing it without adding clutter to the table syntax, but
> specifying merges on a separate line like formulas, like Tom's
>
>   #+TBLCELLMERGE: @2..3$1
>
> (amended here to use the established '..' rather than hyphen for 
> range)
>
> Though if we do add such a line, we might also think of a more 
> general
> solution that could over time be extended with additional 
> formatting
> options, e.g. something like
>
>   #+TBLSTYLE: @2..3$1='(:merge t)::@4$1='(:bgcolor yellow :color 
>   red)
>
> But obviously that could open a can of worms, aka potentially 
> endless
> feature requests requiring different implementations for each 
> backend.
>
> Yours,
> Christian
>
>
>
> Tom Gillespie writes:
>
>> Any support for something like this would need to retain 
>> backward
>> compatibility as well to avoid older versions reformatting the 
>> tables
>> due to e.g. the presence of a double pipe. I also think that 
>> extending
>> the table syntax in ways that makes it more complex than it 
>> already
>> is, will be a non-starter. Thus, an alternate but more likely 
>> approach
>> would be to allow specification of what cells to merge outside 
>> the
>> table as is done for formulas. It is not elegant, but it would 
>> be a
>> layer on top of existing syntax, and it would allow the 
>> fundamental
>> structure of the table to remain the same -- rows of cells. For
>> example #+TBLCELLMERGE: @2-3$1 or something like that. 
>> Thoughts?
>> Tom
>>
>> On Mon, Nov 2, 2020 at 1:37 PM TEC <tecosaur@gmail.com> wrote:
>>>
>>> Hi all,
>>>
>>> This is a pretty major 'feature request', but I think also an
>>> important
>>> one.
>>>
>>> When developing large tables, it can often be /necessary/ to 
>>> start
>>> using
>>> multi-column/row cells for clarity, and sensible exporting
>>> results.
>>>
>>> As far as I am aware, in Org does not currently have any
>>> multi-col/row
>>> syntax. The only viable method seems to be re-implementing the
>>> table
>>> using export blocks in every backend you may want to export to 
>>> (in
>>> my
>>> case, usually TeX + HTML). This is clumsy, difficult to work 
>>> with,
>>> and
>>> could be avoided should org gain support for multi-col/row 
>>> syntax.
>>>
>>> I appreciate that this would constitute a major change both 
>>> the
>>> Org's
>>> syntax and the codebase, but I believe such a change is 
>>> warranted
>>> by the
>>> advantages it would provide.
>>>
>>> Both how this can be implemented while minimising/eliminating 
>>> the
>>> chance
>>> of breaking well-formed current table elements, and what 
>>> syntax
>>> may be
>>> both acceptable and seem sensible to use.
>>>
>>> I would anticipate such a feature working by designating two
>>> characters
>>> to indicate "add row" and "add column". For example "|" and 
>>> "-".
>>> These
>>> characters would take affect when /immediately following/ (no
>>> space) a
>>> cell separator ("|"), and designate the dimensions of the top
>>> right cell.
>>>
>>> Example:
>>> | a | b | c |
>>> |---+---+---|
>>> | a | - | | |
>>> | - | b | . |
>>> | . | | | c |
>>>
>>> Would be interpreted just as any current table is.
>>>
>>> However,
>>>
>>> | hello | there | you  |
>>> |-------+-------+------|
>>> || two column   | cell |
>>>
>>> Contains a 2x1 cell.
>>>
>>> | a little | test |
>>> |----------+------|
>>> |- hello   | hi   |
>>> | two row  | you  |
>>>
>>> Contains a 1x2 cell. In a more complex example:
>>>
>>> | a | b | c |
>>> |---+---+---|
>>> ||-- hi | a |
>>> | two x | . |
>>> | three | b |
>>> | c | - | . |
>>>
>>> Contains a 2x3 cell.
>>>
>>> This is just the first syntax that comes to mind, but 
>>> hopefully
>>> the
>>> general form of this idea seems viable.
>>>
>>> All the best,
>>>
>>> Timothy.
>>>