Fwd: comment lines inside org tables

Fwd: comment lines inside org tables

[Rolf Sander (MPI)]

Hello,

I need to put comments inside a very big table which then should
basically looks like this:

|--------+-------+--------|
| animal | size  | number |
|--------+-------+--------|
| gnus   | big   |      3 |
# don't forget to add elephants here:
| gnats  | small |   1000 |
|--------+-------+--------|

However, if I simply add a line in the middle of the table, org-tbl
splits it into separate tables with different column widths:

|--------+------+--------|
| animal | size | number |
|--------+------+--------|
| gnus   | big  |      3 |
# don't forget to add elephants here:
| gnats | small | 1000 |
|-------+-------+------|

If I start the comment line with "|", the comment becomes an active part
of the table and goes into the first column:

|-------------------------------------+-------+--------|
| animal                              | size  | number |
|-------------------------------------+-------+--------|
| gnus                                | big   |      3 |
| don't forget to add elephants here: |       |        |
| gnats                               | small |   1000 |
|-------------------------------------+-------+--------|

If I start the comment with "|-", it turns into a line separator and the
comment text disappears.

Any suggestions?

       Best regards
              Rolf

-- 
-----------------------------------------------------------------------
    Rolf Sander                              phone: [+49] 6131/305-4610
    Max-Planck Institute of Chemistry        email: rolf.sander@mpic.de
    PO Box 3060, 55020 Mainz, Germany     homepage: www.rolf-sander.net
-----------------------------------------------------------------------
                 www.atmospheric-chemistry-and-physics.net
                  www.geoscientific-model-development.net
-----------------------------------------------------------------------

Re: Fwd: comment lines inside org tables

[John Kitchin]

Use a footnote? It's just a temporary reminder right?

On September 27, 2016, at 6:56 AM, "Rolf Sander (MPI)" <rolf.sander@mpic.de> wrote:

Hello,

I need to put comments inside a very big table which then should
basically looks like this:

|--------+-------+--------|
| animal | size  | number |
|--------+-------+--------|
| gnus   | big   |      3 |
# don't forget to add elephants here:
| gnats  | small |   1000 |
|--------+-------+--------|

However, if I simply add a line in the middle of the table, org-tbl
splits it into separate tables with different column widths:

|--------+------+--------|
| animal | size | number |
|--------+------+--------|
| gnus   | big  |      3 |
# don't forget to add elephants here:
| gnats | small | 1000 |
|-------+-------+------|

If I start the comment line with "|", the comment becomes an active part
of the table and goes into the first column:

|-------------------------------------+-------+--------|
| animal                              | size  | number |
|-------------------------------------+-------+--------|
| gnus                                | big   |      3 |
| don't forget to add elephants here: |       |        |
| gnats                               | small |   1000 |
|-------------------------------------+-------+--------|

If I start the comment with "|-", it turns into a line separator and the
comment text disappears.

Any suggestions?

       Best regards
              Rolf

-- 
-----------------------------------------------------------------------
    Rolf Sander                              phone: [+49] 6131/305-4610
    Max-Planck Institute of Chemistry        email: rolf.sander@mpic.de
    PO Box 3060, 55020 Mainz, Germany     homepage: www.rolf-sander.net
-----------------------------------------------------------------------
                 www.atmospheric-chemistry-and-physics.net
                  www.geoscientific-model-development.net
-----------------------------------------------------------------------

Re: Fwd: comment lines inside org tables

[Nicolas Goaziou]

Hello,

"Rolf Sander (MPI)" <rolf.sander@mpic.de> writes:

> I need to put comments inside a very big table which then should
> basically looks like this:
>
> |--------+-------+--------|
> | animal | size  | number |
> |--------+-------+--------|
> | gnus   | big   |      3 |
> # don't forget to add elephants here:
> | gnats  | small |   1000 |
> |--------+-------+--------|

Try

  |---+-------------------------------------+-------+--------|
  |   | animal                              | size  | number |
  |---+-------------------------------------+-------+--------|
  |   | gnus                                | big   |      3 |
  | / | don't forget to add elephants here: |       |        |
  |   | gnats                               | small |   1000 |
  |---+-------------------------------------+-------+--------|


See (info "(org) Advanced features").

Regards,

-- 
Nicolas Goaziou

Re: Fwd: comment lines inside org tables

[Rolf Sander (MPI)]

Hello Nicolas,

> Try
> |   | animal                              | size  | number |
> | / | don't forget to add elephants here: |       |        |

Thanks for the suggestion. However, this won't help me very much. Even
with the "/", the width of the "animal" column is expanded to the width
of the comment, which can be very long. I could add a narrowing marker
but I want to see the comments, not hide them.

      Best regards
             Rolf

-- 
-----------------------------------------------------------------------
   Rolf Sander                              phone: [+49] 6131/305-4610
   Max-Planck Institute of Chemistry        email: rolf.sander@mpic.de
   PO Box 3060, 55020 Mainz, Germany     homepage: www.rolf-sander.net
-----------------------------------------------------------------------
                www.atmospheric-chemistry-and-physics.net
                 www.geoscientific-model-development.net
-----------------------------------------------------------------------

Re: Fwd: comment lines inside org tables

[Nicolas Goaziou]

Hello,

"Rolf Sander (MPI)" <rolf.sander@mpic.de> writes:

> Thanks for the suggestion. However, this won't help me very much. Even
> with the "/", the width of the "animal" column is expanded to the width
> of the comment, which can be very long. I could add a narrowing marker
> but I want to see the comments, not hide them.

Then I don't understand what you really want. You want to comment a row,
but it's not clear wrt to what action.

Anyway, in the general case, there is no way to comment a row.

Regards,

-- 
Nicolas Goaziou

Re: Fwd: comment lines inside org tables

[Rolf Sander]

Hello Nicolas,

> Then I don't understand what you really want. You want to comment a
> row, but it's not clear wrt to what action.

I want to add a comment line into the file that does not disturb the
table. Like a "|---+---+---|" hline. I don't want any action. All
programming languages have the option to add comments to the source
code. The comment does not trigger any action, and the result of
executing the code is the same, whether with or without comment lines.
It would be nice if org-tbl would allow comment lines as well.

> Anyway, in the general case, there is no way to comment a row.

Okay, so my question turns into a feature request now...

I have started to modify defun org-table-align. With the modification,
org-tbl now ignores all lines starting with "|-/". For me it works fine.
However, please let me know if you see any side-effects. You can
download the full defun here:

http://www.rolf-sander.net/tmp/org-table-align-new.el

And here is a diff to the original code:

*******************************************************
diff -U 1 org-table-align-old.el org-table-align-new.el
@@ -63,4 +63,12 @@
      ;; At the same time, we remove trailing space.
+    (setq lines0 (mapcar (lambda (l)
+                          (cond ((string-match "^ *|-" l)
+                                 nil)
+                                ((string-match "[ \t]+$" l)
+                                 (substring l 0 (match-beginning 0)))
+                                (t l))
+                          )
+			lines))
      (setq lines (mapcar (lambda (l)
-			  (if (string-match "^ *|-" l)
+			  (if (string-match "^ *|-\\($\\|[^/]\\)" l)
  			      nil
@@ -74,3 +82,3 @@
  		    (org-split-string l " *| *"))
-		  (delq nil (copy-sequence lines))))
+                  (delq nil (copy-sequence lines0))))
      ;; How many fields in the longest line?
@@ -164,5 +172,9 @@
  	       (lambda (l)
-		 (if l (apply 'format rfmt
-			      (append (pop fields) emptystrings))
-		   hfmt))
+                 (cond ((eq l nil)
+                        hfmt)
+                       ((string-match "^ *|-/" l)
+                        (concat l "\n"))
+                       (t
+                        (apply 'format rfmt
+                               (append (pop fields) emptystrings)))))
  	       lines ""))
*******************************************************

      Best regards
             Rolf

-- 
-----------------------------------------------------------------------
   Rolf Sander                              phone: [+49] 6131/305-4610
   Max-Planck Institute of Chemistry        email: rolf.sander@mpic.de
   PO Box 3060, 55020 Mainz, Germany     homepage: www.rolf-sander.net
-----------------------------------------------------------------------
                www.atmospheric-chemistry-and-physics.net
                 www.geoscientific-model-development.net
-----------------------------------------------------------------------

Re: Fwd: comment lines inside org tables

[Nicolas Goaziou]

Hello,

Rolf Sander <rolf.sander@mpic.de> writes:

> I want to add a comment line into the file that does not disturb the
> table.

Disturb the table wrt to what action? Alignment? This is a minor issue
if you fix an arbitrary column width, isn't it?

> Like a "|---+---+---|" hline. I don't want any action. All
> programming languages have the option to add comments to the source
> code.

But Org is not a programming language, and an Org document is not source
code.

> Okay, so my question turns into a feature request now...

Allowing to comment anything is a can of worms I really don't want to
open. It complicates a lot of code for little benefit.

> I have started to modify defun org-table-align. With the modification,
> org-tbl now ignores all lines starting with "|-/". For me it works
> fine.

Sorry, but you're barely scratching the surface of the problem. Tables
are not just about alignment. Formulas don't handle these special lines,
neither do exported tables.

Again, this implies a serious work on org-table.el, and it would
introduce inconsistencies with other parts of Org (e.g., lists, where
you cannot comment an item, or even the syntax for commenting table
rows, which would be unusual).

All in all, I think there are more important itches to scratch for the
time being. Of course, YMMV.


Regards,

-- 
Nicolas Goaziou

Re: Fwd: comment lines inside org tables

[Rolf Sander]

Hello Nicolas,

> Allowing to comment anything is a can of worms I really don't want to
> open.

Maybe you are misinterpreting what I wrote. I don't want to comment
"anything". All I want is that a table line starting with "|-/" is not
expanded to "|---+---+---|" on the next re-align. Instead, it should
remain unchanged.

> It complicates a lot of code for little benefit.

Well, regarding the benefit we seem to have different opinions. The
table that I use for my work has 871 lines, 20 columns and a width of
234 characters. With such a big table, I would highly benefit from
comment lines.

Regarding the code, I already wrote a few lines for org-table-align
adding this feature and I don't think it is too complicated.

> Sorry, but you're barely scratching the surface of the problem.

It is quite possible that you are right. Indeed, there is a possibility
that my addition has some negative side effects. To find out if this is
really the case, I have written to this email list, asking for advise.

> Formulas don't handle these special lines, neither do exported tables.

I don't understand why you say this. I have tested formulas and
exporting, taking an example table from the manual and adding two
comment lines. Everything still works fine:

|---+---------+--------+--------+--------+-------+------|
|   | Student | Prob 1 | Prob 2 | Prob 3 | Total | Note |
|---+---------+--------+--------+--------+-------+------|
| ! |         |     P1 |     P2 |     P3 |   Tot |      |
| # | Maximum |     10 |     15 |     25 |    50 | 10.0 |
| ^ |         |     m1 |     m2 |     m3 |    mt |      |
|---+---------+--------+--------+--------+-------+------|
|-/ students in Mr Miller's class:
|---+---------+--------+--------+--------+-------+------|
| # | Peter   |     10 |      8 |     23 |    41 |  8.2 |
| # | Sam     |      2 |      4 |      3 |     9 |  1.8 |
|---+---------+--------+--------+--------+-------+------|
|-/ students in Mr Smith's class:
|---+---------+--------+--------+--------+-------+------|
| # | Tom     |     16 |      8 |     28 |    52 | 10.4 |
| # | Andy    |      8 |      4 |      3 |    15 |  3.0 |
|---+---------+--------+--------+--------+-------+------|
|   | Average |        |        |        |  25.0 |      |
| ^ |         |        |        |        |    at |      |
| $ | max=50  |        |        |        |       |      |
|---+---------+--------+--------+--------+-------+------|
#+TBLFM: $6=vsum($P1..$P3)::$7=10*$Tot/$max;%.1f::$at=vmean(@-II..@-I);%.1f

So I'm asking again: Does anyone see any negative side effects when my
modification is introduced? It only prevents that a table line starting
with "|-/" is expanded to "|---+---+---|" on a re-align.

As mentioned in a previous post, if you want to test my modification,
you can find it here:

http://www.rolf-sander.net/tmp/org-table-align-new.el

      Best regards
             Rolf

-- 
-----------------------------------------------------------------------
                          www.rolf-sander.net
-----------------------------------------------------------------------

Re: Fwd: comment lines inside org tables

[Rolf Sander (MPI)]

Hello Nicolas,

> Allowing to comment anything is a can of worms I really don't want to
> open.

Maybe you are misinterpreting what I wrote. I don't want to comment
"anything". All I want is that a table line starting with "|-/" is not
expanded to "|---+---+---|" on the next re-align. Instead, it should
remain unchanged.

> It complicates a lot of code for little benefit.

Well, regarding the benefit we seem to have different opinions. The
table that I use for my work has 871 lines, 20 columns and a width of
234 characters. With such a big table, I would highly benefit from
comment lines.

Regarding the code, I already wrote a few lines for org-table-align
adding this feature and I don't think it is too complicated.

> Sorry, but you're barely scratching the surface of the problem.

It is quite possible that you are right. Indeed, there is a possibility
that my addition has some negative side effects. To find out if this is
really the case, I have written to this email list, asking for advise.

> Formulas don't handle these special lines, neither do exported tables.

I don't understand why you say this. I have tested formulas and
exporting, taking an example table from the manual and adding two
comment lines. Everything still works fine:

|---+---------+--------+--------+--------+-------+------|
|   | Student | Prob 1 | Prob 2 | Prob 3 | Total | Note |
|---+---------+--------+--------+--------+-------+------|
| ! |         |     P1 |     P2 |     P3 |   Tot |      |
| # | Maximum |     10 |     15 |     25 |    50 | 10.0 |
| ^ |         |     m1 |     m2 |     m3 |    mt |      |
|---+---------+--------+--------+--------+-------+------|
|-/ students in Mr Miller's class:
|---+---------+--------+--------+--------+-------+------|
| # | Peter   |     10 |      8 |     23 |    41 |  8.2 |
| # | Sam     |      2 |      4 |      3 |     9 |  1.8 |
|---+---------+--------+--------+--------+-------+------|
|-/ students in Mr Smith's class:
|---+---------+--------+--------+--------+-------+------|
| # | Tom     |     16 |      8 |     28 |    52 | 10.4 |
| # | Andy    |      8 |      4 |      3 |    15 |  3.0 |
|---+---------+--------+--------+--------+-------+------|
|   | Average |        |        |        |  25.0 |      |
| ^ |         |        |        |        |    at |      |
| $ | max=50  |        |        |        |       |      |
|---+---------+--------+--------+--------+-------+------|
#+TBLFM: $6=vsum($P1..$P3)::$7=10*$Tot/$max;%.1f::$at=vmean(@-II..@-I);%.1f

So I'm asking again: Does anyone see any negative side effects when my
modification is introduced? It only prevents that a table line starting
with "|-/" is expanded to "|---+---+---|" on a re-align.

As mentioned in a previous post, if you want to test my modification,
you can find it here:

http://www.rolf-sander.net/tmp/org-table-align-new.el

       Best regards
              Rolf

-- 
-----------------------------------------------------------------------
                           www.rolf-sander.net
-----------------------------------------------------------------------

Re: Fwd: comment lines inside org tables

[Eric S Fraga]

Sorry to interrupt this conversation but I wonder: could you not simply
use the / label and then have a comment in one of the existing columns,
a column that is width restricted so that a large amount of text doesn't
affect alignent?  You can then use the special edit mode (C-c `) for
long entries to see the full comment.

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.2.50.1, Org release_8.3.6-1184-g29830f

Re: Fwd: comment lines inside org tables

[Rolf Sander (MPI)]

Hello Eric,

> could you not simply use the / label and then have a comment in one of
> the existing columns, a column that is width restricted so that a
> large amount of text doesn't affect alignent? You can then use the
> special edit mode (C-c `) for long entries to see the full comment.

Thanks for the suggestion. Yes, I could do this and see something like
this in emacs:

|---+--------------+-------+--------|
|   | <12>         |       |        |
|   | animal       | size  | number |
|---+--------------+-------+--------|
|   | gnus         | big   |      3 |
| / | don't forg=> |       |        |
|   | gnats        | small |   1000 |
|---+--------------+-------+--------|

The real contents of the file, however, would look very different:

|---+--------------+-------+--------|
|   | <12>         |       |        |
|   | animal       | size  | number |
|---+--------------+-------+--------|
|   | gnus         | big   |      3 |
| / | don't forget to add elephants here: |       |        |
|   | gnats        | small |   1000 |
|---+--------------+-------+--------|

Since I am sharing my table with several colleagues at work, it is
important that the columns are really aligned in the file (not only when
viewing the file in org-mode). Unfortunately, not all of my colleagues
are using emacs...

I am still waiting for an answer to my question if the solution with
"|-/" comments that I have proposed has any negative side effects...

      Best regards
             Rolf

-- 
-----------------------------------------------------------------------
   Rolf Sander                              phone: [+49] 6131/305-4610
   Max-Planck Institute of Chemistry        email: rolf.sander@mpic.de
   PO Box 3060, 55020 Mainz, Germany     homepage: www.rolf-sander.net
-----------------------------------------------------------------------
                www.atmospheric-chemistry-and-physics.net
                 www.geoscientific-model-development.net
-----------------------------------------------------------------------

Re: Fwd: comment lines inside org tables

[Nicolas Goaziou]

Hello,

"Rolf Sander (MPI)" <rolf.sander@mpic.de> writes:

> I am still waiting for an answer to my question if the solution with
> "|-/" comments that I have proposed has any negative side effects...

I already answered to that question, but you discarded my answer.

Again, you're hacking a new syntax that the parser doesn't know about.
Besides, the status of this new row type is undefined (wrt to export, to
formulas, to radio tables...). For these reasons, it is bound to have
negative side effects.

Regards,

-- 
Nicolas Goaziou

Re: Fwd: comment lines inside org tables

[Rolf Sander (MPI)]

Hello Nicolas and everyone else,

I'm sorry to say this but this email list has been the most
disappointing experience for me. I asked a simple question and even
provided the code for a possible solution. The answers I received
included phrases like "can of worms", "little benefit" and "barely
scratching the surface of the problem". And now you even wrote:

>> I am still waiting for an answer to my question if the solution with
>> "|-/" comments that I have proposed has any negative side effects...

> I already answered to that question, but you discarded my answer.

This is not true! You did not answer my question! Without testing, you
just _speculated_ that you expect problems with formulas and export. So
I tested it myself. In my tests I did _not_ experience any problems with
formulas and export. Why do you claim that I discarded your answer?

I still think orgmode is great code and I would have loved to
participate. However, given the way that you treat me here, I lost my
motivation to work with you. Instead, I will have to modify my own *.el
files.

Good bye...
Rolf

-- 
-----------------------------------------------------------------------
   Rolf Sander                              phone: [+49] 6131/305-4610
   Max-Planck Institute of Chemistry        email: rolf.sander@mpic.de
   PO Box 3060, 55020 Mainz, Germany     homepage: www.rolf-sander.net
-----------------------------------------------------------------------
                www.atmospheric-chemistry-and-physics.net
                 www.geoscientific-model-development.net
-----------------------------------------------------------------------

Re: Fwd: comment lines inside org tables

[Thomas S. Dye]

Rolf Sander (MPI) writes:

> I still think orgmode is great code and I would have loved to
> participate. However, given the way that you treat me here, I lost my
> motivation to work with you. Instead, I will have to modify my own *.el
> files.

Git makes this relatively painless.  My sense is that quite a few Org
mode users keep local git branches where they experiment with features
that haven't, for whatever reason, made their way to contrib or core.

All the best,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Fwd: comment lines inside org tables

[Nicolas Goaziou]

Hello,

"Rolf Sander (MPI)" <rolf.sander@mpic.de> writes:

> I'm sorry to say this but this email list has been the most
> disappointing experience for me. I asked a simple question and even
> provided the code for a possible solution. The answers I received
> included phrases like "can of worms", "little benefit" and "barely
> scratching the surface of the problem".

I think that blaming the whole list is uncalled for. You got helpful
answers from other persons than me, yet you only quote parts of my
answers.

> And now you even wrote:

[...]

>> I already answered to that question, but you discarded my answer.
>
> This is not true! You did not answer my question! Without testing, you
> just _speculated_ that you expect problems with formulas and export.

What makes you think I speculated anything? I tried to show you the weak
spots of your model.

> So I tested it myself. In my tests I did _not_ experience any problems
> with formulas and export. Why do you claim that I discarded your
> answer?

Exhibiting one example confirming a theory doesn't validate it.

Anyway, you can do the following with your patch:

 - Move point to your "comment row"
 - M-: (org-element-property :type (org-element-at-point))

If you get `rule', by all means, Org sees your row as a standard table
rule. This is not transparent; this is /not/ a comment. You just created
a degenerate syntax for table rules.

Note that this may be what you really want, but this is not equivalent
to the initial example you gave:

  |--------+-------+--------|
  | animal | size  | number |
  |--------+-------+--------|
  | gnus   | big   |      3 |
  # don't forget to add elephants here:
  | gnats  | small |   1000 |
  |--------+-------+--------|

If such thing existed, the "gnus" and "gnats" rows wouldn't be separated
by a rule, e.g., in export, because Org removes such lines prior to
starting the export process. Another example is the following:

  |--------+-------+--------|
  | animal | size  | number |
  |-/ whatever
  | gnus   | big   |      3 |
  | gnats  | small |   1000 |
  |--------+-------+--------|

In this case, you are creating a table header, so this is not
transparent either. I could also add an example where an additional rule
has an impact on formulas (@I..@II could have a different meaning
depending on the presence or not of another rule).

All in all, I think your problem is ill-defined. You wanted comment
rows, but you didn't specify how it should behave in various situations
involving tables. I stand on my ground : generic comment rows are
difficult to implement and "a can of worms".

Allowing text within a table rule is easier to achieve, as you proved,
but it feels very hackish and limited in use. What if I want to
introduce comments without creating a visible rule in the table? Do
I need yet another syntax?

> I still think orgmode is great code and I would have loved to
> participate. However, given the way that you treat me here,

I don't treat you in any way. I just suggested your idea was wrong.
I understand this can be frustrating. I would be frustrated too.


Regards,

-- 
Nicolas Goaziou