text-only plots

text-only plots

[Thierry Banel]

Hi all Org fans.

What about text-only plots in tables ?
All in Emacs, without any external package.

|  x | x^2 |              |
|----+-----+--------------|
|  0 |   0 |              |
|  1 |   1 | .            |
|  2 |   4 | :            |
|  3 |   9 | u            |
|  4 |  16 | W            |
|  5 |  25 | Wu           |
|  6 |  36 | WW:          |
|  7 |  49 | WWH          |
|  8 |  64 | WWWV         |
|  9 |  81 | WWWWH        |
| 10 | 100 | WWWWWW       |
| 11 | 121 | WWWWWWW-     |
| 12 | 144 | WWWWWWWWl    |
| 13 | 169 | WWWWWWWWWW.  |
| 14 | 196 | WWWWWWWWWWWV |
#+TBLFM: $3='(orgtbl-ascii-draw $2 0 200)

I wrote a package to draw such small-quick-and-text-only graphs.
It is here: http://orgmode.org/worg/org-contrib/orgtbl-ascii-plot.html
Just type C-c p

Comments and enhancements are welcome!
Have fun.

Re: text-only plots

[Suvayu Ali]

On Sat, Dec 07, 2013 at 07:48:51PM +0000, Thierry Banel wrote:
> 
> I wrote a package to draw such small-quick-and-text-only graphs.
> It is here: http://orgmode.org/worg/org-contrib/orgtbl-ascii-plot.html
> Just type C-c p

This looks cool!

-- 
Suvayu

Open source is the future. It sets us free.

Re: text-only plots

[Michael Brand]

[-- Attachment #1: Type: text/plain, Size: 1862 bytes --]

Hi Thierry

On Sat, Dec 7, 2013 at 8:48 PM, Thierry Banel <tbanelwebmin@free.fr> wrote:
> What about text-only plots in tables ?
> All in Emacs, without any external package.
>
> |  x | x^2 |              |
> |----+-----+--------------|
> |  0 |   0 |              |
> |  1 |   1 | .            |
> |  2 |   4 | :            |
> |  3 |   9 | u            |
> |  4 |  16 | W            |
> |  5 |  25 | Wu           |
> |  6 |  36 | WW:          |
> |  7 |  49 | WWH          |
> |  8 |  64 | WWWV         |
> |  9 |  81 | WWWWH        |
> | 10 | 100 | WWWWWW       |
> | 11 | 121 | WWWWWWW-     |
> | 12 | 144 | WWWWWWWWl    |
> | 13 | 169 | WWWWWWWWWW.  |
> | 14 | 196 | WWWWWWWWWWWV |
> #+TBLFM: $3='(orgtbl-ascii-draw $2 0 200)
>
> I wrote a package to draw such small-quick-and-text-only graphs.
> It is here: http://orgmode.org/worg/org-contrib/orgtbl-ascii-plot.html
> Just type C-c p
>
> Comments and enhancements are welcome!
> Have fun.

Your idea is very good. I suggest to have also a unicode variant using
http://en.wikipedia.org/wiki/Block_Elements
with (elt " ▏▎▍▌▋▊▉" [...]) to divide one char into eight widths which
I will use often for time series:

| year |  % | ascii    | unicode  |
|------+----+----------+----------|
| 2009 | 55 | WWWWWWl  | ▉▉▉▉▉▉▌  |
| 2010 | 54 | WWWWWWu  | ▉▉▉▉▉▉▍  |
| 2011 | 60 | WWWWWWW: | ▉▉▉▉▉▉▉▏ |
| 2012 | 62 | WWWWWWW; | ▉▉▉▉▉▉▉▍ |
#+TBLFM: $3 = '(orgtbl-ascii-draw $2 0 100) :: $4 = '(orgtbl-uc-draw $2 0 100)

Attached is a screenshot with DejaVu Sans Mono as an example of a font
that shows the same block height for all block widths (some other
fonts don't show the same block height for all block widths). Remember
"C-u C-x =" too see the name etc. of the char at point.

Michael

[-- Attachment #2: unicode.png --]
[-- Type: image/png, Size: 11488 bytes --]

Re: text-only plots

[Michael Brand]

[-- Attachment #1: Type: text/plain, Size: 1341 bytes --]

On Sun, Dec 8, 2013 at 12:27 PM, Michael Brand
<michael.ch.brand@gmail.com> wrote:
> I suggest to have also a unicode variant using
> http://en.wikipedia.org/wiki/Block_Elements

To make it more complete: Distinguish between the variant with
coordinate grid and the one continuous:

| year |  % | ascii    | grid (0 to 7 8th) | continuous (0 to 8 8th) |
|------+----+----------+-------------------+-------------------------|
| 2009 | 55 | WWWWWWl  | ▉▉▉▉▉▉▌           | ██████▋                 |
| 2010 | 54 | WWWWWWu  | ▉▉▉▉▉▉▍           | ██████▌                 |
| 2011 | 60 | WWWWWWW: | ▉▉▉▉▉▉▉▏          | ███████▎                |
| 2012 | 62 | WWWWWWW; | ▉▉▉▉▉▉▉▍          | ███████▌                |
#+TBLFM: $3 = '(orgtbl-ascii-draw $2 0 100) :: $4 =
'(orgtbl-uc-draw-grid $2 0 100) :: $5 = '(orgtbl-uc-draw-cont $2 0
100)

#+BEGIN_SRC emacs-lisp
  ;; orgtbl-uc-draw-grid
  [...]
  (make-string (floor column) ?\▉)
  (string (elt " ▏▎▍▌▋▊▉" (round (* (- column (floor column)) 7))))))))
  ;; orgtbl-uc-draw-cont
  [...]
  (make-string (floor column) ?\█)
  (string (elt " ▏▎▍▌▋▊▉█" (round (* (- column (floor column)) 8))))))))
#+END_SRC

Michael

[-- Attachment #2: DejaVu_Sans_Mono.png --]
[-- Type: image/png, Size: 16952 bytes --]

Re: text-only plots

[Thierry Banel]

Michael Brand <michael.ch.brand <at> gmail.com> writes:

> To make it more complete: Distinguish between the variant with
> coordinate grid and the one continuous:
> 
> | year |  % | ascii    | grid (0 to 7 8th) | continuous (0 to 8 8th) |
> |------+----+----------+-------------------+-------------------------|
> | 2009 | 55 | WWWWWWl  | ▉▉▉▉▉▉▌           | ██████▋                 |
> | 2010 | 54 | WWWWWWu  | ▉▉▉▉▉▉▍           | ██████▌                 |
> | 2011 | 60 | WWWWWWW: | ▉▉▉▉▉▉▉▏          | ███████▎                |
> | 2012 | 62 | WWWWWWW; | ▉▉▉▉▉▉▉▍          | ███████▌                |
> #+TBLFM: $3 = '(orgtbl-ascii-draw $2 0 100) :: $4 =
> '(orgtbl-uc-draw-grid $2 0 100) :: $5 = '(orgtbl-uc-draw-cont $2 0
> 100)


Hi Michael

I love your extension!
I will integrate it.
The unicode (unicorn?) version should fall-back gracefully to the ascii
version if utf is not supported.

Thierry

Re: text-only plots

[Achim Gratz]

Michael Brand writes:
> Your idea is very good. I suggest to have also a unicode variant using
> http://en.wikipedia.org/wiki/Block_Elements
> with (elt " ▏▎▍▌▋▊▉" [...]) to divide one char into eight widths which

These would be even more useful if all UPPER and RIGHT variants
existed.  Oh well.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: text-only plots

[Thierry Banel]

Hello all.

Following a suggestion from Michael Brand (thanks, Michael!),
I implemented a version able to use unicode block characters.
It is here:
http://orgmode.org/worg/org-contrib/orgtbl-ascii-plot.html

Use =C-u C-c p= in the column to plot, and answer questions.
Without =C-u= prefix, the base ascii, 12 characters wide version is used.

|  x | x^2 | ascii        | uc cont      | uc grid      |
|----+-----+--------------+--------------+--------------|
|  0 |   0 |              |              |              |
|  1 |   1 |              |              |              |
|  2 |   4 | :            | ▏            | ▏            |
|  3 |   9 | u            | ▍            | ▌            |
|  4 |  16 | H            | ▊            | ▉            |
|  5 |  25 | Wu           | ▉▍           | █▌           |
|  6 |  36 | WW.          | ▉▉▏          | ██▏          |
|  7 |  49 | WWH          | ▉▉▉          | ███          |
|  8 |  64 | WWWV         | ▉▉▉▊         | ███▉         |
|  9 |  81 | WWWWV        | ▉▉▉▉▊        | ████▉        |
| 10 | 100 | WWWWWW       | ▉▉▉▉▉▉       | ██████       |
| 11 | 121 | WWWWWWW:     | ▉▉▉▉▉▉▉▎     | ███████▍     |
| 12 | 144 | WWWWWWWWl    | ▉▉▉▉▉▉▉▉▋    | ████████▊    |
| 13 | 169 | WWWWWWWWWW.  | ▉▉▉▉▉▉▉▉▉▉▎  | ██████████▎  |
| 14 | 196 | WWWWWWWWWWWh | ▉▉▉▉▉▉▉▉▉▉▉▉ | ████████████ |
#+TBLFM: $4='(orgtbl-uc-draw-grid $2 0 196 12)::$5='(orgtbl-uc-draw-cont $2
0 196 12)::$3='(orgtbl-ascii-draw $2 0 200)

Have fun
Thierry

Re: text-only plots

[Carsten Dominik]

Hi Thierry,

This is really nice. Thank you!

This is not a lot of code - I would not mind simply integrating this into org-table.el  The only hickup might be that this could be trouble for the exporters - or have you produced the webpage directly from an org file?

- Carsten

On 8.12.2013, at 18:43, Thierry Banel <tbanelwebmin@free.fr> wrote:

> Hello all.
> 
> Following a suggestion from Michael Brand (thanks, Michael!),
> I implemented a version able to use unicode block characters.
> It is here:
> http://orgmode.org/worg/org-contrib/orgtbl-ascii-plot.html
> 
> Use =C-u C-c p= in the column to plot, and answer questions.
> Without =C-u= prefix, the base ascii, 12 characters wide version is used.
> 
> |  x | x^2 | ascii        | uc cont      | uc grid      |
> |----+-----+--------------+--------------+--------------|
> |  0 |   0 |              |              |              |
> |  1 |   1 |              |              |              |
> |  2 |   4 | :            | ▏            | ▏            |
> |  3 |   9 | u            | ▍            | ▌            |
> |  4 |  16 | H            | ▊            | ▉            |
> |  5 |  25 | Wu           | ▉▍           | █▌           |
> |  6 |  36 | WW.          | ▉▉▏          | ██▏          |
> |  7 |  49 | WWH          | ▉▉▉          | ███          |
> |  8 |  64 | WWWV         | ▉▉▉▊         | ███▉         |
> |  9 |  81 | WWWWV        | ▉▉▉▉▊        | ████▉        |
> | 10 | 100 | WWWWWW       | ▉▉▉▉▉▉       | ██████       |
> | 11 | 121 | WWWWWWW:     | ▉▉▉▉▉▉▉▎     | ███████▍     |
> | 12 | 144 | WWWWWWWWl    | ▉▉▉▉▉▉▉▉▋    | ████████▊    |
> | 13 | 169 | WWWWWWWWWW.  | ▉▉▉▉▉▉▉▉▉▉▎  | ██████████▎  |
> | 14 | 196 | WWWWWWWWWWWh | ▉▉▉▉▉▉▉▉▉▉▉▉ | ████████████ |
> #+TBLFM: $4='(orgtbl-uc-draw-grid $2 0 196 12)::$5='(orgtbl-uc-draw-cont $2
> 0 196 12)::$3='(orgtbl-ascii-draw $2 0 200)
> 
> Have fun
> Thierry
> 
> 
> 
> 

Re: text-only plots

[Michael Brand]

Hi Thierry

On Sun, Dec 8, 2013 at 6:43 PM, Thierry Banel <tbanelwebmin@free.fr> wrote:
> Following a suggestion from Michael Brand (thanks, Michael!),
> I implemented a version able to use unicode block characters.
> It is here:
> http://orgmode.org/worg/org-contrib/orgtbl-ascii-plot.html

Thank you for implementing these variations.

Michael

Re: text-only plots

[Thierry Banel]

Carsten Dominik <carsten.dominik <at> gmail.com> writes:

> 
> Hi Thierry,
> 
> This is really nice. Thank you!
> 
> This is not a lot of code - I would not mind simply integrating this into
org-table.el  The only hickup might be
> that this could be trouble for the exporters - or have you produced the
webpage directly from an org file?
> 
> - Carsten
> 


Hi Carsten.

Wonderful!
It will be useful to many.

I'm not sure I understand the exporters issue.
Do you talk about the html exporter, the Odt exporter, and so on ?
Well, once a plot has been added to a table, it is just text.
It will be exported as any other content in the table.
Maybe I'm missing something...

Yes, this webpage:
http://orgmode.org/worg/org-contrib/orgtbl-ascii-plot.html
was produced by the Worg engine, from a regular Org file.

Another issue is the key-binding.
I bound the plotter to C-c p
(p as "plot")
But this can be discussed.

Thierry
PS: thanks for Org-mode, it changed my life.

Re: text-only plots

[Thierry Banel]

Michael Brand <michael.ch.brand <at> gmail.com> writes:

> 
> Thank you for implementing these variations.
> 
> Michael
> 
> 

Thanks for this simple and effective idea.

Thierry

Re: text-only plots

[Suvayu Ali]

[-- Attachment #1: Type: text/plain, Size: 1298 bytes --]

Hi Thierry,

On Sun, Dec 08, 2013 at 11:10:47PM +0000, Thierry Banel wrote:
> Carsten Dominik <carsten.dominik <at> gmail.com> writes:
> > 
> > This is not a lot of code - I would not mind simply integrating this into
> org-table.el  The only hickup might be
> > that this could be trouble for the exporters - or have you produced the
> webpage directly from an org file?
> 
> I'm not sure I understand the exporters issue.
> Do you talk about the html exporter, the Odt exporter, and so on ?
> Well, once a plot has been added to a table, it is just text.
> It will be exported as any other content in the table.
> Maybe I'm missing something...
> 
> Yes, this webpage:
> http://orgmode.org/worg/org-contrib/orgtbl-ascii-plot.html
> was produced by the Worg engine, from a regular Org file.

Technically speaking, it is not exported as a table since you put it as
part of example block.  However a quick test tells me, html export will
work fine (screenshot attached).  However LaTeX export is an entirely
different story.  Unicode support in normal pdflatex is quite
non-trivial.  If you use XeLaTeX, I think you could get it to work with
some appropriate choice of fonts.  I tried libertine and dejavu without
any luck.

Hope this helps,

-- 
Suvayu

Open source is the future. It sets us free.

[-- Attachment #2: ascii-plot.png --]
[-- Type: image/png, Size: 11401 bytes --]

Re: text-only plots

[Ivan Andrus]

On Dec 9, 2013, at 5:19 AM, Suvayu Ali <fatkasuvayu+linux@gmail.com> wrote:

> Hi Thierry,
> 
> On Sun, Dec 08, 2013 at 11:10:47PM +0000, Thierry Banel wrote:
>> Carsten Dominik <carsten.dominik <at> gmail.com> writes:
>>> 
>>> This is not a lot of code - I would not mind simply integrating this into
>> org-table.el  The only hickup might be
>>> that this could be trouble for the exporters - or have you produced the
>> webpage directly from an org file?
>> 
>> I'm not sure I understand the exporters issue.
>> Do you talk about the html exporter, the Odt exporter, and so on ?
>> Well, once a plot has been added to a table, it is just text.
>> It will be exported as any other content in the table.
>> Maybe I'm missing something...
>> 
>> Yes, this webpage:
>> http://orgmode.org/worg/org-contrib/orgtbl-ascii-plot.html
>> was produced by the Worg engine, from a regular Org file.
> 
> Technically speaking, it is not exported as a table since you put it as
> part of example block.  However a quick test tells me, html export will
> work fine (screenshot attached).  However LaTeX export is an entirely
> different story.  Unicode support in normal pdflatex is quite
> non-trivial.  If you use XeLaTeX, I think you could get it to work with
> some appropriate choice of fonts.  I tried libertine and dejavu without
> any luck.

Would it be possible to export it to \rule when using LaTeX?  It wouldn't be text then and you couldn't copy it, but it should work with any TeX engine regardless of fonts etc.  Just a thought.

-Ivan

Re: text-only plots

[Thierry Banel]

Ivan Andrus <darthandrus <at> gmail.com> writes:

> 
> On Dec 9, 2013, at 5:19 AM, Suvayu Ali <fatkasuvayu+linux <at> gmail.com>
wrote:
> > Technically speaking, it is not exported as a table since you put it as
> > part of example block.  However a quick test tells me, html export will
> > work fine (screenshot attached).  However LaTeX export is an entirely
> > different story.  Unicode support in normal pdflatex is quite
> > non-trivial.  If you use XeLaTeX, I think you could get it to work with
> > some appropriate choice of fonts.  I tried libertine and dejavu without
> > any luck.
> 
> Would it be possible to export it to \rule when using LaTeX?  It wouldn't
be text then and you couldn't copy it,
> but it should work with any TeX engine regardless of fonts etc.  Just a
thought.
> 
> -Ivan
> 
> 

Right, Suvayu, Unicodes are not universally supported.
So, what about coming back to basics ?
Just Ascii like that:

|  x |    sin(x/4) | Ascii        |
|----+-------------+--------------|
|  0 |           0 | WWWWWh       |
|  1 |  0.32719470 | WWWWWWWV     |
|  2 |  0.61836980 | WWWWWWWWWl   |
|  3 |  0.84147098 | WWWWWWWWWWW  |
|  4 |  0.97193790 | WWWWWWWWWWWV |
|  5 |  0.99540796 | WWWWWWWWWWWW |
|  6 |  0.90929743 | WWWWWWWWWWW; |
|  7 |  0.72308588 | WWWWWWWWWW-  |
|  8 |  0.45727263 | WWWWWWWWl    |
|  9 |  0.14112001 | WWWWWWl      |
| 10 | -0.19056796 | WWWWl        |
| 11 | -0.50127705 | WWl          |
| 12 | -0.75680250 | W            |
#+TBLFM: $3='(orgtbl-ascii-draw $2 -0.9290145 0.99540796 12)

We remove Unicodes until someone comes with an idea.

Ivan, the LaTex \rule might be a (pretty) solution for the LaTex exporter.
But then we should also take care of all past and future exporters...

Thierry

Re: text-only plots

[Suvayu Ali]

Hi Thiery,

On Mon, Dec 09, 2013 at 10:48:22PM +0000, Thierry Banel wrote:
> Ivan Andrus <darthandrus <at> gmail.com> writes:
> > On Dec 9, 2013, at 5:19 AM, Suvayu Ali <fatkasuvayu+linux <at> gmail.com>
> wrote:
> > > Technically speaking, it is not exported as a table since you put it as
> > > part of example block.  However a quick test tells me, html export will
> > > work fine (screenshot attached).  However LaTeX export is an entirely
> > > different story.  Unicode support in normal pdflatex is quite
> > > non-trivial.  If you use XeLaTeX, I think you could get it to work with
> > > some appropriate choice of fonts.  I tried libertine and dejavu without
> > > any luck.
> > 
> > Would it be possible to export it to \rule when using LaTeX?  It
> > wouldn't be text then and you couldn't copy it, but it should work
> > with any TeX engine regardless of fonts etc.  Just a thought.
> 
> Right, Suvayu, Unicodes are not universally supported.
> So, what about coming back to basics ?
> Just Ascii like that:

 [...chomp...chomp...chomp...]

> We remove Unicodes until someone comes with an idea.
> 
> Ivan, the LaTex \rule might be a (pretty) solution for the LaTex exporter.
> But then we should also take care of all past and future exporters...

Firstly, I think the unicode bit is a terrific improvement; specially
the two options: grid and continuous.  I think there are two distinct
problems here: the plot as shown in org, and export.  We should not
confuse the two.

I will suggest for the moment having the plotting bits is enough.  HTML
export works, plain text export sort of works[1].  As you see, getting
it right for all (most) exporters is non-trivial, so I would say no need
for worrying about supporting past exporters.

Handling this needs some conditional behaviour: plain text -> backend,
or unicode -> backend.  You could use a standardised column title to
selectively choose/ignore columns.  So ASCII export would ignore unicode
columns.  LaTeX export would replace the chars with appropriate symbols
(e.g. \rule, as suggested by Ivan).  Texinfo and markdown supports
unicode, so that should be fine (like HTML).

When we have something that is acceptable, you can publish the exporter
code and the plotting code as part of the same module, say
org-ascii-plot.el (or something along those lines).  Which could then go
in contrib.

WDYT?

Footnotes:

[1] Sort-of, because it "lies" by including unicode chars even if you
    export to ASCII only.


-- 
Suvayu

Open source is the future. It sets us free.

Re: text-only plots

[Thierry Banel]

Suvayu Ali <fatkasuvayu+linux <at> gmail.com> writes:

> Firstly, I think the unicode bit is a terrific improvement; specially
> the two options: grid and continuous.  I think there are two distinct
> problems here: the plot as shown in org, and export.  We should not
> confuse the two.
> 
> I will suggest for the moment having the plotting bits is enough.  HTML
> export works, plain text export sort of works[1].  As you see, getting
> it right for all (most) exporters is non-trivial, so I would say no need
> for worrying about supporting past exporters.
> 
> Handling this needs some conditional behaviour: plain text -> backend,
> or unicode -> backend.  You could use a standardised column title to
> selectively choose/ignore columns.  So ASCII export would ignore unicode
> columns.  LaTeX export would replace the chars with appropriate symbols
> (e.g. \rule, as suggested by Ivan).  Texinfo and markdown supports
> unicode, so that should be fine (like HTML).
> 
> When we have something that is acceptable, you can publish the exporter
> code and the plotting code as part of the same module, say
> org-ascii-plot.el (or something along those lines).  Which could then go
> in contrib.
> 
> WDYT?
> 
> Footnotes:
> 
> [1] Sort-of, because it "lies" by including unicode chars even if you
>     export to ASCII only.
> 


Thanks, Suvayu, for your insight into the export issue.

So, to summarize, we have several options:

1- Keep only the Ascii version | WWWWWWh |
   (+) Simple.
   (+) No problem ahead.
   (-) Lose the clean look provided by unicodes.

2- Put Ascii version in org-core, Unicode version in contrib/
   (+) Simple
   (+) The Org core is on the safe side,
       while contrib/ gives an advanced option which could break exporters.
   (-) The clean look provided by unicodes will remain mostly unknown.

3- Keep Unicode version, do nothing for exporters
   (+) Simple
   (-) Generates long threads in mailing lists:
       "exporter XX is broken, it does not export my table".
   (-) Not all fonts are able to render such unicodes,
       (this is true in Emacs, without even talking about exporters).

4- Mark unicode plot columns as "non-exportable",
   and have exporters ignore them.
   (+) Clean and straightforward.
   (-) Need a new feature in org-tables: "non-exportable column".
   (-) Exporters & users should be made aware of the new feature.

5- Translate unicodes to something sensible, for example
   - LaTeX exporter: \rule
   - Ascii exporter: ignore
   - Odt exporter: pass them on without additional processing
   (+) Exporters take decisions at the cell level rather than at the table
level.
   (+) "Ignore exotic unicodes" or "do nothing special"
       are good default behaviors for all exporters.

6- Another option?

Now we have to decide.
What would be your preference, Org users?

Mine would go to option 1 or 2 for the time being.
Then if people like the plotter, version 2.0 would implement option 5.

Why option 1 or 2 ?
Because it is consistent with the rest of Org.
Table are made of regular Ascii characters (plus, minus, pipe)
|---+---|
| a | b |
although better suited unicode characters exist
http://en.wikipedia.org/wiki/Box_Drawing
┌───┬───┐
│ a │ b │

We all love the vintage look of Org, while being the most advanced orginizer
out there, don't we?

Thierry