Re: Microsoft Excel spreadsheet editing directly from within emacs.

[Jean Louis <bugs@gnu.support>]

Dear Stefan,

You speak of a programming language and what is possible with
programming language. But language is not a spreadsheet, an integrated
program that helps users with handling of data.

The Gnumeric as spreadsheet may be extended by using Python
programming language. In that sense I think there is no limit to
extension of a spreadsheet.

Libreoffice Calc spreadsheet may be extended by using Libreoffice
Basic, Javascript, BeanShell, Python.

Extensibility and programming capabilities do not make software a
"spreadsheet" program. Manual describes it as spreadsheet-like. That
is more accurate.

* Stefan Nobis <stefan-ml@snobis.de> [2020-12-29 14:52]:
> It might be a difficult question, whether Org tables are the best
> solution in a given situation or whether it is the best interchange
> format to use with arbitrary people (but I also doubt that MS Excel is
> a good way - I have seen really nasty and expensive problems caused by
> the use of Excel to move data around).

For interchange, one could choose those open document formats if it
contains formulas or CSV, TSV if it does not. I do not know if CSV/TSV
formats could carry some formula around.

And if data is not that complex one can export it as text or anything
else, maybe as PDF, it all depends how the other side wish to use the
data. Various conversion program may help to convert Org tables to
something else and it is useful.

> But Org tables are very powerful and in many cases even far superior
> than most other spreadsheet software (especially MS Excel - I can't
> count the number of times that Excel tried to be smart and made a
> total mess of my data).

I have a feeling you refer to programming capabilities.

> Here is a short example of what is possible with Org tables:
> 
> #+begin_src org
> ,* Org Spreadsheet
> 
> ,** Example from the Manual
> 
> See [[https://orgmode.org/manual/Advanced-features.html#Advanced-features][The Spreadsheet - Advanced features]].
> 
> |---+-------------+---+-----+--------------------------------------|
> |   | Func        | n | x   | Result                               |
> |---+-------------+---+-----+--------------------------------------|
> | # | exp(x)      | 1 | x   | 1 + x                                |
> | # | exp(x)      | 2 | x   | 1 + x + x^2 / 2                      |
> | # | exp(x)      | 3 | x   | 1 + x + x^2 / 2 + x^3 / 6            |
> | # | x^2+sqrt(x) | 2 | x=0 | x*(0.5 / 0) + x^2 (2 - 0.25 / 0) / 2 |
> | # | x^2+sqrt(x) | 2 | x=1 | 2 + 2.5 x - 2.5 + 0.875 (x - 1)^2    |
> | * | tan(x)      | 3 | x   | x pi / 180 + 5.72e-8 x^3 pi^3        |
> |---+-------------+---+-----+--------------------------------------|
> ,#+TBLFM: $5=taylor($2,$4,$3);n3

I see the above example as demonstration of functional or mathematical
capability. Yet I do not perceive it as spreadsheet capability as the
word spreadsheet is since decades established on the market. A HTML
table with Javascript could do the same as the example above but that
does not make it a spreadsheet.

Spreadsheet is table or tabular view with its rows and cells that
offers entry of data, management of data, especially calculations and
various scripting, charting abilities.

In a spreadsheet program I may visually and nicely presented see a
slice of a whole table. I may move from slice to slice to other pieces
of the whole table of data. Moving from cell to cell is pretty easy
and there are no screen distortions.

Org tables are text, not really tabular information even if so
visualized. But I cannot move easily and nicely from slice to
slice. The header is not there or does not remain fixed to tell me on
whhich row or which column I would be located.

Creating an Org table of 200x300 takes quite some time. I need not
wait when opening a true spreadsheet program. Further there are no
rows unless I start making them by hand.

Spreadsheet programs do not ask me to make a row by my own hand, it is
silly even to think of it.

Org user interface is for zen people who think they can do everything
with specifically complex key bindings.

Spreadsheet user interface is integrated, it does not require user to
remember much, or maybe nothing, just use a mouse. It is intuitive
just as Doug Engelbart envisioned computing how it should be back in
sixties.

Org mode is bringing us back into the era before Doug Engelbart. Back
to history.

Combining various sheets is standard spreadsheet feature. Examples you
have shown are cryptic for spreadsheet users.

But I give you applause for trying so hard. It is just not
realistic. I love Org mode too, but I know what it is. 

> In respect to core features (listing data, using formulas for
> calculation) I doubt that Emacs with Org tables and Calc is missing
> anything.

For example it misses to have rows' named positions by default. Like
from 1 to maybe 100 or from 1 to 10000 or 100000. User has to write it
by hand. No spreadsheet require user to write rows by hand as it would
be absurd. That is core feature of every spreadsheet.

It misses columns' as their named location or position by default. I
would need to name columns, but there are no defined columns. I cannot
say that column H shall contain a count of apples as example as there
is no designated column H.

There are no headers that user can fix to see what and where cursor is
positioned within spreadsheet.

But I see it is better to stop talking about something that is obvious
as we are lovers and enthusiasts and we love our own horse.

The reason why I am writing this is to warn those people that they are
entering the twilight zone of Org mode as if somebody need to do
serious spreadsheet work, Org tables are not spreadsheet. It may help
that subset of users to avoid wasting their time with a tool that is
not meant to be spreadsheet. It will help also those who like Org
mode, to use Org tables. Discussion like this is useful for review by
other people and their decision making.

> Maybe advanced visual presentation of the data is easier with GUI
> Spreadsheets -- then again, it is so easy to combine Org tables with
> the power of Gnuplot, R, Python, Julia, TeX etc. to create
> astonishing visuals, that I prefer this way in many situations.

That is like saying to a user to switch from Emacs to C programming
language as it gives user far more capabilities, for example to make
his own editor or his own programming languages as it offers all the
powers, and if user wish to have more powers than better look at
assembly or machine language.

Users need integration. Not underlying programming capabilities, that
is for programmers, not for users. Users are majority, not
programmers. Programmers already know the powers you mentioned.

Spreadsheet is for users, not really for programmers.

Users need integration, they don't mind of the mentioned powers. They
want to enter rows, columns and click button and get a chart.

I just wish you would see that. Myself I program and I understand you,
but what I wish that you see is the distinction between users and
integrated computing and programmers. We all know programmers have all
powers at ther availability. We are in agreement on that, but you
forget users.

> From my point of view, MS Excel is the toy (I have not too much
> experience with the other GUI spreadsheet programs).

Well, that is what I speak about.

> In Emacs I have the power of Calc (a complete computer algebra
> system) and Lisp (the best programming language, even if Elisp is
> not Common Lisp) at my fingertips.

I like that enthusiasm. I wish Emacs Lisp could handle what I handle
with Common Lisp, but it does not, it blocks, even crushes, there are
many problems. Maybe one day.

Sure I understand you have powers but you are narrowing your own world
and live in the illusion.

I do use Org tables, but if I would recommend somebody who does not
know Emacs, I would never recommend Org tables for reason that I want
person to spare the time, not waste the time. Computer is there to
help human, not human to help computer work, what we do in Org mode,
we constantly wish to make it work as it is not complete software, it
is far from being well integrated.

Well integrated software are today mobile applications. They have
already got about 50% of population using computers if not more. Those
applications are today built mostly out of the foundation that they
need to become integrated, putting things together and helping
users. Not other way around.

> I love that Org tables are fully self-explained, everything is
> explicit and quite obvious.

Org tables are not at all self-explained. It begins from creation of a
table, user creates a table 200x300 and waits maybe 10 seconds that
something happens. In the mean time nothing is told to user about
waiting time, user is implicitly advised to become religious and pray
for good.

Try creating empty 200x300 table and tell me if your interface is not
blocking you?

Spreadsheet interface does not.

> Formulas are easy to inspect. GUI Spreadsheets may be a bit easier
> for the very first steps, but they hide sooo much, that even power
> users with a decade or two of experience have trouble of holding
> everything together.

I do not know what they hide. I was using Lotus 1,2,3 spreadsheet on
Apple //c and it was all open and well documented, but it was
proprietary software. Libreoffice is well documented, Gnumeric is well
documented.

> Are Org tables for everyone? It would be great and IMHO it could
> work, but it will not happen in the foreseeable future.

Org tables are not spreadsheet so it cannot replace spreadsheet unless
it is a simple case. In terms of formulas, anything is possible just
as it is scriptable with any known major free software spreadsheet
today. In terms of being spreadsheet Org tables do not comply neither
with rows, columns nor display of slices of a spreadsheet. It can show
bars on screen but not tell to user where cursor is
positioned. Unspoken on how low usability it has. Just create empty
table 200x300 and see if you may move around it in the empty table. 

> But they are *very* powerful and Emacs + Org + Calc is able to
> replace Spreadsheets in many situation and that solution may even be
> superior in the long run.

I do have same tendencies to take a collection of tools and round them
in my mind as being perfect enough for my work, and justify all the
procrastination I create with it and to avoid anything else. But that
approach, while being self satisfactorily does not help other people
who would be wasting their time maybe for months or year or even more
until they find out that it is not right tool for their job.

Computer programs should spare humans their efforts and
times. Programmers help users to spare their efforts, time, energy,
money, their resources. I do share the opinion that every person
should learn how to program, and Emacs, Emacs Lisp and GNU and
programming languages help so much in that endeavour.

Myself I use database as a spreadsheet, but I cannot say to somebody
who need spreadsheet capabilities to use a database maybe because I
find it personally useful, as it is better to advise people to use
what is better integrated for them or their group of collaborators so
that they spare their working time.

If their group are users who need specific input of data, maybe
spreadsheet is best. If they are handling more critical data most
probably they would never ask about it on the mailings list as they
would have already their consultant hired and managing all issues for
them. Org table users will accommodate themselves, I have no doubts.