Re: Precision in the spreadsheet

[Nick Dokos <ndokos@gmail.com>]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> On Tuesday,  1 Mar 2016 at 11:59, Nick Dokos wrote:
>> Marcin Borkowski <mbork@mbork.pl> writes:
>>
>>> Hi all,
>>>
>>> is it at all possible to set different precision for display and
>>> calculations in Org's spreadsheet?  I mean, things like $1+$2;%.2f (as
>>> in the manual) mean (IIUC) that (1) Org should show 2 decimal figures
>>> and (2) further calculations which use this value also use this
>>> approximation.  Am I right?  If yes, is there a way to change this
>>> behavior?  (I suspect not, but I wanted to make sure.)
>>>
>>
>> I believe that internally, calculations are done in 12-decimal place
>> precision: the %.2f is for display only.
>
> Actually, surprisingly (to me), it appears not.  This simple example
> gives different results depending on the formatting of the first entry:
>
> #+begin_src org
>   ,* table
>   | 0.1 |
>   |  1. |
>   | 10. |
>   ,#+TBLFM: @1$1=1/8;%.1f::@2$1=@-1*10::@3$1=@-1*10
> #+end_src

Yes, I got it wrong: internal calculations may be done in extended
precision, but once the result is in the table, that's that. Achim
explains the situation (and potential solutions) clearly in his
response.

--
Nick