From mboxrd@z Thu Jan  1 00:00:00 1970
From: Nick Dokos <ndokos@gmail.com>
Subject: Re: Precision in the spreadsheet
Date: Tue, 01 Mar 2016 14:19:23 -0500
Message-ID: <87fuwa2eh0.fsf@alphaville.usersys.redhat.com>
References: <87vb56w31l.fsf@mbork.pl>
	<87k2lm2ky5.fsf@alphaville.usersys.redhat.com>
	<87twkqunvx.fsf@ucl.ac.uk>
Mime-Version: 1.0
Content-Type: text/plain
Return-path: <emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org>
Received: from eggs.gnu.org ([2001:4830:134:3::10]:58551)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <geo-emacs-orgmode@m.gmane.org>) id 1aappk-0002wI-4q
	for emacs-orgmode@gnu.org; Tue, 01 Mar 2016 14:19:51 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <geo-emacs-orgmode@m.gmane.org>) id 1aappg-0004tf-Nz
	for emacs-orgmode@gnu.org; Tue, 01 Mar 2016 14:19:48 -0500
Received: from plane.gmane.org ([80.91.229.3]:35063)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <geo-emacs-orgmode@m.gmane.org>) id 1aappg-0004tJ-HP
	for emacs-orgmode@gnu.org; Tue, 01 Mar 2016 14:19:44 -0500
Received: from list by plane.gmane.org with local (Exim 4.69)
	(envelope-from <geo-emacs-orgmode@m.gmane.org>) id 1aappW-0008Q7-IL
	for emacs-orgmode@gnu.org; Tue, 01 Mar 2016 20:19:34 +0100
Received: from nat-pool-bos-t.redhat.com ([66.187.233.206])
	by main.gmane.org with esmtp (Gmexim 0.1 (Debian))
	id 1AlnuQ-0007hv-00
	for <emacs-orgmode@gnu.org>; Tue, 01 Mar 2016 20:19:34 +0100
Received: from ndokos by nat-pool-bos-t.redhat.com with local (Gmexim 0.1
	(Debian)) id 1AlnuQ-0007hv-00
	for <emacs-orgmode@gnu.org>; Tue, 01 Mar 2016 20:19:34 +0100
List-Id: "General discussions about Org-mode." <emacs-orgmode.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/emacs-orgmode>,
	<mailto:emacs-orgmode-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/emacs-orgmode>
List-Post: <mailto:emacs-orgmode@gnu.org>
List-Help: <mailto:emacs-orgmode-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/emacs-orgmode>,
	<mailto:emacs-orgmode-request@gnu.org?subject=subscribe>
Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org
Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org
To: emacs-orgmode@gnu.org

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> On Tuesday,  1 Mar 2016 at 11:59, Nick Dokos wrote:
>> Marcin Borkowski <mbork@mbork.pl> writes:
>>
>>> Hi all,
>>>
>>> is it at all possible to set different precision for display and
>>> calculations in Org's spreadsheet?  I mean, things like $1+$2;%.2f (as
>>> in the manual) mean (IIUC) that (1) Org should show 2 decimal figures
>>> and (2) further calculations which use this value also use this
>>> approximation.  Am I right?  If yes, is there a way to change this
>>> behavior?  (I suspect not, but I wanted to make sure.)
>>>
>>
>> I believe that internally, calculations are done in 12-decimal place
>> precision: the %.2f is for display only.
>
> Actually, surprisingly (to me), it appears not.  This simple example
> gives different results depending on the formatting of the first entry:
>
> #+begin_src org
>   ,* table
>   | 0.1 |
>   |  1. |
>   | 10. |
>   ,#+TBLFM: @1$1=1/8;%.1f::@2$1=@-1*10::@3$1=@-1*10
> #+end_src

Yes, I got it wrong: internal calculations may be done in extended
precision, but once the result is in the table, that's that. Achim
explains the situation (and potential solutions) clearly in his
response.

--
Nick