From mboxrd@z Thu Jan 1 00:00:00 1970 From: Carsten Dominik Subject: Re: Precision setting for spreadsheet calculation Date: Sat, 27 Feb 2010 00:11:31 +0100 Message-ID: <7649555B-447F-486D-94A4-2139971A7AA9@gmail.com> References: <4B882C11.90406@alumni.ethz.ch> <0AEABCAD-CA2E-4F84-99E1-B5670FEC1E8D@gmail.com> <4B88395A.3040501@alumni.ethz.ch> <034F9F11-0888-48F4-9621-B7FC4907850F@gmail.com> <4B88440B.9060203@alumni.ethz.ch> Mime-Version: 1.0 (Apple Message framework v936) Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Return-path: Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Nl9LL-0007Il-Og for emacs-orgmode@gnu.org; Fri, 26 Feb 2010 18:11:35 -0500 Received: from [140.186.70.92] (port=50462 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Nl9LK-0007IS-U4 for emacs-orgmode@gnu.org; Fri, 26 Feb 2010 18:11:35 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1Nl9LK-0007eN-DS for emacs-orgmode@gnu.org; Fri, 26 Feb 2010 18:11:35 -0500 Received: from mail-wy0-f169.google.com ([74.125.82.169]:61504) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Nl9LK-0007eG-9M for emacs-orgmode@gnu.org; Fri, 26 Feb 2010 18:11:34 -0500 Received: by wyb42 with SMTP id 42so328407wyb.0 for ; Fri, 26 Feb 2010 15:11:33 -0800 (PST) In-Reply-To: <4B88440B.9060203@alumni.ethz.ch> List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Michael Brand Cc: emacs-orgmode@gnu.org On Feb 26, 2010, at 10:58 PM, Michael Brand wrote: > Carsten Dominik wrote: >> these are the internals of calc, about which I cannot do much. > > But > > | 0.16666666666666666667 | > #+TBLFM: $1 = 1 / 2 * 3; p20 f20 > > shows that the internals of calc are ok, isn't it? Hi Michael, you are completely right, I was wrong. The other issue that is playing into this is that when using a printf format specifier, the number is first converted into a Lisp number, which is a double, with 16 significant digits... At this step, the additional digits are lost. Maybe you find this interesting enough so that I can persuade you to write a FAQ entry about these issues? :-) - Carsten > >> Michael Brand wrote: >>> But instead of >>> >>> | 0.166666666666666660 | >>> #+TBLFM: $1 = 1 / 2 * 3; %.18f p20 f20 >>> >>> which I get I would expect >>> >>> | 0.166666666666666667 | >>> #+TBLFM: $1 = 1 / 2 * 3; %.18f p20 f20 - Carsten