From mboxrd@z Thu Jan 1 00:00:00 1970 From: Nathaniel Flath Subject: Re: [PATH] Speedups to org-table-recalculate Date: Fri, 10 Oct 2014 12:43:29 -0700 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=047d7b5d3d1a6cc608050516c3df Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:53596) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xcg69-0004lU-Jn for emacs-orgmode@gnu.org; Fri, 10 Oct 2014 15:43:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Xcg67-0004iJ-AG for emacs-orgmode@gnu.org; Fri, 10 Oct 2014 15:43:33 -0400 Received: from mail-oi0-x22f.google.com ([2607:f8b0:4003:c06::22f]:44375) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xcg66-0004i8-VH for emacs-orgmode@gnu.org; Fri, 10 Oct 2014 15:43:31 -0400 Received: by mail-oi0-f47.google.com with SMTP id a141so7963325oig.6 for ; Fri, 10 Oct 2014 12:43:29 -0700 (PDT) In-Reply-To: List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Michael Brand Cc: org-mode List --047d7b5d3d1a6cc608050516c3df Content-Type: text/plain; charset=UTF-8 Mine is a pretty simple table (takes less than a second even in the original case): | Category | Budget | Spent | Remaining | |----------+--------+-------+-----------| | A | 100 | 0 | 100 | | B | 100 | 0 | 100 | | C | 100 | 0 | 100 | | D | 100 | 0 | 100 | | E | 100 | 0 | 100 | | F | 100 | 0 | 100 | | G | 100 | 0 | 100 | | H | 100 | 0 | 100 | | I | 100 | 0 | 100 | | J | 100 | 0 | 100 | | K | 100 | 0 | 100 | | L | 100 | 0 | 100 | | M | 100 | 0 | 100 | | N | 100 | 0 | 100 | | O | 100 | 0 | 100 | | K | 100 | 0 | 100 | |----------+--------+-------+-----------| | Total | 1600 | 0 | 1600 | #+TBLFM: $4=$2-$3::@18$2=vsum(@2$2..@-1)::@18$3=vsum(@2$3..@-1) With the macro: (defmacro time (block) `(let (start end) (setq start (current-time)) ,block (setq end (current-time)) (print (time-subtract end start)))) and running (time (org-table-recalculate t)) Original recalculation: (0 0 396224 0) Version w/ time checks for per-field messages (still always printing at beginning/end of processing):(0 0 56929 0) Version w/ time checks and removing all beginning/end of processing messages: (0 0 22077 0) My patch: (0 0 17405 0) So, it's still a 26% performance degradation to going with the patch and removing the 'global' messaging, but I could probably live with that - qualitatively, there doesn't seem to be too much difference between my patch and doing that, but the original version is obviously slow and with the on-begin/end calculation messages the delay is much more noticable. On Fri, Oct 10, 2014 at 3:35 AM, Michael Brand wrote: > Hi Nathaniel > > On Fri, Oct 10, 2014 at 7:56 AM, Nathaniel Flath > wrote: > > That's still much more slow than not doing it - slightly modifying your > > example,: > > > > (progn > > (setq start (current-time)) > > (let ((row 0) (log (time-add (current-time) '(0 1 0 0)))) > > (while (< row 6543210) > > (setq row (1+ row)) > > (when (time-less-p log (current-time)) > > (setq log (time-add (current-time) '(0 1 0 0))) > > (message "row %d" row)))) > > (setq end (current-time)) > > (print (time-subtract end start))) > > > > prints (0 43 386499 0) on my computer. > > > > Removing the when clause: > > > > (progn > > (setq start (current-time)) > > (let ((row 0) (log (time-add (current-time) '(0 1 0 0)))) > > (while (< row 6543210) > > (setq row (1+ row)))) > > (setq end (current-time)) > > (print (time-subtract end start))) > > > > Results in: > > (0 1 277641 0) > > > > So adding the logging here slows it down by about 43x - It doesn't seem > > worth it. > > Your measurement shows that "(when (time-less-p log (current-time)) > [...]" takes 6.4 microseconds or can run 150'000 times per second. I > would expect it to be negligible compared to what Org has to do for > each row or field like parse, calculate, format etc. Otherwise it > would mean that Org can perform more or not significantly less than > 150'000 rows or fields per second on an appropriate example table. > > Tersely formulated I expect this performance comparison: nothing or > empty loop << a conditional message with time check << Org performs a > simple formula on one row or field << an unconditional message > > Can you make a performance comparison on your table between (a) your > patch and (b) without your patch but with "(when (time-less-p log > (current-time)) [...]" plus describe or share this table? > > Michael > --047d7b5d3d1a6cc608050516c3df Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Mine is a pretty simple table (takes less than a second ev= en in the original case):

| Category | Budget | Spe= nt | Remaining |
|----------+--------+-------+-----------|
<= div>| A =C2=A0 =C2=A0 =C2=A0 =C2=A0| =C2=A0 =C2=A0100 | =C2=A0 =C2=A0 0 | = =C2=A0 =C2=A0 =C2=A0 100 |
| B =C2=A0 =C2=A0 =C2=A0 =C2=A0| =C2= =A0 =C2=A0100 | =C2=A0 =C2=A0 0 | =C2=A0 =C2=A0 =C2=A0 100 |
| C = =C2=A0 =C2=A0 =C2=A0 =C2=A0| =C2=A0 =C2=A0100 | =C2=A0 =C2=A0 0 | =C2=A0 = =C2=A0 =C2=A0 100 |
| D =C2=A0 =C2=A0 =C2=A0 =C2=A0| =C2=A0 =C2= =A0100 | =C2=A0 =C2=A0 0 | =C2=A0 =C2=A0 =C2=A0 100 |
| E =C2=A0 = =C2=A0 =C2=A0 =C2=A0| =C2=A0 =C2=A0100 | =C2=A0 =C2=A0 0 | =C2=A0 =C2=A0 = =C2=A0 100 |
| F =C2=A0 =C2=A0 =C2=A0 =C2=A0| =C2=A0 =C2=A0100 | = =C2=A0 =C2=A0 0 | =C2=A0 =C2=A0 =C2=A0 100 |
| G =C2=A0 =C2=A0 = =C2=A0 =C2=A0| =C2=A0 =C2=A0100 | =C2=A0 =C2=A0 0 | =C2=A0 =C2=A0 =C2=A0 10= 0 |
| H =C2=A0 =C2=A0 =C2=A0 =C2=A0| =C2=A0 =C2=A0100 | =C2=A0 = =C2=A0 0 | =C2=A0 =C2=A0 =C2=A0 100 |
| I =C2=A0 =C2=A0 =C2=A0 = =C2=A0| =C2=A0 =C2=A0100 | =C2=A0 =C2=A0 0 | =C2=A0 =C2=A0 =C2=A0 100 |
| J =C2=A0 =C2=A0 =C2=A0 =C2=A0| =C2=A0 =C2=A0100 | =C2=A0 =C2=A0 0 = | =C2=A0 =C2=A0 =C2=A0 100 |
| K =C2=A0 =C2=A0 =C2=A0 =C2=A0| =C2= =A0 =C2=A0100 | =C2=A0 =C2=A0 0 | =C2=A0 =C2=A0 =C2=A0 100 |
| L = =C2=A0 =C2=A0 =C2=A0 =C2=A0| =C2=A0 =C2=A0100 | =C2=A0 =C2=A0 0 | =C2=A0 = =C2=A0 =C2=A0 100 |
| M =C2=A0 =C2=A0 =C2=A0 =C2=A0| =C2=A0 =C2= =A0100 | =C2=A0 =C2=A0 0 | =C2=A0 =C2=A0 =C2=A0 100 |
| N =C2=A0 = =C2=A0 =C2=A0 =C2=A0| =C2=A0 =C2=A0100 | =C2=A0 =C2=A0 0 | =C2=A0 =C2=A0 = =C2=A0 100 |
| O =C2=A0 =C2=A0 =C2=A0 =C2=A0| =C2=A0 =C2=A0100 | = =C2=A0 =C2=A0 0 | =C2=A0 =C2=A0 =C2=A0 100 |
| K =C2=A0 =C2=A0 = =C2=A0 =C2=A0| =C2=A0 =C2=A0100 | =C2=A0 =C2=A0 0 | =C2=A0 =C2=A0 =C2=A0 10= 0 |
|----------+--------+-------+-----------|
| Total = =C2=A0 =C2=A0| =C2=A0 1600 | =C2=A0 =C2=A0 0 | =C2=A0 =C2=A0 =C2=A01600 |

#+TBLFM: $4=3D$2-$3::@18$2=3Dvsum(@2$2..@-1)::@18$3= =3Dvsum(@2$3..@-1)

With the macro:
(defmacro time (block)
=C2=A0 `(let (start end)
= =C2=A0 =C2=A0 (setq start (current-time))
=C2=A0 =C2=A0 ,block
=C2=A0 =C2=A0 (setq end (current-time))
=C2=A0 =C2=A0 (pr= int (time-subtract end start))))
and running (time (org-tab= le-recalculate t))

Original recalculation: =C2=A0(= 0 0 396224 0)
Version w/ time checks for per-field messages (stil= l always printing at beginning/end of processing):(0 0 56929 0)
V= ersion w/ time checks and removing all beginning/end of processing messages= : (0 0 22077 0)
My patch: =C2=A0(0 0 17405 0)

So, it's still a =C2=A026% performance degradation to going with = the patch and removing the 'global' messaging, but I could probably= live with that - qualitatively, there doesn't seem to be too much diff= erence between my patch and doing that, but the original version is obvious= ly slow and with the on-begin/end calculation messages the delay is much mo= re noticable.

On Fri, Oct 10, 2014 at 3:35 AM, Michael Brand &l= t;michael.c= h.brand@gmail.com> wrote:
H= i Nathaniel

On Fri, Oct 10, 2014 at 7:56 AM, Nathaniel Flath <flat0103@gmail.com> wrote:
> That's still much more slow than not doing it - slightly modifying= your
> example,:
>
> (progn
>=C2=A0 =C2=A0(setq start (current-time))
>=C2=A0 =C2=A0(let ((row 0) (log (time-add (current-time) '(0 1 0 0)= )))
>=C2=A0 =C2=A0 =C2=A0(while (< row 6543210)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0(setq row (1+ row))
>=C2=A0 =C2=A0 =C2=A0 =C2=A0(when (time-less-p log (current-time))
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(setq log (time-add (current-time) &#= 39;(0 1 0 0)))
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(message "row %d" row)))) >=C2=A0 =C2=A0(setq end (current-time))
>=C2=A0 =C2=A0(print (time-subtract end start)))
>
> prints (0 43 386499 0) on my computer.
>
> Removing the when clause:
>
> (progn
>=C2=A0 =C2=A0(setq start (current-time))
>=C2=A0 =C2=A0(let ((row 0) (log (time-add (current-time) '(0 1 0 0)= )))
>=C2=A0 =C2=A0 =C2=A0(while (< row 6543210)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0(setq row (1+ row))))
>=C2=A0 =C2=A0(setq end (current-time))
>=C2=A0 =C2=A0(print (time-subtract end start)))
>
> Results in:
> (0 1 277641 0)
>
> So adding the logging here slows it down by about 43x - It doesn't= seem
> worth it.

Your measurement shows that "(when (time-less-p log (curre= nt-time))
[...]" takes 6.4 microseconds or can run 150'000 times per second.= I
would expect it to be negligible compared to what Org has to do for
each row or field like parse, calculate, format etc. Otherwise it
would mean that Org can perform more or not significantly less than
150'000 rows or fields per second on an appropriate example table.

Tersely formulated I expect this performance comparison: nothing or
empty loop << a conditional message with time check << Org perf= orms a
simple formula on one row or field << an unconditional message

Can you make a performance comparison on your table between (a) your
patch and (b) without your patch but with "(when (time-less-p log
(current-time)) [...]" plus describe or share this table?

Michael

--047d7b5d3d1a6cc608050516c3df--