emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
* Bug: Org Table Performance Issue/Regression
@ 2018-06-10 19:04 Tim Baumgard
  2018-06-26  9:58 ` Nicolas Goaziou
  0 siblings, 1 reply; 5+ messages in thread
From: Tim Baumgard @ 2018-06-10 19:04 UTC (permalink / raw)
  To: emacs-orgmode

I recently updated Emacs from 25.3 (Org mode v. 8.2.10) to 26.1 (Org mode
v. 9.1.9) and noticed a performance issue/regression in one of my Org files that
contains a table that currently has nine columns, around 1,200 rows, and uses
formulas. In 25.3, I could create a new row or use org-table-copy-down without
issues. In 26.1, there’s a multi-second delay each time I try to do either of
these things. As mentioned below, this issue also occurs in the latest commits
to both Emacs and Org mode.

I did some profiling and found that garbage-collect was being called quite a
bit. I set gc-cons-threshold in both versions to a large value to effectively
disable garbage collection and noticed two things: 1) it largely fixed the
performance issues—though performance still wasn’t as good as it was in 25.3—and
2) Emacs/Org mode was using about four times as much memory as it did after
creating a new row and doing calling org-table-copy-down three
times. Specifically, Emacs 25.3 was using a little under 200 MB and Emacs 26.3
was using a around 800 MB.

I also noticed that Org mode is recalculating the table formulas more often than
in past versions. It used to be that it would only recalculate the formulas for
the whole table when I called org-table-recalculate, but after testing some
things and then diff’ing the files, I noticed that other rows were being updated
based on the table formulas I’m using. I’m unsure if this is related.

For what it’s worth, using the Org mode files from Emacs 25.3 in 26.1 doesn’t
display the performance issue, but there are other issues that make it a poor
workaround. What I’m currently doing to work around this for my use case is,
only for this file, setting gc-cons-threshold to a large value and adding a hook
to after-save-hook to call garbage-collect since calling it once when saving
doesn’t take too long.

I’ve inlined a test file and a test init.el file below. The test file includes a
reduced version of the table I’m currently using, including the table TBLFM
line. The init.el file includes the functions required for my table formulas and
how I was using gc-cons-threshold and my workaround. I was still seeing the
issue even without using the formulas, but I thought I'd include them in case
it's helpful.

Some notes:
- All of my testing was done using emacs -Q
- I’m running Emacs on macOS compiled from the sources available on gnu.org. 
- I also tested everything using the latest commits of Emacs
  (efa750e68466fb1bab03028701350a745e0504b6) and Org mode
	(842002f9f7e3246cc285cc9bca1adb715120438d) from the repositories.
- Emacs: GNU Emacs 27.0.50 (build 1, x86_64-apple-darwin17.6.0, NS
  appkit-1561.40 Version 10.13.5 (Build 17F77))
	of 2018-06-10
- Package: Org mode version 9.1.13 (release_9.1.13-791-g842002)


test.org
======

|   | A       | B         | C      | D                         | E                         |        F |   G | H                          |
|---+---------+-----------+--------+---------------------------+---------------------------+----------+-----+----------------------------|
| # | ABCDEFG | HIJKLMNOP | QRSTUV | [2018-06-10 Sun 00:00:00] | [2018-06-10 Sun 02:00:00] | 02:00:00 | 2.0 | ABCDEFGHIJKLMNOPQRSTUVWXYZ |
#+TBLFM: $7='(time-stamp-difference $5 $6)::$8='(time-stamp-difference-hours $5 $6)


init.el
====

(defun time-stamp-difference (earlier-date later-date)
  "Get the difference from two time stamps. This only works when the difference is less than 24 hours."
	(let* ((time-difference
				 (round (- (time-to-seconds (date-to-time later-date))
									 (time-to-seconds (date-to-time earlier-date)))))
				(absolute-time-difference (max time-difference 0)))
		(format-time-string "%T" (encode-time absolute-time-difference 0 0 1 1 1))))

(defun time-stamp-difference-hours (earlier-date later-date)
  "Get the difference from two time stamps in the number of hours."
	(let* ((seconds
				 (- (time-to-seconds (date-to-time later-date))
						(time-to-seconds (date-to-time earlier-date))))
				 (absolute-seconds (max seconds 0))
				 (hours (/ (/ absolute-seconds 60) 60.0)))
		(/ (round (* hours 1000.0)) 1000.0)))

;; This sets the garbage-collection threshold to a large value, effectively
;; disabling it for testing purposes.
;; (setq-default gc-cons-threshold 80000000000)
;; (add-hook 'after-save-hook 'garbage-collect)

^ permalink raw reply	[flat|nested] 5+ messages in thread
end of thread, other threads:[~2018-11-02 18:02 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-06-10 19:04 Bug: Org Table Performance Issue/Regression Tim Baumgard
2018-06-26  9:58 ` Nicolas Goaziou
2018-07-16  2:47   ` Tim Baumgard
2018-09-03  7:59     ` Nicolas Goaziou
2018-11-02 18:02       ` Tim Baumgard

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).