From mboxrd@z Thu Jan 1 00:00:00 1970 From: Karl Voit Subject: Re: [Orgmode] Slow speed of week and month views Date: Sun, 6 Aug 2017 09:54:02 +0200 Message-ID: <2017-08-06T09-41-33@devnull.Karl-Voit.at> References: <87efsre565.fsf@grothesque.org> <878tiyr9tg.fsf@alphapapa.net> <2017-08-05T23-06-00@devnull.Karl-Voit.at> Reply-To: Karl Voit Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:41795) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1deGOG-0007LW-Em for emacs-orgmode@gnu.org; Sun, 06 Aug 2017 03:54:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1deGOD-00035v-Dy for emacs-orgmode@gnu.org; Sun, 06 Aug 2017 03:54:24 -0400 Received: from [195.159.176.226] (port=53435 helo=blaine.gmane.org) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1deGOD-00035T-6u for emacs-orgmode@gnu.org; Sun, 06 Aug 2017 03:54:21 -0400 Received: from list by blaine.gmane.org with local (Exim 4.84_2) (envelope-from ) id 1deGNz-00087H-03 for emacs-orgmode@gnu.org; Sun, 06 Aug 2017 09:54:06 +0200 List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: "Emacs-orgmode" To: emacs-orgmode@gnu.org Hello John, Great to read your thoughts on the topic - I am a huge admirer of your work and we both seem to cope with similar issues with Org-mode. * John Kitchin wrote: > > One is to use the new dynamic module capability to write an org parser in > C, or a dedicated agenda function, which would presumably be faster than in > elisp. This seems hard, and for me would certainly be a multiyear project > I am sure! The downside of this is the need to compile the module. I don't > know how easy it would be to make this work across platforms with the > relatively easy install org-mode currently has. This could have a side > benefit though of a c-lib that could be used by others to expand where > org-mode is used. I'm not a fan of C at all but having the parser in C with the possibility to use this parser for external tools as well sounds awesome to me. After all, I've written a primitive parser for a sub-set of Org-mode for https://github.com/novoid/lazyblorg using Python. > The other way that might work is to rely more heavily on a cached version > of the files, perhaps in a different format than elisp, that is faster to > work with. The approach I have explored in this is to index org files into > a sqlite database. The idea then would be to generate the agenda from a sql > query. I use something like this already to "find stuff in orgmode > anywhere". One of the reasons I wrote this is the org-agenda list of files > isn't practical for me because my files are so scattered on my file system. > I had a need to be able to find TODOs in research projects in a pretty wide > range of locations. > > The code I use is at > https://github.com/jkitchin/scimax/blob/master/org-db.el, and from one > database I can find headlines, contacts, locations, TODO headlines across > my file system, all the files that contain a particular link, and my own > recent org files. I didn't try org-db.el yet. So far, I survived using "git grep" and counsel-grep [0] > I am moderately motivated to switch from sqlite to MongoDB Is org-db.el your standard way of accessing informations or do you use it only for occasional searches where you assume that the usual methods would be slow? > The main point of the database was to get a query language, persistence and > good performance. I have also used caches to speed up using bibtex files, > and my org-contacts with reasonable performance. These have been all elisp, > with no additional dependencies. Maybe one could do something similar to > keep an agenda cache that is persistent and updated via hook functions. Oh yeah. My org-contacts were unusable without at least some minor performance improvements as well. Most important to me: improving manipulation of properties using [1]. For example, org-set-property takes almost 20 seconds to give me its interactive input line in my main Org-mode file. This is a no-go. [1] helped me here a lot. [0] https://github.com/novoid/dot-emacs/blob/master/config.org#optimizing-search-methods [1] https://github.com/novoid/dot-emacs/blob/master/config.org#my-org-region-to-property--my-map-p -- get mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML into Org-mode: > get Memacs from https://github.com/novoid/Memacs < Personal Information Management > http://Karl-Voit.at/tags/pim/ Emacs-related > http://Karl-Voit.at/tags/emacs/