From mboxrd@z Thu Jan 1 00:00:00 1970 From: d@teklibre.org (Dave =?utf-8?Q?T=C3=A4ht?=) Subject: Re: Scaling org-mode Date: Sun, 13 Sep 2009 11:25:00 -0600 Message-ID: <87ab0yg3kj.fsf@mahal.sjds.teklibre.org> References: <87my4zfleg.fsf@mahal.sjds.teklibre.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Mmspm-0001eH-4R for emacs-orgmode@gnu.org; Sun, 13 Sep 2009 13:25:54 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1Mmsph-0001Tg-GZ for emacs-orgmode@gnu.org; Sun, 13 Sep 2009 13:25:53 -0400 Received: from [199.232.76.173] (port=55839 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Mmsph-0001TP-6T for emacs-orgmode@gnu.org; Sun, 13 Sep 2009 13:25:49 -0400 Received: from toutatis.isc.org ([149.20.54.64]:57486 helo=mainmail.teklibre.org) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1Mmspg-0005Cs-NZ for emacs-orgmode@gnu.org; Sun, 13 Sep 2009 13:25:49 -0400 Received: from localhost (localhost [127.0.0.1]) by mainmail.teklibre.org (Postfix) with ESMTP id 590ED12B793 for ; Sun, 13 Sep 2009 10:26:27 -0700 (PDT) Received: from mainmail.teklibre.org ([127.0.0.1]) by localhost (toutatis.sql1.isc.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7UB46kf9C2d4 for ; Sun, 13 Sep 2009 10:26:27 -0700 (PDT) Received: from mail3.teklibre.org (mahal.sjds.teklibre.org [IPv6:2001:470:b9d7::31]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail3.teklibre.org", Issuer "CA Cert Signing Authority" (verified OK)) by mainmail.teklibre.org (Postfix) with ESMTPS id 95CFE12B787 for ; Sun, 13 Sep 2009 10:26:26 -0700 (PDT) In-Reply-To: (Matt Lundin's message of "Sun, 13 Sep 2009 10:04:14 -0400") List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Matt Lundin Cc: emacs-orgmode@gnu.org Matt Lundin writes: > d@teklibre.org (Dave T=C3=A4ht) writes: > >> I have really been enjoying importing my life into org-mode, which I've >> been doing for about two months now.=20 >> >> But. >> >> It currently visits about 100 files and 10k of text to construct the >> agenda. It's starting to get kind of slow and interrupt my workflow, >> particularly the background process that scans them.=20 >> >> While the system is effectively frozen, my message buffer fills up with >> messages about setting the flyspell dictionary to en, etc. This is quite >> annoying with text to speech turned on. I ended up just having appts >> spoken. > > My guess is that this is a flyspell problem. I only get messages about > flyspell if I don't have aspell or the proper dictionary installed. Do > you get these messages when you open buffers in other text modes? What Yes. Opening up a random text file gives me: "Local Ispell dictionary set to en" in the message buffer. It's not annoying when opening a single file. I can hack ispell.el to suppress the message, but I think the core problem is more starting up the aspell process for 100+ files. Can't tell without profiling.=20 When org is running in the background, I actually get two messages, alternating, that and... Darn it, I can't get it to happen right now. > exactly are the messages? What is your flyspell config? It is using aspell. I work in both spanish and english, mostly english. ;; Useful because my LANG variable is usually set to es_ES.UTF-8 ;; Maybe there is another default dictionary I could set or a better way ;; to suppress the message (like, maybe suppressing the language ;; variable somewhere else), or doing this in ispell rather than flyspell (setq flyspell-default-dictionary "en") ;; Relevant hooks ;; Maybe find something that suppresses all these hooks until the buffer ;; is actually displayed on screen would help. (dolist (hook '(text-mode-hook)) (add-hook hook (lambda () (flyspell-mode 1)))) (dolist (hook '(erc-mode-hook emacs-lisp-mode-hook text-mode-hook)) (add-hook hook (lambda () (abbrev-mode 1)) (add-hook hook (lambda () (auto-capitalization-mode 1)) )) > >> Solution #1) cut the number of files down - is a good one. I probably >> can cut those files easily in half right now. The problem is that I have >> about 600 more files to import (scenes from a book), and I really like >> the idea of being able to know what my characters are doing in 2023, and >> separate files was kind of useful at one point. > > One recommendation would be to leave files that are purely notes out of > your agenda. I noticed from your blog posts that you are adding > timestamps to events from your novel. Are these active or inactive > timestamps? My guess is that parsing all that data is slowing org-mode > down. You can always create a timeline from within a particular file > without including it in your normal agenda files. You can also create > custom agenda views to work on different sets of org files. Thank you for reading! I haven't got further than wanting to use embedded logic in the book for various things like calculating relative delta-v, I'd like to sort out some more simple stuff first (like blogging and managing my life. I'm going to get off of blogger, that's certain, but moving to what (and getting pretty html output) remains an open question.=20 Some percentage of 898 blog posts to convert, too. Sigh. I'll want to keep draft blog posts in the agenda. > http://orgmode.org/worg/org-tutorials/org-custom-agenda-commands.php Reading... Brain dumping core...=20 > > While org-mode is exceptionally robust as a plain text tool, if you need > to manage an ever-growing collection of thousands and thousands of > pieces of data, you'll probably be better served by 1) using a database Heck, no, that's what gobs of memory is for! Emacs is, for once, not the biggest process on my system, even with over a hundred buffers loaded. Partial top listing: 764 d 20 0 663m 184m 13m S 0.0 19.2 10:56.43 firefox=20=20= =20=20=20 3964 d 20 0 343m 164m 9112 S 0.0 17.0 11:32.16 emacs=20=20=20= =20=20 :) I have tried to organize my personal collection of messes before using everything from a zillion text files, to database based stuff like evolution, to writing my own XML outliner, to MS-project. All fell down somewhere. What works best is an outliner. I really, really, really missed MORE. Org is better than MORE in almost every way. Yep, I'm willing to adapt to how it works and adapt it to how I work.=20 > or 2) dividing your org-mode files into separate "collections" (i.e., > reserving your regular agenda for only a subset of org files). That strikes me as a good option, but what I'll want to do is add the book-mode collection of files to the normal org-mode files when in "book-mode", so I can stay on task. I think learning how to profile emacs is moving higher up on my list of smart things to learn. > > I keep my org files lean by regularly archiving subtrees. I can always > search the archives when I need to. I am going to be doing that. My current issue with archiving is that I'd like a view that shows my DONE, non-repeating tasks, so I can make sure I've scheduled the next step for all of them before I archive them, and also have a report of what I did for the month that I can review later. (Maybe one exists, I'll go re-read the doc, I remember reading about it...) I LOVE seeing the DONE tasks, it gives me a sense of accomplishment out of a chaotic month, but yea, they need to go OOSOOM as soon as I figure out the above. LOTS of DONE tasks in the last two months. I feel productive. It would be kind of cool to be able to "explode" a set of agenda items into visible on-screen buffers. >> so thought 2) would be to have it only attempt to construct background >> agendas when the system is otherwise idle for a few minutes. I don't >> know how to do that, I figure wrapping this bit with something that >> could detect idleness instead of just running arbitrarily would be good. >> >> (run-at-time nil 3600 'org-agenda-to-appt) >> >> don't know how to detect idleness. > > http://www.gnu.org/software/emacs/elisp/html_node/Idle-Timers.html Bueno. Writing that bit of code right now. Hmmm... Maybe I can tie this int= o erc which also has annoying blocking behavior on netsplits and lost connectivit= y. So much depth to Emacs! Thanks for the thoughts! --=20 Dave Taht http://the-edge.blogspot.com