From mboxrd@z Thu Jan 1 00:00:00 1970 From: jorge.alfaro-murillo@yale.edu (Jorge A. Alfaro-Murillo) Subject: Re: Managing articles in orgmode and collaboration Date: Tue, 23 Sep 2014 09:52:26 -0400 Message-ID: <87bnq6wijp.fsf@yale.edu> References: <87k35dvnu7.fsf@grothesque.org> <87zje8u404.fsf@yale.edu> <87ha0fzrzu.fsf@grothesque.org> <877g1bwnnb.fsf@yale.edu> <87ppem8oo7.fsf@grothesque.org> Mime-Version: 1.0 Content-Type: text/plain; format=flowed Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:39167) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XWQXI-00042c-IU for emacs-orgmode@gnu.org; Tue, 23 Sep 2014 09:53:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XWQXC-0006Fc-AE for emacs-orgmode@gnu.org; Tue, 23 Sep 2014 09:53:44 -0400 Received: from plane.gmane.org ([80.91.229.3]:43319) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XWQXC-0006F8-0N for emacs-orgmode@gnu.org; Tue, 23 Sep 2014 09:53:38 -0400 Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1XWQX5-0003dO-DT for emacs-orgmode@gnu.org; Tue, 23 Sep 2014 15:53:31 +0200 Received: from nat-130-132-173-32.central.yale.edu ([130.132.173.32]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 23 Sep 2014 15:53:31 +0200 Received: from jorge.alfaro-murillo by nat-130-132-173-32.central.yale.edu with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 23 Sep 2014 15:53:31 +0200 List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: emacs-orgmode@gnu.org Christoph Groth writes: > But not all is good: > > - Scaling: Some simple tests seem to indicate that org mode > becomes too sluggish with files of about 50k lines. This is a > dimension that could be easily reached over 10 years if the > file grows by 20 lines per day on average. This is not a > problem in the beginning, but if the scheme does not scale to > a few thousand entries, this renders the whole idea way less > interesting. > - More advanced searching is lacking: AFAIK org mode currently > does not support searching for articles of a given author that > also contain a given keyword in the notes. > Any insights about these two problems? Perhaps the scaling could > be managed by splitting index.org into several files (by year > for example). But how to search then? It's probably not a good > idea to add all the bibliography org-files into the agenda. (Is > there a way to have a secondary list of agenda files?) Perhaps > the solution for both problems would be to write a fast > commandline query tool for such org-databases? The tool could > even use a fast cache if necessary. BibTeX provides bibtex-search-entries (in a bib file it is bound to C-c C-a), which searches for entries with a certain field matching a regexp. Perhaps you want to take advantage of that function to direct your org searches to bib files and back. Check also bibtex-files and bibtex-search-entry-globally, since you would then be able to split the org files and export into several bib files if a single bib file proves too much. My current articles.bib is about 14k lines, and it is not sluggish at all, but I will have to wait a couple of years until I tell you if a 50k one would be. Best, -- Jorge.