emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
From: John Kitchin <jkitchin@andrew.cmu.edu>
To: emacs-orgmode@gnu.org
Cc: Jude DaShiell <jdashiel@panix.com>
Subject: Re: exported contacts problem
Date: Fri, 02 Aug 2019 18:19:47 -0400	[thread overview]
Message-ID: <m2d0hnp5vg.fsf@andrew.cmu.edu> (raw)
In-Reply-To: <20190802160236.GR17561@protected.rcdrun.com>

There are a few options for contacts in org-mode that I have tried.

I agree for a lot of contacts (probably more than a hundred or so), then
native org-contacts might be too slow.

In scimax I have tried a few different approaches to deal with this. The
first is all org/elisp that uses a cache to speed up looking up
contacts. See
https://github.com/jkitchin/scimax/blob/master/contacts.el. The gist of
this is you define a list of files that contacts will be in, and as long
as they haven't changed since the cache was built the cache is used, and
if they have changed the cache for that file is rebuilt. For large
files, org parsing is not that fast in this, and I have not spent any
time optimizing this. There are lots of ways to speed it up, but it
hasn't been so slow to need solving yet. I use this
code regularly, but nothing critical depends on it working. The downside
of this is you have to maintain a list of files that serve as contact
sources. I am pretty sure everything in it is compatible with
org-contacts. This method is fine and I have about 5.8K contacts in this
cache. I use this approach very regularly and have made some nice
org-speedkeys for these contacts and other interesting ideas.

The second approach is also related to caching, and uses a sqlite
database as the cache.
https://github.com/jkitchin/scimax/blob/master/org-db.el This approach
uses a hook function to update the db any time the file is updated. I
can access about 7500 contacts pretty quickly this way (evidently I have
a lot of contacts outside the list of files designated as contacts in
the first approach). I don't use this approach as much for contacts as
for searching all headlines in all my org-files, but I built a contacts
feature because I could. This approach could be expanded to a server
based database but so far sqlite has served all my needs, and it has
about 72K headlines indexed into it right now. The main benefit of this
is mostly set it and forget it; contacts get updated as you make them or
update them.

The main benefit of both of these approaches is it lets you keep org as
the primary way of creating contacts, including keeping all manner of
notes associated with them in sub-headings. You can import contacts from
anywhere just by writing code that creates the headings in org-contacts
form. Since there is a cache, you could probably also write directly to
the cache instead of making intermediate org files. The downside in my
opinion is that I consider it acceptable for the cache to mostly work
most of the time and it is not hard for them to get out of sync (and
neither difficult for them to get resynced). For my work, there is very
little consequence if I can't find a contact. You can organize your
contacts as you see fit all over your file system, which is at times
convenient, and at times hard to deal with. Duplicates, for example, are
a challenge to deal with, and sometimes contacts seem to belong in more
than one place. This is a limitation of org at the moment, there is no
way to "transclude" a heading yet.

You can create groups, these are just headings with multiple emails in
the EMAIL property, as well as use todo states and tags on the contacts
to select them with ivy/helm.

I haven't converged on what the best way to do this kind of persistent
caching. I also use it in org-ref for very large (20K) bibtex databases
and it works fine there too.

I also use mu4e, which indexes everything into a xapian database. It has
about 9K contacts in it right now, and indexes about 100K messages. So
far this works for me. It is more complicated to setup than the previous
two options though.

I wouldn't claim any of these can scale to 192K contacts, but it seems
there are paths towards it. It would certainly take an eye and dedicated
effort towards performance though.


Jean Louis <bugs@gnu.support> writes:

> * Jude DaShiell <jdashiel@panix.com> [2019-08-02 17:48]:
>> I have one email message with several .vcf file attachments.  Has orgmode
>> got any tool or tools I can use to import contacts from such a message
>> into an orgmode table?
>
> And by all means, I would never keep contact in Org file, that is for
> short list fine, but for any future planning, contacts shall be in a
> real database sorted by its lists.
>
> A list could be group of people, or account name, or company name, or
> organization, or interest lists.
>
> Neither bbdb nor Org is suitable for any serious collection of
> contacts. I have 192,000+ contacts, and when they are in database and
> I am using PostgreSQL, it gives me most of benefits, I can sort people
> into lists, groups, I can contact them, count interactions, open up
> their files, emails with a fast command, edit their data, add notes,
> send them faxes and SMS, maintain relations.
>
> Jean


--
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu

  parent reply	other threads:[~2019-08-02 22:19 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-02 15:47 exported contacts problem Jude DaShiell
2019-08-02 16:00 ` Jean Louis
2019-08-02 16:02 ` Jean Louis
2019-08-02 16:06   ` Jude DaShiell
2019-08-02 21:09   ` Eric Abrahamsen
2019-08-02 21:34     ` Jean Louis
2019-08-03  0:26       ` Eric Abrahamsen
2019-08-03 10:33         ` Jean Louis
2019-08-03 15:38           ` Eric Abrahamsen
2019-08-03 16:32             ` Jean Louis
2019-08-03 17:48               ` Neil Jerram
2019-08-03 18:25                 ` Jean Louis
2019-08-03 21:32                 ` Eric Abrahamsen
2019-08-03 22:12                   ` Jean Louis
2019-08-04  1:28                   ` Tim Cross
2019-08-13 22:24                   ` David Masterson
2019-08-14 20:38                     ` Eric Abrahamsen
2019-08-16  0:07                       ` David Masterson
2019-08-03 10:48         ` Jean Louis
2019-08-02 22:19   ` John Kitchin [this message]
2019-08-02 20:33 ` Tim Cross
2019-08-02 20:57   ` Jude DaShiell
2019-08-02 21:25     ` Tim Cross
2019-08-02 21:39       ` Jean Louis
2019-08-02 22:07         ` Tim Cross
2019-08-03  8:25           ` Jean Louis
2019-08-02 21:00   ` Jean Louis

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://www.orgmode.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=m2d0hnp5vg.fsf@andrew.cmu.edu \
    --to=jkitchin@andrew.cmu.edu \
    --cc=emacs-orgmode@gnu.org \
    --cc=jdashiel@panix.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).