* Improving org-contacts performance (and state of development in general) @ 2020-09-06 16:20 Daryl Manning 2020-09-06 16:27 ` Bastien 2020-09-07 2:26 ` Ihor Radchenko 0 siblings, 2 replies; 10+ messages in thread From: Daryl Manning @ 2020-09-06 16:20 UTC (permalink / raw) To: Org-mode [-- Attachment #1: Type: text/plain, Size: 1015 bytes --] Strangely, I've come to rely over the last year on org-contacts as a lightweight, taggable CRM. However, as the file and C-z notes have grown, performance has really started to drag. I know people have used various schemes (caching) etc to try to improve performance and the like, but updates to the file are taking a solid 5 seconds now when making major updates and moving tags around. Is there a solid, forked branch anywhere that focuses on enhancing performance anywhere? I'm tempted to wade in and add features and improvements myself but my elisp-fu is dodgy at best (more golang these days.). I'd be interested in what people are doing to speed it up (and if it is under anything like active development for improvements. It does feel super handy, and feels like it just needs a performance and more modern features overhaul - more on interoperability and less on in-emacs interoperability.). Would love to hear what people have done overall workflow wise if they are using it seriously. thanks, Daryl. [-- Attachment #2: Type: text/html, Size: 1187 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Improving org-contacts performance (and state of development in general) 2020-09-06 16:20 Improving org-contacts performance (and state of development in general) Daryl Manning @ 2020-09-06 16:27 ` Bastien 2020-09-07 8:27 ` Julien Danjou 2020-09-07 2:26 ` Ihor Radchenko 1 sibling, 1 reply; 10+ messages in thread From: Bastien @ 2020-09-06 16:27 UTC (permalink / raw) To: Daryl Manning; +Cc: Julien Danjou, Org-mode Hi Daryl, Daryl Manning <dwm+orgmode@wakatara.com> writes: > Is there a solid, forked branch anywhere that focuses on enhancing > performance anywhere? Note that I'm aware of, but I don't use org-contacts.el, so maybe other will have more information. > I'm tempted to wade in and add features and improvements myself but > my elisp-fu is dodgy at best (more golang these days.). I'm copying Julien, the original author of org-contacts.el, to check if he plans to maintain org-contacts.el further and if he's okay for someone to take over. That someone could be you: better to have a maintainer than no one. Also, org-contacts.el is part of the "contrib/" directory in Org's repository and soon after 9.4 we will strip this directory out of the org-mode.git repo, so having someone taking care of individual contrib packages would be really appreciated. > I'd be interested in what people are doing to speed it up (and if it > is under anything like active development for improvements. It does > feel super handy, and feels like it just needs a performance and more > modern features overhaul - more on interoperability and less on > in-emacs interoperability.). I'm afraid most tricks will be general Org tricks, no directly related to the code in org-contacts.el -- but perhaps I'm wrong. Best, -- Bastien ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Improving org-contacts performance (and state of development in general) 2020-09-06 16:27 ` Bastien @ 2020-09-07 8:27 ` Julien Danjou 2020-09-07 14:41 ` Bastien 0 siblings, 1 reply; 10+ messages in thread From: Julien Danjou @ 2020-09-07 8:27 UTC (permalink / raw) To: Bastien; +Cc: Org-mode, Daryl Manning On Sun, Sep 06 2020, Bastien wrote: Hi everyone, > I'm copying Julien, the original author of org-contacts.el, to check > if he plans to maintain org-contacts.el further and if he's okay for > someone to take over. I don't plan to do anything, feel free to take over! Cheers, -- Julien Danjou # Free Software hacker # https://julien.danjou.info ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Improving org-contacts performance (and state of development in general) 2020-09-07 8:27 ` Julien Danjou @ 2020-09-07 14:41 ` Bastien 0 siblings, 0 replies; 10+ messages in thread From: Bastien @ 2020-09-07 14:41 UTC (permalink / raw) To: Julien Danjou; +Cc: Org-mode, Daryl Manning Julien Danjou <julien@danjou.info> writes: >> I'm copying Julien, the original author of org-contacts.el, to check >> if he plans to maintain org-contacts.el further and if he's okay for >> someone to take over. > > I don't plan to do anything, feel free to take over! Thanks for confirming - Daryl, don't hesitate! -- Bastien ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Improving org-contacts performance (and state of development in general) 2020-09-06 16:20 Improving org-contacts performance (and state of development in general) Daryl Manning 2020-09-06 16:27 ` Bastien @ 2020-09-07 2:26 ` Ihor Radchenko 2020-09-07 2:52 ` Daryl Manning 1 sibling, 1 reply; 10+ messages in thread From: Ihor Radchenko @ 2020-09-07 2:26 UTC (permalink / raw) To: Daryl Manning, Org-mode > However, as the file and C-z notes have grown, > performance has really started to drag. I know people have used various > schemes (caching) etc to try to improve performance and the like, but > updates to the file are taking a solid 5 seconds now when making major > updates and moving tags around. Could you provide some examples what exactly is being slow? Maybe my WIP work on improving performance on large org files [1] might help. Best, Ihor [1] https://www.mail-archive.com/emacs-orgmode@gnu.org/msg127740.html Daryl Manning <dwm+orgmode@wakatara.com> writes: > Strangely, I've come to rely over the last year on org-contacts as a > lightweight, taggable CRM. However, as the file and C-z notes have grown, > performance has really started to drag. I know people have used various > schemes (caching) etc to try to improve performance and the like, but > updates to the file are taking a solid 5 seconds now when making major > updates and moving tags around. > > Is there a solid, forked branch anywhere that focuses on enhancing > performance anywhere? I'm tempted to wade in and add features and > improvements myself but my elisp-fu is dodgy at best (more golang these > days.). > > I'd be interested in what people are doing to speed it up (and if it is > under anything like active development for improvements. It does feel super > handy, and feels like it just needs a performance and more modern features > overhaul - more on interoperability and less on in-emacs interoperability.). > > Would love to hear what people have done overall workflow wise if they are > using it seriously. > > thanks, > Daryl. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Improving org-contacts performance (and state of development in general) 2020-09-07 2:26 ` Ihor Radchenko @ 2020-09-07 2:52 ` Daryl Manning 2020-09-07 8:36 ` Ihor Radchenko 2020-09-10 21:45 ` TRS-80 0 siblings, 2 replies; 10+ messages in thread From: Daryl Manning @ 2020-09-07 2:52 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Org-mode [-- Attachment #1: Type: text/plain, Size: 2707 bytes --] Primary examples would be adding a note (CTRL-z) or changing a tag on a person and then having org-agenda update that. I am assuming it is because the entire file needs to be parsed rather than say, some index of entries. (so perhaps I mischaracterized org-contacts as being slow versus its interaction with other programs.) (for search I use swiper which is very efficient for searching the file whenI need it.). tho quite interested in seeing what perf enhancements you've done on large org files would be interesting. Daryl. PS> As an outside feature though, interoperability of the org-contact formats with other operating system address books, most notable gnome contacts/evolution, goog contacts, and OSX address book would be high on my list in terms of improving org-contacts though. (eg, raw, structued info in all address books, and say perhaps notes or similar maintained and synced in ome manner. On Mon, Sep 7, 2020 at 10:27 AM Ihor Radchenko <yantar92@gmail.com> wrote: > > However, as the file and C-z notes have grown, > > performance has really started to drag. I know people have used various > > schemes (caching) etc to try to improve performance and the like, but > > updates to the file are taking a solid 5 seconds now when making major > > updates and moving tags around. > > Could you provide some examples what exactly is being slow? > Maybe my WIP work on improving performance on large org files [1] might > help. > > Best, > Ihor > > [1] https://www.mail-archive.com/emacs-orgmode@gnu.org/msg127740.html > > > Daryl Manning <dwm+orgmode@wakatara.com> writes: > > > Strangely, I've come to rely over the last year on org-contacts as a > > lightweight, taggable CRM. However, as the file and C-z notes have grown, > > performance has really started to drag. I know people have used various > > schemes (caching) etc to try to improve performance and the like, but > > updates to the file are taking a solid 5 seconds now when making major > > updates and moving tags around. > > > > Is there a solid, forked branch anywhere that focuses on enhancing > > performance anywhere? I'm tempted to wade in and add features and > > improvements myself but my elisp-fu is dodgy at best (more golang these > > days.). > > > > I'd be interested in what people are doing to speed it up (and if it is > > under anything like active development for improvements. It does feel > super > > handy, and feels like it just needs a performance and more modern > features > > overhaul - more on interoperability and less on in-emacs > interoperability.). > > > > Would love to hear what people have done overall workflow wise if they > are > > using it seriously. > > > > thanks, > > Daryl. > [-- Attachment #2: Type: text/html, Size: 3544 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Improving org-contacts performance (and state of development in general) 2020-09-07 2:52 ` Daryl Manning @ 2020-09-07 8:36 ` Ihor Radchenko 2020-09-07 14:43 ` Bastien 2020-09-10 21:45 ` TRS-80 1 sibling, 1 reply; 10+ messages in thread From: Ihor Radchenko @ 2020-09-07 8:36 UTC (permalink / raw) To: Daryl Manning; +Cc: Org-mode > tho quite interested in seeing what perf enhancements you've done on large > org files would be interesting. That's primarily a one single enhancement - use text properties instead of overlays to hide/fold text. Overlays are slow - every time Emacs need to move point across hidden overlays, it takes O(N_overlays). The problem especially affects buffers with many property drawers (every drawer may require an individual overlay) - should be the case for org-contacts files. > Primary examples would be adding a note (CTRL-z) or changing a tag on a > person and then having org-agenda update that. I am assuming it is because > the entire file needs to be parsed rather than say, some index of entries. I did some benchmarks on my agendas with and without my enhancements - my main agenda builds in around 7sec now in comparison with 18-20sec when using overlays. Though 20sec benchmark was not on current master, there were several commits reducing the number of overlays in some cases after I did the benchmark. Also, you may consider native-comp branch. It can push the speed even further. My agenda only took 3-4 seconds on native-comp Emacs branch. > PS> As an outside feature though, interoperability of the org-contact > formats with other operating system address books, most notable gnome > contacts/evolution, goog contacts, and OSX address book would be high on my > list in terms of improving org-contacts though. (eg, raw, structued info in > all address books, and say perhaps notes or similar maintained and synced > in ome manner. I guess it is a job for [future] ox-*.el packages. Best, Ihor Daryl Manning <dwm+orgmode@wakatara.com> writes: > Primary examples would be adding a note (CTRL-z) or changing a tag on a > person and then having org-agenda update that. I am assuming it is because > the entire file needs to be parsed rather than say, some index of entries. > > (so perhaps I mischaracterized org-contacts as being slow versus its > interaction with other programs.) > > (for search I use swiper which is very efficient for searching the file > whenI need it.). > > tho quite interested in seeing what perf enhancements you've done on large > org files would be interesting. > > Daryl. > PS> As an outside feature though, interoperability of the org-contact > formats with other operating system address books, most notable gnome > contacts/evolution, goog contacts, and OSX address book would be high on my > list in terms of improving org-contacts though. (eg, raw, structued info in > all address books, and say perhaps notes or similar maintained and synced > in ome manner. > > > > On Mon, Sep 7, 2020 at 10:27 AM Ihor Radchenko <yantar92@gmail.com> wrote: > >> > However, as the file and C-z notes have grown, >> > performance has really started to drag. I know people have used various >> > schemes (caching) etc to try to improve performance and the like, but >> > updates to the file are taking a solid 5 seconds now when making major >> > updates and moving tags around. >> >> Could you provide some examples what exactly is being slow? >> Maybe my WIP work on improving performance on large org files [1] might >> help. >> >> Best, >> Ihor >> >> [1] https://www.mail-archive.com/emacs-orgmode@gnu.org/msg127740.html >> >> >> Daryl Manning <dwm+orgmode@wakatara.com> writes: >> >> > Strangely, I've come to rely over the last year on org-contacts as a >> > lightweight, taggable CRM. However, as the file and C-z notes have grown, >> > performance has really started to drag. I know people have used various >> > schemes (caching) etc to try to improve performance and the like, but >> > updates to the file are taking a solid 5 seconds now when making major >> > updates and moving tags around. >> > >> > Is there a solid, forked branch anywhere that focuses on enhancing >> > performance anywhere? I'm tempted to wade in and add features and >> > improvements myself but my elisp-fu is dodgy at best (more golang these >> > days.). >> > >> > I'd be interested in what people are doing to speed it up (and if it is >> > under anything like active development for improvements. It does feel >> super >> > handy, and feels like it just needs a performance and more modern >> features >> > overhaul - more on interoperability and less on in-emacs >> interoperability.). >> > >> > Would love to hear what people have done overall workflow wise if they >> are >> > using it seriously. >> > >> > thanks, >> > Daryl. >> ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Improving org-contacts performance (and state of development in general) 2020-09-07 8:36 ` Ihor Radchenko @ 2020-09-07 14:43 ` Bastien 2020-09-07 15:01 ` Ihor Radchenko 0 siblings, 1 reply; 10+ messages in thread From: Bastien @ 2020-09-07 14:43 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Org-mode, Daryl Manning Ihor Radchenko <yantar92@gmail.com> writes: > That's primarily a one single enhancement - use text properties > instead of overlays to hide/fold text. ... or improve the speed for overlays in Emacs :) -- Bastien ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Improving org-contacts performance (and state of development in general) 2020-09-07 14:43 ` Bastien @ 2020-09-07 15:01 ` Ihor Radchenko 0 siblings, 0 replies; 10+ messages in thread From: Ihor Radchenko @ 2020-09-07 15:01 UTC (permalink / raw) To: Bastien; +Cc: Org-mode, Daryl Manning > ... or improve the speed for overlays in Emacs :) My Emacs-C-fu is not good enough ¯ \ _ (ツ) _ / ¯ Bastien <bzg@gnu.org> writes: > Ihor Radchenko <yantar92@gmail.com> writes: > >> That's primarily a one single enhancement - use text properties >> instead of overlays to hide/fold text. > > ... or improve the speed for overlays in Emacs :) > > -- > Bastien ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Improving org-contacts performance (and state of development in general) 2020-09-07 2:52 ` Daryl Manning 2020-09-07 8:36 ` Ihor Radchenko @ 2020-09-10 21:45 ` TRS-80 1 sibling, 0 replies; 10+ messages in thread From: TRS-80 @ 2020-09-10 21:45 UTC (permalink / raw) To: Org-mode On 2020-09-06 22:52, Daryl Manning wrote: > PS> As an outside feature though, interoperability of the org-contact > formats with other operating system address books, most notable gnome > contacts/evolution, goog contacts, and OSX address book would be high > on my list in terms of improving org-contacts though. (eg, raw, > structued info in all address books, and say perhaps notes or similar > maintained and synced in ome manner. Hi Daryl, Good topic. This is essentially the main reason I don't use org-contacts, even though I have become a heavy Orgmode/Emacs user for almost everything else over the past few years. In my case, it's the Contacts on my Android phone, which seems to me to be the primary place where I find myself adding/removing/editing my contacts. In my view, there /must/ be reliable and proper (/two-way/) sync between the two. Since that doesn't exist (that I am aware of) my Android phone won out over org-contacts so far. I haven't looked into it in a while, but at one point I thought that using some common and open format would be the best idea. Something like CardDAV. I actually had it working (syncing) with my NextCloud instance when I had that up and running, but for me NextCloud proved to be to "heavy" in general so I moved away from it, preferring instead lighter weight tools to do the same things. And so far I only implemented Syncthing for the syncing part, and nothing for the CalDAV/CardDAV part (yet) although I have had my eye on Radicale already for a while. Maybe you know all this already, I'm just adding my $0.02. Cheers, TRS-80 ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2020-09-10 21:46 UTC | newest] Thread overview: 10+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2020-09-06 16:20 Improving org-contacts performance (and state of development in general) Daryl Manning 2020-09-06 16:27 ` Bastien 2020-09-07 8:27 ` Julien Danjou 2020-09-07 14:41 ` Bastien 2020-09-07 2:26 ` Ihor Radchenko 2020-09-07 2:52 ` Daryl Manning 2020-09-07 8:36 ` Ihor Radchenko 2020-09-07 14:43 ` Bastien 2020-09-07 15:01 ` Ihor Radchenko 2020-09-10 21:45 ` TRS-80
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).