* [O] Help speeding up Org iCal export @ 2020-05-28 0:24 Ken Mankoff 2020-05-28 8:11 ` Eric S Fraga 2020-05-28 12:17 ` Nicolas Goaziou 0 siblings, 2 replies; 5+ messages in thread From: Ken Mankoff @ 2020-05-28 0:24 UTC (permalink / raw) To: Org-mode Hello, When I eval `(org-icalendar-combine-agenda-files nil)`, it takes about 15 seconds. Some strange messages in the *Messages* buffer include: 22 files scanned, 7 files contains IDs and in total 23 IDs found. But my `org-agenda-files` variable only contains 12 files and no #+INCLUDE statements. There are also several dozen lines of org-babel-exp process sh at position 74770... org-babel-exp process sh at position 74830... org-babel-exp process sh at position 75147... ... org-babel-exp process sqlite at position 27700... org-babel-exp process sqlite at position 27704... org-babel-exp process sh at position 27604... org-babel-exp process sh at position 27617... org-babel-exp process sh at position 27691... org-babel-exp process translate at position 27970... (and with bash, python, emacs-lisp, and other languages). When I view for example the only #BEGIN_SRC translate block in any of my files it is just a test in my notes file: #+BEGIN_SRC translate :src en :dest da This is a test #+END_SRC I cannot eval it with C-c C-c because I have set ":eval no" system-wide with: (setq org-babel-default-header-args (cons '(:eval . "no") (assq-delete-all :eval org-babel-default-header-args))) Sometimes the Messages contain confirmation that a code block is disabled: org-babel-exp process plantuml at position 27719... Evaluation of this plantuml code block is disabled. org-babel-exp process plantuml at position 27762... Evaluation of this plantuml code block is disabled. But those blocks don't have any special header: #+begin_src plantuml :file tryout.png ...misc plantuml code here. #+end_src Can anyone offer suggestions why it reports scanning 22 files, and how to speed it up since no code should be evaluated? Thank you, -k. ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [O] Help speeding up Org iCal export 2020-05-28 0:24 [O] Help speeding up Org iCal export Ken Mankoff @ 2020-05-28 8:11 ` Eric S Fraga 2020-05-28 12:17 ` Nicolas Goaziou 1 sibling, 0 replies; 5+ messages in thread From: Eric S Fraga @ 2020-05-28 8:11 UTC (permalink / raw) To: Ken Mankoff; +Cc: Org-mode On Wednesday, 27 May 2020 at 17:24, Ken Mankoff wrote: > Can anyone offer suggestions why it reports scanning 22 files, and how > to speed it up since no code should be evaluated? Anecdotally, I have the impression that all org buffers in the current Emacs instance are scanned as well as the files noted in the agenda list. But I have no proof of this nor do I know why it would do this. I have the same issue with org-caldav-sync. -- : Eric S Fraga via Emacs 28.0.50, Org release_9.3.6-640-g9bc0cc ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Help speeding up Org iCal export 2020-05-28 0:24 [O] Help speeding up Org iCal export Ken Mankoff 2020-05-28 8:11 ` Eric S Fraga @ 2020-05-28 12:17 ` Nicolas Goaziou 2020-05-28 14:06 ` Ken Mankoff 1 sibling, 1 reply; 5+ messages in thread From: Nicolas Goaziou @ 2020-05-28 12:17 UTC (permalink / raw) To: Ken Mankoff; +Cc: Org-mode Hello, I see a few misconceptions here. Ken Mankoff <mankoff@gmail.com> writes: > When I eval `(org-icalendar-combine-agenda-files nil)`, it takes about > 15 seconds. OK. You may want to profile this, or use ELP. > Some strange messages in the *Messages* buffer include: > > 22 files scanned, 7 files contains IDs and in total 23 IDs found. You may have set `org-icalendar-store-UID' somehow. > But my `org-agenda-files` variable only contains 12 files and no > #+INCLUDE statements. ID scanning is not limited to agenda files. Org also scans files known to contain ID. E.g., if, during the session, you insert an ID in "foo.org", later scans (in the same session) will include "foo.org". See `org-id-files' (internal variable), and `org-id-extra-files', which may contain even more files. > There are also several dozen lines of > > org-babel-exp process sh at position 74770... > org-babel-exp process sh at position 74830... > org-babel-exp process sh at position 75147... > ... > org-babel-exp process sqlite at position 27700... > org-babel-exp process sqlite at position 27704... > org-babel-exp process sh at position 27604... > org-babel-exp process sh at position 27617... > org-babel-exp process sh at position 27691... > org-babel-exp process translate at position 27970... > > (and with bash, python, emacs-lisp, and other languages). When I view for example the only #BEGIN_SRC translate block in any of my files it is just a test in my notes file: > > #+BEGIN_SRC translate :src en :dest da > This is a test > #+END_SRC > > > I cannot eval it with C-c C-c because I have set ":eval no" > system-wide with: This is a red herring, IMO. The message above means the block is being processed for export, not necessarily evaluated. In order to know how to handle the block, Org needs to process it. > (setq org-babel-default-header-args > (cons '(:eval . "no") > (assq-delete-all :eval org-babel-default-header-args))) > > > Sometimes the Messages contain confirmation that a code block is disabled: > > org-babel-exp process plantuml at position 27719... > Evaluation of this plantuml code block is disabled. > org-babel-exp process plantuml at position 27762... > Evaluation of this plantuml code block is disabled. Here, it tries to evaluate the blocks, obviously, but eventually doesn't, per your settings. > But those blocks don't have any special header: > > #+begin_src plantuml :file tryout.png > ...misc plantuml code here. > #+end_src I see no issue here. Org checks if it should evaluate it, and cannot. > Can anyone offer suggestions why it reports scanning 22 files, and how > to speed it up since no code should be evaluated? See above. Instead of wild guesses, you could use an appropriate tool. Regards, -- Nicolas Goaziou ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Help speeding up Org iCal export 2020-05-28 12:17 ` Nicolas Goaziou @ 2020-05-28 14:06 ` Ken Mankoff 2022-01-20 22:57 ` Josh Moller-Mara 0 siblings, 1 reply; 5+ messages in thread From: Ken Mankoff @ 2020-05-28 14:06 UTC (permalink / raw) To: Ken Mankoff, Org-mode [-- Attachment #1: Type: text/plain, Size: 3835 bytes --] Hi Nicolas, Thank you for providing suggestions about the appropriate tool. I've profiled it, examined the variables you suggested, but am not sure what next steps to take. I hope it is OK that I post questions about this on this list... On Thu, May 28, 2020 at 5:17 AM Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote: > Ken Mankoff <mankoff@gmail.com> writes: > > > When I eval `(org-icalendar-combine-agenda-files nil)`, it takes about > > 15 seconds. > > OK. You may want to profile this, or use ELP. > Output from the profiler is here: https://pastebin.com/QvL9w69R but the key lines seems to be: - org-icalendar-combine-agenda-files 15540 83% - apply 15540 83% - org-icalendar--combine-files 15540 83% - mapconcat 15536 83% - #<compiled 0x15879b1> 15536 83% - org-export-as 15162 81% - org-export-data 6493 34% - mapconcat 6493 34% - #<compiled 0x25baf9d> 6493 34% - org-export-data 6493 34% - mapconcat 6164 33% - #<compiled 0x22864b5> 6160 33% - org-export-data 6156 33% - mapconcat 5674 30% with org-export-data and mapconcat repeating, AND - mapc 5672 30% - #<compiled 0x3129c91> 5672 30% - mapc 5672 30% - #<compiled 0x3129c91> 5668 30% - #<compiled 0x15b2141> 4392 23% - org-id-find 4392 23% - org-id-update-id-locations 4376 23% - org-map-entries 3696 19% - org-scan-tags 3668 19% + #<lambda 0x22c5233ea> 990 5% + #<lambda 0x22c5233ea> 879 4% + #<lambda 0x22c5233ea> 830 4% > Some strange messages in the *Messages* buffer include: > > > > 22 files scanned, 7 files contains IDs and in total 23 IDs found. > > You may have set `org-icalendar-store-UID' somehow. > It is set to nil. I have only the *scratch* buffer open, and no other buffers (although some where open earlier in this emacs session). > > But my `org-agenda-files` variable only contains 12 files and no > > #+INCLUDE statements. > > ID scanning is not limited to agenda files. Org also scans files known > to contain ID. E.g., if, during the session, you insert an ID in > "foo.org", later scans (in the same session) will include "foo.org". > > See `org-id-files' (internal variable), and `org-id-extra-files', which > may contain even more files. > `org-id-files` contained a subset of my agenda files plus a few extra (total: 7 files, and these 7 plus my agenda files do not sum to 22). When I set it to nil, nothing significant changed, but it was reset after running `org-icalendar-combine-agenda-files` again. `org-id-extra-files` was set to org-agenda-text-search-extra-files. When I set it to nil, nothing significant changed. It does not get reset when `org-icalendar-combine-agenda-files` is run. -k. [-- Attachment #2: Type: text/html, Size: 6256 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Help speeding up Org iCal export 2020-05-28 14:06 ` Ken Mankoff @ 2022-01-20 22:57 ` Josh Moller-Mara 0 siblings, 0 replies; 5+ messages in thread From: Josh Moller-Mara @ 2022-01-20 22:57 UTC (permalink / raw) To: Ken Mankoff, Org-mode I'm running into this issue now. One thing I noticed is that if you have broken links (and org-export-with-broken-links is t), org-id-update-id-locations will be run for every broken link it finds. This turns out to be quite slow and unnecessary since no files will have changed in between org-id-update-id-locations runs during export. One hacky workaround is to temporarily disable org-id-update-id-locations, like so: (cl-letf (((symbol-function 'org-id-update-id-locations) (lambda (&optional files silent) (message "Not updating locations")))) (let ((org-export-with-broken-links t) (gc-cons-threshold 80000000)) (org-icalendar-combine-agenda-files))) The right solution would probably be to not have broken links in the first place. :P Or maybe org-id-update-id-locations could be debounced or cached in some way. Best, Josh ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2022-01-21 0:07 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2020-05-28 0:24 [O] Help speeding up Org iCal export Ken Mankoff 2020-05-28 8:11 ` Eric S Fraga 2020-05-28 12:17 ` Nicolas Goaziou 2020-05-28 14:06 ` Ken Mankoff 2022-01-20 22:57 ` Josh Moller-Mara
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).