emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
* Discussion request: 15m tangle time, details follow
@ 2014-06-18  0:16 Grant Rettke
  2014-06-18  2:41 ` Aaron Ecay
                   ` (3 more replies)
  0 siblings, 4 replies; 17+ messages in thread
From: Grant Rettke @ 2014-06-18  0:16 UTC (permalink / raw)
  To: emacs-orgmode@gnu.org

Good evening,

Over the past few months I've been working on the same literate
document. It has been a learning
experience for me, trial and error has abounded. The key tenet that
I've adhered too though is to truly
embrace literate programming, and the more I learn the more it makes
sense. The document has
grown quite organically and it has been and continues to be a
wonderful experience. What I need
help, feedback, discussion, and more on is the build time.

The average build takes 15m. It didn't start this way; it was about 3
minutes way back when. The last time it
got kind of big was 9m and I didn't think too much of it. After
literally a day of additions, it shot up to 15m.
I tried upgrading to the latest org release with no change. I also
removed all of the non-tangleable text with no change there, either.To
give a fair picture, I did publish the system here:

https://github.com/grettke/home

My specific request: I need help with pointers on where I should start
looking to speed things up. My goal is to have a full powered literate
programming system in org mode that is blazing fast. This is from a
user perspective, I use it every chance I get now and have barely
scratched the surface. Right now though I'm sort of hobbled by the
build time. That is actually understating it, I can't really be
productive anymore at all. Little
changes take 15m each and if I test it the "right way", 30m. Usually I
would make little changes and every
so often make sure that it can rebuild itself; usually it may :).

This build is documented in the github project; it only loads the
absolute minimum required to do the build.

Ideas: Separate the documents. Hack on org directly.

Non-ideas: Faster hardware. More ram. Newer software.

Details:

Emacs 24.3.1.
Org 8.2.6
OSX 10.9 (software updated)
Darwin orion 13.2.0 Darwin Kernel Version 13.2.0: Thu Apr 17 23:03:13
PDT 2014; root:xnu-2422.100.13~1/RELEASE_X86_64 x86_64
8core 2.x GHz, 16GB ram
Anti-virus turned off (compliance)

Please let me know any comments, questions, or concerns; looking
forward to all and every thought
and idea.

Where I may contribute is with time, effort, patience, cheerfulness,
and experience.

Kind regards,

Grant Rettke | ACM, ASA, FSF, IEEE, SIAM
gcr@wisdomandwonder.com | http://www.wisdomandwonder.com/
“Wisdom begins in wonder.” --Socrates
((λ (x) (x x)) (λ (x) (x x)))
“Life has become immeasurably better since I have been forced to stop
taking it seriously.” --Thompson

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Discussion request: 15m tangle time, details follow
  2014-06-18  0:16 Discussion request: 15m tangle time, details follow Grant Rettke
@ 2014-06-18  2:41 ` Aaron Ecay
  2014-06-18  8:13   ` Sebastien Vauban
                     ` (2 more replies)
  2014-06-18  2:54 ` Nick Dokos
                   ` (2 subsequent siblings)
  3 siblings, 3 replies; 17+ messages in thread
From: Aaron Ecay @ 2014-06-18  2:41 UTC (permalink / raw)
  To: Grant Rettke, emacs-orgmode@gnu.org

[-- Attachment #1: Type: text/plain, Size: 964 bytes --]

Hi Grant,

2014ko ekainak 17an, Grant Rettke-ek idatzi zuen:
> 
> Good evening,
> 
> Over the past few months I've been working on the same literate
> document. It has been a learning
> experience for me, trial and error has abounded. The key tenet that
> I've adhered too though is to truly
> embrace literate programming, and the more I learn the more it makes
> sense. The document has
> grown quite organically and it has been and continues to be a
> wonderful experience. What I need
> help, feedback, discussion, and more on is the build time.
> 
> The average build takes 15m. 

Here you mean time to tangle, correct?  (As opposed to exporting to
HTML/LaTeX/etc.)

I can confirm very long times to tangle a document with a structure like
yours.  I ran the emacs profiler
<https://www.gnu.org/software/emacs/manual/html_node/elisp/Profiling.html>
while tangling the document for 30 secs, then interrupted with C-g and
generated a report.  That is attached.


[-- Attachment #2: profile.txt --]
[-- Type: text/plain, Size: 11000 bytes --]

- ...                                                                                                             29140  96%
 - org-babel-expand-noweb-references                                                                              27746  92%
  - let*                                                                                                          27746  92%
   - let                                                                                                          27746  92%
    - save-current-buffer                                                                                         27746  92%
     - unwind-protect                                                                                             27746  92%
      - progn                                                                                                     27746  92%
       - while                                                                                                    27746  92%
        - funcall                                                                                                 27746  92%
         - save-current-buffer                                                                                    27746  92%
          - save-restriction                                                                                      27746  92%
           - mapconcat                                                                                            27746  92%
            - split-string                                                                                        27746  92%
             - if                                                                                                 27746  92%
              - or                                                                                                27746  92%
               - let                                                                                              27743  92%
                - save-excursion                                                                                  27743  92%
                 - if                                                                                             27743  92%
                  - let*                                                                                          27743  92%
                   - let                                                                                          27743  92%
                    - unwind-protect                                                                              27743  92%
                     - progn                                                                                      27743  92%
                      - while                                                                                     27743  92%
                       - if                                                                                       27696  92%
                        - progn                                                                                   27475  91%
                         - let                                                                                    27457  91%
                          - let                                                                                   27411  91%
                           - org-babel-get-src-block-info                                                         27218  90%
                            - org-babel-parse-src-block-match                                                     27093  90%
                             - org-babel-params-from-properties                                                   26595  88%
                              - let                                                                               26595  88%
                               - unwind-protect                                                                   26592  88%
                                - progn                                                                           26585  88%
                                 - list                                                                           26585  88%
                                  - let                                                                           26218  87%
                                   - org-babel-parse-multiple-vars                                                26218  87%
                                    - delq                                                                        26212  87%
                                     - mapcar                                                                     26205  87%
                                      - org-babel-params-from-properties-inner1                                   26184  86%
                                       - let                                                                      26178  86%
                                        - and                                                                     26175  86%
                                         - setq                                                                   26152  86%
                                          - org-entry-get                                                         26135  86%
                                           - let                                                                  26119  86%
                                            - save-excursion                                                      26095  86%
                                             - save-excursion                                                     26051  86%
                                              - if                                                                26041  86%
                                               - org-entry-get-with-inheritance                                   25257  83%
                                                - byte-code                                                       25179  83%
                                                 - org-entry-get                                                  20968  69%
                                                  - let                                                           20951  69%
                                                   - save-excursion                                               20898  69%
                                                    - save-excursion                                              20859  69%
                                                     - if                                                         20834  69%
                                                      - if                                                        20821  69%
                                                       - save-excursion                                           20699  68%
                                                        - save-restriction                                        20674  68%
                                                         - let                                                    20640  68%
                                                          - org-get-property-block                                18232  60%
                                                           - byte-code                                            15555  51%
                                                            + org-back-to-heading                                  2511   8%
                                                              outline-next-heading                                 2105   6%
                                                              org-before-first-heading-p                           1764   5%
                                                          + if                                                     2325   7%
                                                     + goto-char                                                      3   0%
                                                 + org-up-heading-safe                                             4078  13%
                                                 + org-back-to-heading                                                8   0%
                                               + and                                                                 14   0%
                                                goto-char                                                             3   0%
                                               if                                                                     6   0%
                                           cons                                                                      11   0%
                                      + mapcar                                                                       12   0%
                                      mapc                                                                            6   0%
                                  + if                                                                              198   0%
                                  + org-babel-parse-header-arguments                                                169   0%
                             + apply                                                                                188   0%
                             + byte-code                                                                            163   0%
                               generate-new-buffer                                                                   61   0%
                             + org-do-remove-indentation                                                             29   0%
                             + org-unescape-code-in-string                                                           22   0%
                              org-babel-where-is-src-block-head                                                      15   0%
                              last                                                                                    3   0%
                           + if                                                                                      73   0%
                        + org-babel-active-location-p                                                               221   0%
               + save-excursion                                                                                       3   0%
   Automatic GC                                                                                                    1394   4%
+ command-execute                                                                                                   900   2%
+ timer-event-handler                                                                                                63   0%

[-- Attachment #3: Type: text/plain, Size: 3448 bytes --]


I did two non-standard things to this profile.  The first was:

(setq profiler-report-cpu-line-format
  '((100 left)
    ;; The 100 above is increased from the default of 50
    ;; to allow the deeply nested call tree to be seen
    (24 right ((19 right)
	       (5 right)))))

The second was to convert an anonymous lambda found in
org-babel-params-from-properties into a named function, so that it would
show up in the profiling results on its own line:

(defun org-babel-params-from-properties-inner1 (header-arg)
  (let (val)
    (and (setq val (org-entry-get (point) header-arg t))
	 (cons (intern (concat ":" header-arg))
	       (org-babel-read val)))))

The profile shows that most of the slowdown is in org-entry-get.  Indeed,
org-babel-params-from-properties calls this function ~30 times per source
block.  When called with the inherit arg set to t (as here), this function
takes time (at least) proportional to the number of headings dominating
the source block, which in your document can be up to 5.

I think there are two problems here.  The first is the situation where
babel needs to fetch 30 properties per source block.  Indeed, this is
marked “deprecated” in the source, in favor of a system where there is
only one header arg.  This has been marked deprecated for almost exactly
a year in the code (Achim’s commit 90b16870 of 2013-06-23), but I don’t
know of any prominent announcement of the deprecation.  So I doubt the
old slow code could be removed without breaking many people’s setups,
although possibly a customization variable could be introduced to allow
users to opt in to the new, faster system.  You’d then have to update
your file:

  :PROPERTIES:
  :exports: none
  :tangle: no
  :END:

becomes

  :PROPERTIES:
  :header-args: :exports none :tangle no
  :END:

The new system is also a bit inferior, in that it doesn’t allow header
arg inheritance as easily.  So with the one-prop-per-arg system the
following works as expected:

  * foo
    :PROPERTIES:
    :exports: none
    :END:
  ** bar
     :PROPERTIES:
     :tangle: no
     :END:

  (src block here)

On the other hand, in the new system there’s no way to specify some
header args at foo and some at bar; the lowest header-args property
wins.  (At least as far as I can see)

The second issue is that it’s desirable to memoize calls to
org-entry-get.  Probably the easiest way to do this is to use the
org-element cache.  Indeed, a quick and hacky test that I did seemed to
confirm that this yields some speedup.  There are conceptual issues
though – org-element forces all property keys to be uppercase, whereas
org-entry-get (as near as I can tell...) follows the user’s
customization of case-fold-search to determine its case sensitivity.  So
one has to think carefully about how a rewrite to use org-element might
affect the case-sensitivity of the property API (although code relying
on the API to be sensitive to case of property keys might be rare in
practice).

TL;DR:

1. I see the same slowness you report
2. It seems like an architectural issue rather than one of
   (mis)configuration
3. There are broad fixes available, but they require potentially
   compatibility-breaking changes to Org
4. (maybe with this analysis someone can come up with a more targeted
   fix for your use case)

Hope this is helpful,

-- 
Aaron Ecay

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Discussion request: 15m tangle time, details follow
  2014-06-18  0:16 Discussion request: 15m tangle time, details follow Grant Rettke
  2014-06-18  2:41 ` Aaron Ecay
@ 2014-06-18  2:54 ` Nick Dokos
  2014-06-18  8:18 ` Thorsten Jolitz
  2014-06-18  8:20 ` Andreas Leha
  3 siblings, 0 replies; 17+ messages in thread
From: Nick Dokos @ 2014-06-18  2:54 UTC (permalink / raw)
  To: emacs-orgmode

Grant Rettke <gcr@wisdomandwonder.com> writes:

>
> This build is documented in the github project; it only loads the
> absolute minimum required to do the build.
>
> Ideas: Separate the documents. Hack on org directly.
>
> Non-ideas: Faster hardware. More ram. Newer software.
>
> Details:
>
> Emacs 24.3.1.
> Org 8.2.6
> OSX 10.9 (software updated)
> Darwin orion 13.2.0 Darwin Kernel Version 13.2.0: Thu Apr 17 23:03:13
> PDT 2014; root:xnu-2422.100.13~1/RELEASE_X86_64 x86_64
> 8core 2.x GHz, 16GB ram
> Anti-virus turned off (compliance)
>
> Please let me know any comments, questions, or concerns; looking
> forward to all and every thought
> and idea.
>
> Where I may contribute is with time, effort, patience, cheerfulness,
> and experience.
>

Have you profiled the build? Do you know where the time is spent?
If not, that's where I'd start:

   (info "(elisp) Profiling")

Nick

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Discussion request: 15m tangle time, details follow
  2014-06-18  2:41 ` Aaron Ecay
@ 2014-06-18  8:13   ` Sebastien Vauban
  2014-06-18 17:47     ` Aaron Ecay
  2014-06-18 20:59   ` Eric Schulte
  2014-06-19  0:56   ` Grant Rettke
  2 siblings, 1 reply; 17+ messages in thread
From: Sebastien Vauban @ 2014-06-18  8:13 UTC (permalink / raw)
  To: emacs-orgmode-mXXj517/zsQ

Hi Aaron,

Aaron Ecay wrote:
> [...]
> babel needs to fetch 30 properties per source block.  Indeed, this is
> marked “deprecated” in the source, in favor of a system where there is
> only one header arg.  This has been marked deprecated for almost exactly
> a year in the code (Achim’s commit 90b16870 of 2013-06-23), but I don’t
> know of any prominent announcement of the deprecation.

I neither was aware of such a deprecation.

Are you talking of the comment in function
`org-babel-params-from-properties' (in ob-core.el)?

Thought, I can't parse it yet the way you do -- without understanding
much more of that code, as the comments differ in "at point of
definition" vs "at point of call":

     ;; DEPRECATED header arguments specified as separate property at
     ;; point of definition

     ;; header arguments specified with the header-args property at
     ;; point of call

What you're talking about is for specifying header arguments in
a subtree, anyway always at the same point:

> [...] You’d then have to update your file:
>
>   :PROPERTIES:
>   :exports: none
>   :tangle: no
>   :END:
>
> becomes
>
>   :PROPERTIES:
>   :header-args: :exports none :tangle no
>   :END:
>
> The new system is also a bit inferior, in that it doesn’t allow header
> arg inheritance as easily.  So with the one-prop-per-arg system the
> following works as expected:
>
>   * foo
>     :PROPERTIES:
>     :exports: none
>     :END:
>   ** bar
>      :PROPERTIES:
>      :tangle: no
>      :END:
>
>   (src block here)
>
> On the other hand, in the new system there’s no way to specify some
> header args at foo and some at bar; the lowest header-args property
> wins.  (At least as far as I can see)

Maybe the "+" mechanism for concatenating property strings would help
here?

Best regards,
  Seb

-- 
Sebastien Vauban

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Discussion request: 15m tangle time, details follow
  2014-06-18  0:16 Discussion request: 15m tangle time, details follow Grant Rettke
  2014-06-18  2:41 ` Aaron Ecay
  2014-06-18  2:54 ` Nick Dokos
@ 2014-06-18  8:18 ` Thorsten Jolitz
  2014-06-18  9:34   ` Thorsten Jolitz
  2014-06-19  1:00   ` Grant Rettke
  2014-06-18  8:20 ` Andreas Leha
  3 siblings, 2 replies; 17+ messages in thread
From: Thorsten Jolitz @ 2014-06-18  8:18 UTC (permalink / raw)
  To: emacs-orgmode

Grant Rettke <gcr@wisdomandwonder.com> writes:

> The average build takes 15m. 

Which file do you mean - TC3F.org?
Thats some 4400 lines, thus not _that_ big really. I once used a giant
init.el copied from the web that had some 8000 lines (now I'm back to
2500 again ...). 

I converted that file to outshine, since its mainly one programming
language (emacs-lisp in this case):

[with-current-buffer "TC3F.org]
,-------------------------------------------------
| (benchmark-run (outorg-convert-org-to-outshine))
| (1.7926752110000002 8 1.042145478000009)
`-------------------------------------------------

After this is done *once*, you can always switch between emacs-lisp and
org-mode with outorg, It takes 0.4 sec to convert the whole file to org
again

,------------------------------------------
| (benchmark-run (outorg-edit-as-org '(4)))
| (0.365756325 1 0.13800897400000167)
`------------------------------------------

and 1.6 sec to convert it back to outshine (I have to fix this speed
difference ;)

,---------------------------------------------
| (benchmark-run (outorg-copy-edits-and-exit))
| (1.616835235 8 1.106696710999998)
`---------------------------------------------

But normally you do not convert the whole buffer to Org with outorg,
just the subtree at hand, and thats instantly.

Then productivity means that your init file *is* in a programming
language mode (TC3F.el) and you can modify and eval your code
on-the-fly. Whenever you need to edit the comment text, you do

M-# M-#
,-----------------------
| M-x outorg-edit-as-org
`-----------------------

and when you are done, M-#

,-------------------------------
| M-x outorg-copy-edits-and-exit
`-------------------------------

Its just the reverse of Org-mode with souce-blocks, and in cases like an
Emacs init-file, when its mostly one programming language and the
source-code is more important than the text (and frequently modified),
this reverse approach might be more productive. 

PS 

I just figured that I ran the benchmarks on an outorg testing branch,
which is faster than master but not yet ready. So things might be a bit
slower with master branch, but in terms of seconds (maybe 2 or 3 sec to
convert the whole file?)

-- 
cheers,
Thorsten

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Discussion request: 15m tangle time, details follow
  2014-06-18  0:16 Discussion request: 15m tangle time, details follow Grant Rettke
                   ` (2 preceding siblings ...)
  2014-06-18  8:18 ` Thorsten Jolitz
@ 2014-06-18  8:20 ` Andreas Leha
  2014-06-19  0:58   ` Grant Rettke
  3 siblings, 1 reply; 17+ messages in thread
From: Andreas Leha @ 2014-06-18  8:20 UTC (permalink / raw)
  To: emacs-orgmode

Hi Grant,

Grant Rettke <gcr@wisdomandwonder.com> writes:

> Good evening,
>
> Over the past few months I've been working on the same literate
> document. It has been a learning
> experience for me, trial and error has abounded. The key tenet that
> I've adhered too though is to truly
> embrace literate programming, and the more I learn the more it makes
> sense. The document has
> grown quite organically and it has been and continues to be a
> wonderful experience. What I need
> help, feedback, discussion, and more on is the build time.
>
> The average build takes 15m. It didn't start this way; it was about 3
> minutes way back when. The last time it
> got kind of big was 9m and I didn't think too much of it. After
> literally a day of additions, it shot up to 15m.
> I tried upgrading to the latest org release with no change. I also
> removed all of the non-tangleable text with no change there, either.To
> give a fair picture, I did publish the system here:
>
> https://github.com/grettke/home
>
> My specific request: I need help with pointers on where I should start
> looking to speed things up. My goal is to have a full powered literate
> programming system in org mode that is blazing fast. This is from a
> user perspective, I use it every chance I get now and have barely
> scratched the surface. Right now though I'm sort of hobbled by the
> build time. That is actually understating it, I can't really be
> productive anymore at all. Little
> changes take 15m each and if I test it the "right way", 30m. Usually I
> would make little changes and every
> so often make sure that it can rebuild itself; usually it may :).
>
> This build is documented in the github project; it only loads the
> absolute minimum required to do the build.
>
> Ideas: Separate the documents. Hack on org directly.
>
> Non-ideas: Faster hardware. More ram. Newer software.
>
> Details:
>
> Emacs 24.3.1.
> Org 8.2.6
> OSX 10.9 (software updated)
> Darwin orion 13.2.0 Darwin Kernel Version 13.2.0: Thu Apr 17 23:03:13
> PDT 2014; root:xnu-2422.100.13~1/RELEASE_X86_64 x86_64
> 8core 2.x GHz, 16GB ram
> Anti-virus turned off (compliance)
>
> Please let me know any comments, questions, or concerns; looking
> forward to all and every thought
> and idea.

I do not have the time to look into this, but fortunately, other
have done so already ;-)

Just one quick idea:  Have you tried [fn:1]
(setq org-babel-use-quick-and-dirty-noweb-expansion t)

This can lead to dramatic speedups in my experience.

Best,

Andreas

>
> Where I may contribute is with time, effort, patience, cheerfulness,
> and experience.
>
> Kind regards,
>
> Grant Rettke | ACM, ASA, FSF, IEEE, SIAM
> gcr@wisdomandwonder.com | http://www.wisdomandwonder.com/
> “Wisdom begins in wonder.” --Socrates
> ((λ (x) (x x)) (λ (x) (x x)))
> “Life has become immeasurably better since I have been forced to stop
> taking it seriously.” --Thompson

Footnotes:

[fn:1] 
(see article.gmane.org/gmane.emacs.orgmode/50625/match=problem+noweb+ref+property)

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Discussion request: 15m tangle time, details follow
  2014-06-18  8:18 ` Thorsten Jolitz
@ 2014-06-18  9:34   ` Thorsten Jolitz
  2014-06-19  1:00   ` Grant Rettke
  1 sibling, 0 replies; 17+ messages in thread
From: Thorsten Jolitz @ 2014-06-18  9:34 UTC (permalink / raw)
  To: emacs-orgmode

Thorsten Jolitz <tjolitz@gmail.com> writes:

> Grant Rettke <gcr@wisdomandwonder.com> writes:
>
>> The average build takes 15m. 

> [with-current-buffer "TC3F.org]

> After this is done *once*, you can always switch between emacs-lisp and
> org-mode with outorg, It takes 0.4 sec to convert the whole file to org
> again
>
> ,------------------------------------------
> | (benchmark-run (outorg-edit-as-org '(4)))
> | (0.365756325 1 0.13800897400000167)
> `------------------------------------------
>
> and 1.6 sec to convert it back to outshine (I have to fix this speed
> difference ;)
>
> ,---------------------------------------------
> | (benchmark-run (outorg-copy-edits-and-exit))
> | (1.616835235 8 1.106696710999998)
> `---------------------------------------------

This thread inspired me to profile these two commands - they do roughly
the same thing, only in the opposite direction, and I found it strange
why converting from Source to Org should be 4x faster than converting
from Org to Source. 

It turned out that 

,------------------------------------------------------------
| kill-whole-line is an interactive compiled Lisp function in
| `simple.el'.
`------------------------------------------------------------

was the sole culprit (used for killing the source-block
ddelimiters). Its a lisp function and does too many things besides
killing the line. Replacing it with C function

,------------------------------------------------------------------
| delete-region is an interactive built-in function in `editfns.c'.
`------------------------------------------------------------------

improves performance to (on whole file TC3):

,---------------------------------------------
| (benchmark-run (outorg-copy-edits-and-exit))
| (0.66580701 2 0.28894333600000266)
`---------------------------------------------

The remaining speed difference is partly because I undo the indendation 
introduced in the Org source-blocks before the conversion back to
source, and thus need to process the file twice. 

-- 
cheers,
Thorsten

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Discussion request: 15m tangle time, details follow
  2014-06-18  8:13   ` Sebastien Vauban
@ 2014-06-18 17:47     ` Aaron Ecay
  0 siblings, 0 replies; 17+ messages in thread
From: Aaron Ecay @ 2014-06-18 17:47 UTC (permalink / raw)
  To: Sebastien Vauban, emacs-orgmode

Hi Sebastien,

2014ko ekainak 18an, Sebastien Vauban-ek idatzi zuen:
>
> Hi Aaron,
>
> Aaron Ecay wrote:
>> [...]
>> babel needs to fetch 30 properties per source block.  Indeed, this is
>> marked “deprecated” in the source, in favor of a system where there is
>> only one header arg.  This has been marked deprecated for almost exactly
>> a year in the code (Achim’s commit 90b16870 of 2013-06-23), but I don’t
>> know of any prominent announcement of the deprecation.
>
> I neither was aware of such a deprecation.
>
> Are you talking of the comment in function
> `org-babel-params-from-properties' (in ob-core.el)?
>
> Thought, I can't parse it yet the way you do -- without understanding
> much more of that code, as the comments differ in "at point of
> definition" vs "at point of call":
>
>      ;; DEPRECATED header arguments specified as separate property at
>      ;; point of definition
>
>      ;; header arguments specified with the header-args property at
>      ;; point of call

That also differs between the two methods.

,----
|
| * foo
|   :PROPERTIES:...
|
| #+name: xyz
| #+begin_src
|   ...
| #+end_src
|
| * bar
|   :PROPERTIES:...
|
| #+call: xyz()
`----

The current code looks for individual header arg properties at foo, but
for the property header-args at bar (for the #+call line)

>
> What you're talking about is for specifying header arguments in
> a subtree, anyway always at the same point:
>
>> [...] You’d then have to update your file:
>>
>> :PROPERTIES:
>> :exports: none
>> :tangle: no
>> :END:
>>
>> becomes
>>
>> :PROPERTIES:
>> :header-args: :exports none :tangle no
>> :END:
>>
>> The new system is also a bit inferior, in that it doesn’t allow header
>> arg inheritance as easily.  So with the one-prop-per-arg system the
>> following works as expected:
>>
>> * foo
>> :PROPERTIES:
>> :exports: none
>> :END:
>> ** bar
>> :PROPERTIES:
>> :tangle: no
>> :END:
>>
>> (src block here)
>>
>> On the other hand, in the new system there’s no way to specify some
>> header args at foo and some at bar; the lowest header-args property
>> wins.  (At least as far as I can see)
>
> Maybe the "+" mechanism for concatenating property strings would help
> here?

No, org-entry-get-with-inheritance will not continue climbing the tree
once it finds one instance of the property in question, as demonstrated
here:

,----
| * foo
|   :PROPERTIES:
|   :quux+:    abc
|   :END:
| ** bar
|    :PROPERTIES:
|    :quux+:    foo
|    :quux+:    bar
|    :END:
|
| #+BEGIN_SRC emacs-lisp
|   (org-entry-get nil "quux" t)
| #+END_SRC
|
| #+RESULTS:
| : foo bar
`----


--
Aaron Ecay

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Discussion request: 15m tangle time, details follow
  2014-06-18  2:41 ` Aaron Ecay
  2014-06-18  8:13   ` Sebastien Vauban
@ 2014-06-18 20:59   ` Eric Schulte
  2014-06-19  1:05     ` Grant Rettke
  2014-06-19  0:56   ` Grant Rettke
  2 siblings, 1 reply; 17+ messages in thread
From: Eric Schulte @ 2014-06-18 20:59 UTC (permalink / raw)
  To: Grant Rettke; +Cc: emacs-orgmode@gnu.org

Aaron Ecay <aaronecay@gmail.com> writes:

> Hi Grant,
>
> 2014ko ekainak 17an, Grant Rettke-ek idatzi zuen:
>> 
>> Good evening,
>> 
>> Over the past few months I've been working on the same literate
>> document. It has been a learning
>> experience for me, trial and error has abounded. The key tenet that
>> I've adhered too though is to truly
>> embrace literate programming, and the more I learn the more it makes
>> sense. The document has
>> grown quite organically and it has been and continues to be a
>> wonderful experience. What I need
>> help, feedback, discussion, and more on is the build time.
>> 
>> The average build takes 15m. 
>
> Here you mean time to tangle, correct?  (As opposed to exporting to
> HTML/LaTeX/etc.)
>
> I can confirm very long times to tangle a document with a structure like
> yours.  I ran the emacs profiler
> <https://www.gnu.org/software/emacs/manual/html_node/elisp/Profiling.html>
> while tangling the document for 30 secs, then interrupted with C-g and
> generated a report.  That is attached.
>
>
>
> I did two non-standard things to this profile.  The first was:
>
> (setq profiler-report-cpu-line-format
>   '((100 left)
>     ;; The 100 above is increased from the default of 50
>     ;; to allow the deeply nested call tree to be seen
>     (24 right ((19 right)
> 	       (5 right)))))
>
> The second was to convert an anonymous lambda found in
> org-babel-params-from-properties into a named function, so that it would
> show up in the profiling results on its own line:
>
> (defun org-babel-params-from-properties-inner1 (header-arg)
>   (let (val)
>     (and (setq val (org-entry-get (point) header-arg t))
> 	 (cons (intern (concat ":" header-arg))
> 	       (org-babel-read val)))))
>
> The profile shows that most of the slowdown is in org-entry-get.  Indeed,
> org-babel-params-from-properties calls this function ~30 times per source
> block.  When called with the inherit arg set to t (as here), this function
> takes time (at least) proportional to the number of headings dominating
> the source block, which in your document can be up to 5.
>

Thanks for taking the time to profile this.  It's nice to have more
evidence that the use of properties is definitely the culprit here.

>
> I think there are two problems here.  The first is the situation where
> babel needs to fetch 30 properties per source block.  Indeed, this is
> marked “deprecated” in the source, in favor of a system where there is
> only one header arg.  This has been marked deprecated for almost exactly
> a year in the code (Achim’s commit 90b16870 of 2013-06-23), but I don’t
> know of any prominent announcement of the deprecation.  So I doubt the
> old slow code could be removed without breaking many people’s setups,
> although possibly a customization variable could be introduced to allow
> users to opt in to the new, faster system.  You’d then have to update
> your file:
>
>   :PROPERTIES:
>   :exports: none
>   :tangle: no
>   :END:
>
> becomes
>
>   :PROPERTIES:
>   :header-args: :exports none :tangle no
>   :END:
>
> The new system is also a bit inferior, in that it doesn’t allow header
> arg inheritance as easily.  So with the one-prop-per-arg system the
> following works as expected:
>
>   * foo
>     :PROPERTIES:
>     :exports: none
>     :END:
>   ** bar
>      :PROPERTIES:
>      :tangle: no
>      :END:
>
>   (src block here)
>
> On the other hand, in the new system there’s no way to specify some
> header args at foo and some at bar; the lowest header-args property
> wins.  (At least as far as I can see)
>

As I recall this inheritance issue is the wall that we ran up against.
The deprecation comment in the code was premature.

>
> The second issue is that it’s desirable to memoize calls to
> org-entry-get.  Probably the easiest way to do this is to use the
> org-element cache.  Indeed, a quick and hacky test that I did seemed to
> confirm that this yields some speedup.  There are conceptual issues
> though – org-element forces all property keys to be uppercase, whereas
> org-entry-get (as near as I can tell...) follows the user’s
> customization of case-fold-search to determine its case sensitivity.  So
> one has to think carefully about how a rewrite to use org-element might
> affect the case-sensitivity of the property API (although code relying
> on the API to be sensitive to case of property keys might be rare in
> practice).
>

Thanks, it does sound like org-element cache could be useful here, I
don't believe this existed last time we wrestled with this performance
issue.

The only other options I can think of are;

- introduce a customization variable to eliminate or limit the use of
  property lookup for code blocks to either perform none or to only
  search for a limited set of properties

- possibly extend org-element-get (or provide an alternative) which
  takes multiple keys (which may be more efficient depending on the
  implementation)

>
> TL;DR:
>
> 1. I see the same slowness you report
> 2. It seems like an architectural issue rather than one of
>    (mis)configuration
> 3. There are broad fixes available, but they require potentially
>    compatibility-breaking changes to Org
> 4. (maybe with this analysis someone can come up with a more targeted
>    fix for your use case)
>
> Hope this is helpful,

Very helpful, thanks for providing both empirical data and useful
analysis.

Best,
Eric

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D (see https://u.fsf.org/yw)

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Discussion request: 15m tangle time, details follow
  2014-06-18  2:41 ` Aaron Ecay
  2014-06-18  8:13   ` Sebastien Vauban
  2014-06-18 20:59   ` Eric Schulte
@ 2014-06-19  0:56   ` Grant Rettke
  2 siblings, 0 replies; 17+ messages in thread
From: Grant Rettke @ 2014-06-19  0:56 UTC (permalink / raw)
  To: Grant Rettke, emacs-orgmode@gnu.org

On Tue, Jun 17, 2014 at 9:41 PM, Aaron Ecay <aaronecay@gmail.com> wrote:
> Here you mean time to tangle, correct?

Yes, thank you for digging in.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Discussion request: 15m tangle time, details follow
  2014-06-18  8:20 ` Andreas Leha
@ 2014-06-19  0:58   ` Grant Rettke
  0 siblings, 0 replies; 17+ messages in thread
From: Grant Rettke @ 2014-06-19  0:58 UTC (permalink / raw)
  To: Andreas Leha; +Cc: emacs-orgmode@gnu.org

On Wed, Jun 18, 2014 at 3:20 AM, Andreas Leha
<andreas.leha@med.uni-goettingen.de> wrote:
> Just one quick idea:  Have you tried [fn:1]
> (setq org-babel-use-quick-and-dirty-noweb-expansion t)
>
> This can lead to dramatic speedups in my experience.

I have not because I am utilizing property inheritance; it makes it
really, really pleasant to work with the document.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Discussion request: 15m tangle time, details follow
  2014-06-18  8:18 ` Thorsten Jolitz
  2014-06-18  9:34   ` Thorsten Jolitz
@ 2014-06-19  1:00   ` Grant Rettke
  1 sibling, 0 replies; 17+ messages in thread
From: Grant Rettke @ 2014-06-19  1:00 UTC (permalink / raw)
  To: Thorsten Jolitz; +Cc: emacs-orgmode@gnu.org

On Wed, Jun 18, 2014 at 3:18 AM, Thorsten Jolitz <tjolitz@gmail.com> wrote:
> Grant Rettke <gcr@wisdomandwonder.com> writes:
>
>> The average build takes 15m.
>
> Which file do you mean - TC3F.org?
> Thats some 4400 lines, thus not _that_ big really. I once used a giant
> init.el copied from the web that had some 8000 lines (now I'm back to
> 2500 again ...).

Yes. Thank you for digging in. Thanks for the tip, I had not learned
out-org yet.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Discussion request: 15m tangle time, details follow
  2014-06-18 20:59   ` Eric Schulte
@ 2014-06-19  1:05     ` Grant Rettke
  2014-06-19 13:44       ` Eric Schulte
  0 siblings, 1 reply; 17+ messages in thread
From: Grant Rettke @ 2014-06-19  1:05 UTC (permalink / raw)
  To: Eric Schulte; +Cc: emacs-orgmode@gnu.org

I still want to be using org,, so my plan for now then is to write a
pre-processing script for before org-babel-tangle is run that:

1. Checks each headline
2. Checks if there is a source block
3. If that source block doesn't have a noweb-ref name, and there is
another source block under that
    headline, then "merge" it with the next source block of the same
nature in that heading.

That is off the top of my head, and I will try it by hand, first and
see if it tangles the same result.

Thoughts?

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Discussion request: 15m tangle time, details follow
  2014-06-19  1:05     ` Grant Rettke
@ 2014-06-19 13:44       ` Eric Schulte
  2014-06-20  0:01         ` Grant Rettke
  0 siblings, 1 reply; 17+ messages in thread
From: Eric Schulte @ 2014-06-19 13:44 UTC (permalink / raw)
  To: Grant Rettke; +Cc: emacs-orgmode@gnu.org

Grant Rettke <gcr@wisdomandwonder.com> writes:

> I still want to be using org,, so my plan for now then is to write a
> pre-processing script for before org-babel-tangle is run that:
>
> 1. Checks each headline
> 2. Checks if there is a source block
> 3. If that source block doesn't have a noweb-ref name, and there is
> another source block under that
>     headline, then "merge" it with the next source block of the same
> nature in that heading.
>
> That is off the top of my head, and I will try it by hand, first and
> see if it tangles the same result.
>
> Thoughts?

Perhaps an easier workaround would be to customize the value of
`org-babel-common-header-args-w-values'.  This variable determines which
header arguments are checked for in properties.  If you remove all
header arguments from this variable which you know are not used in a
property, it may dramatically speed up tangle time.

This may reduce the utility of some convenience functions, but should
not have too large of an impact on overall functionality.

Best,
Eric

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D (see https://u.fsf.org/yw)

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Discussion request: 15m tangle time, details follow
  2014-06-19 13:44       ` Eric Schulte
@ 2014-06-20  0:01         ` Grant Rettke
  2014-06-20  1:26           ` Grant Rettke
  0 siblings, 1 reply; 17+ messages in thread
From: Grant Rettke @ 2014-06-20  0:01 UTC (permalink / raw)
  To: Eric Schulte; +Cc: emacs-orgmode@gnu.org

Understood. I will start testing.

Just out of curiosity, knowing full well that it will break the
tangle, and also since I hadn't tried
the quick and dirty flag, I ran it with this to see how long it took:

(setq org-babel-common-header-args-w-values nil)

It went from 936 seconds (15 minutes)

Down to 279 (4 minutes).

Now I am wondering if it is worth converting my document *not* to use
property inheritance...

What would you folks do?
Grant Rettke | ACM, ASA, FSF, IEEE, SIAM
gcr@wisdomandwonder.com | http://www.wisdomandwonder.com/
“Wisdom begins in wonder.” --Socrates
((λ (x) (x x)) (λ (x) (x x)))
“Life has become immeasurably better since I have been forced to stop
taking it seriously.” --Thompson


On Thu, Jun 19, 2014 at 8:44 AM, Eric Schulte <schulte.eric@gmail.com> wrote:
> Grant Rettke <gcr@wisdomandwonder.com> writes:
>
>> I still want to be using org,, so my plan for now then is to write a
>> pre-processing script for before org-babel-tangle is run that:
>>
>> 1. Checks each headline
>> 2. Checks if there is a source block
>> 3. If that source block doesn't have a noweb-ref name, and there is
>> another source block under that
>>     headline, then "merge" it with the next source block of the same
>> nature in that heading.
>>
>> That is off the top of my head, and I will try it by hand, first and
>> see if it tangles the same result.
>>
>> Thoughts?
>
> Perhaps an easier workaround would be to customize the value of
> `org-babel-common-header-args-w-values'.  This variable determines which
> header arguments are checked for in properties.  If you remove all
> header arguments from this variable which you know are not used in a
> property, it may dramatically speed up tangle time.
>
> This may reduce the utility of some convenience functions, but should
> not have too large of an impact on overall functionality.
>
> Best,
> Eric
>
> --
> Eric Schulte
> https://cs.unm.edu/~eschulte
> PGP: 0x614CA05D (see https://u.fsf.org/yw)

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Discussion request: 15m tangle time, details follow
  2014-06-20  0:01         ` Grant Rettke
@ 2014-06-20  1:26           ` Grant Rettke
  2014-06-20 14:50             ` Grant Rettke
  0 siblings, 1 reply; 17+ messages in thread
From: Grant Rettke @ 2014-06-20  1:26 UTC (permalink / raw)
  To: Eric Schulte; +Cc: emacs-orgmode@gnu.org

Just excluded what seemed excludable and got build time down to 8
minutes from 15 minutes.

Kept the following as it seems that it would allow the most important things:

    (session    . :any)
    (noweb-ref  . :any)
    (noweb      . ((yes no tangle no-export strip-export)))
    (file       . :any)
    (tangle     . ((tangle yes no :any)))
    (results    . ((file list vector table scalar verbatim)
                   (raw html latex org code pp drawer)
                   (replace silent none append prepend)
                   (output value)))

Here is what I excluded:

(setq org-babel-common-header-args-w-values
      (assq-delete-all 'cache org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'cmdline org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'colnames org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'comments org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'dir org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'eval org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'exports org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'epilogue org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'file-desc org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'hlines org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'mkdirp org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'no-expand org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'noeval org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'noweb-sep org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'padline org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'post org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'prologue org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'rownames org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'sep org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'shebang org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'tangle-mode org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'var org-babel-common-header-args-w-values))
(setq org-babel-common-header-args-w-values
      (assq-delete-all 'wrap org-babel-common-header-args-w-values))
Grant Rettke | ACM, ASA, FSF, IEEE, SIAM
gcr@wisdomandwonder.com | http://www.wisdomandwonder.com/
“Wisdom begins in wonder.” --Socrates
((λ (x) (x x)) (λ (x) (x x)))
“Life has become immeasurably better since I have been forced to stop
taking it seriously.” --Thompson


On Thu, Jun 19, 2014 at 7:01 PM, Grant Rettke <gcr@wisdomandwonder.com> wrote:
> Understood. I will start testing.
>
> Just out of curiosity, knowing full well that it will break the
> tangle, and also since I hadn't tried
> the quick and dirty flag, I ran it with this to see how long it took:
>
> (setq org-babel-common-header-args-w-values nil)
>
> It went from 936 seconds (15 minutes)
>
> Down to 279 (4 minutes).
>
> Now I am wondering if it is worth converting my document *not* to use
> property inheritance...
>
> What would you folks do?
> Grant Rettke | ACM, ASA, FSF, IEEE, SIAM
> gcr@wisdomandwonder.com | http://www.wisdomandwonder.com/
> “Wisdom begins in wonder.” --Socrates
> ((λ (x) (x x)) (λ (x) (x x)))
> “Life has become immeasurably better since I have been forced to stop
> taking it seriously.” --Thompson
>
>
> On Thu, Jun 19, 2014 at 8:44 AM, Eric Schulte <schulte.eric@gmail.com> wrote:
>> Grant Rettke <gcr@wisdomandwonder.com> writes:
>>
>>> I still want to be using org,, so my plan for now then is to write a
>>> pre-processing script for before org-babel-tangle is run that:
>>>
>>> 1. Checks each headline
>>> 2. Checks if there is a source block
>>> 3. If that source block doesn't have a noweb-ref name, and there is
>>> another source block under that
>>>     headline, then "merge" it with the next source block of the same
>>> nature in that heading.
>>>
>>> That is off the top of my head, and I will try it by hand, first and
>>> see if it tangles the same result.
>>>
>>> Thoughts?
>>
>> Perhaps an easier workaround would be to customize the value of
>> `org-babel-common-header-args-w-values'.  This variable determines which
>> header arguments are checked for in properties.  If you remove all
>> header arguments from this variable which you know are not used in a
>> property, it may dramatically speed up tangle time.
>>
>> This may reduce the utility of some convenience functions, but should
>> not have too large of an impact on overall functionality.
>>
>> Best,
>> Eric
>>
>> --
>> Eric Schulte
>> https://cs.unm.edu/~eschulte
>> PGP: 0x614CA05D (see https://u.fsf.org/yw)

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: Discussion request: 15m tangle time, details follow
  2014-06-20  1:26           ` Grant Rettke
@ 2014-06-20 14:50             ` Grant Rettke
  0 siblings, 0 replies; 17+ messages in thread
From: Grant Rettke @ 2014-06-20 14:50 UTC (permalink / raw)
  To: Eric Schulte; +Cc: emacs-orgmode@gnu.org

This is a nicer way, makes it obvious what *is* being used; uses dash.el:

(let* ((allowed '(exports
                  file
                  noweb
                  noweb-ref
                  session
                  tangle))
       (new-ls
        (--filter (member (car it) allowed)
                  org-babel-common-header-args-w-values)))
  (setq org-babel-common-header-args-w-values new-ls))
Grant Rettke | ACM, ASA, FSF, IEEE, SIAM
gcr@wisdomandwonder.com | http://www.wisdomandwonder.com/
“Wisdom begins in wonder.” --Socrates
((λ (x) (x x)) (λ (x) (x x)))
“Life has become immeasurably better since I have been forced to stop
taking it seriously.” --Thompson


On Thu, Jun 19, 2014 at 8:26 PM, Grant Rettke <gcr@wisdomandwonder.com> wrote:
> Just excluded what seemed excludable and got build time down to 8
> minutes from 15 minutes.
>
> Kept the following as it seems that it would allow the most important things:
>
>     (session    . :any)
>     (noweb-ref  . :any)
>     (noweb      . ((yes no tangle no-export strip-export)))
>     (file       . :any)
>     (tangle     . ((tangle yes no :any)))
>     (results    . ((file list vector table scalar verbatim)
>                    (raw html latex org code pp drawer)
>                    (replace silent none append prepend)
>                    (output value)))
>
> Here is what I excluded:
>
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'cache org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'cmdline org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'colnames org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'comments org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'dir org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'eval org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'exports org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'epilogue org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'file-desc org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'hlines org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'mkdirp org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'no-expand org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'noeval org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'noweb-sep org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'padline org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'post org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'prologue org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'rownames org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'sep org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'shebang org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'tangle-mode org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'var org-babel-common-header-args-w-values))
> (setq org-babel-common-header-args-w-values
>       (assq-delete-all 'wrap org-babel-common-header-args-w-values))
> Grant Rettke | ACM, ASA, FSF, IEEE, SIAM
> gcr@wisdomandwonder.com | http://www.wisdomandwonder.com/
> “Wisdom begins in wonder.” --Socrates
> ((λ (x) (x x)) (λ (x) (x x)))
> “Life has become immeasurably better since I have been forced to stop
> taking it seriously.” --Thompson
>
>
> On Thu, Jun 19, 2014 at 7:01 PM, Grant Rettke <gcr@wisdomandwonder.com> wrote:
>> Understood. I will start testing.
>>
>> Just out of curiosity, knowing full well that it will break the
>> tangle, and also since I hadn't tried
>> the quick and dirty flag, I ran it with this to see how long it took:
>>
>> (setq org-babel-common-header-args-w-values nil)
>>
>> It went from 936 seconds (15 minutes)
>>
>> Down to 279 (4 minutes).
>>
>> Now I am wondering if it is worth converting my document *not* to use
>> property inheritance...
>>
>> What would you folks do?
>> Grant Rettke | ACM, ASA, FSF, IEEE, SIAM
>> gcr@wisdomandwonder.com | http://www.wisdomandwonder.com/
>> “Wisdom begins in wonder.” --Socrates
>> ((λ (x) (x x)) (λ (x) (x x)))
>> “Life has become immeasurably better since I have been forced to stop
>> taking it seriously.” --Thompson
>>
>>
>> On Thu, Jun 19, 2014 at 8:44 AM, Eric Schulte <schulte.eric@gmail.com> wrote:
>>> Grant Rettke <gcr@wisdomandwonder.com> writes:
>>>
>>>> I still want to be using org,, so my plan for now then is to write a
>>>> pre-processing script for before org-babel-tangle is run that:
>>>>
>>>> 1. Checks each headline
>>>> 2. Checks if there is a source block
>>>> 3. If that source block doesn't have a noweb-ref name, and there is
>>>> another source block under that
>>>>     headline, then "merge" it with the next source block of the same
>>>> nature in that heading.
>>>>
>>>> That is off the top of my head, and I will try it by hand, first and
>>>> see if it tangles the same result.
>>>>
>>>> Thoughts?
>>>
>>> Perhaps an easier workaround would be to customize the value of
>>> `org-babel-common-header-args-w-values'.  This variable determines which
>>> header arguments are checked for in properties.  If you remove all
>>> header arguments from this variable which you know are not used in a
>>> property, it may dramatically speed up tangle time.
>>>
>>> This may reduce the utility of some convenience functions, but should
>>> not have too large of an impact on overall functionality.
>>>
>>> Best,
>>> Eric
>>>
>>> --
>>> Eric Schulte
>>> https://cs.unm.edu/~eschulte
>>> PGP: 0x614CA05D (see https://u.fsf.org/yw)

^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2014-06-20 14:50 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-06-18  0:16 Discussion request: 15m tangle time, details follow Grant Rettke
2014-06-18  2:41 ` Aaron Ecay
2014-06-18  8:13   ` Sebastien Vauban
2014-06-18 17:47     ` Aaron Ecay
2014-06-18 20:59   ` Eric Schulte
2014-06-19  1:05     ` Grant Rettke
2014-06-19 13:44       ` Eric Schulte
2014-06-20  0:01         ` Grant Rettke
2014-06-20  1:26           ` Grant Rettke
2014-06-20 14:50             ` Grant Rettke
2014-06-19  0:56   ` Grant Rettke
2014-06-18  2:54 ` Nick Dokos
2014-06-18  8:18 ` Thorsten Jolitz
2014-06-18  9:34   ` Thorsten Jolitz
2014-06-19  1:00   ` Grant Rettke
2014-06-18  8:20 ` Andreas Leha
2014-06-19  0:58   ` Grant Rettke

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).