Re: Exporting large documents

[Carsten Dominik <carsten.dominik@gmail.com>]

Hi Achim,

this is an interesting experiment, thank you!

I think it would also be interesting to use elp to see which
function are taking up the non-linear time.

- Carsten

On 27.4.2013, at 21:28, Achim Gratz <Stromeko@nexgo.de> wrote:

> I've been looking at export runtimes for large documents with the new
> exporter.  The example I've used is the orgmanual.org from Tom.  I first
> exported each subtree standalone, then the document as a whole to
> texinfo.  The startup of Emacs takes about 1 s of user time and 1.5 s of
> wall time, these have not been subtracted in the table below.  The table
> shows the individual runtimes for each subtree export, their total and
> the last line is for the export of the full document.
> 
> |    user |   sys |   wall |  util |
> |---------+-------+--------+-------|
> |   4.856 | 0.048 |   5.52 | 88.5% |
> |  13.748 | 0.160 |  15.04 | 92.4% |
> |  15.004 | 0.036 |  16.06 | 93.5% |
> |   8.464 | 0.068 |  10.37 | 82.1% |
> |   8.420 | 0.088 |  13.29 | 63.9% |
> |   5.568 | 0.052 |   8.03 | 69.8% |
> |   7.648 | 0.064 |   9.26 | 83.1% |
> |  12.020 | 0.056 |  14.16 | 85.2% |
> |   7.796 | 0.044 |  11.00 | 71.1% |
> |  27.352 | 0.068 |  33.71 | 81.3% |
> |   6.564 | 0.044 |   7.00 | 94.2% |
> |  17.124 | 0.108 |  19.17 | 89.8% |
> |   6.124 | 0.068 |   6.79 | 91.0% |
> |  10.632 | 0.068 |  11.73 | 91.1% |
> |  15.932 | 0.052 |  17.33 | 92.2% |
> |   6.836 | 0.080 |   7.61 | 90.8% |
> |   3.964 | 0.040 |   4.54 | 88.1% |
> |   5.076 | 0.160 |   6.01 | 87.0% |
> |   3.488 | 0.060 |   4.06 | 87.1% |
> |   3.532 | 0.056 |   4.14 | 86.4% |
> |   3.516 | 0.044 |   4.20 | 84.5% |
> |   3.576 | 0.064 |   4.17 | 87.0% |
> |   3.552 | 0.064 |   4.12 | 87.6% |
> |   6.528 | 0.176 |  10.73 | 62.3% |
> |---------+-------+--------+-------|
> | 207.320 | 1.768 | 248.04 | 84.3% |
> | 386.384 | 0.392 | 415.94 | 92.9% |
> 
> As you can see, the export gets slower (a lot) the larger the scope of
> the export gets.  I would hope that something can be done about it, I've
> earlier tried to profile the export (posted over in the Orgmanual
> thread), but I don't think the result was conclusive.
> 
> So as an additional experiment, I just used the preamble and
> Introduction of orgmanual.org and then doubled the copies of the
> Introduction subtress with each iteration.  I runtime was linear in
> size, you'd expect to see the runtimes about double on each iteration,
> too.
> 
> |    user |   sys |   wall |  util | size |
> |---------+-------+--------+-------+------|
> |   2.500 | 0.064 |   3.14 | 81.5% |  18K |
> |   3.740 | 0.056 |   4.37 | 86.7% |  33K |
> |   6.224 | 0.112 |   6.98 | 90.6% |  63K |
> |  11.524 | 0.060 |  12.53 | 92.4% | 122K |
> |  22.860 | 0.084 |  24.35 | 94.2% | 241K |
> |  48.760 | 0.100 |  51.87 | 94.1% | 479K |
> | 110.804 | 0.124 | 120.64 | 91.9% | 955K |
> | 280.084 | 0.360 | 304.48 | 92.1% | 1.9M |
> | 868.712 | 0.768 | 930.24 | 93.4% | 3.8M |
> 
> Octave thinks that y = 1.725 x^2 + 1.025 x + 0.009 is a good fit to that
> data, so O(N^2) behaviour overall as suspected.
> 
> 
> Regards,
> Achim.
> -- 
> +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+
> 
> Factory and User Sound Singles for Waldorf rackAttack:
> http://Synth.Stromeko.net/Downloads.html#WaldorfSounds
> 
>