From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ken Mankoff Subject: Re: Painfully Slow Export Date: Tue, 12 Jun 2018 23:30:38 +0200 Message-ID: References: <73844D54-30A4-48CB-8303-6DA3C932A444@ucsd.edu> <04337B1C-B1CB-4349-87E1-5C7051B38BA7@ucsd.edu> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="00000000000051d806056e788f7c" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:33315) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fSqr0-00039z-8o for emacs-orgmode@gnu.org; Tue, 12 Jun 2018 17:29:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fSqqy-000710-0n for emacs-orgmode@gnu.org; Tue, 12 Jun 2018 17:29:25 -0400 Received: from mail-qk0-x231.google.com ([2607:f8b0:400d:c09::231]:38383) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fSqqx-00070q-OO for emacs-orgmode@gnu.org; Tue, 12 Jun 2018 17:29:23 -0400 Received: by mail-qk0-x231.google.com with SMTP id y4-v6so309528qka.5 for ; Tue, 12 Jun 2018 14:29:23 -0700 (PDT) In-Reply-To: List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: "Emacs-orgmode" To: Charles Berry Cc: Org-mode --00000000000051d806056e788f7c Content-Type: text/plain; charset="UTF-8" As an MWE, I can test with "emacs -Q" and the following org file: #+PROPERTY: header-args :eval never-export # (setq org-export-use-babel t) #+BEGIN_SRC emacs-lisp :results none (org-babel-do-load-languages 'org-babel-load-languages '((shell . t))) #+END_SRC #+BEGIN_SRC sh :results org raw for i in $(seq 200); do echo "" echo "#+BEGIN_SRC sh :results output :exports results" echo "echo 'hello, world'" echo "#+END_SRC" echo "#+RESULTS:" echo ": goodbye" done #+END_SRC #+RESULTS: Execute the first code block to allow execution of the second. Execute the second code block to generate 200 more. Export. It takes ~6 seconds on my machine, which seems long but is bearable. Based on the profiling, I think the reason it takes 1 minute is because the code blocks use <> blocks?. I can verify this with the following MWE: #+PROPERTY: header-args :eval never-export # (setq org-export-use-babel t) #+NAME: foo #+BEGIN_SRC sh :results verbatim ls #+END_SRC #+RESULTS: #+NAME: bar #+BEGIN_SRC sh :results verbatim :noweb yes <> #+END_SRC #+RESULTS: #+NAME: baz #+BEGIN_SRC sh :results verbatim :noweb yes <> #+END_SRC #+RESULTS: Repeat the BAZ block with the <> about 100 times and export this, and it takes a long time. I killed it after ~30 seconds, because I think I have my evidence that :noweb yes is the culprit for the slow export. On Tue, Jun 12, 2018 at 11:15 PM Ken Mankoff wrote: > > On Tue, Jun 12, 2018 at 11:07 PM Berry, Charles wrote: > >> >> > On Jun 12, 2018, at 12:41 PM, Ken Mankoff wrote: >> > >> > Hi Chuck, >> > >> > It looks fine to me. I see: >> > >> > Name: fig:foo >> > Lang: python >> > Properties: >> > :header-args :eval never-export >> > :header-args:python nil >> > Header Arguments: >> > :cache no >> > :eval never-export >> > :exports results >> > :hlines no >> > :noweb yes >> > :results file replace >> > :session discharge_ts >> > :tangle no >> > >> > >> >> And still the src blocks execute? >> >> > No. The source blocks never execute. It just takes an entire minute to > export the file with 200 source blocks. If they executed it would probably > take a few hours. > > -k. > --00000000000051d806056e788f7c Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
As an MWE, I can test with "emacs -Q" and the fo= llowing org file:








#+P= ROPERTY: header-args :eval never-export
# (setq org-export-use-ba= bel t)

#+BEGIN_SRC emacs-lisp :results none
<= div>(org-babel-do-load-languages
=C2=A0'org-babel-load-langua= ges
=C2=A0'((shell . t)))
#+END_SRC

<= /div>
#+BEGIN_SRC sh :results org raw
for i in $(seq 200); do=
echo ""
echo "#+BEGIN_SRC sh :results o= utput :exports results"
echo "echo 'hello, world= 9;"
echo "#+END_SRC"
echo "#+RESULT= S:"
echo ": goodbye"
done
#+EN= D_SRC
#+RESULTS:


Execute the first code block to allow execution of the second.= Execute the second code block to generate 200 more. Export. It takes ~6 se= conds on my machine, which seems long but is bearable. Based on the profili= ng, I think the reason it takes 1 minute is because the code blocks use <= ;<noweb>> blocks?. I can verify this with the following MWE:
=


#+PROPERTY: header-args :eval never= -export
# (setq org-export-use-babel t)

= #+NAME: foo
#+BEGIN_SRC sh :results verbatim
ls
#+END_SRC
#+RESULTS:

#+NAME: bar
#+BEGIN_SRC sh :results verbatim :noweb yes
<<foo>= >
#+END_SRC
#+RESULTS:

#+NA= ME: baz
#+BEGIN_SRC sh :results verbatim :noweb yes
<= ;<bar>>
#+END_SRC
#+RESULTS:
<= br>
Repeat the BAZ block with the <<bar>> about 100 t= imes and export this, and it takes a long time. I killed it after ~30 secon= ds, because I think I have my evidence that :noweb yes is the culprit for t= he slow export.



On Tue, Jun 12, 2018 at 11:15 PM Ken Mankoff = <mankoff@gmail.com> wrote:

On Tue, Jun 12, 2018 at 11:07 PM Berry, Charle= s <ccberry@ucsd.ed= u> wrote:

> On Jun 12, 2018, at 12:41 PM, Ken Mankoff <mankoff@gmail.com> wrote:
>
> Hi Chuck,
>
> It looks fine to me. I see:
>
> Name: fig:foo
> Lang: python
> Properties:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0:header-args=C2=A0 =C2=A0 :eval never-export=
>=C2=A0 =C2=A0 =C2=A0 =C2=A0:header-args:python=C2=A0 =C2=A0 =C2=A0nil > Header Arguments:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0:cache=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 no<= br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0:eval=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0never-export
>=C2=A0 =C2=A0 =C2=A0 =C2=A0:exports=C2=A0 =C2=A0 =C2=A0 =C2=A0 results<= br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0:hlines=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0no<= br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0:noweb=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 yes=
>=C2=A0 =C2=A0 =C2=A0 =C2=A0:results=C2=A0 =C2=A0 =C2=A0 =C2=A0 file rep= lace
>=C2=A0 =C2=A0 =C2=A0 =C2=A0:session=C2=A0 =C2=A0 =C2=A0 =C2=A0 discharg= e_ts
>=C2=A0 =C2=A0 =C2=A0 =C2=A0:tangle=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0no<= br> >
>=C2=A0

And still the src blocks execute?


N= o. The source blocks never execute. It just takes an entire minute to expor= t the file with 200 source blocks. If they executed it would probably take = a few hours.

=C2=A0 =C2=A0-k.=C2=A0
--00000000000051d806056e788f7c--