[RFC] Slight change to asynchronous export

[RFC] Slight change to asynchronous export

[Nicolas Goaziou]

Hello,

At the moment, by default, external emacs process for asynchronous
export is called with:

  /path/to/emacs -Q --batch -l org-export-async-init-file ... export stuff...

where `org-export-async-init-file' defaults to `user-init-file'. It is,
by default, not the same as calling "emacs", because of the "-Q".

I suggest allowing a nil (default) value for
`org-export-async-init-file'. In that case, the command will become:

  /path/to/emacs --batch ... export stuff...

If it is a file, it will be the same as before.

WDYT?


Regards,

-- 
Nicolas Goaziou

Re: [RFC] Slight change to asynchronous export

[Sebastien Vauban]

Hello Nicolas,

Nicolas Goaziou wrote:
> At the moment, by default, external emacs process for asynchronous
> export is called with:
>
>   /path/to/emacs -Q --batch -l org-export-async-init-file ... export stuff...
>
> where `org-export-async-init-file' defaults to `user-init-file'. It is,
> by default, not the same as calling "emacs", because of the "-Q".
>
> I suggest allowing a nil (default) value for
> `org-export-async-init-file'. In that case, the command will become:
>
>   /path/to/emacs --batch ... export stuff...
>
> If it is a file, it will be the same as before.
>
> WDYT?

IIUC, the default `user-init-file' still gets read because you drop the `-Q'
option. And you allow adding a supplementary configuration file for the export
process.

Right?

The only drawback would be for people having a long .emacs config file (which
takes a long time to load): they would experience a longer running time for the
async export, but it's async, so they wouldn't mind much.

That seems good to me.

BTW, is there no possibility of doing the same with emacsclient (but with no
visible buffer)?  That'd drop the startup time of Emacs. But, once again,
that's not a problem per se, as it's async. It's for my own understanding.

Best regards,
  Seb

-- 
Sebastien Vauban

Re: [RFC] Slight change to asynchronous export

[Nicolas Goaziou]

Hello,

"Sebastien Vauban" <sva-news-D0wtAvR13HarG/iDocfnWg@public.gmane.org>
writes:

> Nicolas Goaziou wrote:
>> At the moment, by default, external emacs process for asynchronous
>> export is called with:
>>
>>   /path/to/emacs -Q --batch -l org-export-async-init-file ... export stuff...
>>
>> where `org-export-async-init-file' defaults to `user-init-file'. It is,
>> by default, not the same as calling "emacs", because of the "-Q".
>>
>> I suggest allowing a nil (default) value for
>> `org-export-async-init-file'. In that case, the command will become:
>>
>>   /path/to/emacs --batch ... export stuff...
>>
>> If it is a file, it will be the same as before.
>>
>> WDYT?
>
> IIUC, the default `user-init-file' still gets read because you drop the `-Q'
> option. And you allow adding a supplementary configuration file for the export
> process.

With the proposed change, you get `user-init-file' and all site-lisp/
family, which is equivalent to a regular Emacs call.

Actually, by default, you only get `user-init-file'.

> Right?

Correct.

> The only drawback would be for people having a long .emacs config file (which
> takes a long time to load): they would experience a longer running time for the
> async export, but it's async, so they wouldn't mind much.

If you specify a specific config file, which is what
`org-export-async-init-file' suggests, the load time will be decreased.

> BTW, is there no possibility of doing the same with emacsclient (but with no
> visible buffer)?  That'd drop the startup time of Emacs. But, once again,
> that's not a problem per se, as it's async. It's for my own
> understanding.

No, that's not possible. Emacsclient runs in the same process as the
server: this is not asynchronous anymore. To convince yourself, try
spawning two clients from the same server, run (sleep-for 30) in one of
them and switch to the other one.


Regards,

-- 
Nicolas Goaziou

Re: [RFC] Slight change to asynchronous export

[Carsten Dominik]

Hi Nicolas,

this sounds OK for me.

- Carsten
On 27.10.2013, at 10:35, Nicolas Goaziou <n.goaziou@gmail.com> wrote:

> Hello,
> 
> At the moment, by default, external emacs process for asynchronous
> export is called with:
> 
>  /path/to/emacs -Q --batch -l org-export-async-init-file ... export stuff...
> 
> where `org-export-async-init-file' defaults to `user-init-file'. It is,
> by default, not the same as calling "emacs", because of the "-Q".
> 
> I suggest allowing a nil (default) value for
> `org-export-async-init-file'. In that case, the command will become:
> 
>  /path/to/emacs --batch ... export stuff...
> 
> If it is a file, it will be the same as before.
> 
> WDYT?
> 
> 
> Regards,
> 
> -- 
> Nicolas Goaziou
> 

Re: [RFC] Slight change to asynchronous export

[Aaron Ecay]

2013ko urriak 27an, Nicolas Goaziou-ek idatzi zuen:
> 
> Hello,
> 
> At the moment, by default, external emacs process for asynchronous
> export is called with:
> 
>   /path/to/emacs -Q --batch -l org-export-async-init-file ... export stuff...
> 
> where `org-export-async-init-file' defaults to `user-init-file'. It is,
> by default, not the same as calling "emacs", because of the "-Q".
> 
> I suggest allowing a nil (default) value for
> `org-export-async-init-file'. In that case, the command will become:
> 
>   /path/to/emacs --batch ... export stuff...
> 
> If it is a file, it will be the same as before.
> 
> WDYT?

Hi Nicolas,

My init.el file does (server-start), and also opens all the buffers
(usually 100+) I had opened the last time I quit emacs (by using emacs’s
included desktop library).  The buffers in turn spawn child processes
(aspell, some python IDE-ish autocomplete server, ...)  I think these
things are relatively common in init.el files, but not things that
should be done by the async export process.  Other examples might
include gnus/other email/rss fetching, or automatic update of installed
elpa packages.

Of course this would not be an issue for me in particular, but I think
it would be very easy for a user to get unwanted results.

I guess you want to make it so that users don’t have to configure async
export (beyond just flipping it on) in order to use it.  But I don’t
know if there is a sensible way to parcel out benign bits of
initialization code from dangerous ones without user intervention.
(“Dangerous” only in the sense of “should not be run by the export
process”.)

An interesting approach might be to extend the approach of
org-export--generate-copy-script in order to generate an init file
containing all relevant variable settings that can be read by the async
exporter, but given that export hooks can call into arbitrary
user-defined elisp, I don’t know if this could get off the ground.

-- 
Aaron Ecay

Re: [RFC] Slight change to asynchronous export

[Nicolas Goaziou]

Hello,

Aaron Ecay <aaronecay@gmail.com> writes:

> My init.el file does (server-start), and also opens all the buffers
> (usually 100+) I had opened the last time I quit emacs (by using emacs’s
> included desktop library).  The buffers in turn spawn child processes
> (aspell, some python IDE-ish autocomplete server, ...)  I think these
> things are relatively common in init.el files, but not things that
> should be done by the async export process.  Other examples might
> include gnus/other email/rss fetching, or automatic update of installed
> elpa packages.
>
> Of course this would not be an issue for me in particular, but I think
> it would be very easy for a user to get unwanted results.

At the moment, default configuration will load init.el (or an
equivalent) so my proposal is not different in that regard.

> I guess you want to make it so that users don’t have to configure async
> export (beyond just flipping it on) in order to use it.

I want the default value to be less surprising, even though I strongly
encourage users to customize `org-export-async-init-file'.

> But I don’t know if there is a sensible way to parcel out benign bits
> of initialization code from dangerous ones without user intervention.

I don't know either, and that is well out of the scope of my proposal.


Regards,

-- 
Nicolas Goaziou

Re: [RFC] Slight change to asynchronous export

[Aaron Ecay]

2013ko urriak 28an, Nicolas Goaziou-ek idatzi zuen:
> 
> At the moment, default configuration will load init.el (or an
> equivalent) so my proposal is not different in that regard.

Oh I see.  I didn’t read your original email carefully enough; I focused
on the removal of -Q, which I just associate with not loading init.el.

Sorry for the noise.

-- 
Aaron Ecay

Re: [RFC] Slight change to asynchronous export

[Achim Gratz]

Nicolas Goaziou writes:
> At the moment, by default, external emacs process for asynchronous
> export is called with:
>
>   /path/to/emacs -Q --batch -l org-export-async-init-file ... export stuff...
>
> where `org-export-async-init-file' defaults to `user-init-file'. It is,
> by default, not the same as calling "emacs", because of the "-Q".

That's a good thing, I'd say.  There is a lot of stuff going on in init
files that you most likely don't want done when firing Emacs off in the
background.  That is especially true of some site-init files that you
typically have no control over.

> I suggest allowing a nil (default) value for
> `org-export-async-init-file'. In that case, the command will become:
>
>   /path/to/emacs --batch ... export stuff...
>
> If it is a file, it will be the same as before.
>
> WDYT?

If you don't make this the default then no harm is done, I guess.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: [RFC] Slight change to asynchronous export

[Nicolas Goaziou]

Hello,

Achim Gratz <Stromeko@nexgo.de> writes:

> Nicolas Goaziou writes:
>> At the moment, by default, external emacs process for asynchronous
>> export is called with:
>>
>>   /path/to/emacs -Q --batch -l org-export-async-init-file ... export stuff...
>>
>> where `org-export-async-init-file' defaults to `user-init-file'. It is,
>> by default, not the same as calling "emacs", because of the "-Q".
>
> That's a good thing, I'd say.  There is a lot of stuff going on in init
> files that you most likely don't want done when firing Emacs off in the
> background.  That is especially true of some site-init files that you
> typically have no control over.

As I said, it wouldn't be worse than the current situation.

> If you don't make this the default then no harm is done, I guess.

The whole point of this change is to change the default value. If
there's no interest in it, I won't bother making it.


Regards,

-- 
Nicolas Goaziou

Re: [RFC] Slight change to asynchronous export

[Rasmus]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Hello,
>> BTW, is there no possibility of doing the same with emacsclient (but with no
>> visible buffer)?  That'd drop the startup time of Emacs. But, once again,
>> that's not a problem per se, as it's async. It's for my own
>> understanding.
>
> No, that's not possible. Emacsclient runs in the same process as the
> server: this is not asynchronous anymore. To convince yourself, try
> spawning two clients from the same server, run (sleep-for 30) in one of
> them and switch to the other one.

My understanding of Sebastien's question is whether you could run
async with a new Emacs server, with `server-name' "slave" say.  It
would only allow one process at the time, but it would also only load
init once.

–Rasmus

-- 
C is for Cookie

Re: [RFC] Slight change to asynchronous export

[Rasmus]

Nicolas Goaziou writes:

> At the moment, by default, external emacs process for asynchronous
> export is called with:

>   /path/to/emacs -Q --batch -l org-export-async-init-file ... export stuff...

> where `org-export-async-init-file' defaults to `user-init-file'. It is,
> by default, not the same as calling "emacs", because of the "-Q".

> I suggest allowing a nil (default) value for
> `org-export-async-init-file'. In that case, the command will become:

>   /path/to/emacs --batch ... export stuff...

> If it is a file, it will be the same as before.

> WDYT?

Fine with me.  I wouldn't use it as I have customizations in my Emacs
file though.  I'm guessing this is the case for most people. . .  

Aaron Ecay <aaronecay@gmail.com> writes:

> 2013ko urriak 27an, Nicolas Goaziou-ek idatzi zuen:

> I guess you want to make it so that users don’t have to configure async
> export (beyond just flipping it on) in order to use it.  But I don’t
> know if there is a sensible way to parcel out benign bits of
> initialization code from dangerous ones without user intervention.
> (“Dangerous” only in the sense of “should not be run by the export
> process”.)

The issue is parsing out settings like org-latex-tables-booktabs etc.

I have a pretty structured init file and am able to pick out the right
parts of my config file by only evaluating the part of the init file
that starts with ";;*+ Org" and end with "**? [A-Za-z0-9]".  

–Rasmus


-- 
Vote for proprietary math!

Re: [RFC] Slight change to asynchronous export

[Nicolas Goaziou]

Hello,

Rasmus <rasmus@gmx.us> writes:

> My understanding of Sebastien's question is whether you could run
> async with a new Emacs server, with `server-name' "slave" say.  It
> would only allow one process at the time, but it would also only load
> init once.

It would also leave another emacs server open all the time.

But, seriously, with a dedicated asynchronous init file, init time is
negligible. Here, it is 3 milliseconds. You can measure it with:

  (org-export-async-start
   (lambda (result) (message result))
   '(format-time-string "%S,%3N s"
                        (time-subtract after-init-time before-init-time)))


Regards,

-- 
Nicolas Goaziou

Re: [RFC] Slight change to asynchronous export

[Rick Frankel]

On 2013-10-29 04:35, Nicolas Goaziou wrote:
> Achim Gratz <Stromeko@nexgo.de> writes:
> 
> Nicolas Goaziou writes:
> At the moment, by default, external emacs process for asynchronous
> export is called with:
> 
> /path/to/emacs -Q --batch -l org-export-async-init-file ... export 
> stuff...
> 
> where `org-export-async-init-file' defaults to `user-init-file'. It is,
> by default, not the same as calling "emacs", because of the "-Q".
> 
> That's a good thing, I'd say.  There is a lot of stuff going on in init
> files that you most likely don't want done when firing Emacs off in the
> background.  That is especially true of some site-init files that you
> typically have no control over.
> 
> As I said, it wouldn't be worse than the current situation.
> 
> If you don't make this the default then no harm is done, I guess.
> 
> The whole point of this change is to change the default value. If
> there's no interest in it, I won't bother making it.

I'm all for it. The current default is, IMHO, the worst of both worlds
since, by loading my init but leaving out site-lisp i end up w/ the
wrong org loaded (the one from the default distribution, not
site-lisp) unless I make explicit reference to the development org in
site-lisp.

For Aaron and others concerned about e.g, `server-start', it's easy to
check if emacs is not running in batch or, in the case of server
start, if emacs is running in a multi-window environment.

rick

Re: [RFC] Slight change to asynchronous export

[Sebastien Vauban]

Rick Frankel wrote:
> For Aaron and others concerned about e.g, `server-start', it's easy to
> check if emacs is not running in batch or, in the case of server
> start, if emacs is running in a multi-window environment.

The "server-start" thing was a question of mine, trying to see if using
emacsclient would be possible. It isn't. So, please forget about this.

The async process launches a new Emacs process, with no interaction whatsoever
with the Emacs you edit in.

Best regards,
  Seb

-- 
Sebastien Vauban

Re: [RFC] Slight change to asynchronous export

[Nicolas Goaziou]

[-- Attachment #1: Type: text/plain, Size: 1432 bytes --]

Helo,

Rick Frankel <rick@rickster.com> writes:

> On 2013-10-29 04:35, Nicolas Goaziou wrote:
>> Achim Gratz <Stromeko@nexgo.de> writes:
>>
>> Nicolas Goaziou writes:
>> At the moment, by default, external emacs process for asynchronous
>> export is called with:
>>
>> /path/to/emacs -Q --batch -l org-export-async-init-file ... export
>> stuff...
>>
>> where `org-export-async-init-file' defaults to `user-init-file'. It is,
>> by default, not the same as calling "emacs", because of the "-Q".
>>
>> That's a good thing, I'd say.  There is a lot of stuff going on in init
>> files that you most likely don't want done when firing Emacs off in the
>> background.  That is especially true of some site-init files that you
>> typically have no control over.
>>
>> As I said, it wouldn't be worse than the current situation.
>>
>> If you don't make this the default then no harm is done, I guess.
>>
>> The whole point of this change is to change the default value. If
>> there's no interest in it, I won't bother making it.
>
> I'm all for it. The current default is, IMHO, the worst of both worlds
> since, by loading my init but leaving out site-lisp i end up w/ the
> wrong org loaded (the one from the default distribution, not
> site-lisp) unless I make explicit reference to the development org in
> site-lisp.

For reference, here is the suggested patch.

Is there any strong point against it?


Regards,

-- 
Nicolas Goaziou

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ox-Change-default-asynchronous-export-setup.patch --]
[-- Type: text/x-diff, Size: 2853 bytes --]

From 993105a2bdf70a146c302e10884da1f0de406a65 Mon Sep 17 00:00:00 2001
From: Nicolas Goaziou <n.goaziou@gmail.com>
Date: Mon, 28 Oct 2013 18:56:04 +0100
Subject: [PATCH] ox: Change default asynchronous export setup

* lisp/ox.el (org-export-async-init-file): Change default value and
  allowed values.
(org-export-async-start): Apply change to the variable.
---
 lisp/ox.el | 37 +++++++++++++++++++++++++------------
 1 file changed, 25 insertions(+), 12 deletions(-)

diff --git a/lisp/ox.el b/lisp/ox.el
index 141abc4..20d7107 100644
--- a/lisp/ox.el
+++ b/lisp/ox.el
@@ -839,15 +839,23 @@ automatically.  But you can retrieve them with \\[org-export-stack]."
   :package-version '(Org . "8.0")
   :type 'boolean)
 
-(defcustom org-export-async-init-file user-init-file
+(defcustom org-export-async-init-file nil
   "File used to initialize external export process.
-Value must be an absolute file name.  It defaults to user's
-initialization file.  Though, a specific configuration makes the
-process faster and the export more portable."
+
+Value must be either nil or an absolute file name.  When nil, the
+external process is launched like a regular Emacs session,
+loading user's initialization file and any site specific
+configuration.  If a file is provided, it, and only it, is loaded
+at start-up.
+
+Therefore, using a specific configuration makes the process to
+load faster and the export more portable."
   :group 'org-export-general
   :version "24.4"
-  :package-version '(Org . "8.0")
-  :type '(file :must-match t))
+  :package-version '(Org . "8.3")
+  :type '(choice
+	  (const :tag "Regular startup" nil)
+	  (file :tag "Specific start-up file" :must-match t)))
 
 (defcustom org-export-dispatch-use-expert-ui nil
   "Non-nil means using a non-intrusive `org-export-dispatch'.
@@ -5552,12 +5560,17 @@ and `org-export-to-file' for more specialized functions."
          (let* ((process-connection-type nil)
                 (,proc-buffer (generate-new-buffer-name "*Org Export Process*"))
                 (,process
-                 (start-process
-                  "org-export-process" ,proc-buffer
-                  (expand-file-name invocation-name invocation-directory)
-                  "-Q" "--batch"
-                  "-l" org-export-async-init-file
-                  "-l" ,temp-file)))
+		 (apply
+		  #'start-process
+		  (append
+		   (list "org-export-process"
+			 ,proc-buffer
+			 (expand-file-name invocation-name invocation-directory)
+			 "--batch")
+		   (if org-export-async-init-file
+		       (list "-Q" "-l" org-export-async-init-file)
+		     (list "-l" user-init-file))
+		   (list "-l" ,temp-file)))))
            ;; Register running process in stack.
            (org-export-add-to-stack (get-buffer ,proc-buffer) nil ,process)
            ;; Set-up sentinel in order to catch results.
-- 
1.8.4.2

Re: [RFC] Slight change to asynchronous export

[Rick Frankel]

On 2013-10-30 06:00, Nicolas Goaziou wrote:
> Helo,
> For reference, here is the suggested patch.
> 
> Is there any strong point against it?

looks great to me.

rick

Re: [RFC] Slight change to asynchronous export

[Nicolas Goaziou]

Hello,

Rick Frankel <rick@rickster.com> writes:

> On 2013-10-30 06:00, Nicolas Goaziou wrote:
>> Helo,
>> For reference, here is the suggested patch.
>>
>> Is there any strong point against it?
>
> looks great to me.

Applied.


Regards,

-- 
Nicolas Goaziou