External compilation

External compilation

[Rasmus]

Hi,

I want to translate some documents to pdf.  I'm currently doing it with
something like this

    OX-FUN    = emacs --batch --no-init-file --load conf.el $1 --funcall $2 --kill
    OX-LATEX  = $(call OX-FUN, $1, org-latex-export-to-pdf)

But this is very limiting (I am told) as it require me to maintain a
Makefile.  It's not as easy as latex.

I can move the call to  Emacs as file variable s.t.

    # Local Variables:
    # eval:  (unless (or user-init-file (window-system)) (load-file "conf.el"))
    # End:

And export by calling:

    emacs --batch --no-init-file --eval="(progn (setq enable-local-eval t) (require 'ox-latex))" $1 --funcall=org-latex-export-to-pdf

But this is kind of annoying as it disturbs opening the file

I tried to port my config to conf.org and let babel handle it, but
conf.org also defines the document class so I never get to the babel
evaluation stage.

Has anybody come up with a better method?

Ideally, what I would like would be to specify a per-file/project init
file.  Sort of like org-export-async-init-file, but as part of
org-export-options-alist to get the "init.el-dependency" inside the
Org-file.  It would be read when Org exports async or via batch.

—Rasmus

-- 
This message is brought to you by the department of redundant departments

Re: External compilation

[Sebastien Vauban]

Hi Rasmus,

Rasmus wrote:
> I want to translate some documents to pdf.  I'm currently doing it with
> something like this
>
>     OX-FUN    = emacs --batch --no-init-file --load conf.el $1 --funcall $2 --kill
>     OX-LATEX  = $(call OX-FUN, $1, org-latex-export-to-pdf)

One side question.

I guess that you meant "--kill-emacs" above (I don't see any function
called "kill"); but, anyway, why do you need to kill Emacs?  Isn't that
redundant with the "--batch" option?

Best regards,
  Seb

-- 
Sebastien Vauban

Re: External compilation

[Rasmus]

Sebastien Vauban <sva-news@mygooglest.com>
writes:

> One side question.
>
> I guess that you meant "--kill-emacs" above (I don't see any function
> called "kill"); but, anyway, why do you need to kill Emacs?  Isn't that
> redundant with the "--batch" option?

Googles told me to put it there.  I obeyed.

In my experience it has no effect.

—Rasmus

-- 
You people at the NSA are becoming my new best friends!

Re: External compilation

[Sebastien Vauban]

Rasmus wrote:
> Sebastien Vauban writes:
>
>> One side question.
>>
>> I guess that you meant "--kill-emacs" above (I don't see any function
>> called "kill"); but, anyway, why do you need to kill Emacs?  Isn't
>> that redundant with the "--batch" option?
>
> Googles told me to put it there.  I obeyed.
>
> In my experience it has no effect.

I have the same both experiences (Google + no effect). Thanks for
confirming!

Best regards,
  Seb

-- 
Sebastien Vauban

[ox, patch] external compilation (was: External compilation)

[Rasmus]

[-- Attachment #1: Type: text/plain, Size: 1007 bytes --]

Rasmus <rasmus@gmx.us> writes:

> I can move the call to  Emacs as file variable s.t.
>
>     # Local Variables:
>     # eval:  (unless (or user-init-file (window-system)) (load-file "conf.el"))
>     # End:

A much better variable to check is `noninteractive', cf
(info "(elisp) Batch mode").

> Ideally, what I would like would be to specify a per-file/project init
> file.  Sort of like org-export-async-init-file, but as part of
> org-export-options-alist to get the "init.el-dependency" inside the
> Org-file.  It would be read when Org exports async or via batch.

This patch moves org-export-async-init-file to org-export-options-alist
and introduces #+INIT for setting it in a file.

It also automatically "switches" to async export if running Emacs in
batch.  I don't know if this is a good idea!  E.g. I guess it would starts
a new Emacs process from a Emacs batch process...

WDYT?  Would something like this be appropriate?

—Rasmus

-- 
One thing that is clear: it's all down hill from here 

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ox.el-Introduce-file-specific-init-files.patch --]
[-- Type: text/x-diff, Size: 6337 bytes --]

From 796c6a55e2ece026a17268bd9a021895fa1be273 Mon Sep 17 00:00:00 2001
From: rasmus <rasmus@gmx.us>
Date: Fri, 20 Feb 2015 12:56:06 +0100
Subject: [PATCH] ox.el: Introduce file-specific init-files.

* ox.el (org-export-options-alist): Add
  :init-file.  (org-export-async-start): Use :init-file.
* ox-publish.el (org-publish) (org-publish-all)
  (org-publish-current-file): Use :init-file if noninteractive.
* ox-icalendar.el (org-icalendar-export-agenda-files),
  (org-icalendar-combine-agenda-files): Use :init-file if
  noninteractive.

Allows specifying a file-specific init-file via #+INIT.
---
 lisp/ox-icalendar.el |  4 ++--
 lisp/ox-odt.el       |  2 +-
 lisp/ox-publish.el   |  6 +++---
 lisp/ox.el           | 18 +++++++++++-------
 4 files changed, 17 insertions(+), 13 deletions(-)

diff --git a/lisp/ox-icalendar.el b/lisp/ox-icalendar.el
index b473f11..3a489f7 100644
--- a/lisp/ox-icalendar.el
+++ b/lisp/ox-icalendar.el
@@ -832,7 +832,7 @@ Return ICS file name."
 When optional argument ASYNC is non-nil, export happens in an
 external process."
   (interactive)
-  (if async
+  (if (or async noninteractive)
       ;; Asynchronous export is not interactive, so we will not call
       ;; `org-check-agenda-file'.  Instead we remove any non-existent
       ;; agenda file from the list.
@@ -870,7 +870,7 @@ the `org-export-stack' interface.
 The file is stored under the name chosen in
 `org-icalendar-combined-agenda-file'."
   (interactive)
-  (if async
+  (if (or async noninteractive)
       (let ((files (org-remove-if-not 'file-exists-p (org-agenda-files t))))
 	(org-export-async-start
 	    (lambda (dummy)
diff --git a/lisp/ox-odt.el b/lisp/ox-odt.el
index 4a94de1..a3fd43b 100644
--- a/lisp/ox-odt.el
+++ b/lisp/ox-odt.el
@@ -4232,7 +4232,7 @@ file-local settings.
 Return output file's name."
   (interactive)
   (let ((outfile (org-export-output-file-name ".odt" subtreep)))
-    (if async
+    (if (or async noninteractive)
 	(org-export-async-start (lambda (f) (org-export-add-to-stack f 'odt))
 	  `(expand-file-name
 	    (org-odt--export-wrap
diff --git a/lisp/ox-publish.el b/lisp/ox-publish.el
index d450c0e..c5b0a76 100644
--- a/lisp/ox-publish.el
+++ b/lisp/ox-publish.el
@@ -884,7 +884,7 @@ publishing will be done asynchronously, in another process."
 		   (assoc project org-publish-project-alist))))
     (cond
      ((not project))
-     (async
+     ((or async noninteractive)
       (org-export-async-start (lambda (results) nil)
 	`(let ((org-publish-use-timestamps-flag
 		,(and (not force) org-publish-use-timestamps-flag)))
@@ -905,7 +905,7 @@ directory and force publishing all projects.  With a non-nil
 optional argument ASYNC, publishing will be done asynchronously,
 in another process."
   (interactive "P")
-  (if async
+  (if (or async noninteractive)
       (org-export-async-start (lambda (results) nil)
 	`(progn
 	   (when ',force (org-publish-remove-all-timestamps))
@@ -927,7 +927,7 @@ optional argument ASYNC is non-nil, publishing will be done
 asynchronously, in another process."
   (interactive "P")
   (let ((file (buffer-file-name (buffer-base-buffer))))
-    (if async
+    (if (or async noninteractive)
 	(org-export-async-start (lambda (results) nil)
 	  `(let ((org-publish-use-timestamps-flag
 		  (if ',force nil ,org-publish-use-timestamps-flag)))
diff --git a/lisp/ox.el b/lisp/ox.el
index 6a16997..6cf728b 100644
--- a/lisp/ox.el
+++ b/lisp/ox.el
@@ -108,6 +108,7 @@
     (:select-tags "SELECT_TAGS" nil org-export-select-tags split)
     (:exclude-tags "EXCLUDE_TAGS" nil org-export-exclude-tags split)
     (:creator "CREATOR" nil org-export-creator-string)
+    (:init-file "INIT" nil org-export-async-init-file t)
     (:headline-levels nil "H" org-export-headline-levels)
     (:preserve-breaks nil "\\n" org-export-preserve-breaks)
     (:section-numbers nil "num" org-export-with-section-numbers)
@@ -859,7 +860,7 @@ automatically.  But you can retrieve them with \\[org-export-stack]."
   :type 'boolean)
 
 (defcustom org-export-async-init-file nil
-  "File used to initialize external export process.
+  "File used to initialize external and batch export processes.
 
 Value must be either nil or an absolute file name.  When nil, the
 external process is launched like a regular Emacs session,
@@ -868,7 +869,9 @@ configuration.  If a file is provided, it, and only it, is loaded
 at start-up.
 
 Therefore, using a specific configuration makes the process to
-load faster and the export more portable."
+load faster and the export more portable.
+
+In ox-publish projects the value :init-file takes precedence."
   :group 'org-export-general
   :version "24.4"
   :package-version '(Org . "8.0")
@@ -5646,7 +5649,8 @@ and `org-export-to-file' for more specialized functions."
                        ;; Sexp to evaluate in the buffer.
                        (print (progn ,,@body))))))
          ;; Start external process.
-         (let* ((process-connection-type nil)
+         (let* ((init (plist-get :init (org-export--get-inbuffer-options)))
+		(process-connection-type nil)
                 (,proc-buffer (generate-new-buffer-name "*Org Export Process*"))
                 (,process
 		 (apply
@@ -5656,8 +5660,8 @@ and `org-export-to-file' for more specialized functions."
 			 ,proc-buffer
 			 (expand-file-name invocation-name invocation-directory)
 			 "--batch")
-		   (if org-export-async-init-file
-		       (list "-Q" "-l" org-export-async-init-file)
+		   (if init
+		       (list "-Q" "-l" init)
 		     (list "-l" user-init-file))
 		   (list "-l" ,temp-file)))))
            ;; Register running process in stack.
@@ -5724,7 +5728,7 @@ use it to set a major mode there, e.g,
 
 This function returns BUFFER."
   (declare (indent 2))
-  (if async
+  (if (or async noninteractive)
       (org-export-async-start
 	  `(lambda (output)
 	     (with-current-buffer (get-buffer-create ,buffer)
@@ -5790,7 +5794,7 @@ or FILE."
   (if (not (file-writable-p file)) (error "Output file not writable")
     (let ((ext-plist (org-combine-plists `(:output-file ,file) ext-plist))
 	  (encoding (or org-export-coding-system buffer-file-coding-system)))
-      (if async
+      (if (or async noninteractive)
           (org-export-async-start
 	      `(lambda (file)
 		 (org-export-add-to-stack (expand-file-name file) ',backend))
-- 
2.3.0

Re: [ox, patch] external compilation

[Nicolas Goaziou]

Hello,

Rasmus <rasmus@gmx.us> writes:

>> Ideally, what I would like would be to specify a per-file/project init
>> file.  Sort of like org-export-async-init-file, but as part of
>> org-export-options-alist to get the "init.el-dependency" inside the
>> Org-file.  It would be read when Org exports async or via batch.
>
> This patch moves org-export-async-init-file to org-export-options-alist
> and introduces #+INIT for setting it in a file.
>SW
> It also automatically "switches" to async export if running Emacs in
> batch.  I don't know if this is a good idea!  E.g. I guess it would starts
> a new Emacs process from a Emacs batch process...
>
> WDYT?  Would something like this be appropriate?

Can't you simply set-up a publishing project?

You can also use 

  (let ((org-export-async-init-file "whatever.el"))
    (org-latex-export-to-file ...))


Regards,

-- 
Nicolas Goaziou

Re: [ox, patch] external compilation

[Rasmus]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Hello,
>
> Rasmus <rasmus@gmx.us> writes:
>
>>> Ideally, what I would like would be to specify a per-file/project init
>>> file.  Sort of like org-export-async-init-file, but as part of
>>> org-export-options-alist to get the "init.el-dependency" inside the
>>> Org-file.  It would be read when Org exports async or via batch.
>>
>> This patch moves org-export-async-init-file to org-export-options-alist
>> and introduces #+INIT for setting it in a file.
>>SW
>> It also automatically "switches" to async export if running Emacs in
>> batch.  I don't know if this is a good idea!  E.g. I guess it would starts
>> a new Emacs process from a Emacs batch process...
>>
>> WDYT?  Would something like this be appropriate?
>
> Can't you simply set-up a publishing project?

No this has the same issue as the Makefile.  "It's not as easy as latex".
I want *one* coherent file with all the necessary instructions, that can
be edited in different program from Emacs, and be easily compiled.

E.g. I wrote syntax highlight for Org in texworks, but I also want to make
the "green button" work easily as latex without too many ugly hacks.

A makefile or a ox-project is great for the pros.  Unfortunately not
everyone is...

Makefiles and projects is the tool better for multi-file projects.

—Rasmus 

-- 
A clever person solves a problem. A wise person avoids it

Re: [ox, patch] external compilation

[Nicolas Goaziou]

Rasmus <rasmus@gmx.us> writes:

>> Can't you simply set-up a publishing project?
>
> No this has the same issue as the Makefile.  "It's not as easy as latex".

Setting up a publishing project is easy and well documented.

> I want *one* coherent file with all the necessary instructions, that can
> be edited in different program from Emacs, and be easily compiled.
>
> E.g. I wrote syntax highlight for Org in texworks, but I also want to make
> the "green button" work easily as latex without too many ugly hacks.

Isn't a file local variable enough then? 

Also, what about the (let ((org-export-async-init-file ...))) suggested
earlier?


Regards,

Re: [ox, patch] external compilation

[Rasmus]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

>> No this has the same issue as the Makefile.  "It's not as easy as latex".
>
> Setting up a publishing project is easy and well documented.

So is Makefile.

>> I want *one* coherent file with all the necessary instructions, that can
>> be edited in different program from Emacs, and be easily compiled.
>>
>> E.g. I wrote syntax highlight for Org in texworks, but I also want to make
>> the "green button" work easily as latex without too many ugly hacks.
>
> Isn't a file local variable enough then? 

It's not interpret in a batch call by default (which may be a feature that
is not wise to mess with).

> Also, what about the (let ((org-export-async-init-file ...))) suggested
> earlier?

How to I generalize this to a emacs --batch call from a random org file?

—Rasmus

-- 
The second rule of Fight Club is: You do not talk about Fight Club

Re: [ox, patch] external compilation

[Nicolas Goaziou]

Rasmus <rasmus@gmx.us> writes:

> How to I generalize this to a emacs --batch call from a random org
> file?

What's wrong with

  emacs --batch -Q -l conf.el --visit toexport.org -f org-latex-export-to-pdf 


Regards,

Re: [ox, patch] external compilation

[Rasmus]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

>   emacs --batch -Q -l conf.el --visit toexport.org -f org-latex-export-to-pdf 

That it hardcodes conf.el.  So it doesn't toexport1.org which depends on
config/conf.el.  I want to specify the dependency in in toexport.org
explicitly so that

     emacs --batch -Q --visit toexport.org -f org-latex-export-to-pdf

generally works with custom-config files.  Again, think about supporting a
Org from a editor where you just have "one button".

—Rasmus

-- 
Need more coffee. . .

Re: [ox, patch] external compilation

[Nicolas Goaziou]

Rasmus <rasmus@gmx.us> writes:

> That it hardcodes conf.el.  So it doesn't toexport1.org which depends on
> config/conf.el.  I want to specify the dependency in in toexport.org
> explicitly so that
>
>      emacs --batch -Q --visit toexport.org -f org-latex-export-to-pdf
>
> generally works with custom-config files.  Again, think about supporting a
> Org from a editor where you just have "one button".

Then

  # Local Variables:
  # org-export-async-init-file: "config/config.el"
  # End:

at the end of your file and

  emacs --batch -Q --eval="(setq enable-local-variables :all)" --visit toexport.org --eval="(org-export-latex-to-pdf t)"

Regards,

Re: [ox, patch] external compilation

[Rasmus]

Hi,

>> That it hardcodes conf.el.  So it doesn't toexport1.org which depends on
>> config/conf.el.  I want to specify the dependency in in toexport.org
>> explicitly so that
>>
>>      emacs --batch -Q --visit toexport.org -f org-latex-export-to-pdf
>>
>> generally works with custom-config files.  Again, think about supporting a
>> Org from a editor where you just have "one button".
>
> Then
>
>   # Local Variables:
>   # org-export-async-init-file: "config/config.el"
>   # End:
>
> at the end of your file and
>
>   emacs --batch -Q --eval="(setq enable-local-variables :all)" --visit
> toexport.org --eval="(org-export-latex-to-pdf t)"

That's more or less what I do cf. the first post, namely

# Local Variables:
# eval:  (unless noninteractive (load-file "org-conf.el"))
# End:

[Which gives me the annoying prompt].

I call Emacsfor like this. For some reason I had to explicitly require
'ox, maybe just me doing a mistake.

emacs --batch --no-init-file --eval="(progn (setq enable-local-eval t) (require 'ox-latex))" $1 --funcall=org-latex-export-to-pdf

The whole point is that I would like to cut it down to:

    emacs --batch --no-init-file $1 --funcall=org-latex-export-to-pdf

With the dependency specified in an #+INIT or #+PREAMBLE or #+LISP-SETUP
or whatever. For the sake of making ox export for arbitrary $1 " as easy
as latex".  (Even better would be a dedicated binary: "ox --latex $1").

I don't know if it's a security issue, and if you don't see a benefit from
the above, I won't push further.

—Rasmus

-- 
Slowly unravels in a ball of yarn and the devil collects it

Re: [ox, patch] external compilation

[Nicolas Goaziou]

Rasmus <rasmus@gmx.us> writes:

> That's more or less what I do cf. the first post, namely
>
> # Local Variables:
> # eval:  (unless noninteractive (load-file "org-conf.el"))
> # End:
>
> [Which gives me the annoying prompt].

The solution I gave you doesn't prompt anything.

> With the dependency specified in an #+INIT or #+PREAMBLE or #+LISP-SETUP
> or whatever. For the sake of making ox export for arbitrary $1 " as easy
> as latex".  (Even better would be a dedicated binary: "ox --latex $1").
>
> I don't know if it's a security issue, and if you don't see a benefit from
> the above, I won't push further.

I don't see the point of adding a feature that will _only_ benefit to
other editors than Emacs.

Also, I have trouble understanding why you would need to switch init
file per document, instead of using, e.g., #+SETUPFILE.

To solve a portability issue, which is a related problem, then you don't
want to change init file for a give Org document, but instead attach it
to fixed set-up. We could work it out by defining a new file format
".borg" (for Bundled Org) containing both the init file and the
document, in a compressed directory (created, e.g., with
`org-assimilate').

Regards,

Re: [ox, patch] external compilation

[Rasmus]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Rasmus <rasmus@gmx.us> writes:
>
>> That's more or less what I do cf. the first post, namely
>>
>> # Local Variables:
>> # eval:  (unless noninteractive (load-file "org-conf.el"))
>> # End:
>>
>> [Which gives me the annoying prompt].
>
> The solution I gave you doesn't prompt anything.

You are right.  So yours (slightly) better.

>> With the dependency specified in an #+INIT or #+PREAMBLE or #+LISP-SETUP
>> or whatever. For the sake of making ox export for arbitrary $1 " as easy
>> as latex".  (Even better would be a dedicated binary: "ox --latex $1").
>>
>> I don't know if it's a security issue, and if you don't see a benefit from
>> the above, I won't push further.
>
> I don't see the point of adding a feature that will _only_ benefit to
> other editors than Emacs.

It also makes it easier for Emacs users.  I compile org with a Makefile
sometimes.

> Also, I have trouble understanding why you would need to switch init
> file per document, instead of using, e.g., #+SETUPFILE.

The document is shared between a couple of people.  My init.el is
opinionated and > 3000 lines and only assumes Emacs-git.  It would be a
distraction to circulate it for this purpose.

By having a separate init file I ensure stability and coherency, hopefully
also across time.  For the same reason, I copy bib entries from my
"global" lit.bib to a project lit.bib.

Why not #+SETUP: 'Cause I also define filters, links,
org-latex-package-alist etc.  I even use a separate "org-cite" library.

> To solve a portability issue, which is a related problem, then you don't
> want to change init file for a give Org document, but instead attach it
> to fixed set-up. 

Even better.  And more ambitious!

> We could work it out by defining a new file format ".borg" (for Bundled
> Org) containing both the init file and the document, in a compressed
> directory (created, e.g., with `org-assimilate').

Hmm, I guess that would be nice.  Do you have like a tar in mind?  Would
.borg be tar since it contains both "libraries" and a document?

Unless we can somehow sandbox execution I guess it holds the same security
issues as a #+INIT?

BTW: If it can hold libraries, this could solve another pity: manually
copying ox-koma-letter.el to vanilla Emacs setups.

—Rasmus

-- 
Enough with the bla bla!