emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
* emacs build command for org-files
@ 2020-01-26 18:10 John Kitchin
  2020-01-26 19:38 ` briangpowell .
                   ` (2 more replies)
  0 siblings, 3 replies; 11+ messages in thread
From: John Kitchin @ 2020-01-26 18:10 UTC (permalink / raw)
  To: org-mode-email

[-- Attachment #1: Type: text/plain, Size: 2645 bytes --]

Hi everyone,

This is only semi-on-topic. I am looking for something like M-x compile for
my org-files, but I don't want it to necessarily use Makefiles. I am
looking for suggestions of existing solutions to this, or thoughts on how
to implement this.

Actually, if it was possible to get M-x compile to run an elisp function
instead of a make file, it might be all I need, but it looks like it runs
shell commands.

For most of my files, simple exporting is totally adequate. But, some files
are more complicated, and what I usually do in these cases is write an
elisp code block in a section that is tagged :noexport: and then I run that
block to build the result. A recent example was a proposal where I needed a
pdf of the body, and separate pdf of the references.

I have separate elisp functions that generate these, and then I added some
custom cleanup code in the block to delete some intermediate directories. I
don't want to put these in a makefile because they are specific to this
document. While this works, in a large document I find it a little
inconvenient to make a small change say at the top, and then to jump to the
bottom to run the build block to see how it changed. What I would prefer is
to just run a command like M-x org-compile that would know about this build
block and run it. That block could of course be a shell block that runs a
makefile, but it would most often be an elisp block. I could even imagine
that there is a makefile block that is tangled before running a shell block
that runs a make command.

What I do in a function now is something like this in a save-excursion:

(when (not (stringp (org-babel-goto-named-src-block "build")))
      (org-babel-execute-src-block))

I don't use this in these projects, but they highlight some of the
complexities I am trying to simplify. These are book like projects with
special formatting needs, and multiple outputs.

In this project (
https://github.com/jkitchin/pycse/blob/master/pycse.org#L15096), I have
multiple output targets that I would run.

In this project (
https://github.com/jkitchin/dft-book/blob/master/dft.org#build) I use some
temporary filters to save the src blocks to files, and to embed them in the
pdf so they can be opened.

Anyway, it feels like I am reinventing something here, and that there might
be some better approach already out there. Maybe some elisp equivalent of a
makefile or something?

Thoughts?

John

-----------------------------------
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu

[-- Attachment #2: Type: text/html, Size: 3405 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: emacs build command for org-files
  2020-01-26 18:10 emacs build command for org-files John Kitchin
@ 2020-01-26 19:38 ` briangpowell .
  2020-01-26 20:05   ` John Kitchin
  2020-01-26 21:13   ` briangpowell .
  2020-01-27  8:16 ` Stefan Nobis
  2020-01-27 12:40 ` Neil Jerram
  2 siblings, 2 replies; 11+ messages in thread
From: briangpowell . @ 2020-01-26 19:38 UTC (permalink / raw)
  To: John Kitchin; +Cc: org-mode-email

[-- Attachment #1: Type: text/plain, Size: 4054 bytes --]

"don't want it to necessarily use Makefiles"

Why the hey not Dr. Kitchin!?

Make is an extremely powerful language

Problem is people make makefiles that are often hard for others to
read--suggest you try hard to keep it simple & avoid the fancy ways of
doing things

Make has flow control & is a very rich language

I use makefiles for every project Then Ccc to compile and/or run
everything--in Emacs--can't get easier and faster than that--as long as you
make the makefiles so easy that anyone can read & understand exactly what
it does

This is my default, which I may edit on-the-fly and just put in another
project name:

make --ignore-errors --jobs=555 -w --keep-going --warn-undefined-variables
--environment-overrides -f ~/n/n/etc/1cv2tex2doc-project_mak.makefile
1cv2tex2doc-project-exe

All slightly interesting ways to improve make have failed slightly, in one
way or another--they catch on for a few years until they're mostly
abandoned--for very good reasons

Look at it this way: Is there anything wrong with make?  Where does it fail?

Everyone should learn & use make

--Brian G. Powell, M.S.--that guy that still loves Makefiles and still
loves using asterisks ("***") as bullets in OrgMode

On Sun, Jan 26, 2020 at 1:11 PM John Kitchin <jkitchin@andrew.cmu.edu>
wrote:

> Hi everyone,
>
> This is only semi-on-topic. I am looking for something like M-x compile
> for my org-files, but I don't want it to necessarily use Makefiles. I am
> looking for suggestions of existing solutions to this, or thoughts on how
> to implement this.
>
> Actually, if it was possible to get M-x compile to run an elisp function
> instead of a make file, it might be all I need, but it looks like it runs
> shell commands.
>
> For most of my files, simple exporting is totally adequate. But, some
> files are more complicated, and what I usually do in these cases is write
> an elisp code block in a section that is tagged :noexport: and then I run
> that block to build the result. A recent example was a proposal where I
> needed a pdf of the body, and separate pdf of the references.
>
> I have separate elisp functions that generate these, and then I added some
> custom cleanup code in the block to delete some intermediate directories. I
> don't want to put these in a makefile because they are specific to this
> document. While this works, in a large document I find it a little
> inconvenient to make a small change say at the top, and then to jump to the
> bottom to run the build block to see how it changed. What I would prefer is
> to just run a command like M-x org-compile that would know about this build
> block and run it. That block could of course be a shell block that runs a
> makefile, but it would most often be an elisp block. I could even imagine
> that there is a makefile block that is tangled before running a shell block
> that runs a make command.
>
> What I do in a function now is something like this in a save-excursion:
>
> (when (not (stringp (org-babel-goto-named-src-block "build")))
>       (org-babel-execute-src-block))
>
> I don't use this in these projects, but they highlight some of the
> complexities I am trying to simplify. These are book like projects with
> special formatting needs, and multiple outputs.
>
> In this project (
> https://github.com/jkitchin/pycse/blob/master/pycse.org#L15096), I have
> multiple output targets that I would run.
>
> In this project (
> https://github.com/jkitchin/dft-book/blob/master/dft.org#build) I use
> some temporary filters to save the src blocks to files, and to embed them
> in the pdf so they can be opened.
>
> Anyway, it feels like I am reinventing something here, and that there
> might be some better approach already out there. Maybe some elisp
> equivalent of a makefile or something?
>
> Thoughts?
>
> John
>
> -----------------------------------
> Professor John Kitchin
> Doherty Hall A207F
> Department of Chemical Engineering
> Carnegie Mellon University
> Pittsburgh, PA 15213
> 412-268-7803
> @johnkitchin
> http://kitchingroup.cheme.cmu.edu
>
>

[-- Attachment #2: Type: text/html, Size: 5274 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: emacs build command for org-files
  2020-01-26 19:38 ` briangpowell .
@ 2020-01-26 20:05   ` John Kitchin
  2020-01-26 21:13   ` briangpowell .
  1 sibling, 0 replies; 11+ messages in thread
From: John Kitchin @ 2020-01-26 20:05 UTC (permalink / raw)
  To: briangpowell .; +Cc: org-mode-email

[-- Attachment #1: Type: text/plain, Size: 6129 bytes --]

On Sun, Jan 26, 2020 at 2:39 PM briangpowell . <briangpowellms@gmail.com>
wrote:

> "don't want it to necessarily use Makefiles"
>
> Why the hey not Dr. Kitchin!?
>

1. I like all the build code in one file that also contains the document I
am building. I know I could put a makefile in a src block and tangle it
before running it, but make is not meeting my needs for this.
2. all the build code is mostly written in elisp
3. my emacs startup is slow and I don't use emacsclient so sequential build
commands that launch emacs with shell arguments are too slow compared to
the build sequences in src blocks.


>
> Make is an extremely powerful language
>

It is not as easy or readable for me to use as elisp, and probably not as
powerful. I don't like powerful programming in shell (or make) languages, I
prefer to do that in something like elisp or python where I my skills are
more developed and I have better tools for debugging.


>
> Problem is people make makefiles that are often hard for others to
> read--suggest you try hard to keep it simple & avoid the fancy ways of
> doing things
>

Sometimes you have fancy needs that require fancy ways of doing things.


>
> Make has flow control & is a very rich language
>

I don't really need flow control for this need.


> I use makefiles for every project Then Ccc to compile and/or run
> everything--in Emacs--can't get easier and faster than that--as long as you
> make the makefiles so easy that anyone can read & understand exactly what
> it does
>

When they are this easy, it is nearly equivalent to simply publishing the
org file to the target, so I am C-c C-e lo away from what I want.


>
> This is my default, which I may edit on-the-fly and just put in another
> project name:
>
> make --ignore-errors --jobs=555 -w --keep-going --warn-undefined-variables
> --environment-overrides -f ~/n/n/etc/1cv2tex2doc-project_mak.makefile
> 1cv2tex2doc-project-exe
>

This is exactly the kind of command I might put in a build source block. I
suppose you might save that as a file local variable so you can just M-x
compile away though.


>
> All slightly interesting ways to improve make have failed slightly, in one
> way or another--they catch on for a few years until they're mostly
> abandoned--for very good reasons
>
> Look at it this way: Is there anything wrong with make?  Where does it
> fail?
>

Make is perfectly fine for sequences of shell commands that operate on
files. With straight latex projects, make would be my goto.  With my org
files though, it is not a straight export to latex and carry on approach I
use. So, I am looking for something that is more aligned with sequences of
elisp commands on a buffer, and not in an asynchronous way. These command
sequences are usually customized on a per document basis, otherwise I would
already have wrapped them up into general export commands.


>
> Everyone should learn & use make
>

I know how to use make, and I don't like it for this purpose. The src
blocks serve my needs just fine, I am just looking for an easier way to
call them in large documents.


>
> --Brian G. Powell, M.S.--that guy that still loves Makefiles and still
> loves using asterisks ("***") as bullets in OrgMode
>
> On Sun, Jan 26, 2020 at 1:11 PM John Kitchin <jkitchin@andrew.cmu.edu>
> wrote:
>
>> Hi everyone,
>>
>> This is only semi-on-topic. I am looking for something like M-x compile
>> for my org-files, but I don't want it to necessarily use Makefiles. I am
>> looking for suggestions of existing solutions to this, or thoughts on how
>> to implement this.
>>
>> Actually, if it was possible to get M-x compile to run an elisp function
>> instead of a make file, it might be all I need, but it looks like it runs
>> shell commands.
>>
>> For most of my files, simple exporting is totally adequate. But, some
>> files are more complicated, and what I usually do in these cases is write
>> an elisp code block in a section that is tagged :noexport: and then I run
>> that block to build the result. A recent example was a proposal where I
>> needed a pdf of the body, and separate pdf of the references.
>>
>> I have separate elisp functions that generate these, and then I added
>> some custom cleanup code in the block to delete some intermediate
>> directories. I don't want to put these in a makefile because they are
>> specific to this document. While this works, in a large document I find it
>> a little inconvenient to make a small change say at the top, and then to
>> jump to the bottom to run the build block to see how it changed. What I
>> would prefer is to just run a command like M-x org-compile that would know
>> about this build block and run it. That block could of course be a shell
>> block that runs a makefile, but it would most often be an elisp block. I
>> could even imagine that there is a makefile block that is tangled before
>> running a shell block that runs a make command.
>>
>> What I do in a function now is something like this in a save-excursion:
>>
>> (when (not (stringp (org-babel-goto-named-src-block "build")))
>>       (org-babel-execute-src-block))
>>
>> I don't use this in these projects, but they highlight some of the
>> complexities I am trying to simplify. These are book like projects with
>> special formatting needs, and multiple outputs.
>>
>> In this project (
>> https://github.com/jkitchin/pycse/blob/master/pycse.org#L15096), I have
>> multiple output targets that I would run.
>>
>> In this project (
>> https://github.com/jkitchin/dft-book/blob/master/dft.org#build) I use
>> some temporary filters to save the src blocks to files, and to embed them
>> in the pdf so they can be opened.
>>
>> Anyway, it feels like I am reinventing something here, and that there
>> might be some better approach already out there. Maybe some elisp
>> equivalent of a makefile or something?
>>
>> Thoughts?
>>
>> John
>>
>> -----------------------------------
>> Professor John Kitchin
>> Doherty Hall A207F
>> Department of Chemical Engineering
>> Carnegie Mellon University
>> Pittsburgh, PA 15213
>> 412-268-7803
>> @johnkitchin
>> http://kitchingroup.cheme.cmu.edu
>>
>>

[-- Attachment #2: Type: text/html, Size: 9111 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: emacs build command for org-files
  2020-01-26 19:38 ` briangpowell .
  2020-01-26 20:05   ` John Kitchin
@ 2020-01-26 21:13   ` briangpowell .
  2020-01-27 15:53     ` John Kitchin
  1 sibling, 1 reply; 11+ messages in thread
From: briangpowell . @ 2020-01-26 21:13 UTC (permalink / raw)
  To: John Kitchin; +Cc: org-mode-email

[-- Attachment #1: Type: text/plain, Size: 6118 bytes --]

"Actually, if it was possible to get M-x compile to run an elisp function
instead of a make file, it might be all I need, but it looks like it runs
shell commands."

You probably know this but just in case:

"You can call simulate an EmacsLisp-based script by putting this in a batch
script:

emacs -batch -l ~/.emacs.editor --eval="(require 'foo)" \
--eval="(require 'bar)" \
--eval="(some-function $*)"
Starting with the Emacs 22, you can write Emacs scripts just as if you were
writing Bash or Perl scripts, when you include this at the top of your
script file:

#!/usr/bin/emacs --script'

--I mean, Elisp functions can be called as batch files from shell commands
called by make makefiles

But I believe you're looking for something better, Python is great, but, it
shouldn't be applied to every problem

"Python will be the Emacs Lisp of the 1990's"--Guido Van Rossum, in one of
his 1st meetings on his new language Python--I was there, I wrote down this
quote in an Emacs Lisp book

Python is a masterpiece developed at the Stichting Mathematisch Centrum in
Amsterdam, Holland--during a Christmas break

But Python is, strictly speaking, very similar to Lisp & ELisp

There are reasons why functional languages like Lisp, Python, Erlang,
Haskell, Clojure, etc. Pure Functional Languages, are still not fully
dominating--though Python is very popular

For such purposes, time reveals what's best--make makefiles & shell
programming will continue to be best for many things

Have you looked into emacs-lisp shell--i.e. the "eshell"?

--and I believe you can call the "eshell" in batch mode from a shell
command--and the "eshell" can execute emacs lisp functions of course

--and you can even do all that from the Python interactive shell if you
like--and call remote shells from it and/or use "IPython"
https://en.wikipedia.org/wiki/IPython





On Sun, Jan 26, 2020 at 2:38 PM briangpowell . <briangpowellms@gmail.com>
wrote:

> "don't want it to necessarily use Makefiles"
>
> Why the hey not Dr. Kitchin!?
>
> Make is an extremely powerful language
>
> Problem is people make makefiles that are often hard for others to
> read--suggest you try hard to keep it simple & avoid the fancy ways of
> doing things
>
> Make has flow control & is a very rich language
>
> I use makefiles for every project Then Ccc to compile and/or run
> everything--in Emacs--can't get easier and faster than that--as long as you
> make the makefiles so easy that anyone can read & understand exactly what
> it does
>
> This is my default, which I may edit on-the-fly and just put in another
> project name:
>
> make --ignore-errors --jobs=555 -w --keep-going --warn-undefined-variables
> --environment-overrides -f ~/n/n/etc/1cv2tex2doc-project_mak.makefile
> 1cv2tex2doc-project-exe
>
> All slightly interesting ways to improve make have failed slightly, in one
> way or another--they catch on for a few years until they're mostly
> abandoned--for very good reasons
>
> Look at it this way: Is there anything wrong with make?  Where does it
> fail?
>
> Everyone should learn & use make
>
> --Brian G. Powell, M.S.--that guy that still loves Makefiles and still
> loves using asterisks ("***") as bullets in OrgMode
>
> On Sun, Jan 26, 2020 at 1:11 PM John Kitchin <jkitchin@andrew.cmu.edu>
> wrote:
>
>> Hi everyone,
>>
>> This is only semi-on-topic. I am looking for something like M-x compile
>> for my org-files, but I don't want it to necessarily use Makefiles. I am
>> looking for suggestions of existing solutions to this, or thoughts on how
>> to implement this.
>>
>> Actually, if it was possible to get M-x compile to run an elisp function
>> instead of a make file, it might be all I need, but it looks like it runs
>> shell commands.
>>
>> For most of my files, simple exporting is totally adequate. But, some
>> files are more complicated, and what I usually do in these cases is write
>> an elisp code block in a section that is tagged :noexport: and then I run
>> that block to build the result. A recent example was a proposal where I
>> needed a pdf of the body, and separate pdf of the references.
>>
>> I have separate elisp functions that generate these, and then I added
>> some custom cleanup code in the block to delete some intermediate
>> directories. I don't want to put these in a makefile because they are
>> specific to this document. While this works, in a large document I find it
>> a little inconvenient to make a small change say at the top, and then to
>> jump to the bottom to run the build block to see how it changed. What I
>> would prefer is to just run a command like M-x org-compile that would know
>> about this build block and run it. That block could of course be a shell
>> block that runs a makefile, but it would most often be an elisp block. I
>> could even imagine that there is a makefile block that is tangled before
>> running a shell block that runs a make command.
>>
>> What I do in a function now is something like this in a save-excursion:
>>
>> (when (not (stringp (org-babel-goto-named-src-block "build")))
>>       (org-babel-execute-src-block))
>>
>> I don't use this in these projects, but they highlight some of the
>> complexities I am trying to simplify. These are book like projects with
>> special formatting needs, and multiple outputs.
>>
>> In this project (
>> https://github.com/jkitchin/pycse/blob/master/pycse.org#L15096), I have
>> multiple output targets that I would run.
>>
>> In this project (
>> https://github.com/jkitchin/dft-book/blob/master/dft.org#build) I use
>> some temporary filters to save the src blocks to files, and to embed them
>> in the pdf so they can be opened.
>>
>> Anyway, it feels like I am reinventing something here, and that there
>> might be some better approach already out there. Maybe some elisp
>> equivalent of a makefile or something?
>>
>> Thoughts?
>>
>> John
>>
>> -----------------------------------
>> Professor John Kitchin
>> Doherty Hall A207F
>> Department of Chemical Engineering
>> Carnegie Mellon University
>> Pittsburgh, PA 15213
>> 412-268-7803
>> @johnkitchin
>> http://kitchingroup.cheme.cmu.edu
>>
>>

[-- Attachment #2: Type: text/html, Size: 8064 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: emacs build command for org-files
  2020-01-26 18:10 emacs build command for org-files John Kitchin
  2020-01-26 19:38 ` briangpowell .
@ 2020-01-27  8:16 ` Stefan Nobis
  2020-01-27 13:33   ` John Kitchin
  2020-01-27 12:40 ` Neil Jerram
  2 siblings, 1 reply; 11+ messages in thread
From: Stefan Nobis @ 2020-01-27  8:16 UTC (permalink / raw)
  To: emacs-orgmode

John Kitchin <jkitchin@andrew.cmu.edu> writes:

> Hi everyone,

> This is only semi-on-topic. I am looking for something like M-x compile for
> my org-files, but I don't want it to necessarily use Makefiles. I am
> looking for suggestions of existing solutions to this, or thoughts on how
> to implement this.

This may not be the solution you are looking for, but maybe a good
source of ideas:

   https://github.com/doublep/eldev

Another idea: Just use a (configurable) function name or source block
name to look for in a document. Then some magic function (say
org-compile-document) can look for a custom function/block inside the
document (e.g. look for a marked source block) and execute it, if
found. If no custom function/block is found, some default action will
be executed (e.g. ask user what to do, run pre-configured default
export action etc.).

-- 
Until the next mail...,
Stefan.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: emacs build command for org-files
  2020-01-26 18:10 emacs build command for org-files John Kitchin
  2020-01-26 19:38 ` briangpowell .
  2020-01-27  8:16 ` Stefan Nobis
@ 2020-01-27 12:40 ` Neil Jerram
  2020-01-27 13:16   ` John Kitchin
  2 siblings, 1 reply; 11+ messages in thread
From: Neil Jerram @ 2020-01-27 12:40 UTC (permalink / raw)
  To: John Kitchin; +Cc: org-mode-email

[-- Attachment #1: Type: text/plain, Size: 664 bytes --]

On Sun, 26 Jan 2020 at 18:11, John Kitchin <jkitchin@andrew.cmu.edu> wrote:

> Hi everyone,
>
> This is only semi-on-topic. I am looking for something like M-x compile
> for my org-files, but I don't want it to necessarily use Makefiles. I am
> looking for suggestions of existing solutions to this, or thoughts on how
> to implement this.
>
> Actually, if it was possible to get M-x compile to run an elisp function
> instead of a make file, it might be all I need, but it looks like it runs
> shell commands.
>

So how about typing "M-: <elisp>" instead of "M-x compile" ?  Why involve
other layers when Emacs can directly evaluate elisp?

Best wishes,
    Neil

[-- Attachment #2: Type: text/html, Size: 1202 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: emacs build command for org-files
  2020-01-27 12:40 ` Neil Jerram
@ 2020-01-27 13:16   ` John Kitchin
  0 siblings, 0 replies; 11+ messages in thread
From: John Kitchin @ 2020-01-27 13:16 UTC (permalink / raw)
  To: Neil Jerram; +Cc: org-mode-email


Neil Jerram <neiljerram@gmail.com> writes:

> On Sun, 26 Jan 2020 at 18:11, John Kitchin <jkitchin@andrew.cmu.edu> wrote:
>
>> Hi everyone,
>>
>> This is only semi-on-topic. I am looking for something like M-x compile
>> for my org-files, but I don't want it to necessarily use Makefiles. I am
>> looking for suggestions of existing solutions to this, or thoughts on how
>> to implement this.
>>
>> Actually, if it was possible to get M-x compile to run an elisp function
>> instead of a make file, it might be all I need, but it looks like it runs
>> shell commands.
>>
>
> So how about typing "M-: <elisp>" instead of "M-x compile" ?  Why involve
> other layers when Emacs can directly evaluate elisp?

It is just a reflection of how long some of the build blocks are. They
frequently let-bind variables and functions to achieve some particular
goal for the specific project. Since it is a one-time block, I don't
usually make it a function, and the code exists in a src block. "M-:"
isn't convenient for me in this scenario.

>
> Best wishes,
>     Neil


--
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: emacs build command for org-files
  2020-01-27  8:16 ` Stefan Nobis
@ 2020-01-27 13:33   ` John Kitchin
  0 siblings, 0 replies; 11+ messages in thread
From: John Kitchin @ 2020-01-27 13:33 UTC (permalink / raw)
  To: emacs-orgmode


Stefan Nobis <stefan-ml@snobis.de> writes:

> John Kitchin <jkitchin@andrew.cmu.edu> writes:
>
>> Hi everyone,
>
>> This is only semi-on-topic. I am looking for something like M-x compile for
>> my org-files, but I don't want it to necessarily use Makefiles. I am
>> looking for suggestions of existing solutions to this, or thoughts on how
>> to implement this.
>
> This may not be the solution you are looking for, but maybe a good
> source of ideas:
>
>    https://github.com/doublep/eldev

Thanks for this tip. This is the kind of thing I was hoping this thread
would unearth. This seems kind of like rake from ruby, where the build
scripts are written in the language you are building from. That is more
aligned with what I am trying to do here.

>
> Another idea: Just use a (configurable) function name or source block
> name to look for in a document. Then some magic function (say
> org-compile-document) can look for a custom function/block inside the
> document (e.g. look for a marked source block) and execute it, if
> found. If no custom function/block is found, some default action will
> be executed (e.g. ask user what to do, run pre-configured default
> export action etc.).

This is what I do now. I thought (hoped?) maybe someone else had worked
out something similar, and had a better idea.


--
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: emacs build command for org-files
  2020-01-26 21:13   ` briangpowell .
@ 2020-01-27 15:53     ` John Kitchin
  2020-01-27 17:38       ` Jack Kamm
  0 siblings, 1 reply; 11+ messages in thread
From: John Kitchin @ 2020-01-27 15:53 UTC (permalink / raw)
  To: briangpowell .; +Cc: org-mode-email


briangpowell . <briangpowellms@gmail.com> writes:

> "Actually, if it was possible to get M-x compile to run an elisp function
> instead of a make file, it might be all I need, but it looks like it runs
> shell commands."
>
> You probably know this but just in case:
>
> "You can call simulate an EmacsLisp-based script by putting this in a batch
> script:
>
> emacs -batch -l ~/.emacs.editor --eval="(require 'foo)" \
> --eval="(require 'bar)" \
> --eval="(some-function $*)"
> Starting with the Emacs 22, you can write Emacs scripts just as if you were
> writing Bash or Perl scripts, when you include this at the top of your
> script file:
>
> #!/usr/bin/emacs --script'
>
> --I mean, Elisp functions can be called as batch files from shell commands
> called by make makefiles

I have tried this before, but my emacs startup time is pretty slow, so
this takes longer than running a src block, and there is no shared state
between commands. With src blocks, there can be shared state, e.g. one
block computes something another uses.

If anyone is interested in how to use emacs in scripts, see http://kitchingroup.cheme.cmu.edu/blog/2014/08/11/Using-org-mode-outside-of-Emacs-sort-of/

It just doesn't make sense to me to have to write an emacs shell script/command
like that, when there is already a src block with just the elisp
commands in them that does just that. In that post, I show how you could
call that block from a shell command, but I don't think that is really
what I want.

What I am doing is more aligned with the org-publish capability,
where you effectively can have targets with specific directions on how
to build each component. You can publish one component, or one file, or
the whole project. It has at least some primitive checking of
whether an org-file is newer than the output file expected. I have to
confess I have struggled to get a project defined that does what I need
here though.

For example, in my proposal project, I have to submit a pdf of the body
with no references in it, and a pdf of just the references. That is done
by two separate exports. One is straightforward, it just uses
\nobibliography to get just the body with all the correct citations, but
no references at the end. That step generates a .bbl file that is used
in the second export, where I remove the body in a copy of the generated
tex file, insert the contents of the bbl file, and then build the pdf
from that. I don't know of a more straightforward way to do this.

>
> But I believe you're looking for something better, Python is great, but, it
> shouldn't be applied to every problem

I am just looking for another way to use src blocks in org-mode to build
documents that require complicated manipulations, powered by elisp. I am
not advocating for or suggesting Python be used for every build problem.

It is a preference to not add more files, e.g. by tangling the src
blocks just so I can use make. Even when I have done this, it is always
triggered by running a src block in an org file that first tangles
everything, and then runs a shell command. That is usually for literate
programming projects though, where the desired output is compiled code.

>
> "Python will be the Emacs Lisp of the 1990's"--Guido Van Rossum, in one of
> his 1st meetings on his new language Python--I was there, I wrote down this
> quote in an Emacs Lisp book
>
> Python is a masterpiece developed at the Stichting Mathematisch Centrum in
> Amsterdam, Holland--during a Christmas break
>
> But Python is, strictly speaking, very similar to Lisp & ELisp

It isn't close in the sense that in a Python shell, you don't
create buffers (or anything like them), and then use Python functions to
transform them to other forms.

Emacs is practically a lisp machine, where you can do just that, for
better or worse.

>
> There are reasons why functional languages like Lisp, Python, Erlang,
> Haskell, Clojure, etc. Pure Functional Languages, are still not fully
> dominating--though Python is very popular
>
> For such purposes, time reveals what's best--make makefiles & shell
> programming will continue to be best for many things

I agree this might be best for compiling code via command line programs
that were built with a pipe philosophy, and where shared state is either
not needed, or is communicated via files.

I am not yet convinced it is necessary to think of building org-files
that way just because it works so well for code. You can do this all
within the elisp environment where you are editing the org file. Once
that works, why abstract it out to a shell command? Why not have some
easy way to just run the blocks that build what you want?

make/shell programs fall flat on their face in the following way: In an
elisp script we have the finest tooling around to see the docstrings of
any function, definitions of variables, to navigate to function
definitions, and to debug and step through the code. This makes this
approach so superior to anything else in my opinion, it is hard to argue
any other feature of make/shell is worth giving it up for. I will
concede this is just a better tooling argument though, but these tools
do not exist as far as I know in a shell. You might get tab-completion,
but not easy access to documentation and code. If there was a lispy
version of make that made this possible, I would use it.

Anyway, I am not arguing make is not good for what it does. It is fine
for that. It isn't what I want to use for this though.

>
> Have you looked into emacs-lisp shell--i.e. the "eshell"?
>
> --and I believe you can call the "eshell" in batch mode from a shell
> command--and the "eshell" can execute emacs lisp functions of course
>
> --and you can even do all that from the Python interactive shell if you
> like--and call remote shells from it and/or use "IPython"
> https://en.wikipedia.org/wiki/IPython
>
>
>
>
>
> On Sun, Jan 26, 2020 at 2:38 PM briangpowell . <briangpowellms@gmail.com>
> wrote:
>
>> "don't want it to necessarily use Makefiles"
>>
>> Why the hey not Dr. Kitchin!?
>>
>> Make is an extremely powerful language
>>
>> Problem is people make makefiles that are often hard for others to
>> read--suggest you try hard to keep it simple & avoid the fancy ways of
>> doing things
>>
>> Make has flow control & is a very rich language
>>
>> I use makefiles for every project Then Ccc to compile and/or run
>> everything--in Emacs--can't get easier and faster than that--as long as you
>> make the makefiles so easy that anyone can read & understand exactly what
>> it does
>>
>> This is my default, which I may edit on-the-fly and just put in another
>> project name:
>>
>> make --ignore-errors --jobs=555 -w --keep-going --warn-undefined-variables
>> --environment-overrides -f ~/n/n/etc/1cv2tex2doc-project_mak.makefile
>> 1cv2tex2doc-project-exe
>>
>> All slightly interesting ways to improve make have failed slightly, in one
>> way or another--they catch on for a few years until they're mostly
>> abandoned--for very good reasons
>>
>> Look at it this way: Is there anything wrong with make?  Where does it
>> fail?
>>
>> Everyone should learn & use make
>>
>> --Brian G. Powell, M.S.--that guy that still loves Makefiles and still
>> loves using asterisks ("***") as bullets in OrgMode
>>
>> On Sun, Jan 26, 2020 at 1:11 PM John Kitchin <jkitchin@andrew.cmu.edu>
>> wrote:
>>
>>> Hi everyone,
>>>
>>> This is only semi-on-topic. I am looking for something like M-x compile
>>> for my org-files, but I don't want it to necessarily use Makefiles. I am
>>> looking for suggestions of existing solutions to this, or thoughts on how
>>> to implement this.
>>>
>>> Actually, if it was possible to get M-x compile to run an elisp function
>>> instead of a make file, it might be all I need, but it looks like it runs
>>> shell commands.
>>>
>>> For most of my files, simple exporting is totally adequate. But, some
>>> files are more complicated, and what I usually do in these cases is write
>>> an elisp code block in a section that is tagged :noexport: and then I run
>>> that block to build the result. A recent example was a proposal where I
>>> needed a pdf of the body, and separate pdf of the references.
>>>
>>> I have separate elisp functions that generate these, and then I added
>>> some custom cleanup code in the block to delete some intermediate
>>> directories. I don't want to put these in a makefile because they are
>>> specific to this document. While this works, in a large document I find it
>>> a little inconvenient to make a small change say at the top, and then to
>>> jump to the bottom to run the build block to see how it changed. What I
>>> would prefer is to just run a command like M-x org-compile that would know
>>> about this build block and run it. That block could of course be a shell
>>> block that runs a makefile, but it would most often be an elisp block. I
>>> could even imagine that there is a makefile block that is tangled before
>>> running a shell block that runs a make command.
>>>
>>> What I do in a function now is something like this in a save-excursion:
>>>
>>> (when (not (stringp (org-babel-goto-named-src-block "build")))
>>>       (org-babel-execute-src-block))
>>>
>>> I don't use this in these projects, but they highlight some of the
>>> complexities I am trying to simplify. These are book like projects with
>>> special formatting needs, and multiple outputs.
>>>
>>> In this project (
>>> https://github.com/jkitchin/pycse/blob/master/pycse.org#L15096), I have
>>> multiple output targets that I would run.
>>>
>>> In this project (
>>> https://github.com/jkitchin/dft-book/blob/master/dft.org#build) I use
>>> some temporary filters to save the src blocks to files, and to embed them
>>> in the pdf so they can be opened.
>>>
>>> Anyway, it feels like I am reinventing something here, and that there
>>> might be some better approach already out there. Maybe some elisp
>>> equivalent of a makefile or something?
>>>
>>> Thoughts?
>>>
>>> John
>>>
>>> -----------------------------------
>>> Professor John Kitchin
>>> Doherty Hall A207F
>>> Department of Chemical Engineering
>>> Carnegie Mellon University
>>> Pittsburgh, PA 15213
>>> 412-268-7803
>>> @johnkitchin
>>> http://kitchingroup.cheme.cmu.edu
>>>
>>>


--
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: emacs build command for org-files
  2020-01-27 15:53     ` John Kitchin
@ 2020-01-27 17:38       ` Jack Kamm
  2020-01-27 19:13         ` John Kitchin
  0 siblings, 1 reply; 11+ messages in thread
From: Jack Kamm @ 2020-01-27 17:38 UTC (permalink / raw)
  To: John Kitchin, briangpowell .; +Cc: org-mode-email

John Kitchin <jkitchin@andrew.cmu.edu> writes:

> If there was a lispy version of make that made this possible, I would
> use it.

This is an aside, but GNU Make can use Guile Scheme as an embedded
extension language:

https://www.gnu.org/software/make/manual/html_node/Guile-Integration.html

I don't think this solves any issues in this thread, but I think it's
pretty neat, and might scratch your itch if you're looking for a lispy
make.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: emacs build command for org-files
  2020-01-27 17:38       ` Jack Kamm
@ 2020-01-27 19:13         ` John Kitchin
  0 siblings, 0 replies; 11+ messages in thread
From: John Kitchin @ 2020-01-27 19:13 UTC (permalink / raw)
  To: Jack Kamm; +Cc: org-mode-email

that is also interesting to know, thanks!

Jack Kamm <jackkamm@gmail.com> writes:

> John Kitchin <jkitchin@andrew.cmu.edu> writes:
>
>> If there was a lispy version of make that made this possible, I would
>> use it.
>
> This is an aside, but GNU Make can use Guile Scheme as an embedded
> extension language:
>
> https://www.gnu.org/software/make/manual/html_node/Guile-Integration.html
>
> I don't think this solves any issues in this thread, but I think it's
> pretty neat, and might scratch your itch if you're looking for a lispy
> make.


--
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2020-01-27 19:14 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-26 18:10 emacs build command for org-files John Kitchin
2020-01-26 19:38 ` briangpowell .
2020-01-26 20:05   ` John Kitchin
2020-01-26 21:13   ` briangpowell .
2020-01-27 15:53     ` John Kitchin
2020-01-27 17:38       ` Jack Kamm
2020-01-27 19:13         ` John Kitchin
2020-01-27  8:16 ` Stefan Nobis
2020-01-27 13:33   ` John Kitchin
2020-01-27 12:40 ` Neil Jerram
2020-01-27 13:16   ` John Kitchin

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).