From mboxrd@z Thu Jan 1 00:00:00 1970 From: John Kitchin Subject: Re: emacs build command for org-files Date: Mon, 27 Jan 2020 10:53:23 -0500 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:52958) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iw6hd-0003IC-RF for emacs-orgmode@gnu.org; Mon, 27 Jan 2020 10:53:32 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iw6hb-0005tq-RU for emacs-orgmode@gnu.org; Mon, 27 Jan 2020 10:53:29 -0500 Received: from mail-qt1-x829.google.com ([2607:f8b0:4864:20::829]:44785) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iw6hb-0005tg-Mc for emacs-orgmode@gnu.org; Mon, 27 Jan 2020 10:53:27 -0500 Received: by mail-qt1-x829.google.com with SMTP id w8so7692361qts.11 for ; Mon, 27 Jan 2020 07:53:27 -0800 (PST) In-reply-to: List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane-mx.org@gnu.org Sender: "Emacs-orgmode" To: "briangpowell ." Cc: org-mode-email briangpowell . writes: > "Actually, if it was possible to get M-x compile to run an elisp function > instead of a make file, it might be all I need, but it looks like it runs > shell commands." > > You probably know this but just in case: > > "You can call simulate an EmacsLisp-based script by putting this in a batch > script: > > emacs -batch -l ~/.emacs.editor --eval="(require 'foo)" \ > --eval="(require 'bar)" \ > --eval="(some-function $*)" > Starting with the Emacs 22, you can write Emacs scripts just as if you were > writing Bash or Perl scripts, when you include this at the top of your > script file: > > #!/usr/bin/emacs --script' > > --I mean, Elisp functions can be called as batch files from shell commands > called by make makefiles I have tried this before, but my emacs startup time is pretty slow, so this takes longer than running a src block, and there is no shared state between commands. With src blocks, there can be shared state, e.g. one block computes something another uses. If anyone is interested in how to use emacs in scripts, see http://kitchingroup.cheme.cmu.edu/blog/2014/08/11/Using-org-mode-outside-of-Emacs-sort-of/ It just doesn't make sense to me to have to write an emacs shell script/command like that, when there is already a src block with just the elisp commands in them that does just that. In that post, I show how you could call that block from a shell command, but I don't think that is really what I want. What I am doing is more aligned with the org-publish capability, where you effectively can have targets with specific directions on how to build each component. You can publish one component, or one file, or the whole project. It has at least some primitive checking of whether an org-file is newer than the output file expected. I have to confess I have struggled to get a project defined that does what I need here though. For example, in my proposal project, I have to submit a pdf of the body with no references in it, and a pdf of just the references. That is done by two separate exports. One is straightforward, it just uses \nobibliography to get just the body with all the correct citations, but no references at the end. That step generates a .bbl file that is used in the second export, where I remove the body in a copy of the generated tex file, insert the contents of the bbl file, and then build the pdf from that. I don't know of a more straightforward way to do this. > > But I believe you're looking for something better, Python is great, but, it > shouldn't be applied to every problem I am just looking for another way to use src blocks in org-mode to build documents that require complicated manipulations, powered by elisp. I am not advocating for or suggesting Python be used for every build problem. It is a preference to not add more files, e.g. by tangling the src blocks just so I can use make. Even when I have done this, it is always triggered by running a src block in an org file that first tangles everything, and then runs a shell command. That is usually for literate programming projects though, where the desired output is compiled code. > > "Python will be the Emacs Lisp of the 1990's"--Guido Van Rossum, in one of > his 1st meetings on his new language Python--I was there, I wrote down this > quote in an Emacs Lisp book > > Python is a masterpiece developed at the Stichting Mathematisch Centrum in > Amsterdam, Holland--during a Christmas break > > But Python is, strictly speaking, very similar to Lisp & ELisp It isn't close in the sense that in a Python shell, you don't create buffers (or anything like them), and then use Python functions to transform them to other forms. Emacs is practically a lisp machine, where you can do just that, for better or worse. > > There are reasons why functional languages like Lisp, Python, Erlang, > Haskell, Clojure, etc. Pure Functional Languages, are still not fully > dominating--though Python is very popular > > For such purposes, time reveals what's best--make makefiles & shell > programming will continue to be best for many things I agree this might be best for compiling code via command line programs that were built with a pipe philosophy, and where shared state is either not needed, or is communicated via files. I am not yet convinced it is necessary to think of building org-files that way just because it works so well for code. You can do this all within the elisp environment where you are editing the org file. Once that works, why abstract it out to a shell command? Why not have some easy way to just run the blocks that build what you want? make/shell programs fall flat on their face in the following way: In an elisp script we have the finest tooling around to see the docstrings of any function, definitions of variables, to navigate to function definitions, and to debug and step through the code. This makes this approach so superior to anything else in my opinion, it is hard to argue any other feature of make/shell is worth giving it up for. I will concede this is just a better tooling argument though, but these tools do not exist as far as I know in a shell. You might get tab-completion, but not easy access to documentation and code. If there was a lispy version of make that made this possible, I would use it. Anyway, I am not arguing make is not good for what it does. It is fine for that. It isn't what I want to use for this though. > > Have you looked into emacs-lisp shell--i.e. the "eshell"? > > --and I believe you can call the "eshell" in batch mode from a shell > command--and the "eshell" can execute emacs lisp functions of course > > --and you can even do all that from the Python interactive shell if you > like--and call remote shells from it and/or use "IPython" > https://en.wikipedia.org/wiki/IPython > > > > > > On Sun, Jan 26, 2020 at 2:38 PM briangpowell . > wrote: > >> "don't want it to necessarily use Makefiles" >> >> Why the hey not Dr. Kitchin!? >> >> Make is an extremely powerful language >> >> Problem is people make makefiles that are often hard for others to >> read--suggest you try hard to keep it simple & avoid the fancy ways of >> doing things >> >> Make has flow control & is a very rich language >> >> I use makefiles for every project Then Ccc to compile and/or run >> everything--in Emacs--can't get easier and faster than that--as long as you >> make the makefiles so easy that anyone can read & understand exactly what >> it does >> >> This is my default, which I may edit on-the-fly and just put in another >> project name: >> >> make --ignore-errors --jobs=555 -w --keep-going --warn-undefined-variables >> --environment-overrides -f ~/n/n/etc/1cv2tex2doc-project_mak.makefile >> 1cv2tex2doc-project-exe >> >> All slightly interesting ways to improve make have failed slightly, in one >> way or another--they catch on for a few years until they're mostly >> abandoned--for very good reasons >> >> Look at it this way: Is there anything wrong with make? Where does it >> fail? >> >> Everyone should learn & use make >> >> --Brian G. Powell, M.S.--that guy that still loves Makefiles and still >> loves using asterisks ("***") as bullets in OrgMode >> >> On Sun, Jan 26, 2020 at 1:11 PM John Kitchin >> wrote: >> >>> Hi everyone, >>> >>> This is only semi-on-topic. I am looking for something like M-x compile >>> for my org-files, but I don't want it to necessarily use Makefiles. I am >>> looking for suggestions of existing solutions to this, or thoughts on how >>> to implement this. >>> >>> Actually, if it was possible to get M-x compile to run an elisp function >>> instead of a make file, it might be all I need, but it looks like it runs >>> shell commands. >>> >>> For most of my files, simple exporting is totally adequate. But, some >>> files are more complicated, and what I usually do in these cases is write >>> an elisp code block in a section that is tagged :noexport: and then I run >>> that block to build the result. A recent example was a proposal where I >>> needed a pdf of the body, and separate pdf of the references. >>> >>> I have separate elisp functions that generate these, and then I added >>> some custom cleanup code in the block to delete some intermediate >>> directories. I don't want to put these in a makefile because they are >>> specific to this document. While this works, in a large document I find it >>> a little inconvenient to make a small change say at the top, and then to >>> jump to the bottom to run the build block to see how it changed. What I >>> would prefer is to just run a command like M-x org-compile that would know >>> about this build block and run it. That block could of course be a shell >>> block that runs a makefile, but it would most often be an elisp block. I >>> could even imagine that there is a makefile block that is tangled before >>> running a shell block that runs a make command. >>> >>> What I do in a function now is something like this in a save-excursion: >>> >>> (when (not (stringp (org-babel-goto-named-src-block "build"))) >>> (org-babel-execute-src-block)) >>> >>> I don't use this in these projects, but they highlight some of the >>> complexities I am trying to simplify. These are book like projects with >>> special formatting needs, and multiple outputs. >>> >>> In this project ( >>> https://github.com/jkitchin/pycse/blob/master/pycse.org#L15096), I have >>> multiple output targets that I would run. >>> >>> In this project ( >>> https://github.com/jkitchin/dft-book/blob/master/dft.org#build) I use >>> some temporary filters to save the src blocks to files, and to embed them >>> in the pdf so they can be opened. >>> >>> Anyway, it feels like I am reinventing something here, and that there >>> might be some better approach already out there. Maybe some elisp >>> equivalent of a makefile or something? >>> >>> Thoughts? >>> >>> John >>> >>> ----------------------------------- >>> Professor John Kitchin >>> Doherty Hall A207F >>> Department of Chemical Engineering >>> Carnegie Mellon University >>> Pittsburgh, PA 15213 >>> 412-268-7803 >>> @johnkitchin >>> http://kitchingroup.cheme.cmu.edu >>> >>> -- Professor John Kitchin Doherty Hall A207F Department of Chemical Engineering Carnegie Mellon University Pittsburgh, PA 15213 412-268-7803 @johnkitchin http://kitchingroup.cheme.cmu.edu