Re: [PATCH] ob-maxima.el, etc. (was Re: [MAINTENANCE] On how much we can expose internals into defcustom)

[Leo Butler <Leo.Butler@umanitoba.ca>]

[-- Attachment #1: Type: text/plain, Size: 4421 bytes --]

On Wed, Sep 20 2023, Ihor Radchenko <yantar92@posteo.net> wrote:

> Leo Butler <Leo.Butler@umanitoba.ca> writes:
>
>>>>> May you clarify the purpose of "linenum"?
>>> Do I understand correctly that the above will simply affect debug output
>>> when maxima references where a problematic line is located in the source?
>>
>> No, it affects how output labels are printed. With this value, the
>> "first" line in the source block receives the line number 1; without it,
>> it would get 2.
>
> Ok. This should be documented in the commit message and possibly in the
> code itself.

I think it makes most sense to put in a source-code comment, explaining
the why. That is what I have done.

>
>>>
>>>>>>                                (unless (or (string-match "batch" line)
>>>>>>                                            (string-match "^rat: replaced .*$" line)
>>>>>>                                            (string-match "^;;; Loading #P" line)
>>>>>> +                                          (string-match "^read and interpret" line)
>>>>>> +                                          (string-match "^(%\\([io]-?[0-9]+\\))[ ]+$" line)
>>>>>
>>>>> May you explain why you added these two conditions?
>>>>
>>
>> <snip>
>>
>>>
>>> May empty lines be intentional in some maxima code?
>>
>> A blank line is simply skipped by the maxima reader; an empty input line
>> is a syntax error.
>>
>> However, "empty" output may appear, so I have modified that filter.
>
>        +                                          (string-match "^(%\\([io]-?[0-9]+\\))[ ]+$" line)
>
> It would be nice to add a comment line explaining what this regexp
> matches against.

Ok. I think it makes the most sense to explain the purpose of each
regexp, new and old, and to make this somewhat extensible.

What I have done is cut out that undocumented filter, put the regexps
into a new variable (`org-babel-maxima--output-filter-regexps'), added
source-code comments to explain what each one does and added a new
function (`org-babel-maxima--output-filter') that replaces the old
filter in `org-babel-execute:maxima'.


>
>> * (org-babel-maxima--graphic-package-options): an alist of Maxima
>> graphics packages and the Maxima code to set up that package.
>
> This is a bit confusing. I'd say "A new variable storing alist of Maxima ..."
>
>> * (org-babel-maxima--default-epilogue): an alist of the clean-up code
>> that is run at end of a `graphical-output' or `non-graphical-output'
>> source block.
>
> Same here.
> Basically, it should be clear from the commit message when you add new
> variables and when you modify existing.

Points taken.

>
>> * (org-babel-maxima-expand): prepare the source block for
>> execution.
>
> This is vague. Please explain what is changed in more details.

Done.

>
>> * (org-babel-execute:maxima): use the :batch header argument and
>> `org-babel-maxima--command-arguments-default' to execute the source
>> block.  Add a couple extra regexps to filter the output of a batch-ed
>> source block.
>
> This reads better compared to the above.
>
>> +(defconst org-babel-header-args:maxima
>> +  '((batch               . :any)
>
> Why :any? Only two values are allowed here.
>
>> +    (graphics-pkg        . :any))
>
> Same. The value is a closed list.

I have made the change requested.

However, beyond the documentation value, I don't see this variable's
values being used.

>
>> +(defvar org-babel-maxima--command-arguments-default
>> +  "--very-quiet"
>> +  "Command-line arguments sent to Maxima by default.  If the
>
> Please keep the first line as a single sentence.
> See https://www.gnu.org/software/emacs/manual/html_node/elisp/Documentation-Tips.html
>
>> +(defvar org-babel-maxima--default-epilogue
>> +  '((graphical-output . "gnuplot_close ()$")
>> +    (non-graphical-output . ""))
>> +  "The final Maxima code executed in a source block.  An alist with
>
> Same problem with the first line.

Fixed. I don't use `apropos', so I didn't see the problem (and I had
failed to absorb everything in §D.6).

Please see the attached, revised patch. It fixes the problems that you
have identified–except for the way that the gnuplot terminal is
determined. I would like to leave that for a subsequent patch, mainly
because this one has become quite large.

Best regards,
Leo


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-lisp-ob-maxima.el-enable-use-of-batch-loader-and-dra.patch --]
[-- Type: text/x-diff; name="0001-lisp-ob-maxima.el-enable-use-of-batch-loader-and-dra.patch", Size: 17046 bytes --]

From 078e29f13f15305e291b1c0e46e306d633a8b93a Mon Sep 17 00:00:00 2001
From: Leo Butler <leo.butler@umanitoba.ca>
Date: Tue, 19 Sep 2023 13:36:06 -0500
Subject: [PATCH] * lisp/ob-maxima.el: enable use of `batch' loader and `draw'

* (org-babel-header-args:maxima): Document the two new header
arguments (batch and graphics-pkg) that are specific to ob-maxima.
* (org-babel-maxima--command-arguments-default): A new variable
storing the default command-line argument(s).  This value was
hard-coded in `org-babel-maxima:execute'.
* (org-babel-maxima--graphic-package-options): A new variable that
stores an alist of Maxima graphics packages and the Maxima code to set
up that package.
* (org-babel-maxima--default-epilogue): A new variable that stores an
alist of the clean-up code that is run at end of a `graphical-output'
or `non-graphical-output' source block.
* (org-babel-maxima--output-filter-regexps): A new variable that
stores a list of regexps to identify "bad" output lines to be removed
from Maxima's output.  Source-code comments document each regexp's
purpose.  Two additional regexps have been added: one filters
un-wanted output from `batch' and the other removes empty input lines
that `batch'-ed output may spuriously produce (actual empty input
lines are syntax errors, see the new tests below).
* (org-babel-maxima--output-filter): A new function that takes a
single line of Maxima output.  It returns nil if the line has zero
length or matches a regexp in
`org-babel-maxima--output-filter-regexps'; otherwise, it returns the
line.  This function and regexp replace the hard-coded filter in
`org-babel-execute:maxima'.
* (org-babel-maxima-expand): Prepare the source block for execution,
depending on whether it is producing graphical output or not.  In case
of graphical output, use the `graphics-pkg' header to set the graphics
package and use `org-babel-maxima--graphic-package-options' to set-up
the package.  Grovel the graphics terminal from the output filename.
* (org-babel-execute:maxima): Use the :batch header argument and
`org-babel-maxima--command-arguments-default' to execute the source
block.  Replace the existing, in-line output filter and its regexps
with `org-babel-maxima--output-filter' and
`org-babel-maxima--output-filter-regexps'.

* testing/examples/ob-maxima-test.org: Add test examples.

Include examples of the batch-related tests from
testing/lisp/test-ob-maxima.el.  Provide an example of the
`:graphics-pkg' header argument with the `draw' package.

* testing/lisp/test-ob-maxima.el: Introduce six new, batch-related
test function.  Each test exercises the :batch header argument.  The
response to unusual inputs is tested (empty strings, strings with just
whitespace, input with the `:lisp' reader, and two syntax-related
errors).
---
 lisp/ob-maxima.el                   | 115 +++++++++++++++++++++-------
 testing/examples/ob-maxima-test.org |  54 +++++++++++++
 testing/lisp/test-ob-maxima.el      | 107 ++++++++++++++++++++++++++
 3 files changed, 248 insertions(+), 28 deletions(-)

diff --git a/lisp/ob-maxima.el b/lisp/ob-maxima.el
index d1d7c7424..eac44ff9a 100644
--- a/lisp/ob-maxima.el
+++ b/lisp/ob-maxima.el
@@ -37,6 +37,11 @@
 
 (require 'ob)
 
+(defconst org-babel-header-args:maxima
+  '((batch               . (batchload batch load))
+    (graphics-pkg        . (plot draw)))
+  "Maxima-specific header arguments.")
+
 (defvar org-babel-tangle-lang-exts)
 (add-to-list 'org-babel-tangle-lang-exts '("maxima" . "max"))
 
@@ -48,31 +53,81 @@
   :group 'org-babel
   :type 'string)
 
+(defvar org-babel-maxima--command-arguments-default
+  "--very-quiet"
+  "Command-line arguments sent to Maxima by default.
+If the `:batch' header argument is set to `batchload' or unset,
+then the `:cmdline' header argument is appended to this default;
+otherwise, if the `:cmdline' argument is set, it over-rides this
+default.  See `org-babel-maxima-command' and
+`org-babel-execute:maxima'.")
+
+(defvar org-babel-maxima--graphic-package-options
+  '((plot . "(set_plot_option ('[gnuplot_term, %s]), set_plot_option ('[gnuplot_out_file, %S]))$")
+    (draw . "(load(draw), set_draw_defaults(terminal='%s,file_name=%S))$"))
+  "An alist of graphics packages and Maxima code.
+Each element is a cons (PACKAGE-NAME . FORMAT-STRING).
+FORMAT-STRING contains Maxima code to configure the graphics
+package; it must contain `%s' to set the terminal and `%S' to set
+the filename, in that order.  The default graphics package is
+`plot'; `draw' is also supported.  See
+`org-babel-maxima-expand'.")
+
+(defvar org-babel-maxima--default-epilogue
+  '((graphical-output . "gnuplot_close ()$")
+    (non-graphical-output . ""))
+  "The final Maxima code executed in a source block.
+An alist with the epilogue for graphical and non-graphical
+output.  See `org-babel-maxima-expand'.")
+
 (defun org-babel-maxima-expand (body params)
   "Expand a block of Maxima code according to its header arguments."
-  (let ((vars (org-babel--get-vars params))
-	(epilogue (cdr (assq :epilogue params)))
-	(prologue (cdr (assq :prologue params))))
+  (let* ((vars (org-babel--get-vars params))
+         (graphic-file (ignore-errors (org-babel-graphical-output-file params)))
+	 (epilogue (cdr (assq :epilogue params)))
+	 (prologue (cdr (assq :prologue params))))
     (mapconcat 'identity
-	       (list
-		;; Any code from the specified prologue at the start.
-		prologue
-		;; graphic output
-		(let ((graphic-file (ignore-errors (org-babel-graphical-output-file params))))
-		  (if graphic-file
-		      (format
-		       "set_plot_option ([gnuplot_term, png]); set_plot_option ([gnuplot_out_file, %S]);"
-		       graphic-file)
-		    ""))
-		;; variables
-		(mapconcat 'org-babel-maxima-var-to-maxima vars "\n")
-		;; body
-		body
-		;; Any code from the specified epilogue at the end.
-		epilogue
-		"gnuplot_close ()$")
+               (delq nil
+	             (list
+		      ;; Any code from the specified prologue at the start.
+		      prologue
+		      ;; graphic output
+		      (if graphic-file
+                          (let* ((graphics-pkg (intern (or (cdr (assq :graphics-pkg params)) "plot")))
+                                 (graphic-format-string (cdr (assq graphics-pkg org-babel-maxima--graphic-package-options)))
+                                 (graphic-terminal (file-name-extension graphic-file))
+                                 (graphic-file (if (eq graphics-pkg 'plot) graphic-file (file-name-sans-extension graphic-file))))
+                            (format graphic-format-string graphic-terminal graphic-file)))
+		      ;; variables
+		      (mapconcat 'org-babel-maxima-var-to-maxima vars "\n")
+		      ;; body
+		      body
+		      ;; Any code from the specified epilogue at the end.
+		      epilogue
+		      (if graphic-file
+                          (cdr (assq :graphical-output org-babel-maxima--default-epilogue))
+                        (cdr (assq :non-graphical-output org-babel-maxima--default-epilogue)))))
 	       "\n")))
 
+(defvar org-babel-maxima--output-filter-regexps
+  '("batch"                     ;; remove the `batch' or `batchload' line
+    "^rat: replaced .*$"        ;; remove notices from `rat'
+    "^;;; Loading #P"           ;; remove notices from the lisp implementation
+    "^read and interpret"       ;; remove notice from `batch'
+    "^(%\\([i]-?[0-9]+\\))[ ]$" ;; remove empty input lines from `batch'-ing
+    )
+  "Regexps to remove extraneous lines from Maxima's output.
+See `org-babel-maxima--output-filter'.")
+
+(defun org-babel-maxima--output-filter (line)
+  "Filter empty or undesired lines from Maxima output.
+Return nil if LINE is zero-length or it matches a regexp in
+`org-babel-maxima--output-filter'; otherwise, return LINE."
+  (unless (or (= 0 (length line))
+              (cl-some #'(lambda(r) (string-match r line))
+                       org-babel-maxima--output-filter-regexps))
+    line))
+
 (defun org-babel-execute:maxima (body params)
   "Execute a block of Maxima entries with org-babel.
 This function is called by `org-babel-execute-src-block'."
@@ -80,11 +135,20 @@ This function is called by `org-babel-execute-src-block'."
   (let ((result-params (split-string (or (cdr (assq :results params)) "")))
 	(result
 	 (let* ((cmdline (or (cdr (assq :cmdline params)) ""))
+                (batch/load (or (cdr (assq :batch params)) "batchload"))
+                (cmdline (if (or (equal cmdline "") (equal batch/load "batchload"))
+                             ;; legacy behaviour:
+                             ;; ensure that --very-quiet is on command-line by default
+                             (concat cmdline " " org-babel-maxima--command-arguments-default)
+                           ;; if using an alternate loader, :cmdline overwrites default
+                           cmdline))
 		(in-file (org-babel-temp-file "maxima-" ".max"))
-		(cmd (format "%s --very-quiet -r %s %s"
+ 		(cmd (format "%s -r %s %s"
 			     org-babel-maxima-command
                              (shell-quote-argument
-                              (format "batchload(%S)$" in-file))
+                              ;; bind linenum to 0 so the first line
+                              ;; of in-file has line number 1
+                              (format "(linenum:0, %s(%S))$" batch/load in-file))
                              cmdline)))
 	   (with-temp-file in-file (insert (org-babel-maxima-expand body params)))
 	   (message cmd)
@@ -93,12 +157,7 @@ This function is called by `org-babel-execute-src-block'."
              (mapconcat
               #'identity
               (delq nil
-                    (mapcar (lambda (line)
-                              (unless (or (string-match "batch" line)
-                                          (string-match "^rat: replaced .*$" line)
-                                          (string-match "^;;; Loading #P" line)
-                                          (= 0 (length line)))
-                                line))
+                    (mapcar #'org-babel-maxima--output-filter
                             (split-string raw "[\r\n]"))) "\n")))))
     (if (ignore-errors (org-babel-graphical-output-file params))
 	nil
diff --git a/testing/examples/ob-maxima-test.org b/testing/examples/ob-maxima-test.org
index b83114a4f..c7847d959 100644
--- a/testing/examples/ob-maxima-test.org
+++ b/testing/examples/ob-maxima-test.org
@@ -23,6 +23,14 @@ plot2d(sin(a*x), [x, 0, 2*%pi])$
 #+begin_src maxima  :results graphics :file maxima-test-3d.png
 plot3d (2^(-u^2 + v^2), [u, -3, 3], [v, -2, 2])$
 #+end_src
+
+** Use the ~draw~ package
+This test exercises the ~:graphics-pkg~ header argument.
+#+name: ob-maxima/draw
+#+begin_src maxima  :var a=0.5 :results graphics file :file ./maxima-test-cos.png :graphics-pkg draw
+draw2d(explicit(cos(a*x), x, -%pi, %pi))$
+#+end_src
+
 * Output to a file
 Output to a file
 #+begin_src maxima :file maxima-test-ouput.out
@@ -89,3 +97,49 @@ tex(ratsimp(diff(%e^(a*x), x)));
 #+BEGIN_LaTeX
 $$a\,e^{a\,x}$$
 #+END_LaTeX
+
+* Batch
+:PROPERTIES:
+:header-args:maxima: :exports both :results verbatim :batch batch
+:END:
+
+Exercise the ~:batch~ header argument. These tests are also defined in
+~testing/lisp/test-ob-maxima.el~. The test name is name of the ~ert~
+test.
+
+#+name: ob-maxima/batch+verbatim
+#+begin_src maxima
+(assume(z>0),
+integrate(exp(-t)*t^z, t, 0, inf));
+#+end_src
+
+#+name: ob-maxima/batch+verbatim+quiet
+#+begin_src maxima :cmdline --quiet
+(assume(z>0),
+integrate(exp(-t)*t^z, t, 0, inf));
+#+end_src
+
+#+name: ob-maxima/batch+verbatim+:lisp
+#+begin_src maxima :cmdline --quiet
+:lisp #$(assume(z>0),integrate(exp(-t)*t^z, t, 0, inf));#$
+#+end_src
+
+#+name: ob-maxima/batch+verbatim+empty-string
+#+begin_src maxima :cmdline --quiet
+"";
+#+end_src
+
+#+name: ob-maxima/batch+verbatim+whitespace-string
+#+begin_src maxima :cmdline --quiet
+" ";
+#+end_src
+
+#+name: ob-maxima/batch+verbatim+syntax-error
+#+begin_src maxima :cmdline --quiet
+;
+#+end_src
+
+#+name: ob-maxima/batch+verbatim+eof-error
+#+begin_src maxima :cmdline --quiet
+x:
+#+end_src
diff --git a/testing/lisp/test-ob-maxima.el b/testing/lisp/test-ob-maxima.el
index ae9fdc775..211b70e06 100644
--- a/testing/lisp/test-ob-maxima.el
+++ b/testing/lisp/test-ob-maxima.el
@@ -66,6 +66,113 @@
      (equal
       '((1 2 3) (2 3 4) (3 4 5)) (org-babel-execute-src-block)))))
 
+
+;; 6 tests to test the :batch header argument
+(ert-deftest ob-maxima/batch+verbatim ()
+  "Exercise the `:batch' header argument.  Since `--very-quiet'
+is set, the input and output are printed without labels."
+  (unwind-protect
+    (org-test-with-temp-text
+        (format "#+begin_src maxima :results verbatim :batch batch
+(assume(z>0),
+integrate(exp(-t)*t^z, t, 0, inf));
+#+end_src")
+      (should (equal (org-babel-execute-src-block)
+                     "(assume(z > 0),integrate(exp(-t)*t^z,t,0,inf))\n                                 gamma(z + 1)")))))
+
+(ert-deftest ob-maxima/batch+verbatim+quiet ()
+  "Exercise the `:batch' header argument.  Since `--quiet' is
+set, the input and output are printed with labels."
+  (unwind-protect
+    (org-test-with-temp-text
+        (format "#+name: ob-maxima/batch+verbatim
+#+begin_src maxima :results verbatim :batch batch :cmdline --quiet
+(assume(z>0),
+integrate(exp(-t)*t^z, t, 0, inf));
+#+end_src")
+      (should (equal (org-babel-execute-src-block)
+                     "(%i1) (assume(z > 0),integrate(exp(-t)*t^z,t,0,inf))\n(%o1)                            gamma(z + 1)")))))
+
+(ert-deftest ob-maxima/batch+verbatim+:lisp ()
+  "Exercise the `:batch' header argument with `:lisp' reader.
+Since `--quiet' is set, the output is printed (as a lisp form)."
+  (unwind-protect
+    (org-test-with-temp-text
+        (format "#+name: ob-maxima/batch+verbatim+:lisp
+#+begin_src maxima :results verbatim :batch batch :cmdline --quiet
+:lisp #$(assume(z>0),integrate(exp(-t)*t^z, t, 0, inf));#$
+#+end_src
+")
+      (should (equal (org-babel-execute-src-block)
+                     "((%GAMMA SIMP) ((MPLUS SIMP) 1 $Z))")))))
+
+(ert-deftest ob-maxima/batch+verbatim+empty-string-vq ()
+  "Exercise the `:batch' header argument.  Send an empty string to
+Maxima.  Since `--very-quiet' is set, the output is printed."
+  (unwind-protect
+    (org-test-with-temp-text
+        (format "#+name: ob-maxima/batch+verbatim+empty-string-vq
+#+begin_src maxima :results verbatim :batch batch :cmdline --very-quiet
+\"\";
+#+end_src
+")
+      (should (equal (org-babel-execute-src-block) "\"\"\n ")))))
+
+(ert-deftest ob-maxima/batch+verbatim+empty-string ()
+  "Exercise the `:batch' header argument.  Send an empty string to
+Maxima.  Since `--quiet' is set, the input and output are printed
+with labels."
+  (unwind-protect
+    (org-test-with-temp-text
+        (format "#+name: ob-maxima/batch+verbatim+empty-string
+#+begin_src maxima :results verbatim :batch batch :cmdline --quiet
+\"\";
+#+end_src
+")
+      (should (equal (org-babel-execute-src-block) "(%i1) \"\"\n(%o1) ")))))
+
+(ert-deftest ob-maxima/batch+verbatim+whitespace-string ()
+  "Exercise the `:batch' header argument.  Send an empty string to
+Maxima.  Since `--quiet' is set, the input and output are printed
+with labels."
+  (unwind-protect
+    (org-test-with-temp-text
+        (format "#+name: ob-maxima/batch+verbatim+whitespace-string
+#+begin_src maxima :results verbatim :batch batch :cmdline --quiet
+\" \";
+#+end_src
+")
+      (should (equal (org-babel-execute-src-block)
+                     "(%i1) \" \"\n(%o1)                                   ")))))
+
+(ert-deftest ob-maxima/batch+verbatim+syntax-error ()
+  "Exercise the `:batch' header argument.  Send invalid input to
+Maxima."
+  (unwind-protect
+    (org-test-with-temp-text
+        (format "#+name: ob-maxima/batch+verbatim+syntax-error
+#+begin_src maxima  :results verbatim :batch batch :cmdline --quiet
+;
+#+end_src
+")
+      (should (string-match "incorrect syntax: Premature termination of input at ;\\."
+                            (org-babel-execute-src-block))))))
+
+(ert-deftest ob-maxima/batch+verbatim+eof-error ()
+  "Exercise the `:batch' header argument.  Send invalid input to
+Maxima."
+  (unwind-protect
+    (org-test-with-temp-text
+        (format "#+name: ob-maxima/batch+verbatim+eof-error
+#+begin_src maxima  :results verbatim :batch batch :cmdline --quiet
+x:
+#+end_src
+")
+      (should (string-match "end of file while scanning expression\\."
+                            (org-babel-execute-src-block))))))
+
+
+
 (provide 'test-ob-maxima)
 
 ;;; test-ob-maxima.el ends here
-- 
2.40.1