* ob-octave: improve MATLAB support
@ 2024-11-08 19:30 John C
2024-11-09 9:32 ` Ihor Radchenko
0 siblings, 1 reply; 2+ messages in thread
From: John C @ 2024-11-08 19:30 UTC (permalink / raw)
To: emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 981 bytes --]
Hi
Please see the attached patch which makes MATLAB work with org babel
eval. Here's the commit message:
ob-octave.el: improve MATLAB support
* lisp/ob-octave.el (org-babel-octave-evaluate): Improve MATLAB support
- Eliminated the code related to MATLAB Emacs Link. This capability was
removed from MATLAB release R2009a, 15 years ago.
- Fixed the following type of org block evaluation:
1) #+begin_src matlab :results verbatim
2) #+begin_src matlab :results output
3) #+begin_src matlab :results output latex
4) #+begin_src matlab :results file graphics
which aid in writing scientific papers.
- Minor point, the correct spelling of MATLAB when referencing the product is
all upper case.
* lisp/ob-matlab.el (header): Update URL for MATLAB
* etc/ORG-NEWS (New functions and changes in function arguments):
Added entry "ob-octave: improved MATLAB support"
Note, I have sent in the paper work for the FSF copyright assignment.
Thanks
John
[-- Attachment #2: ob-octave.el.patch --]
[-- Type: text/x-patch, Size: 14725 bytes --]
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 5d421172f..a086ce792 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -162,6 +162,84 @@ bibliography format requires them to be written in title-case.
# This also includes changes in function behavior from Elisp perspective.
+*** ob-octave: improved MATLAB support
+
+Improved ob-octave MATLAB (https://www.mathworks.com) code blocks processing.
+The prior version of ob-octave didn't correctly support MATLAB. To use MATLAB
+with org, you need https://github.com/MathWorks/Emacs-MATLAB-Mode.
+
+Example ~verbatim~ code block:
+
+#+begin_src org
+,#+begin_src matlab :results verbatim
+ a = 2 + 3;
+ ans = magic(a);
+,#+end_src
+
+,#+RESULTS:
+| 17 | 24 | 1 | 8 | 15 |
+| 23 | 5 | 7 | 14 | 16 |
+| 4 | 6 | 13 | 20 | 22 |
+| 10 | 12 | 19 | 21 | 3 |
+| 11 | 18 | 25 | 2 | 9 |
+#+end_src
+
+Example ~output~ code block:
+
+#+begin_src org
+,#+begin_src matlab :results output
+ disp('The results are:')
+ a = [1, 2; 3, 4]
+ b = a * 2
+,#+end_src
+
+,#+RESULTS:
+,#+begin_example
+The results are:
+
+a =
+
+ 1 2
+ 3 4
+
+b =
+
+ 2 4
+ 6 8
+,#+end_example
+
+#+end_src
+
+Example ~latex~ code block:
+
+#+begin_src org
+,#+begin_src matlab :results output latex
+ m = [4*pi, 3*pi; 2*pi, pi];
+ result = latex(sym(m));
+ disp(result)
+,#+end_src
+
+,#+RESULTS:
+,#+begin_export latex
+\left(\begin{array}{cc} 4\,\pi & 3\,\pi \\ 2\,\pi & \pi \end{array}\right)
+,#+end_export
+#+end_src
+
+Example ~graphics~ code block:
+
+#+begin_src org
+,#+header: :file sinewave.png
+,#+begin_src matlab :results file graphics
+ t = [0 : 0.1 : 2*pi];
+ y = sin(t);
+ plot(t, y);
+ set(gcf, 'PaperUnits', 'inches', 'PaperPosition', [0 0 4 3]) % Set the size to 4" x 3"
+,#+end_src
+
+,#+RESULTS:
+[[file:sinewave.png]]
+#+end_src
+
*** ob-sqlite: Added ability to open a database in readonly mode
Added option :readonly to ob-sqlite.
diff --git a/lisp/ob-matlab.el b/lisp/ob-matlab.el
index de8deadbe..bea40482b 100644
--- a/lisp/ob-matlab.el
+++ b/lisp/ob-matlab.el
@@ -28,11 +28,10 @@
;;; Requirements:
-;; Matlab
-
-;; matlab.el required for interactive emacs sessions and matlab-mode
-;; major mode for source code editing buffer
-;; https://matlab-emacs.sourceforge.net/
+;; 1) MATLAB from https://www.mathworks.com
+;; 2) https://github.com/mathworks/Emacs-MATLAB-Mode
+;; For matlab-shell to run MATLAB within Emacs and matlab-mode
+;; major mode for source code editing buffer
;;; Code:
diff --git a/lisp/ob-octave.el b/lisp/ob-octave.el
index 005990f20..39a20b57f 100644
--- a/lisp/ob-octave.el
+++ b/lisp/ob-octave.el
@@ -1,4 +1,4 @@
-;;; ob-octave.el --- Babel Functions for Octave and Matlab -*- lexical-binding: t; -*-
+;;; ob-octave.el --- Babel Functions for Octave and MATLAB -*- lexical-binding: t; -*-
;; Copyright (C) 2010-2024 Free Software Foundation, Inc.
@@ -30,6 +30,8 @@
;;; Code:
+(require 'cl-seq)
+
(require 'org-macs)
(org-assert-version)
@@ -39,7 +41,9 @@
(declare-function matlab-shell "ext:matlab-mode")
(declare-function matlab-shell-run-region "ext:matlab-mode")
-(defvar org-babel-default-header-args:matlab '())
+;; Use the session "*MATLAB*" buffer created by `matlab-shell` for code evaluation.
+(defvar org-babel-default-header-args:matlab '((:session . "*MATLAB*")))
+
(defvar org-babel-default-header-args:octave '())
(defvar org-babel-matlab-shell-command "matlab -nosplash"
@@ -47,18 +51,36 @@
(defvar org-babel-octave-shell-command "octave -q"
"Shell command to run octave as an external process.")
-(defvar org-babel-matlab-with-emacs-link nil
- "If non-nil use matlab-shell-run-region for session evaluation.
-This will use EmacsLink if (matlab-with-emacs-link) evaluates
-to a non-nil value.")
-
-(defvar org-babel-matlab-emacs-link-wrapper-method
- "%s
-if ischar(ans), fid = fopen('%s', 'w'); fprintf(fid, '%%s\\n', ans); fclose(fid);
-else, save -ascii %s ans
-end
-delete('%s')
+(defvar org-babel-matlab-print "print(\"-dpng\", %S);\nans=%S;"
+ ;; MATLAB command-function duality requires that the file name be specified
+ ;; without quotes. Using: print -dpng "file.png", would produce a file with
+ ;; the quotes in the file name on disk. Therefore, use the functional form
+ ;; to handle files with spaces, print("-dpng", "file.png").
+ ;; Example:
+ ;; #+begin_src matlab :file "sine wave.png" :results file graphics
+ ;; t = [0 : 0.1 : 2*pi];
+ ;; y = sin(t);
+ ;; plot(t, y);
+ ;; set(gcf, 'PaperUnits', 'inches', 'PaperPosition', [0 0 4 3]) % Set the size to 4" x 3"
+ ;; #+end_src
+ ;;
+ ;; #+RESULTS:
+ ;; [[file:sine wave.png]]
+ "MATLAB format specifier to print current figure to a file.")
+
+(defvar org-babel-octave-print "print -dpng %S\nans=%S"
+ "Octave format specifier to print current figure to a file.")
+
+(defvar org-babel-matlab-wrapper-method
+ (concat "\
+cd('%s');
+%s
+if ~exist('ans', 'var') ans = ''; end; \
+writematrix(ans, '%s', 'Delimiter', 'tab');
")
+ "Format specifier used when evaluating MATLAB code blocks.
+Arguments are the `default-directory', the MATLAB code, and a result file.txt.")
+
(defvar org-babel-octave-wrapper-method
"%s
if ischar(ans), fid = fopen('%s', 'w'); fdisp(fid, ans); fclose(fid);
@@ -92,7 +114,10 @@ When MATLABP is non-nil, execute Matlab. Otherwise, execute Octave."
(list
"set (0, \"defaultfigurevisible\", \"off\");"
full-body
- (format "print -dpng %S\nans=%S" gfx-file gfx-file))
+ (format (if matlabp
+ org-babel-matlab-print
+ org-babel-octave-print)
+ gfx-file gfx-file))
"\n")
full-body)
result-type matlabp)))
@@ -153,6 +178,23 @@ If there is not a current inferior-process-buffer in SESSION then
create. Return the initialized session. PARAMS are src block parameters."
(org-babel-octave-initiate-session session params 'matlab))
+(defun org-babel-matlab-shell ()
+ "Start and/or wait for MATLAB shell."
+ (require 'matlab-shell) ;; make `matlab-shell-busy-checker' available
+ (cond
+ ((fboundp 'matlab-shell-busy-checker)
+ ;; Start the shell if needed. `matlab-shell' will reuse existing if already running.
+ (matlab-shell)
+ ;; If we just started the matlab-shell, wait for the prompt. If we do not
+ ;; wait, then the startup messages will show up in the evaluation results.
+ (matlab-shell-busy-checker 'wait-for-prompt))
+ (t
+ (message (concat "You version of matlab-mode is old.\n"
+ "Please update, see https://github.com/mathworks/Emacs-MATLAB-Mode\n"
+ "Updating will eliminate unexpected output in your results\n"))
+ (sit-for 3)
+ (matlab-shell))))
+
(defun org-babel-octave-initiate-session (&optional session _params matlabp)
"Create an octave inferior process buffer.
If there is not a current inferior-process-buffer in SESSION then
@@ -165,9 +207,15 @@ Octave session, unless MATLABP is non-nil."
(unless (string= session "none")
(let ((session (or session
(if matlabp "*Inferior Matlab*" "*Inferior Octave*"))))
- (if (org-babel-comint-buffer-livep session) session
+ (if (org-babel-comint-buffer-livep session)
+ (progn
+ (when (and matlabp (fboundp 'matlab-shell-busy-checker))
+ ;; Can't evaluate if the matlab-shell is currently running code
+ (matlab-shell-busy-checker 'error-if-busy))
+ session)
(save-window-excursion
- (if matlabp (unless org-babel-matlab-with-emacs-link (matlab-shell))
+ (if matlabp
+ (org-babel-matlab-shell)
(run-octave))
(rename-buffer (if (bufferp session) (buffer-name session)
(if (stringp session) session (buffer-name))))
@@ -183,66 +231,107 @@ value of the last statement in BODY, as elisp."
(org-babel-octave-evaluate-session session body result-type matlabp)
(org-babel-octave-evaluate-external-process body result-type matlabp)))
+(defun org-babel-octave-wrapper-tmp-file (matlabp)
+ "Return a local tmp file with name adjusted for MATLABP."
+ (if matlabp
+ ;; writematrix requires a file ending with '.txt'
+ (org-babel-temp-file "matlab-" ".txt")
+ (org-babel-temp-file "octave-")))
+
+(defun org-babel-octave-get-code-to-eval (body tmp-file matlabp)
+ "Format BODY of the code block for evaluation saving results to TMP-FILE.
+If MATLABP, format for MATLAB, else format for Octave."
+ (if matlabp
+ (format org-babel-matlab-wrapper-method default-directory body tmp-file)
+ (format org-babel-octave-wrapper-method body tmp-file tmp-file)))
+
(defun org-babel-octave-evaluate-external-process (body result-type matlabp)
- "Evaluate BODY in an external Octave or Matalab process.
+ "Evaluate BODY in an external Octave or MATLAB process.
Process the result as RESULT-TYPE. Use Octave, unless MATLABP is non-nil."
(let ((cmd (if matlabp
org-babel-matlab-shell-command
org-babel-octave-shell-command)))
(pcase result-type
(`output (org-babel-eval cmd body))
- (`value (let ((tmp-file (org-babel-temp-file "octave-")))
+ (`value (let ((tmp-file (org-babel-process-file-name
+ (org-babel-octave-wrapper-tmp-file matlabp)
+ 'noquote)))
(org-babel-eval
cmd
- (format org-babel-octave-wrapper-method body
- (org-babel-process-file-name tmp-file 'noquote)
- (org-babel-process-file-name tmp-file 'noquote)))
+ (org-babel-octave-get-code-to-eval body tmp-file matlabp))
(org-babel-octave-import-elisp-from-file tmp-file))))))
+(defun org-babel-body-for-output (body matlabp)
+ "If MATLABP, fixup BODY for MATLAB output result-type."
+ (when matlabp
+ ;; When we send multi-line input to `matlab-shell', we'll see the "body"
+ ;; code lines echoed in the output which is not what one would expect. To
+ ;; remove these unwanted lines, we append a comment "%-<org-eval>" to each
+ ;; line in the body MATLAB code. After we collect the results from
+ ;; evaluation, we leverage the "%-<org-eval>" to remove the unwanted lines.
+ ;; Example of desired behavior:
+ ;; #+begin_src matlab :results output
+ ;; disp('The results are:')
+ ;; a = [1, 2; 3, 4]
+ ;; b = a * 2
+ ;; #+end_src
+ ;;
+ ;; #+RESULTS:
+ ;; #+begin_example
+ ;; The results are:
+ ;;
+ ;; a =
+ ;;
+ ;; 1 2
+ ;; 3 4
+ ;;
+ ;; b =
+ ;;
+ ;; 2 4
+ ;; 6 8
+ ;; #+end_example
+
+ (setq body (replace-regexp-in-string "\n" " %-<org-eval>-\n" body))
+ (when (not (string-match "\n\\'" body))
+ (setq body (concat body " %-<org-eval>-"))))
+ body)
+
+(defun org-babel-fix-up-output (results)
+ "Fix up RESULTS for output result-type."
+ ;; When we send multi-line input to `matlab-shell', we'll see the "body" code
+ ;; lines echoed in the output. Therefore, leverage the "%-<org-eval>" to
+ ;; remove the unnecessary lines.
+ (let ((fixed-results (replace-regexp-in-string "^[^\n]*%-<org-eval>-\n" ""
+ results)))
+ ;; Remove unnecessary starting blank line caused by stripping %-<org-eval>
+ (replace-regexp-in-string "\\`[[:space:]\r\n]+" "" fixed-results)))
+
(defun org-babel-octave-evaluate-session
(session body result-type &optional matlabp)
"Evaluate BODY in SESSION."
- (let* ((tmp-file (org-babel-temp-file (if matlabp "matlab-" "octave-")))
- (wait-file (org-babel-temp-file "matlab-emacs-link-wait-signal-"))
+ (let* ((tmp-file (org-babel-octave-wrapper-tmp-file matlabp))
(full-body
(pcase result-type
(`output
(mapconcat
#'org-babel-chomp
- (list body org-babel-octave-eoe-indicator) "\n"))
+ (list (org-babel-body-for-output body matlabp)
+ org-babel-octave-eoe-indicator)
+ "\n"))
(`value
- (if (and matlabp org-babel-matlab-with-emacs-link)
- (concat
- (format org-babel-matlab-emacs-link-wrapper-method
- body
- (org-babel-process-file-name tmp-file 'noquote)
- (org-babel-process-file-name tmp-file 'noquote) wait-file) "\n")
- (mapconcat
- #'org-babel-chomp
- (list (format org-babel-octave-wrapper-method
- body
- (org-babel-process-file-name tmp-file 'noquote)
- (org-babel-process-file-name tmp-file 'noquote))
- org-babel-octave-eoe-indicator) "\n")))))
- (raw (if (and matlabp org-babel-matlab-with-emacs-link)
- (save-window-excursion
- (with-temp-buffer
- (insert full-body)
- (write-region "" 'ignored wait-file nil nil nil 'excl)
- (matlab-shell-run-region (point-min) (point-max))
- (message "Waiting for Matlab Emacs Link")
- (while (file-exists-p wait-file) (sit-for 0.01))
- "")) ;; matlab-shell-run-region doesn't seem to
- ;; make *matlab* buffer contents easily
- ;; available, so :results output currently
- ;; won't work
- (org-babel-comint-with-output
- (session
- (if matlabp
- org-babel-octave-eoe-indicator
- org-babel-octave-eoe-output)
- t full-body)
- (insert full-body) (comint-send-input nil t))))
+ (mapconcat
+ #'org-babel-chomp
+ (list (org-babel-octave-get-code-to-eval body tmp-file matlabp)
+ org-babel-octave-eoe-indicator)
+ "\n"))))
+ (raw-results
+ (org-babel-comint-with-output
+ (session
+ (if matlabp
+ org-babel-octave-eoe-indicator
+ org-babel-octave-eoe-output)
+ t full-body)
+ (insert full-body) (comint-send-input nil t)))
results)
(pcase result-type
(`value
@@ -250,12 +339,13 @@ Process the result as RESULT-TYPE. Use Octave, unless MATLABP is non-nil."
(`output
(setq results
(if matlabp
- (cdr (reverse (delete "" (mapcar #'org-strip-quotes
- (mapcar #'org-trim raw)))))
+ (cdr (reverse (delete "" (mapcar #'org-strip-quotes
+ (mapcar #'org-trim raw-results)))))
(cdr (member org-babel-octave-eoe-output
(reverse (mapcar #'org-strip-quotes
- (mapcar #'org-trim raw)))))))
- (mapconcat #'identity (reverse results) "\n")))))
+ (mapcar #'org-trim raw-results)))))))
+ (org-babel-fix-up-output
+ (mapconcat #'identity (reverse results) "\n"))))))
(defun org-babel-octave-import-elisp-from-file (file-name)
"Import data from FILE-NAME.
^ permalink raw reply related [flat|nested] 2+ messages in thread* Re: ob-octave: improve MATLAB support
2024-11-08 19:30 ob-octave: improve MATLAB support John C
@ 2024-11-09 9:32 ` Ihor Radchenko
0 siblings, 0 replies; 2+ messages in thread
From: Ihor Radchenko @ 2024-11-09 9:32 UTC (permalink / raw)
To: John C; +Cc: emacs-orgmode
John C <john.ciolfi.32@gmail.com> writes:
> Please see the attached patch which makes MATLAB work with org babel
> eval. Here's the commit message:
Thanks a lot for the patch!
The patch introduces major changes. Would it be possible to add some
tests as well?
> Note, I have sent in the paper work for the FSF copyright assignment.
Let us know when it is done. Also, if there are any problems or delays
in the process, we can try helping.
My initial comments on the patch are below.
> +*** ob-octave: improved MATLAB support
"fixed" maybe :)
> +Improved ob-octave MATLAB (https://www.mathworks.com) code blocks processing.
> +The prior version of ob-octave didn't correctly support MATLAB. To use MATLAB
> +with org, you need https://github.com/MathWorks/Emacs-MATLAB-Mode.
> +
> +Example ~verbatim~ code block:
> +
> +#+begin_src org
> +,#+begin_src matlab :results verbatim
I am not sure if examples are necessary in the NEWS. Maybe we can simply
mention that :results verbatim, output, and latex are supported now.
As for examples, please put them into
https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-octave-matlab.html
The source code for that page is at https://git.sr.ht/~bzg/worg
> -;; Matlab
> -
> -;; matlab.el required for interactive emacs sessions and matlab-mode
> -;; major mode for source code editing buffer
> -;; https://matlab-emacs.sourceforge.net/
> +;; 1) MATLAB from https://www.mathworks.com
Please remove all the links to mathworks.com - it is FSF policy to not
link to proprietary software pages. We cannot have these links in Org
code.
> +;; 2) https://github.com/mathworks/Emacs-MATLAB-Mode
This one is GPL-licensed and hence fine.
> -(defvar org-babel-default-header-args:matlab '())
> +;; Use the session "*MATLAB*" buffer created by `matlab-shell` for code evaluation.
> +(defvar org-babel-default-header-args:matlab '((:session . "*MATLAB*")))
The new default implies that sessions will be enabled by default. I am
not sure if it is what you intended to do.
> -(defvar org-babel-matlab-with-emacs-link nil
> - "If non-nil use matlab-shell-run-region for session evaluation.
> -This will use EmacsLink if (matlab-with-emacs-link) evaluates
> -to a non-nil value.")
> -
> -(defvar org-babel-matlab-emacs-link-wrapper-method
> - "%s
> -if ischar(ans), fid = fopen('%s', 'w'); fprintf(fid, '%%s\\n', ans); fclose(fid);
> -else, save -ascii %s ans
> -end
> -delete('%s')
I understand that these variable no longer make sense after we remove
support of MATLAB Emacs Link. However, I'd prefer to not remove
variables without warning. Instead, please deprecate them.
> + (when matlabp
> + ;; When we send multi-line input to `matlab-shell', we'll see the "body"
> + ;; code lines echoed in the output which is not what one would expect. To
Maybe you can somehow make use of REMOVE-ECHO in `org-babel-comint-with-output'?
--
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2024-11-09 9:32 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-11-08 19:30 ob-octave: improve MATLAB support John C
2024-11-09 9:32 ` Ihor Radchenko
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).