bug with :wrap when exporting

bug with :wrap when exporting

[Alan Schmitt]

Hello,

I think I have found a bug with ":wrap" when exporting a block. Consider
the following org file:

--8<---------------cut here---------------start------------->8---
#+name: mywrap
#+BEGIN_SRC sh :exports none :results raw
echo "(+ 1 2)"
#+END_SRC

Exporting with the result absent:

#+name: testmywrap
#+call: mywrap() :wrap src emacs-lisp

#+results: testmywrap

Exporting with the result present:

#+name: testmywrap2
#+call: mywrap() :wrap src emacs-lisp

#+results: testmywrap2
#+BEGIN_src emacs-lisp
(+ 1 2)
#+END_src
--8<---------------cut here---------------end--------------->8---

If you export it, you'll see that the results for the second call are
duplicated. Note that this only happens during export: evaluating the
second call to mywrap does not duplicate the results below.

Alan

Re: bug with :wrap when exporting

[Nicolas Goaziou]

[-- Attachment #1: Type: text/plain, Size: 910 bytes --]

Hello,

Alan Schmitt <alan.schmitt@polytechnique.org> writes:

> I think I have found a bug with ":wrap" when exporting a block. Consider
> the following org file:
>
> #+name: mywrap
> #+BEGIN_SRC sh :exports none :results raw
> echo "(+ 1 2)"
> #+END_SRC
>
> Exporting with the result absent:
>
> #+name: testmywrap
> #+call: mywrap() :wrap src emacs-lisp
>
> #+results: testmywrap
>
> Exporting with the result present:
>
> #+name: testmywrap2
> #+call: mywrap() :wrap src emacs-lisp
>
> #+results: testmywrap2
> #+BEGIN_src emacs-lisp
> (+ 1 2)
> #+END_src
>
> If you export it, you'll see that the results for the second call are
> duplicated. Note that this only happens during export: evaluating the
> second call to mywrap does not duplicate the results below.

The following patch (from maint) fixed the problem shown in the ECM.
Does it still work in the general case?


Regards,

-- 
Nicolas Goaziou

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-ob-exp-Fix-duplicate-evaluation-with-wrap-src.patch --]
[-- Type: text/x-diff, Size: 12130 bytes --]

From 2b79fc4f9940abfd97d83103ab1de39cf2c44ede Mon Sep 17 00:00:00 2001
From: Nicolas Goaziou <n.goaziou@gmail.com>
Date: Wed, 4 Dec 2013 23:23:33 +0100
Subject: [PATCH] ob-exp: Fix duplicate evaluation with :wrap src

* lisp/ob-exp.el (org-babel-exp-process-buffer): Fix duplicate
  evaluation with :wrap src.
(org-babel-exp-non-block-elements): Removed function.
* testing/lisp/test-ob-lob.el (test-ob-lob/export-lob-lines): Fix test.
---
 lisp/ob-exp.el              | 266 ++++++++++++++++++++------------------------
 testing/lisp/test-ob-lob.el |   1 +
 2 files changed, 119 insertions(+), 148 deletions(-)

diff --git a/lisp/ob-exp.el b/lisp/ob-exp.el
index d6d4566..746504e 100644
--- a/lisp/ob-exp.el
+++ b/lisp/ob-exp.el
@@ -150,19 +150,17 @@ this template."
   :type 'string)
 
 (defvar org-babel-default-lob-header-args)
-(defun org-babel-exp-non-block-elements (start end)
-  "Process inline source and call lines between START and END for export."
+(defun org-babel-exp-process-buffer ()
+  "Execute all Babel blocks in current buffer."
   (interactive)
-  (save-excursion
-    (goto-char start)
-    (unless (markerp end)
-      (let ((m (make-marker)))
-	(set-marker m end (current-buffer))
-	(setq end m)))
-    (let ((rx (concat "\\(?:"  org-babel-inline-src-block-regexp
-		      "\\|" org-babel-lob-one-liner-regexp "\\)")))
-      (while (re-search-forward rx end t)
-	(save-excursion
+  (save-window-excursion
+    (save-excursion
+      (let ((case-fold-search t)
+	    (regexp (concat org-babel-inline-src-block-regexp "\\|"
+			    org-babel-lob-one-liner-regexp "\\|"
+			    "^[ \t]*#\\+BEGIN_SRC")))
+	(goto-char (point-min))
+	(while (re-search-forward regexp nil t)
 	  (let* ((element (save-excursion
 			    ;; If match is inline, point is at its
 			    ;; end.  Move backward so
@@ -170,145 +168,117 @@ this template."
 			    ;; object, not the following one.
 			    (backward-char)
 			    (save-match-data (org-element-context))))
-		 (type (org-element-type element)))
-	    (when (memq type '(babel-call inline-babel-call inline-src-block))
-	      (let ((beg-el (org-element-property :begin element))
-		    (end-el (org-element-property :end element)))
-		(case type
-		  (inline-src-block
-		   (let* ((info (org-babel-parse-inline-src-block-match))
-			  (params (nth 2 info)))
-		     (setf (nth 1 info)
-			   (if (and (cdr (assoc :noweb params))
-				    (string= "yes" (cdr (assoc :noweb params))))
-			       (org-babel-expand-noweb-references
-				info (org-babel-exp-get-export-buffer))
-			     (nth 1 info)))
-		     (goto-char beg-el)
-		     (let ((replacement (org-babel-exp-do-export info 'inline)))
-		       (if (equal replacement "")
-			   ;; Replacement code is empty: completely
-			   ;; remove inline src block, including extra
-			   ;; white space that might have been created
-			   ;; when inserting results.
-			   (delete-region beg-el
-					  (progn (goto-char end-el)
-						 (skip-chars-forward " \t")
-						 (point)))
-			 ;; Otherwise: remove inline src block but
-			 ;; preserve following white spaces.  Then
-			 ;; insert value.
-			 (delete-region beg-el
-					(progn (goto-char end-el)
-					       (skip-chars-backward " \t")
-					       (point)))
-			 (insert replacement)))))
-		  ((babel-call inline-babel-call)
-		   (let* ((lob-info (org-babel-lob-get-info))
-			  (results
-			   (org-babel-exp-do-export
-			    (list "emacs-lisp" "results"
-				  (apply #'org-babel-merge-params
-					 org-babel-default-header-args
-					 org-babel-default-lob-header-args
-					 (append
-					  (org-babel-params-from-properties)
-					  (list
-					   (org-babel-parse-header-arguments
-					    (org-no-properties
-					     (concat
-					      ":var results="
-					      (mapconcat 'identity
-							 (butlast lob-info 2)
-							 " ")))))))
-				  "" (nth 3 lob-info) (nth 2 lob-info))
-			    'lob))
-			  (rep (org-fill-template
-				org-babel-exp-call-line-template
-				`(("line"  . ,(nth 0 lob-info))))))
-		     ;; If replacement is empty, completely remove the
-		     ;; object/element, including any extra white space
-		     ;; that might have been created when including
-		     ;; results.
-		     (if (equal rep "")
-			 (delete-region
-			  beg-el
-			  (progn (goto-char end-el)
-				 (if (not (eq type 'babel-call))
-				     (progn (skip-chars-forward " \t") (point))
-				   (skip-chars-forward " \r\t\n")
-				   (line-beginning-position))))
-		       ;; Otherwise, preserve following white
-		       ;; spaces/newlines and then, insert replacement
-		       ;; string.
-		       (goto-char beg-el)
+		 (type (org-element-type element))
+		 (beg-el (org-element-property :begin element))
+		 (end-el (org-element-property :end element)))
+	    (case type
+	      (inline-src-block
+	       (let* ((info (org-babel-parse-inline-src-block-match))
+		      (params (nth 2 info)))
+		 (setf (nth 1 info)
+		       (if (and (cdr (assoc :noweb params))
+				(string= "yes" (cdr (assoc :noweb params))))
+			   (org-babel-expand-noweb-references
+			    info (org-babel-exp-get-export-buffer))
+			 (nth 1 info)))
+		 (goto-char beg-el)
+		 (let ((replacement (org-babel-exp-do-export info 'inline)))
+		   (if (equal replacement "")
+		       ;; Replacement code is empty: remove inline src
+		       ;; block, including extra white space that
+		       ;; might have been created when inserting
+		       ;; results.
 		       (delete-region beg-el
 				      (progn (goto-char end-el)
-					     (skip-chars-backward " \r\t\n")
+					     (skip-chars-forward " \t")
 					     (point)))
-		       (insert rep)))))))))))))
-
-(defvar org-src-preserve-indentation)	; From org-src.el
-(defun org-babel-exp-process-buffer ()
-  "Execute all blocks in visible part of buffer."
-  (interactive)
-  (save-window-excursion
-    (let ((case-fold-search t)
-	  (pos (point-min)))
-      (goto-char pos)
-      (while (re-search-forward "^[ \t]*#\\+BEGIN_SRC" nil t)
-        (let ((element (save-match-data (org-element-at-point))))
-          (when (eq (org-element-type element) 'src-block)
-            (let* ((match-start (copy-marker (match-beginning 0)))
-                   (begin (copy-marker (org-element-property :begin element)))
-                   ;; Make sure we don't remove any blank lines after
-                   ;; the block when replacing it.
-                   (block-end (save-excursion
-				(goto-char (org-element-property :end element))
-				(skip-chars-backward " \r\t\n")
-				(copy-marker (line-end-position))))
-                   (ind (org-get-indentation))
-                   (headers
-		    (cons
-		     (org-element-property :language element)
-		     (let ((params (org-element-property :parameters element)))
-		       (and params (org-split-string params "[ \t]+"))))))
-              ;; Execute all non-block elements between POS and
-              ;; current block.
-              (org-babel-exp-non-block-elements pos begin)
-	      ;; Take care of matched block: compute replacement
-	      ;; string. In particular, a nil REPLACEMENT means the
-	      ;; block should be left as-is while an empty string
-	      ;; should remove the block.
-              (let ((replacement (progn (goto-char match-start)
-					(org-babel-exp-src-block headers))))
-                (cond ((not replacement) (goto-char block-end))
-		      ((equal replacement "")
-		       (delete-region begin
-				      (progn (goto-char block-end)
-					     (skip-chars-forward " \r\t\n")
-					     (if (eobp) (point)
-					       (line-beginning-position)))))
-		      (t
-		       (goto-char match-start)
-		       (delete-region (point) block-end)
-		       (insert replacement)
-		       (if (org-element-property :preserve-indent element)
-			   ;; Indent only the code block markers.
-			   (save-excursion (skip-chars-backward " \r\t\n")
-					   (indent-line-to ind)
-					   (goto-char match-start)
-					   (indent-line-to ind))
-			 ;; Indent everything.
-			 (indent-rigidly match-start (point) ind)))))
-	      (setq pos (line-beginning-position))
-              ;; Cleanup markers.
-	      (set-marker match-start nil)
-	      (set-marker begin nil)
-              (set-marker block-end nil)))))
-      ;; Eventually execute all non-block Babel elements between last
-      ;; src-block and end of buffer.
-      (org-babel-exp-non-block-elements pos (point-max)))))
+		     ;; Otherwise: remove inline src block but
+		     ;; preserve following white spaces.  Then insert
+		     ;; value.
+		     (delete-region beg-el
+				    (progn (goto-char end-el)
+					   (skip-chars-backward " \t")
+					   (point)))
+		     (insert replacement)))))
+	      ((babel-call inline-babel-call)
+	       (let* ((lob-info (org-babel-lob-get-info))
+		      (results
+		       (org-babel-exp-do-export
+			(list "emacs-lisp" "results"
+			      (apply #'org-babel-merge-params
+				     org-babel-default-header-args
+				     org-babel-default-lob-header-args
+				     (append
+				      (org-babel-params-from-properties)
+				      (list
+				       (org-babel-parse-header-arguments
+					(org-no-properties
+					 (concat
+					  ":var results="
+					  (mapconcat 'identity
+						     (butlast lob-info 2)
+						     " ")))))))
+			      "" (nth 3 lob-info) (nth 2 lob-info))
+			'lob))
+		      (rep (org-fill-template
+			    org-babel-exp-call-line-template
+			    `(("line"  . ,(nth 0 lob-info))))))
+		 ;; If replacement is empty, completely remove the
+		 ;; object/element, including any extra white space
+		 ;; that might have been created when including
+		 ;; results.
+		 (if (equal rep "")
+		     (delete-region
+		      beg-el
+		      (progn (goto-char end-el)
+			     (if (not (eq type 'babel-call))
+				 (progn (skip-chars-forward " \t") (point))
+			       (skip-chars-forward " \r\t\n")
+			       (line-beginning-position))))
+		   ;; Otherwise, preserve following white
+		   ;; spaces/newlines and then, insert replacement
+		   ;; string.
+		   (goto-char beg-el)
+		   (delete-region beg-el
+				  (progn (goto-char end-el)
+					 (skip-chars-backward " \r\t\n")
+					 (point)))
+		   (insert rep))))
+	      (src-block
+	       (let* ((match-start (match-beginning 0))
+		      ;; Make sure we don't remove any blank lines
+		      ;; after the block when replacing it.
+		      (block-end (save-excursion
+				   (goto-char end-el)
+				   (skip-chars-backward " \r\t\n")
+				   (line-end-position)))
+		      (ind (org-get-indentation))
+		      (headers
+		       (cons
+			(org-element-property :language element)
+			(let ((params (org-element-property :parameters element)))
+			  (and params (org-split-string params "[ \t]+"))))))
+		 ;; Take care of matched block: compute replacement
+		 ;; string.  In particular, a nil REPLACEMENT means
+		 ;; the block should be left as-is while an empty
+		 ;; string should remove the block.
+		 (let ((replacement (progn (goto-char match-start)
+					   (org-babel-exp-src-block headers))))
+		   (cond ((not replacement) (goto-char block-end))
+			 ((equal replacement "")
+			  (delete-region beg-el end-el))
+			 (t
+			  (goto-char match-start)
+			  (delete-region (point) block-end)
+			  (insert replacement)
+			  (if (org-element-property :preserve-indent element)
+			      ;; Indent only the code block markers.
+			      (save-excursion (skip-chars-backward " \r\t\n")
+					      (indent-line-to ind)
+					      (goto-char match-start)
+					      (indent-line-to ind))
+			    ;; Indent everything.
+			    (indent-rigidly match-start (point) ind))))))))))))))
 
 (defun org-babel-in-example-or-verbatim ()
   "Return true if point is in example or verbatim code.
diff --git a/testing/lisp/test-ob-lob.el b/testing/lisp/test-ob-lob.el
index db3b7c8..828eef2 100644
--- a/testing/lisp/test-ob-lob.el
+++ b/testing/lisp/test-ob-lob.el
@@ -86,6 +86,7 @@
 	(let ((org-current-export-file buf))
 	  (org-babel-exp-process-buffer))
 	(message (buffer-string))
+	(goto-char (point-min))
 	(should (re-search-forward "^: 0" nil t))
 	(should (re-search-forward "call =2= stuck" nil t))
 	(should (re-search-forward
-- 
1.8.4.2

Re: bug with :wrap when exporting

[Alan Schmitt]

Hello Nicolas,

n.goaziou@gmail.com writes:

> The following patch (from maint) fixed the problem shown in the ECM.

Off topic question: what does ECM stands for? (I understand it's a small
example showing a bug, but I could not find the meaning of the acronym.)

> Does it still work in the general case?

I'd gladly test it, but I'm not sure how to do it. I understand I need
to switch branch to the maint branch on my git clone of the repository.
But then what is the simplest way to test this different org version
without changing my current installation? Should I simply launch "emacs
-Q" the evaluate something like:

(add-to-list 'load-path "~/src/org-mode/lisp") 
(require 'org)

?

Thanks,

Alan

Re: bug with :wrap when exporting

[Josiah Schwab]

Alan Schmitt writes:

> Off topic question: what does ECM stands for? (I understand it's a small
> example showing a bug, but I could not find the meaning of the
> acronym.)

Exemple Complet Minimal

http://orgmode.org/worg/org-faq.html#ecm

Best,
Josiah

Re: bug with :wrap when exporting

[Alan Schmitt]

jschwab@gmail.com writes:

> Alan Schmitt writes:
>
>> Off topic question: what does ECM stands for? (I understand it's a small
>> example showing a bug, but I could not find the meaning of the
>> acronym.)
>
> Exemple Complet Minimal
>
> http://orgmode.org/worg/org-faq.html#ecm

Very interesting, I was not expecting French here.

Thanks,

Alan

Re: bug with :wrap when exporting

[Alan Schmitt]

alan.schmitt@polytechnique.org writes:

> I'd gladly test it, but I'm not sure how to do it. I understand I need
> to switch branch to the maint branch on my git clone of the repository.
> But then what is the simplest way to test this different org version
> without changing my current installation? Should I simply launch "emacs
> -Q" the evaluate something like:
>
> (add-to-list 'load-path "~/src/org-mode/lisp") 
> (require 'org)

I tried to do this, and do a "make clean" to make sure old elc files
would not be picked up, but then export fails with

Symbol's function definition is void: org-export-dispatch

In fact it tells me more than this: I later get

Org-mode version 8.2.3c (release_8.2.3c-20-gaf6f14.dirty-git @ mixed
installation!
/usr/local/Cellar/emacs-mac/emacs-24.3-mac-4.5/share/emacs/24.3/lisp/org/
and /Users/schmitta/projets/org-mode/lisp/)

Do I need to compile the org files for them to be taken into account?

Thanks,

Alan

Re: bug with :wrap when exporting

[Nick Dokos]

Alan Schmitt <alan.schmitt@polytechnique.org> writes:

> jschwab@gmail.com writes:
>
>> Alan Schmitt writes:
>>
>>> Off topic question: what does ECM stands for? (I understand it's a small
>>> example showing a bug, but I could not find the meaning of the
>>> acronym.)
>>
>> Exemple Complet Minimal
>>
>> http://orgmode.org/worg/org-faq.html#ecm
>
> Very interesting, I was not expecting French here.
>

"Nobody expects the Spanish Inquisition^W^W^WSebastien Vauban" :-)

Re: bug with :wrap when exporting

[Achim Gratz]

Alan Schmitt writes:
> I tried to do this, and do a "make clean" to make sure old elc files
> would not be picked up, but then export fails with

You would want to do "make uncompiled" and also "(require 'org-loaddefs)".


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves

Re: bug with :wrap when exporting

[Sebastien Vauban]

Nick Dokos wrote:
> Alan Schmitt <alan.schmitt-o/5/jSaJEHk+NdeTPqioyti2O/JbrIOy@public.gmane.org> writes:
>> jschwab-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org writes:
>>> Alan Schmitt writes:
>>>
>>>> Off topic question: what does ECM stands for? (I understand it's a small
>>>> example showing a bug, but I could not find the meaning of the
>>>> acronym.)
>>>
>>> Exemple Complet Minimal
>>>
>>> http://orgmode.org/worg/org-faq.html#ecm
>>
>> Very interesting, I was not expecting French here.
>
> "Nobody expects the Spanish Inquisition^W^W^WSebastien Vauban" :-)

My 15 minutes of fame... ;-)

Best regards,
  Seb

-- 
Sebastien Vauban

Re: bug with :wrap when exporting

[Alan Schmitt]

Stromeko@nexgo.de writes:

> Alan Schmitt writes:
>> I tried to do this, and do a "make clean" to make sure old elc files
>> would not be picked up, but then export fails with
>
> You would want to do "make uncompiled" and also "(require 'org-loaddefs)".

Thanks a lot, with this I was able to test the patch.

Alan

Re: bug with :wrap when exporting

[Alan Schmitt]

n.goaziou@gmail.com writes:

> Hello,
>
> Alan Schmitt <alan.schmitt@polytechnique.org> writes:
>
>> I think I have found a bug with ":wrap" when exporting a block. Consider
>> the following org file:
>>
>> #+name: mywrap
>> #+BEGIN_SRC sh :exports none :results raw
>> echo "(+ 1 2)"
>> #+END_SRC
>>
>> Exporting with the result absent:
>>
>> #+name: testmywrap
>> #+call: mywrap() :wrap src emacs-lisp
>>
>> #+results: testmywrap
>>
>> Exporting with the result present:
>>
>> #+name: testmywrap2
>> #+call: mywrap() :wrap src emacs-lisp
>>
>> #+results: testmywrap2
>> #+BEGIN_src emacs-lisp
>> (+ 1 2)
>> #+END_src
>>
>> If you export it, you'll see that the results for the second call are
>> duplicated. Note that this only happens during export: evaluating the
>> second call to mywrap does not duplicate the results below.
>
> The following patch (from maint) fixed the problem shown in the ECM.
> Does it still work in the general case?

Yes, it works great on the following example (which includes "direct
printing", in the sense of echoing the #+BEGIN_SRC lines) as well as in
the ":wrap" case.

--8<---------------cut here---------------start------------->8---
#+name: fetchcoq
#+BEGIN_SRC sh :exports none :results raw :var f="foo.v"
echo "#+BEGIN_SRC coq"
head $f
echo
echo "#+END_SRC"
#+END_SRC

Test using direct printing

#+name: test1
#+call: fetchcoq(f="demo.v") :results raw

#+results: test1
#+BEGIN_SRC coq
Definition toto : forall x, exists y, x = y.

Lemma foo: forall x, x=x.
#+END_SRC

#+name: fetchcoq2
#+BEGIN_SRC sh :exports none :results raw :var f="foo.v"
head $f
echo
#+END_SRC

Test using wrap

#+name: test2
#+call: fetchcoq2(f="demo.v") :wrap src coq

#+results: test2
#+BEGIN_src coq test
Definition toto : forall x, exists y, x = y.

Lemma foo: forall x, x=x.
#+END_src
--8<---------------cut here---------------end--------------->8---

Thanks,

Alan

Re: bug with :wrap when exporting

[Nicolas Goaziou]

Hello,

Alan Schmitt <alan.schmitt@polytechnique.org> writes:

> Yes, it works great on the following example (which includes "direct
> printing", in the sense of echoing the #+BEGIN_SRC lines) as well as in
> the ":wrap" case.

Applied, then. Thank you for your feedback.


Regards,

-- 
Nicolas Goaziou