source blocks mangled when edited

source blocks mangled when edited

[Michael Gauland]

[-- Attachment #1: Type: text/plain, Size: 926 bytes --]

I've stared having trouble editing source blocks.    When I use C-c ' to
edit block, the editing behaves as expected, but when I C-c ' to return
to the main org file, the code is altered in strange ways.  I've had
trouble coming up with a really small example, but the attached file
seems to consistently demonstrate the problem, even when running emacs
without my settings.

The file has two identical source blocks. The first generally behaves
fine, though some lines get extra indentation.

The second suffers more serious distortions. For example, the first line
changes from "digraph G {" to "aph G {".

I've seen this with other types of code as well (at least SQL and json),
so it's not specific to dot.

I'm running emacs 26.1 on Debian 10.9, with org build from git.

I'm not even sure how to start tracking this down. Any help would be
greatly appreciated!

Kind regards,

Mike Gauland


[-- Attachment #2: bug.org --]
[-- Type: text/plain, Size: 1326 bytes --]

#+BEGIN_SRC dot :file diagram1.png :cmdline -Kdot -Tpng
  digraph G {
      rankdir = "LR"
       pre [label="Preprocessor"]                          
       subgraph cluster_inputs {
           label="Inputs"
           rankdir = "TB"
           input1 [label="input"]
           input2 [label="input"]
           input3 [label="∙\n∙\n∙",shape=none]
           input4 [label="input"]
       } -> pre

               pre -> final

               subgraph users {
         rankdir="TB"
         "End Users"
         "Resellers"
         "Corporate Users"
      }

      final -> "End Users"
      final -> "Resellers"
      final -> "Corporate Users"
  }
#+END_SRC

#+BEGIN_SRC dot :file diagram2.png :cmdline -Kdot -Tpng
  digraph G {
      rankdir = "LR"
       pre [label="Preprocessor"]                          
       subgraph cluster_inputs {
           label="Inputs"
           rankdir = "TB"
           input1 [label="input"]
           input2 [label="input"]
           input3 [label="∙\n∙\n∙",shape=none]
           input4 [label="input"]
       } -> pre

       pre -> final

       subgraph users {
         rankdir="TB"
         "End Users"
         "Resellers"
         "Corporate Users"
      }

      final -> "End Users"
      final -> "Resellers"
      final -> "Corporate Users"
  }
#+END_SRC

Re: source blocks mangled when edited

[Samuel Wales]

idk if this will help as you probably know all of it already.  butg
you asked for any help so here goes.  i run maint, not master or any
emacs versions.  weird that you are missing 3 cols?

===

here are notes on my workaround which wfm in the meantime if you are
willing to do the same thing.  i was having trouble getting exported
indentation for example blocks.

;; fix default for source and example block indentation
;;
;; when you edit a source or data block, org will by default
;; remove some leading whitespace and then add 2 spaces, so the
;; whole block in org is indented by 2.
;;
;; (the deleted rectangle seems to be the maximum amount of
;; whitespace common to all lines in each block.)
;;
;; initial tabs get converted to spaces.  example blocks get
;; unindented during export incorrectly.  mistakes are possible.
;; this as non-nil prevents all that.  see led for more.
;;
;; we EITHER unindent all and set this non-nil, OR put up with
;; stripping of leading whitespace from blocks where we don't
;; want the indentation stripped.
;;
;; /i prefer t./ it is not possible in babel to indent without
;; having the special feature of removing the whitespace
;; rectangle.
;;
;; you can use -i to locally do t.  there isn't an arg that
;; locally does the reverse of -i.  i.e. to do nil.
;;
;; i have gotten comfortable with no indentation and imo it is a good
default for newcomers and does not surprise.
;;
;; source blocks might actually work without indentation even
;; with the default value, but then when you edit it indents.
(setq org-src-preserve-indentation t)
;; (setq org-src-preserve-indentation nil)
;;
;; only has effect if org-src-preserve-indentation is nil
;; (setq org-edit-src-content-indentation 0)


On 5/30/21, Michael Gauland <mikelygee@gmail.com> wrote:
> I've stared having trouble editing source blocks.    When I use C-c ' to
> edit block, the editing behaves as expected, but when I C-c ' to return
> to the main org file, the code is altered in strange ways.  I've had
> trouble coming up with a really small example, but the attached file
> seems to consistently demonstrate the problem, even when running emacs
> without my settings.
>
> The file has two identical source blocks. The first generally behaves
> fine, though some lines get extra indentation.
>
> The second suffers more serious distortions. For example, the first line
> changes from "digraph G {" to "aph G {".
>
> I've seen this with other types of code as well (at least SQL and json),
> so it's not specific to dot.
>
> I'm running emacs 26.1 on Debian 10.9, with org build from git.
>
> I'm not even sure how to start tracking this down. Any help would be
> greatly appreciated!
>
> Kind regards,
>
> Mike Gauland
>
>


-- 
The Kafka Pandemic

Please learn what misopathy is.
https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html

Re: source blocks mangled when edited

[Michael Gauland]

On 31/05/21 4:21 pm, Samuel Wales wrote:
> idk if this will help as you probably know all of it already.  butg
> you asked for any help so here goes.  i run maint, not master or any
> emacs versions.  weird that you are missing 3 cols?
Switching to the maint branch didn't change anything, but I found that I
was unable to reproduce the problem with commit
9802877fbe442a52b4e63782b84921f46cf5d56b, but *could* reproduce it from
commit d02ad1f207e1579aff8f36f740a065d71472c182. Since that commit
affects org-src.el, I may be on the right track.

Re: source blocks mangled when edited

[Sébastien Miquel]

Hi Michael,

Michael Gauland writes:
> The file has two identical source blocks. The first generally behaves
> fine, though some lines get extra indentation.
>
> The second suffers more serious distortions. For example, the first line
> changes from "digraph G {" to "aph G {".

I'm unable to reproduce with a recent emacs version.

> I'm not even sure how to start tracking this down. Any help would be
> greatly appreciated!
The relevant functions are `org-edit-src-exit` and perhaps
`org-src--contents-for-write-back`.

Can you instrument these functions to see what's happening ? Is
`org-src--contents-for-write-back` populating the buffer correctly ?
Does the `replace-buffer-contents` call in `org-edit-src-exit` return
`t` ?

Regards,

-- 
Sébastien Miquel

Re: source blocks mangled when edited

[Michael Gauland]

On 31/05/21 8:15 pm, Sébastien Miquel wrote:
> The relevant functions are `org-edit-src-exit` and perhaps
> `org-src--contents-for-write-back`.
>
> Can you instrument these functions to see what's happening ? Is
> `org-src--contents-for-write-back` populating the buffer correctly ?
> Does the `replace-buffer-contents` call in `org-edit-src-exit` return
> `t` ?
>
I didn't instrument the functions, but found that there are two places
that test '(if (version< emacs-version "26.1"...'. If I change that to
use "version<=", the problem goes away (I'm still running 26.1). I don't
know whether this is the right fix (the underlying problem may be a
quirk in my system, and this is just masking it).  I'm hesitant to
submit a patch unless someone else can replicate the problem.

Re: source blocks mangled when edited

[Sébastien Miquel]

Michael Gauland writes:
> I didn't instrument the functions, but found that there are two places
> that test '(if (version< emacs-version "26.1"...'. If I change that to
> use "version<=", the problem goes away (I'm still running 26.1). I don't
> know whether this is the right fix (the underlying problem may be a
> quirk in my system, and this is just masking it).  I'm hesitant to
> submit a patch unless someone else can replicate the problem.
The commit `d02ad1f207e1579aff8f36f740a065d71472c182` introduced the
use of `replace-buffer-contents` when available. This function was
introduced in emacs 26.1, hence the version tests.

There must be an issue with the `replace-buffer-contents` calls. Can
you call `trace-function` on `replace-buffer-contents` and trigger the
behaviour again to see what it returns ?

You said it also happens with `emacs -q`, right ? I'll try to see if I
can reproduce with emacs 26.1 tomorrow.

-- 
Sébastien Miquel

Re: source blocks mangled when edited

[Michael Gauland]

On 31/05/21 9:42 pm, Sébastien Miquel wrote:
> There must be an issue with the `replace-buffer-contents` calls. Can
> you call `trace-function` on `replace-buffer-contents` and trigger the
> behaviour again to see what it returns ?
>
This is all the *trace-output* buffer shows:

======================================================================
1 -> (replace-buffer-contents #<buffer *org-src-write-back*>)
1 <- replace-buffer-contents: nil

Re: [PATCH] source blocks mangled when edited

[Sébastien Miquel]

[-- Attachment #1: Type: text/plain, Size: 736 bytes --]

Michael Gauland writes:
> This is all the*trace-output*  buffer shows:
>
> ======================================================================
> 1 -> (replace-buffer-contents #<buffer*org-src-write-back*>)
> 1 <- replace-buffer-contents: nil
Indeed, the `replace-buffer-contents` call is failing.

I've been able to reproduce with earlier versions of emacs 26.1. With
later versions of emacs 26, the problem goes away.

It seems earlier versions of `replace-buffer-contents` are not quite
reliable. It was patched prior to 27.1 and the new documentation
string makes some guarantees of correctness, so let's just change the
minimal version to 27.1.

Thank you for the report.

Regards,

-- 
Sébastien Miquel

X-Woof-Bug: confirmed


[-- Attachment #2: 0001-org-src.el-Use-replace-buffer-contents-only-for-emac.patch --]
[-- Type: text/x-patch, Size: 1976 bytes --]

From 8ebdbc5eca92de4429d3994a3663d8aa3b9877fe Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= <sebastien.miquel@posteo.eu>
Date: Tue, 1 Jun 2021 08:56:48 +0200
Subject: [PATCH] org-src.el: Use `replace-buffer-contents' only for emacs >=
 27

* lisp/org-src.el: Use `replace-buffer-contents' only for emacs >= 27.

It was introduced in emacs 26.1, but earlier versions made no
guarantees of correctness.
---
 lisp/org-src.el | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/lisp/org-src.el b/lisp/org-src.el
index 79f002e56..4698c6dd2 100644
--- a/lisp/org-src.el
+++ b/lisp/org-src.el
@@ -1199,12 +1199,12 @@ Throw an error if there is no such buffer."
       ;; insert new contents.
       (delete-overlay overlay)
       (let ((expecting-bol (bolp)))
-	(if (version< emacs-version "26.1")
+	(if (version< emacs-version "27.1")
 	    (progn (delete-region beg end)
 		   (insert (with-current-buffer write-back-buf (buffer-string))))
 	    (save-restriction
 	      (narrow-to-region beg end)
-	      (replace-buffer-contents write-back-buf)
+	      (replace-buffer-contents write-back-buf 0.1 nil)
 	      (goto-char (point-max))))
 	(when (and expecting-bol (not (bolp))) (insert "\n")))
       (kill-buffer write-back-buf)
@@ -1246,13 +1246,13 @@ Throw an error if there is no such buffer."
        (undo-boundary)
        (goto-char beg)
        (let ((expecting-bol (bolp)))
-	 (if (version< emacs-version "26.1")
+	 (if (version< emacs-version "27.1")
 	     (progn (delete-region beg end)
 		    (insert (with-current-buffer write-back-buf
                               (buffer-string))))
 	     (save-restriction
 	       (narrow-to-region beg end)
-	       (replace-buffer-contents write-back-buf)
+	       (replace-buffer-contents write-back-buf 0.1 nil)
 	       (goto-char (point-max))))
 	 (when (and expecting-bol (not (bolp))) (insert "\n")))))
     (when write-back-buf (kill-buffer write-back-buf))
-- 
2.31.1

Re: [PATCH] source blocks mangled when edited

[Nicolas Goaziou]

Hello,

Sébastien Miquel <sebastien.miquel@posteo.eu> writes:

> Subject: [PATCH] org-src.el: Use `replace-buffer-contents' only for emacs >=
>  27
>
> * lisp/org-src.el: Use `replace-buffer-contents' only for emacs >= 27.
>
> It was introduced in emacs 26.1, but earlier versions made no
> guarantees of correctness.

I tweaked the commit message and applied your patch. Thank you.

Regards,
-- 
Nicolas Goaziou