Re: Bug: inconsistent escaping of coderef regexp

[Tom Gillespie <tgbugs@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2746 bytes --]

Hi Nicolas,
    After a bit of investigation I understand the issue better now.
There are two problems here. One is an easy single line change,
the other is a deeper issue, which is that it is impossible for the
user to specify their own coderef regexp that can be used in both
cases. No matter what change we make we are likely to break
existing org files if users relied on one behavior and not the other.

Given this, I would say that it is worse to break tangling behavior
than it is to break coderef search because it is obvious to the user
when coderef search breaks, whereas a change in tanging
behavior is a silent change that users will not be aware of.

If we want a temporary fix, a patch is attached, but I would suggest
against changing the behavior right now and instead work toward
a new, more consistent system using header args.

I think that moving to use header args to control these is an
opportunity to resolve both issues, and to make a start toward
eventually deprecating the switches. The only question that I
have right now regarding that implementation is whether we
provide header args for just the coderef regexp or also for the
coderef format, with the coderef regexp taking precedence. The
deeper issue is that the format string that appears in the org-src
snippit below is hard coded, and if we allow users to set the
coderef format, then it may make sense to let them set that
format string. However, this would duplicate the simpler
functionality of simply allowing the user to provide their own
coderef regexp.

At the moment I have two proposed header args which are
:coderef-regexp with default matching the current output of the
org-src snippit below, and :coderef-tangle which defaults to yes
matching the behavior of the existing =-r= switch. There is an
option for a 3rd header arg that would directly replaced the =-l=
switch :coderef-format, however as mentioned above it adds
significant complexity and requires a fourth argument
:coderef-surround or something like that which is the
hard coded format string in the org-src snippit.

I'm working on a basic implementation and will respond in this
thread again when I have something worth looking at. Best!
Tom

For the record there are at least 3 different inconsistent regex
that are used to detect coderefs.

org-element:
(string-match "-l +\"\\([^\"\n]+\\)\"" switches)
is duplicated between org-element-src-block-parser and
org-element-example-block-parser

org-src:
(format "\\([ \t]*\\(%s\\)[ \t]*\\)$"
      (replace-regexp-in-string
       "%s"
       (if label (regexp-quote label) "\\([-a-zA-Z0-9_][-a-zA-Z0-9_ ]*\\)")
       (regexp-quote fmt)
       nil t))
ob-tangle:
(re-search-forward (replace-regexp-in-string "%s" ".+" cref-fmt) nil t)

[-- Attachment #2: cref-fmt.patch --]
[-- Type: text/x-patch, Size: 545 bytes --]

diff --git a/lisp/ob-tangle.el b/lisp/ob-tangle.el
index aa0373ab8..677d9d8ba 100644
--- a/lisp/ob-tangle.el
+++ b/lisp/ob-tangle.el
@@ -446,7 +446,7 @@ non-nil, return the full association list to be used by
 	      (when (string-match "-r" extra)
 		(goto-char (point-min))
 		(while (re-search-forward
-			(replace-regexp-in-string "%s" ".+" cref-fmt) nil t)
+			(replace-regexp-in-string "%s" ".+" (org-src-coderef-regexp cref-fmt)) nil t)
 		  (replace-match "")))
 	      (run-hooks 'org-babel-tangle-body-hook)
 	      (buffer-string))))