words starting with call_ confuse C-c C-c and export

words starting with call_ confuse C-c C-c and export

[Daniel Clemente]

  Hi, with org-mode from today on Emacs 23.4.1 and with this 2-line file:

- [ ] call_me
- [ ] try funcall_lambda (maybe)


1. Go to the „me“ and press C-c C-c. You get „C-c C-c can do nothing useful at this location“. I expected to switch the checkbox.

2. Go to the „maybe“ and press C-c C-c. I got:

Debugger entered--Lisp error: (void-function maybe)
  (maybe)
  eval((maybe))
  org-babel-read("(maybe)")
  org-babel-ref-parse("results=(maybe)")
  #[(el) …
  mapcar(#[(el) …
  org-babel-process-params(((:comments . "") (:shebang . "") (:cache . "no") (:padline . "") (:noweb . "no") (:tangle . "no") (:exports . "code") (:results . "replace") (:var . "results=(maybe)") (:hlines . "no") (:session . "none")))
  org-babel-lob-execute(("lambda (maybe)" nil 13 nil))
  org-babel-lob-execute-maybe()
  org-babel-execute-maybe()
  org-babel-execute-safely-maybe()
  run-hook-with-args-until-success(org-babel-execute-safely-maybe)
  org-ctrl-c-ctrl-c(nil)


3. Similar confusions happen on export; the word Fcall_interactively which appeared in a gdb backtrace was crashing the HTML exportation.


  I think something similar happened to me years ago, and I had to avoid all call_ words!

Re: words starting with call_ confuse C-c C-c and export

[Nicolas Goaziou]

Hello,

Daniel Clemente <n142857@gmail.com> writes:

>   Hi, with org-mode from today on Emacs 23.4.1 and with this 2-line file:
>
> - [ ] call_me
> - [ ] try funcall_lambda (maybe)
>
>
> 1. Go to the „me“ and press C-c C-c. You get „C-c C-c can do nothing
> useful at this location“. I expected to switch the checkbox.

This should be fixed.

> 2. Go to the „maybe“ and press C-c C-c. I got:
>
> Debugger entered--Lisp error: (void-function maybe)
>   (maybe)
>   eval((maybe))
>   org-babel-read("(maybe)")
>   org-babel-ref-parse("results=(maybe)")
>   #[(el) …
>   mapcar(#[(el) …
>   org-babel-process-params(((:comments . "") (:shebang . "") (:cache .
> "no") (:padline . "") (:noweb . "no") (:tangle . "no") (:exports .
> "code") (:results . "replace") (:var . "results=(maybe)") (:hlines .
> "no") (:session . "none")))
>   org-babel-lob-execute(("lambda (maybe)" nil 13 nil))
>   org-babel-lob-execute-maybe()
>   org-babel-execute-maybe()
>   org-babel-execute-safely-maybe()
>   run-hook-with-args-until-success(org-babel-execute-safely-maybe)
>   org-ctrl-c-ctrl-c(nil)
>
>
> 3. Similar confusions happen on export; the word Fcall_interactively which appeared in a gdb backtrace was crashing the HTML exportation.
>
>
>   I think something similar happened to me years ago, and I had to
>   avoid all call_ words!

We may tweak `org-babel-inline-lob-one-liner-regexp' in order to make it
harder to trigger it unwillingly. 

Cc'ing Eric Schulte to know his opinion.


Regards,

-- 
Nicolas Goaziou

Re: words starting with call_ confuse C-c C-c and export

[Eric Schulte]

>
> We may tweak `org-babel-inline-lob-one-liner-regexp' in order to make it
> harder to trigger it unwillingly. 
>

The trade-off here is between raising an error when e.g., a like
matching the call line syntax has a problem or failing silently.  The
former is preferable in the case where you intend the syntax to be a
call and you *do* want to know if there is a problem, and the latter is
preferable if you aren't trying to issue a call and just stumbled upon
the syntax.

I'm open to either solution, it's just a question of which use case is
more important and which failure condition is more onerous.

Best,

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

Re: words starting with call_ confuse C-c C-c and export

[Nicolas Goaziou]

Hello,

Eric Schulte <schulte.eric@gmail.com> writes:

>> We may tweak `org-babel-inline-lob-one-liner-regexp' in order to make it
>> harder to trigger it unwillingly. 
>>
>
> The trade-off here is between raising an error when e.g., a like
> matching the call line syntax has a problem or failing silently.  The
> former is preferable in the case where you intend the syntax to be a
> call and you *do* want to know if there is a problem, and the latter is
> preferable if you aren't trying to issue a call and just stumbled upon
> the syntax.
>
> I'm open to either solution, it's just a question of which use case is
> more important and which failure condition is more onerous.

Just to be clear, I thought about making parens mandatory in inline
Babel call syntax. Underscore is overloaded already: underline,
subscript...

Note that OP's problem can be solved once we have escaped underscores
(again). But I'm not there yet.


Regards,

-- 
Nicolas Goaziou

Re: words starting with call_ confuse C-c C-c and export

[Eric Schulte]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Hello,
>
> Eric Schulte <schulte.eric@gmail.com> writes:
>
>>> We may tweak `org-babel-inline-lob-one-liner-regexp' in order to make it
>>> harder to trigger it unwillingly. 
>>>
>>
>> The trade-off here is between raising an error when e.g., a like
>> matching the call line syntax has a problem or failing silently.  The
>> former is preferable in the case where you intend the syntax to be a
>> call and you *do* want to know if there is a problem, and the latter is
>> preferable if you aren't trying to issue a call and just stumbled upon
>> the syntax.
>>
>> I'm open to either solution, it's just a question of which use case is
>> more important and which failure condition is more onerous.
>
> Just to be clear, I thought about making parens mandatory in inline
> Babel call syntax. Underscore is overloaded already: underline,
> subscript...
>

I'm open to this change.

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

Re: words starting with call_ confuse C-c C-c and export

[Nicolas Goaziou]

Hello,

Eric Schulte <schulte.eric@gmail.com> writes:

> Nicolas Goaziou <n.goaziou@gmail.com> writes:

>> Just to be clear, I thought about making parens mandatory in inline
>> Babel call syntax. Underscore is overloaded already: underline,
>> subscript...
>>
>
> I'm open to this change.

In fact, they are already mandatory. The problem is different. Current
regexp is:

  "\\([^\n]*?\\)call_\\([^()\n]+?\\)\\(\\[\\(.*?\\)\\]\\|\\(\\)\\)(\\([^\n]*?\\))\\(\\[\\(.*?\\)\\]\\)?"

In particular, name is \\([^()\n]+?\\), and can include whitespace
characters. Therefore "call_name (args)" is valid. Isn't it too much
permissive in the context of an Org (i.e. textual) document?

Also, couldn't we limit names to alphanumeric characters and, maybe,
some puctuation (e.g. hypen)?


Regards,

-- 
Nicolas Goaziou

Re: words starting with call_ confuse C-c C-c and export

[Eric Schulte]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Hello,
>
> Eric Schulte <schulte.eric@gmail.com> writes:
>
>> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>
>>> Just to be clear, I thought about making parens mandatory in inline
>>> Babel call syntax. Underscore is overloaded already: underline,
>>> subscript...
>>>
>>
>> I'm open to this change.
>
> In fact, they are already mandatory. The problem is different. Current
> regexp is:
>
>   "\\([^\n]*?\\)call_\\([^()\n]+?\\)\\(\\[\\(.*?\\)\\]\\|\\(\\)\\)(\\([^\n]*?\\))\\(\\[\\(.*?\\)\\]\\)?"
>
> In particular, name is \\([^()\n]+?\\), and can include whitespace
> characters. Therefore "call_name (args)" is valid. Isn't it too much
> permissive in the context of an Org (i.e. textual) document?
>
> Also, couldn't we limit names to alphanumeric characters and, maybe,
> some puctuation (e.g. hypen)?
>

Why don't we exclude whitespace from names.  Do you think that would be
sufficient?

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

Re: words starting with call_ confuse C-c C-c and export

[Nicolas Goaziou]

Hello,

Eric Schulte <schulte.eric@gmail.com> writes:

> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>> In fact, they are already mandatory. The problem is different. Current
>> regexp is:
>>
>>   "\\([^\n]*?\\)call_\\([^()\n]+?\\)\\(\\[\\(.*?\\)\\]\\|\\(\\)\\)(\\([^\n]*?\\))\\(\\[\\(.*?\\)\\]\\)?"
>>
>> In particular, name is \\([^()\n]+?\\), and can include whitespace
>> characters. Therefore "call_name (args)" is valid. Isn't it too much
>> permissive in the context of an Org (i.e. textual) document?
>>
>> Also, couldn't we limit names to alphanumeric characters and, maybe,
>> some puctuation (e.g. hypen)?
>>
>
> Why don't we exclude whitespace from names.  Do you think that would be
> sufficient?

It would solve the current problem, but there are still many problematic
characters allowed (e.g., commas, curly brackets). I think there's no
point in allowing "call_{i=1}()" as a valid inline Babel call.

Furthermore, I don't think it's a real limitation to use alphanumeric
characters (and hyphen) only for a function name. I would even require
an alphabetic character as the first char, to avoid calls like:
"call_1()".

BTW, later in that regexp, there is a dubious "(\\([^\n]*?\\))". Is
there any reason not to use "(\\(.*?\\))" instead? Also, what about the
empty matcher "\\(\\)"? I don't get its use.


Regards,

-- 
Nicolas Goaziou

Re: words starting with call_ confuse C-c C-c and export

[Eric Schulte]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Hello,
>
> Eric Schulte <schulte.eric@gmail.com> writes:
>
>> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>>> In fact, they are already mandatory. The problem is different. Current
>>> regexp is:
>>>
>>>   "\\([^\n]*?\\)call_\\([^()\n]+?\\)\\(\\[\\(.*?\\)\\]\\|\\(\\)\\)(\\([^\n]*?\\))\\(\\[\\(.*?\\)\\]\\)?"
>>>
>>> In particular, name is \\([^()\n]+?\\), and can include whitespace
>>> characters. Therefore "call_name (args)" is valid. Isn't it too much
>>> permissive in the context of an Org (i.e. textual) document?
>>>
>>> Also, couldn't we limit names to alphanumeric characters and, maybe,
>>> some puctuation (e.g. hypen)?
>>>
>>
>> Why don't we exclude whitespace from names.  Do you think that would be
>> sufficient?
>
> It would solve the current problem, but there are still many problematic
> characters allowed (e.g., commas, curly brackets). I think there's no
> point in allowing "call_{i=1}()" as a valid inline Babel call.
>
> Furthermore, I don't think it's a real limitation to use alphanumeric
> characters (and hyphen) only for a function name. I would even require
> an alphabetic character as the first char, to avoid calls like:
> "call_1()".
>

I often use "/" in function names, and in general I'd prefer not to
remove characters (e.g., "{") unless there is a specific reason.  I've
changed the name portion of the regular expression to disallow space
characters.

>
> BTW, later in that regexp, there is a dubious "(\\([^\n]*?\\))". Is
> there any reason not to use "(\\(.*?\\))" instead?

I've made this change and all tests continue to pass, so we can stick
with the simpler and less explicit option.

> Also, what about the empty matcher "\\(\\)"? I don't get its use.
>

I'm hesitant to make changes that could affect the match data, it is
possible this is present so that match data is similar to other regular
expressions matching callable elements.

The heavy use of regular expressions in Babel code is difficult to
maintain, but I'm not sure if there is any practical alternative.
Regardless this particular two-lines regular expression could really use
many more than two lines of comments...

Best,

>
>
> Regards,

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

Re: words starting with call_ confuse C-c C-c and export

[Nicolas Goaziou]

Hello,

Eric Schulte <schulte.eric@gmail.com> writes:

> Nicolas Goaziou <n.goaziou@gmail.com> writes:

>> It would solve the current problem, but there are still many problematic
>> characters allowed (e.g., commas, curly brackets). I think there's no
>> point in allowing "call_{i=1}()" as a valid inline Babel call.
>>
>> Furthermore, I don't think it's a real limitation to use alphanumeric
>> characters (and hyphen) only for a function name. I would even require
>> an alphabetic character as the first char, to avoid calls like:
>> "call_1()".
>>
>
> I often use "/" in function names, and in general I'd prefer not to
> remove characters (e.g., "{") unless there is a specific reason.

I think there's a specific reason to disallow naming a function "{i=1}":
it is too close to underline syntax. We should make syntax less prone to
ambiguity (see recent thread about underline and subscript) ; this
doesn't help. Think also about call_~my-verbatim-function-name~(),
call_@@latex:my-export-snippet@@()...

We're talking about function names, not free-form text, so limitations
are understandable. For example, macro names only allow alphanumeric
characters or hyphens and have to start with an alphabetic character.

I suggest to select a few symbols allowed in names (e.g., "/") as
a starter. It is always possible to discuss on a case by case basis if
other symbols should be added.


Regards,

-- 
Nicolas Goaziou

Re: words starting with call_ confuse C-c C-c and export

[Eric Schulte]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Hello,
>
> Eric Schulte <schulte.eric@gmail.com> writes:
>
>> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>
>>> It would solve the current problem, but there are still many problematic
>>> characters allowed (e.g., commas, curly brackets). I think there's no
>>> point in allowing "call_{i=1}()" as a valid inline Babel call.
>>>
>>> Furthermore, I don't think it's a real limitation to use alphanumeric
>>> characters (and hyphen) only for a function name. I would even require
>>> an alphabetic character as the first char, to avoid calls like:
>>> "call_1()".
>>>
>>
>> I often use "/" in function names, and in general I'd prefer not to
>> remove characters (e.g., "{") unless there is a specific reason.
>
> I think there's a specific reason to disallow naming a function "{i=1}":
> it is too close to underline syntax. We should make syntax less prone to
> ambiguity (see recent thread about underline and subscript) ; this
> doesn't help. Think also about call_~my-verbatim-function-name~(),
> call_@@latex:my-export-snippet@@()...
>
> We're talking about function names, not free-form text, so limitations
> are understandable. For example, macro names only allow alphanumeric
> characters or hyphens and have to start with an alphabetic character.
>

Having considered this, unless there are user objections I'd be happy
limiting function names to the same restricted alphabet as macro names.

This would be a breaking change though, and should be mentioned as such
in the notes.

Best,

>
> I suggest to select a few symbols allowed in names (e.g., "/") as
> a starter. It is always possible to discuss on a case by case basis if
> other symbols should be added.
>
>
> Regards,

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

Re: words starting with call_ confuse C-c C-c and export

[Nicolas Goaziou]

Hello,

Eric Schulte <schulte.eric@gmail.com> writes:

> Nicolas Goaziou <n.goaziou@gmail.com> writes:

>> We're talking about function names, not free-form text, so limitations
>> are understandable. For example, macro names only allow alphanumeric
>> characters or hyphens and have to start with an alphabetic character.
>
> Having considered this, unless there are user objections I'd be happy
> limiting function names to the same restricted alphabet as macro names.
>
> This would be a breaking change though, and should be mentioned as such
> in the notes.

Once you have settled for a regexp (do you want to include "/"
character?), please let me know, I'll update org-element accordingly.


Regards,

-- 
Nicolas Goaziou

Re: words starting with call_ confuse C-c C-c and export

[Eric Schulte]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Hello,
>
> Eric Schulte <schulte.eric@gmail.com> writes:
>
>> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>
>>> We're talking about function names, not free-form text, so limitations
>>> are understandable. For example, macro names only allow alphanumeric
>>> characters or hyphens and have to start with an alphabetic character.
>>
>> Having considered this, unless there are user objections I'd be happy
>> limiting function names to the same restricted alphabet as macro names.
>>
>> This would be a breaking change though, and should be mentioned as such
>> in the notes.
>
> Once you have settled for a regexp (do you want to include "/"
> character?), please let me know, I'll update org-element accordingly.
>

Is "/" allowed in macro names?  I think the biggest benefit here is
unification between macro and function names.  Is there a macro name
regexp which could be used directly (to ensure that these two stay
unified)?  I don't see one, so I expect we'll want to add an
org-babel-function-name regexp along the lines of "[a-zA-Z0-9\-\/]+".
This would then be used in the following variables.
- org-babel-src-block-regexp
- org-babel-src-name-w-name-regexp

And the following functions should be updated to ensure that the name
only includes the allowed characters.
- org-babel-named-data-regexp-for-name
- org-babel-named-src-block-regexp-for-name

Does this sound about right?

>
> Regards,

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

Re: words starting with call_ confuse C-c C-c and export

[Nicolas Goaziou]

Hello,

Eric Schulte <schulte.eric@gmail.com> writes:

> Is "/" allowed in macro names?

No. Macro names use the following regexp:

  [a-zA-Z][-a-zA-Z0-9_]*

> I think the biggest benefit here is
> unification between macro and function names.  Is there a macro name
> regexp which could be used directly (to ensure that these two stay
> unified)?  I don't see one, so I expect we'll want to add an
> org-babel-function-name regexp along the lines of "[a-zA-Z0-9\-\/]+".
> This would then be used in the following variables.
> - org-babel-src-block-regexp
> - org-babel-src-name-w-name-regexp
>
> And the following functions should be updated to ensure that the name
> only includes the allowed characters.
> - org-babel-named-data-regexp-for-name
> - org-babel-named-src-block-regexp-for-name
>
> Does this sound about right?

Note there is no limitation on the contents of NAME keywords. Unless the
same limitation propagates to those (but should it?), Babel calls will
be ignored if forbidden characters are used.


Regards,

-- 
Nicolas Goaziou

Re: words starting with call_ confuse C-c C-c and export

[Eric Schulte]

>>
>> Does this sound about right?
>
> Note there is no limitation on the contents of NAME keywords. Unless the
> same limitation propagates to those (but should it?), Babel calls will
> be ignored if forbidden characters are used.
>

I think it is more important that code block names resemble data names
than macro names, so I'm afraid that I'm back to my original position of
preferring to keep as many characters as possible in function names.

Best,

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

Re: words starting with call_ confuse C-c C-c and export

[Nicolas Goaziou]

Eric Schulte <schulte.eric@gmail.com> writes:

>> Note there is no limitation on the contents of NAME keywords. Unless the
>> same limitation propagates to those (but should it?), Babel calls will
>> be ignored if forbidden characters are used.
>>
>
> I think it is more important that code block names resemble data names
> than macro names, so I'm afraid that I'm back to my original position of
> preferring to keep as many characters as possible in function names.

You can still have forbidden characters even with the current permissive
regexp:

  #+NAME: how do you call me?

is a valid NAME value but not a valid Babel name.

Another option is, as noted before, to restrict NAME values accordingly.


Regards,

-- 
Nicolas Goaziou