strange behaviour with $PROP_r1 value in a table

strange behaviour with $PROP_r1 value in a table

[Fraga, Eric]

Hello,

I am trying to have an entry in a table use a value of a PROPERTY.
Every time I try to enter, say,

:=$PROP_r1

into a table, the r1 becomes @1$18.  I have tried this with emacs -Q.  I
am using emacs from git (as of a week or two ago) and org that comes
with it.

Here is what I get:

--8<---------------cut here---------------start------------->8---
#+property: r1 0.95

| value | #ERROR |
#+TBLFM: @1$2=$PROP_@1$18
--8<---------------cut here---------------end--------------->8---

The @1$18, by the way, is consistent regardless of where in a table I
might be trying to insert the expression.

If I change the property name to 'recovery' instead of 'r1', everything
works as expected.

Has anybody seen anything similar?

Thank you,
eric
-- 
: Eric S Fraga, with org 9.7.10-5ebd97 in Emacs 31.0.50

Re: strange behaviour with $PROP_r1 value in a table

[Rens Oliemans]

[-- Attachment #1: Type: text/plain, Size: 1183 bytes --]

Confirmed, this is indeed strange behaviour, I think a bug.

It happens because Org supports both letter/number references to cells (like A7, or in
your case r1) and also Org's default representation, something like @7$1 (equivalent to
A7) or @1$18 (equivalent to r1). Wherever appropriate, it converts A7-like references to
@7$1-like references. See more in

    (info "(org) References")

, and specifically the "Named references" section. You can evaluate that info sexp, or
find it in section 3.5.1 otherwise :).

It seems like a mistake to convert such a cell reference in the '$PROP_x' case. The
conversion is done by 'org-table-convert-refs-to-rc', which is done before
'org-table-formula-substitute-names' is called. Simply changing the order will not work,
since the property value might also be a letter/number-like reference.

My added patch fixes this by not replacing such a representation when it is preceded by
'$PROP_'.

I have two questions about it:

- The function '$PROP_<remote' still gives '#ERROR', is this related to this bug, or is
  the property name '<remote' invalid in any way and is this intended?

- Should I add tests for this behaviour?

Best,
Rens


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-lisp-org-table.el-Don-t-convert-table-references-for.patch --]
[-- Type: text/x-patch, Size: 1138 bytes --]

From b1b57f263853d28777f4067e782d5cf401e73691 Mon Sep 17 00:00:00 2001
From: Rens Oliemans <hallo@rensoliemans.nl>
Date: Fri, 6 Sep 2024 11:36:57 +0200
Subject: [PATCH] lisp/org-table.el: Don't convert table references for
 properties

* lisp/org-table.el (org-table-convert-refs-to-rc): don't convert refs
to rc for properties

Reported-by: Eric S Fraga <e.fraga@ucl.ac.uk>
Link: https://list.orgmode.org/87jzfqjsyr.fsf@ucl.ac.uk
---
 lisp/org-table.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org-table.el b/lisp/org-table.el
index a0708e3ec..b1c5f8fbc 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -3451,7 +3451,7 @@ See menu for more commands.")))))
 Works for single references, but also for entire formulas and even the
 full TBLFM line."
   (let ((start 0))
-    (while (string-match "\\<\\([a-zA-Z]+\\)\\([0-9]+\\>\\|&\\)\\|\\(;[^\r\n:]+\\|\\<remote([^,)]*)\\)" s start)
+    (while (string-match "\\(\\$PROP_\\)\\{0\\}\\<\\([a-zA-Z]+\\)\\([0-9]+\\>\\|&\\)\\|\\(;[^\r\n:]+\\|\\<remote([^,)]*)\\)" s start)
       (cond
        ((match-end 3)
 	;; format match, just advance
-- 
2.46.0

Re: strange behaviour with $PROP_r1 value in a table

[Fraga, Eric]

Response below/inline for email Rens Oliemans wrote:
> (original email sent  6 Sep 2024 at 11:40)
> 
> Confirmed, this is indeed strange behaviour, I think a bug.
>
> It happens because Org supports both letter/number references to cells (like A7, or in
> your case r1)

Ah, this makes sense and hence the 18 (18th letter of alphabet)!  Thank
you for figuring this out.  I've not use letter/number references ever
in org tables so had forgotten that this was possible.

eric

-- 
: Eric S Fraga, with org 9.7.10-5ebd97 in Emacs 31.0.50

Re: strange behaviour with $PROP_r1 value in a table

[Ihor Radchenko]

Rens Oliemans <hallo@rensoliemans.nl> writes:

> Confirmed, this is indeed strange behaviour, I think a bug.
> ...
> It seems like a mistake to convert such a cell reference in the '$PROP_x' case. The
> conversion is done by 'org-table-convert-refs-to-rc', which is done before
> 'org-table-formula-substitute-names' is called. Simply changing the order will not work,
> since the property value might also be a letter/number-like reference.

I just tried

#+property: r1 0.96

| value | 0.96 |
#+TBLFM: @1$2=$PROP_r1

followed by C-c C-c on TBLFM line, and I cannot reproduce.

I _can_ reproduce via C-c = -> `org-table-formula-from-user'/`org-table-formula-to-user'
where is it a result of the default value of `org-table-use-standard-references'

> -    (while (string-match "\\<\\([a-zA-Z]+\\)\\([0-9]+\\>\\|&\\)\\|\\(;[^\r\n:]+\\|\\<remote([^,)]*)\\)" s start)
> +    (while (string-match "\\(\\$PROP_\\)\\{0\\}\\<\\([a-zA-Z]+\\)\\([0-9]+\\>\\|&\\)\\|\\(;[^\r\n:]+\\|\\<remote([^,)]*)\\)" s start)

This is doing much more than fixing the reported bug and might affect
valid uses. I'd rather fix the bug (if it is a bug) explicitly, without
risking breaking other things.

> I have two questions about it:
>
> - The function '$PROP_<remote' still gives '#ERROR', is this related to this bug, or is
>   the property name '<remote' invalid in any way and is this intended?
>
> - Should I add tests for this behaviour?

Before discussing any further, may you please check if 

#+property: r1 0.96

| value | 0.96 |
#+TBLFM: @1$2=$PROP_r1

works on your side?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: strange behaviour with $PROP_r1 value in a table

[Rens Oliemans]

Ihor Radchenko <yantar92@posteo.net> writes:

> I just tried
>
> #+property: r1 0.96
>
> | value | 0.96 |
> #+TBLFM: @1$2=$PROP_r1
>
> followed by C-c C-c on TBLFM line, and I cannot reproduce.

Indeed, when simply executing the formula nothing goes wrong. You highlighted it below,
the "problem" occurs via `org-table-formula-from-user'.

> I _can_ reproduce via C-c = -> `org-table-formula-from-user'/`org-table-formula-to-user'
> where is it a result of the default value of `org-table-use-standard-references'

The same for me: I can reproduce via 'C-c =', or via the formula editor in the cell
(':=$PROP_r1'). `org-table-use-standard-references' is interesting, I did not see it
before.

>> -    (while (string-match "\\<\\([a-zA-Z]+\\)\\([0-9]+\\>\\|&\\)\\|\\(;[^\r\n:]+\\|\\<remote([^,)]*)\\)" s start)
>> +    (while (string-match "\\(\\$PROP_\\)\\{0\\}\\<\\([a-zA-Z]+\\)\\([0-9]+\\>\\|&\\)\\|\\(;[^\r\n:]+\\|\\<remote([^,)]*)\\)" s start)
>
> This is doing much more than fixing the reported bug and might affect
> valid uses. I'd rather fix the bug (if it is a bug) explicitly, without
> risking breaking other things.

Hm, looking at the manual ("(org) References"), a constant of the form '$PROP_Xyz' will be
used to search for property 'Xyz', and it seems like a bug to convert 'Xyz' to another
format in this case.

The only use-cases that are affected with this change seem to be when a table formula
starts with '$PROP_' and has a spreadsheet-like reference after it, no? Perhaps I am
wrong: I searched for the usages of `org-table-convert-refs-to-rc', but perhaps I missed
something important.

Do you agree that it is a bug, even with `org-table-use-standard-references' set to `t' or
`from'?

-- 
Best,
Rens

Re: strange behaviour with $PROP_r1 value in a table

[Ihor Radchenko]

Rens Oliemans <hallo@rensoliemans.nl> writes:

>>> -    (while (string-match "\\<\\([a-zA-Z]+\\)\\([0-9]+\\>\\|&\\)\\|\\(;[^\r\n:]+\\|\\<remote([^,)]*)\\)" s start)
>>> +    (while (string-match "\\(\\$PROP_\\)\\{0\\}\\<\\([a-zA-Z]+\\)\\([0-9]+\\>\\|&\\)\\|\\(;[^\r\n:]+\\|\\<remote([^,)]*)\\)" s start)
>>
>> This is doing much more than fixing the reported bug and might affect
>> valid uses. I'd rather fix the bug (if it is a bug) explicitly, without
>> risking breaking other things.
>
> Hm, looking at the manual ("(org) References"), a constant of the form '$PROP_Xyz' will be
> used to search for property 'Xyz', and it seems like a bug to convert 'Xyz' to another
> format in this case.
>
> The only use-cases that are affected with this change seem to be when a table formula
> starts with '$PROP_' and has a spreadsheet-like reference after it, no? Perhaps I am
> wrong: I searched for the usages of `org-table-convert-refs-to-rc', but perhaps I missed
> something important.

Another potential example is
#+CONSTANTS: valC2=299792458.
and then $valC2 reference.

> Do you agree that it is a bug, even with `org-table-use-standard-references' set to `t' or
> `from'?

Currently, the order of reference expansion is undefined. So, it is
_technically_ not a bug, but undefined behavior.

However, it is indeed confusing.
So, we may consider specifying the order of reference expansion.
But what should be that order? We need to think about all possible
expansions and how they might interact in edge cases (e.g. what if we
have :CrazyPropertyName@1$2:)

Also, it is confusing that the manual does not clearly explain that "B4"
expansion only happens when editing formulas interactively via formula
editor.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>