* [BUG?] Matching tags: & operator no more implicit between tags and special property @ 2023-08-23 7:57 Samuel Loury 2023-08-23 10:21 ` Jens Schmidt 2023-08-23 10:31 ` Ihor Radchenko 0 siblings, 2 replies; 31+ messages in thread From: Samuel Loury @ 2023-08-23 7:57 UTC (permalink / raw) To: emacs-orgmode [-- Attachment #1.1: Type: text/plain, Size: 926 bytes --] Hi. I've been using org-mode as PKMS for 13 years. Thanks for that awesome tool. After upgrading on main yesterday, I realized a change of behavior. I'm not sure whether it is a bug or not. The following tags query used to implicitly behave as a AND "-tag-TODO=\"TODO\"". It does not anymore. To reproduce this, simply use the following content. ``` * TODO todo and tag :tag: * TODO simply todo * nothing * only tag :tag: ``` And run the following code. `(org-tags-view nil "-tag-todo=\"TODO\"")` Before commit f689eb44f175fbbdc4e8ef0ad6f5201b10863438, this showed only "nothing", as expected. Now, it shows all the entries. To get back to the old behavior, I need to explicitly add the boolean operator & in between. I attached the code of the example in this mail. Hope that this bug report is useful. [-- Attachment #1.2: tag-match-implicit-and.tar.xz --] [-- Type: application/x-xz, Size: 528 bytes --] [-- Attachment #1.3: Type: text/plain, Size: 104 bytes --] -- Konubinix GPG Key : 7439106A Fingerprint: 5993 BE7A DA65 E2D9 06CE 5C36 75D2 3CED 7439 106A [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 487 bytes --] ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [BUG?] Matching tags: & operator no more implicit between tags and special property 2023-08-23 7:57 [BUG?] Matching tags: & operator no more implicit between tags and special property Samuel Loury @ 2023-08-23 10:21 ` Jens Schmidt 2023-08-23 10:37 ` Ihor Radchenko 2023-08-23 10:31 ` Ihor Radchenko 1 sibling, 1 reply; 31+ messages in thread From: Jens Schmidt @ 2023-08-23 10:21 UTC (permalink / raw) To: Samuel Loury, emacs-orgmode, Ihor Radchenko Thanks for the detailed report! On 2023-08-23 09:57, Samuel Loury wrote: > The following tags query used to implicitly behave as a AND > "-tag-TODO=\"TODO\"". > Before commit f689eb44f175fbbdc4e8ef0ad6f5201b10863438, this showed only > "nothing", as expected. That was my short-sighted attempt to allow for unquoted minus signs in property names, and your query right now matches all entries having a property "tag-TODO" different from "TODO". TBH, I didn't think about your use case, where two negations are concatenated directly after each other. OTOH, the unquoting ("\\-") that has been in place before above commit was broken, which might serve as a hint that few to none actually use minus signs in property names, at least not in queries. Ihor, WDYT? I can (with the necessary delay to allow for proper testing) revert to some sort of quoting in property names which would allow for Samuel's use case again. ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [BUG?] Matching tags: & operator no more implicit between tags and special property 2023-08-23 10:21 ` Jens Schmidt @ 2023-08-23 10:37 ` Ihor Radchenko 0 siblings, 0 replies; 31+ messages in thread From: Ihor Radchenko @ 2023-08-23 10:37 UTC (permalink / raw) To: Jens Schmidt; +Cc: Samuel Loury, emacs-orgmode Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes: > OTOH, the unquoting ("\\-") that has been in place before above commit > was broken, which might serve as a hint that few to none actually use > minus signs in property names, at least not in queries. > > Ihor, WDYT? I can (with the necessary delay to allow for proper > testing) revert to some sort of quoting in property names which would > allow for Samuel's use case again. Agree. In case of ambiguity TAG1-TAG2-TAG3-PROP="" like we see here, we should prefer multiple shorter conditions: TAG1 && -TAG2 && -TAG3 && PROP="" When "TAG1-TAG2" is an actual tag name with dash, we may allow escaping. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [BUG?] Matching tags: & operator no more implicit between tags and special property 2023-08-23 7:57 [BUG?] Matching tags: & operator no more implicit between tags and special property Samuel Loury 2023-08-23 10:21 ` Jens Schmidt @ 2023-08-23 10:31 ` Ihor Radchenko 2023-08-23 10:38 ` Jens Schmidt 1 sibling, 1 reply; 31+ messages in thread From: Ihor Radchenko @ 2023-08-23 10:31 UTC (permalink / raw) To: Samuel Loury, Jens Schmidt; +Cc: emacs-orgmode Samuel Loury <konubinix@gmail.com> writes: > The following tags query used to implicitly behave as a AND > "-tag-TODO=\"TODO\"". > > It does not anymore. Thanks for reporting! Confirmed. Jens, we got an edge case with "-", after all. What is happening here is that the new parser (https://orgmode.org/list/9132e58f-d89e-f7df-bbe4-43d53a2367d2@vodafonemail.de) treats the above match string as property "tag-TODO" not equal "TODO" instead of not tag equal "tag" and not TODO equal "TODO" -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [BUG?] Matching tags: & operator no more implicit between tags and special property 2023-08-23 10:31 ` Ihor Radchenko @ 2023-08-23 10:38 ` Jens Schmidt 2023-08-23 14:00 ` [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] Jens Schmidt 0 siblings, 1 reply; 31+ messages in thread From: Jens Schmidt @ 2023-08-23 10:38 UTC (permalink / raw) To: Ihor Radchenko, Samuel Loury; +Cc: emacs-orgmode On 2023-08-23 12:31, Ihor Radchenko wrote: > Jens, we got an edge case with "-", after all. > > What is happening here is that the new parser > (https://orgmode.org/list/9132e58f-d89e-f7df-bbe4-43d53a2367d2@vodafonemail.de) > treats the above match string as > > property "tag-TODO" not equal "TODO" > > instead of > > not tag equal "tag" and not TODO equal "TODO" Mails crossing. Let's continue your thread after some cool down. ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-08-23 10:38 ` Jens Schmidt @ 2023-08-23 14:00 ` Jens Schmidt 2023-08-23 15:55 ` Jens Schmidt 2023-08-24 7:32 ` Ihor Radchenko 0 siblings, 2 replies; 31+ messages in thread From: Jens Schmidt @ 2023-08-23 14:00 UTC (permalink / raw) To: Ihor Radchenko, Samuel Loury; +Cc: emacs-orgmode > On 2023-08-23 12:31, Ihor Radchenko wrote: > >> Jens, we got an edge case with "-", after all. >> >> What is happening here is that the new parser >> (https://orgmode.org/list/9132e58f-d89e-f7df-bbe4-43d53a2367d2@vodafonemail.de) >> treats the above match string as >> >> property "tag-TODO" not equal "TODO" >> >> instead of >> >> not tag equal "tag" and not TODO equal "TODO" Right. And you already provided in the other branch of this discussion a direction how to resolve that: > [...] In case of ambiguity TAG1-TAG2-TAG3-PROP="" like we see here, we > should prefer multiple shorter conditions: TAG1 && -TAG2 && -TAG3 && PROP="" > > When "TAG1-TAG2" is an actual tag name with dash, we may allow escaping. But I'd like to clarify that. From the Org syntax https://orgmode.org/worg/org-syntax.html#Headings I understood that tag names cannot contain minus characters, and `org-tag-re' does not match any, either. So we are talking about property names only, right? At least I'll do so in the following ... Then the question is what quoting scheme to use for property names. The previous one used before my commit f689eb4 (A) "\\-" => "-" never has been documented and never has been working properly, since the *matching* of these was done on *prop* names, but the *unescaping* on *tag* names. So we are basically free to come up with something new. Some obvious choice would be a simpler single backslash (B) "\-" => "-" And when I have been "fixing" the parser, I also thought that (C) ":...:" => "..." would give a nice and somewhat logical quoting scheme. That is, if a property name contains "special" characters, it could be surrounded by colons to quote these. So we have so far quoting schemes A, B, C, with my preference being C. Any other proposals? I'll focus on C, where there is one big cave-at: The main regexp in `org-make-tags-matcher' also allows for (undocumented) colons as inclusive search term separator: ;; exclusion and inclusion (the latter being implicit) "\\(?1:[-+:]\\)?" ^ Which collides with the colons used by quoting scheme C. I'd rather sacrifice that use of colon as search term separator, TBH. Given the property name syntax in https://orgmode.org/worg/org-syntax.html#Node_Properties the subre in `org-make-tags-matcher' to match property names should then look similar to "\\(?5:[[:alnum:]_]+\\|:[^[:space:]]+?:\\)" , right (colons being stripped off later)? I'm really asking about the trailing plus signs here, but these do not seem to make sense in property queries. What do you think? ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-08-23 14:00 ` [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] Jens Schmidt @ 2023-08-23 15:55 ` Jens Schmidt 2023-08-24 7:30 ` Ihor Radchenko 2023-08-24 7:32 ` Ihor Radchenko 1 sibling, 1 reply; 31+ messages in thread From: Jens Schmidt @ 2023-08-23 15:55 UTC (permalink / raw) To: Ihor Radchenko, Samuel Loury; +Cc: emacs-orgmode On 2023-08-23 16:00, Jens Schmidt wrote: > Given the property name syntax in > > https://orgmode.org/worg/org-syntax.html#Node_Properties > > the subre in `org-make-tags-matcher' to match property names > should then look similar to > > "\\(?5:[[:alnum:]_]+\\|:[^[:space:]]+?:\\)" > > , right (colons being stripped off later)? I'm really asking > about the trailing plus signs here, but these do not seem to > make sense in property queries. Yet another edge-case: In the Org syntax definition a property name is really delimited by a "colon-whitespace" sequence. This takes out the ambiguity whether a property name is defined non-greedily :[^[:space:]]+?: or greedily :[^[:space:]]+: I'm too lazy right now to think about the consequences, but I hope this makes a difference only for really weird property matches, like this: :foo:=1+:bar:=2 This would be parsed non-greedily as FOO == 1 && BAR == 2 or greedily as FOO:=1+:BAR == 2 I'd go for non-greedily defined property names. ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-08-23 15:55 ` Jens Schmidt @ 2023-08-24 7:30 ` Ihor Radchenko 0 siblings, 0 replies; 31+ messages in thread From: Ihor Radchenko @ 2023-08-24 7:30 UTC (permalink / raw) To: Jens Schmidt; +Cc: Samuel Loury, emacs-orgmode Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes: > Yet another edge-case: In the Org syntax definition a property > name is really delimited by a "colon-whitespace" sequence. This > takes out the ambiguity whether a property name is defined > non-greedily > > :[^[:space:]]+?: > > or greedily > > :[^[:space:]]+: > > I'm too lazy right now to think about the consequences, but I > hope this makes a difference only for really weird property > matches, like this: > > :foo:=1+:bar:=2 Thus, -1 to this idea with ":" delimiter. If possible, we should avoid edge cases. Otherwise, we are getting rid of the edge case with "-" by introducing yet another edge case. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-08-23 14:00 ` [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] Jens Schmidt 2023-08-23 15:55 ` Jens Schmidt @ 2023-08-24 7:32 ` Ihor Radchenko 2023-08-24 8:52 ` Jens Schmidt 1 sibling, 1 reply; 31+ messages in thread From: Ihor Radchenko @ 2023-08-24 7:32 UTC (permalink / raw) To: Jens Schmidt; +Cc: Samuel Loury, emacs-orgmode Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes: >> When "TAG1-TAG2" is an actual tag name with dash, we may allow escaping. > > But I'd like to clarify that. From the Org syntax > > https://orgmode.org/worg/org-syntax.html#Headings > > I understood that tag names cannot contain minus characters, > and `org-tag-re' does not match any, either. So we are talking > about property names only, right? You are right. > Then the question is what quoting scheme to use for property > names. The previous one used before my commit f689eb4 > > (A) "\\-" => "-" > > never has been documented and never has been working properly, > since the *matching* of these was done on *prop* names, but > the *unescaping* on *tag* names. So we are basically free to > come up with something new. > > Some obvious choice would be a simpler single backslash > > (B) "\-" => "-" I prefer (B). And we will need to allow escaping of the "\" itself. Like \\. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-08-24 7:32 ` Ihor Radchenko @ 2023-08-24 8:52 ` Jens Schmidt 2023-08-25 18:46 ` Jens Schmidt 0 siblings, 1 reply; 31+ messages in thread From: Jens Schmidt @ 2023-08-24 8:52 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Samuel Loury, emacs-orgmode On 2023-08-24 09:32, Ihor Radchenko wrote: > Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes: >> Some obvious choice would be a simpler single backslash >> >> (B) "\-" => "-" > > I prefer (B). And we will need to allow escaping of the "\" itself. Like > \\. OK. Since backslash is not used yet in the rest of the regexp, (B) should be rather edge-case-free. So the corresponding subre to match property names would look like \(?5:\(?:[[:alnum:]]+\|\\[^[:space:]]\)+\) IOW, backslash quotes everything except whitespace, which by definition cannot be part of a property name. Will start on this, but with tests and documentation this might take some time. ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-08-24 8:52 ` Jens Schmidt @ 2023-08-25 18:46 ` Jens Schmidt 2023-08-26 10:16 ` Ihor Radchenko 0 siblings, 1 reply; 31+ messages in thread From: Jens Schmidt @ 2023-08-25 18:46 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Samuel Loury, emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 644 bytes --] On 2023-08-24 10:52, Jens Schmidt wrote: > On 2023-08-24 09:32, Ihor Radchenko wrote: >> I prefer (B). And we will need to allow escaping of the "\" itself. Like >> \\. > > OK. Since backslash is not used yet in the rest of the regexp, (B) > should be rather edge-case-free. So the corresponding subre to match > property names would look like > > \(?5:\(?:[[:alnum:]]+\|\\[^[:space:]]\)+\) > > IOW, backslash quotes everything except whitespace, which by definition > cannot be part of a property name. > > Will start on this, but with tests and documentation this might take > some time. Here comes a first patch ... please check. [-- Attachment #2: 0001-org-make-tags-matcher-Re-add-quoting-of-property-nam.patch --] [-- Type: text/x-patch, Size: 8910 bytes --] From 830750bfa10b52ac77abe8af1f2789057f9101c1 Mon Sep 17 00:00:00 2001 From: Jens Schmidt <jschmidt4gnu@vodafonemail.de> Date: Thu, 24 Aug 2023 22:38:02 +0200 Subject: [PATCH] org-make-tags-matcher: Re-add quoting of property names * lisp/org.el (org-make-tags-matcher): * testing/lisp/test-org.el (test-org/map-entries): Move special cased handling of LEVEL properties. Add tests for other special cased properties TODO and CATEGORY. * lisp/org.el (org-make-tags-matcher): * doc/org-manual.org (Matching tags and properties): * testing/lisp/test-org.el (test-org/map-entries): Re-add and extend quoting of property names in search strings. Link: https://orgmode.org/list/87h6oq2nu1.fsf@gmail.com --- doc/org-manual.org | 20 +++++++-------- lisp/org.el | 55 ++++++++++++++++++++++++---------------- testing/lisp/test-org.el | 34 +++++++++++++++++-------- 3 files changed, 66 insertions(+), 43 deletions(-) diff --git a/doc/org-manual.org b/doc/org-manual.org index 17b25fef4..10a03b344 100644 --- a/doc/org-manual.org +++ b/doc/org-manual.org @@ -9320,21 +9320,19 @@ With the regular =<= operator, the search would handle entries without an =EFFORT= property as having a zero effort and would include them in the result as well. -Currently, you can use only property names including alphanumeric -characters, underscores, and minus characters in search strings. In -addition, if you want to search for a property whose name starts with -a minus character, you have to "quote" that leading minus character -with an explicit positive selection plus character, like this: +You can use all characters valid in property names when matching +properties. However, you have to quote some characters in property +names with backslashes when using them in search strings, namely all +characters different from alphanumerics and underscores[fn:: If you +quote alphanumeric characters or underscores with a backslash, that +backslash is ignored.]. For example, to search for all entries having +a property =-long=and\twisted:property.name*= with value =foo=, use +search string #+begin_example -+-long-and-twisted-property-name-="foo" +\-long\=and\\twisted\:property\.name\*="foo" #+end_example -#+texinfo: @noindent -Without that extra plus character, the minus character would be taken -to indicate a negative selection on search term -=long-and-twisted-property-name-="foo"=. - You can configure Org mode to use property inheritance during a search, but beware that this can slow down searches considerably. See [[*Property Inheritance]], for details. diff --git a/lisp/org.el b/lisp/org.el index 50df1b2d9..78f8eb2e9 100644 --- a/lisp/org.el +++ b/lisp/org.el @@ -11328,15 +11328,14 @@ See also `org-scan-tags'." "\\(?2:" ;; tag regexp match "{[^}]+}\\|" - ;; LEVEL property match. For sake of consistency, - ;; recognize starred operators here as well. We do - ;; not need to process them below, however, since - ;; the LEVEL property is always present. - "LEVEL\\(?3:" opre "\\)\\*?\\(?4:[0-9]+\\)\\|" - ;; regular property match + ;; property match. Try to keep this subre generic + ;; and rather handle special properties like LEVEL + ;; and CATEGORY further below. This ensures that + ;; the same quoting mechanics can be used for all + ;; property names. "\\(?:" ;; property name [1] - "\\(?5:[[:alnum:]_-]+\\)" + "\\(?5:\\(?:[[:alnum:]_]+\\|\\\\[^[:space:]]\\)+\\)" ;; operator, optionally starred "\\(?6:" opre "\\)\\(?7:\\*\\)?" ;; operand (regexp, double-quoted string, @@ -11353,13 +11352,19 @@ See also `org-scan-tags'." (start 0) tagsmatch todomatch tagsmatcher todomatcher) - ;; [1] The minus characters in property names do *not* conflict - ;; with the exclusion operator above, since the mandatory - ;; following operator distinguishes these both cases. - ;; Accordingly, minus characters do not need any special quoting, - ;; even if https://orgmode.org/list/87jzv67k3p.fsf@localhost and - ;; commit 19b0e03f32c6032a60150fc6cb07c6f766cb3f6c suggest - ;; otherwise. + ;; [1] The history of this particular subre: + ;; - \\([[:alnum:]_]+\\) [pre-19b0e03] + ;; Does not allow for minus characters in property names. + ;; - "\\(\\(?:[[:alnum:]_]+\\(?:\\\\-\\)*\\)+\\)" [19b0e03] + ;; Incomplete fix of above issue, still resulting in, e.g., + ;; https://orgmode.org/list/87jzv67k3p.fsf@localhost. + ;; - "\\(?5:[[:alnum:]_-]+\\)" [f689eb4] + ;; Allows for unquoted minus characters in property names, but + ;; conflicts with searches like -TAG-PROP="VALUE". See + ;; https://orgmode.org/list/87h6oq2nu1.fsf@gmail.com. + ;; - current subre + ;; Like second solution, but with proper unquoting and allowing + ;; for all possible characters in property names to be quoted. ;; Expand group tags. (setq match (org-tags-expand match)) @@ -11404,22 +11409,28 @@ See also `org-scan-tags'." ;; exact tag match in [3]. (tag (match-string 2 term)) (regexp (eq (string-to-char tag) ?{)) - (levelp (match-end 4)) (propp (match-end 5)) (mm (cond (regexp ; [2] `(with-syntax-table org-mode-tags-syntax-table (org-match-any-p ,(substring tag 1 -1) tags-list))) - (levelp - `(,(org-op-to-function (match-string 3 term)) - level - ,(string-to-number (match-string 4 term)))) (propp - (let* (;; Convert property name to an Elisp + (let* (;; Determine property name. + (pn (upcase + (save-match-data + (replace-regexp-in-string + "\\\\\\(.\\)" "\\1" + (match-string 5 term) + t nil)))) + ;; Convert property name to an Elisp ;; accessor for that property (aka. as - ;; getter value). - (gv (pcase (upcase (match-string 5 term)) + ;; getter value). Symbols LEVEL and TODO + ;; referenced below get bound by the + ;; matcher that this function returns. + (gv (pcase pn + ("LEVEL" + '(number-to-string level)) ("CATEGORY" '(org-get-category (point))) ("TODO" 'todo) diff --git a/testing/lisp/test-org.el b/testing/lisp/test-org.el index e33f500a3..8355e2d77 100644 --- a/testing/lisp/test-org.el +++ b/testing/lisp/test-org.el @@ -2862,11 +2862,24 @@ test <point> (equal '(11) (org-test-with-temp-text "* Level 1\n** Level 2" (let (org-odd-levels-only) (org-map-entries #'point "LEVEL>1"))))) - ;; Level match with (ignored) starred operator. + ;; Category match. (should - (equal '(11) - (org-test-with-temp-text "* Level 1\n** Level 2" - (let (org-odd-levels-only) (org-map-entries #'point "LEVEL>*1"))))) + (equal '(59) + (org-test-with-temp-text " +#+CATEGORY: foo + +* H1 +:PROPERTIES: +:CATEGORY: bar +:END: + +* H2" + (org-map-entries #'point "CATEGORY=\"foo\"")))) + ;; Todo match. + (should + (equal '(6) + (org-test-with-temp-text "* H1\n* TODO H2\n* DONE H3" + (org-map-entries #'point "TODO=\"TODO\"")))) ;; Tag match. (should (equal '(11) @@ -2948,7 +2961,7 @@ SCHEDULED: <2014-03-04 tue.>" :END: * H3" (org-map-entries #'point "TEST!=*1")))) - ;; Property matches on names including minus characters. + ;; Property matches on names containing quoted characters. (org-test-with-temp-text " * H1 :BAR: @@ -2967,11 +2980,12 @@ SCHEDULED: <2014-03-04 tue.>" :PROPERTIES: :-FOO: 2 :END: -* H5" - (should (equal '(2) (org-map-entries #'point "TEST-FOO!=*0-FOO"))) - (should (equal '(2) (org-map-entries #'point "-FOO+TEST-FOO!=*0"))) - (should (equal '(88) (org-map-entries #'point "+-FOO!=*0-FOO"))) - (should (equal '(88) (org-map-entries #'point "-FOO+-FOO!=*0")))) +* H5 :TEST:" + (should (equal '(2) (org-map-entries #'point "TEST\\-FOO!=*0-FOO"))) + (should (equal '(2) (org-map-entries #'point "-FOO+TEST\\-FOO!=*0"))) + (should (equal '(88) (org-map-entries #'point "\\-FOO!=*0-FOO"))) + (should (equal '(88) (org-map-entries #'point "-FOO+\\-FOO!=*0"))) + (should (equal '(88) (org-map-entries #'point "-TEST-FOO-TEST\\-FOO=1")))) ;; Multiple criteria. (should (equal '(23) -- 2.30.2 ^ permalink raw reply related [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-08-25 18:46 ` Jens Schmidt @ 2023-08-26 10:16 ` Ihor Radchenko 2023-08-26 11:53 ` Jens Schmidt 0 siblings, 1 reply; 31+ messages in thread From: Ihor Radchenko @ 2023-08-26 10:16 UTC (permalink / raw) To: Jens Schmidt; +Cc: Samuel Loury, emacs-orgmode Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes: >> IOW, backslash quotes everything except whitespace, which by definition >> cannot be part of a property name. >> >> Will start on this, but with tests and documentation this might take >> some time. > > Here comes a first patch ... please check. Thanks! > #+begin_example > -+-long-and-twisted-property-name-="foo" > +\-long\=and\\twisted\:property\.name\*="foo" > #+end_example This actually feels rather cubersome. I am now wondering if we could instead do something like "-long=and\twisted:property.name*"="foo" Then, we will just have to quote the " itself, not all non-alphanumerics. Although, using " might be tricky. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-08-26 10:16 ` Ihor Radchenko @ 2023-08-26 11:53 ` Jens Schmidt 2023-08-26 12:00 ` Ihor Radchenko 0 siblings, 1 reply; 31+ messages in thread From: Jens Schmidt @ 2023-08-26 11:53 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Samuel Loury, emacs-orgmode On 2023-08-26 12:16, Ihor Radchenko wrote: > Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes: >> #+begin_example >> -+-long-and-twisted-property-name-="foo" >> +\-long\=and\\twisted\:property\.name\*="foo" >> #+end_example > > This actually feels rather cubersome. That particular example looks awkward, but pls don't forget that users (mostly) have been happy without any option of quoting so far. IOW, there shouldn't be much to quote in real life. I can provide a simpler example in the manual, but that's probably not what you meant :-) > I am now wondering if we could instead do something like > "-long=and\twisted:property.name*"="foo" > Then, we will just have to quote the " itself, not all > non-alphanumerics. > > Although, using " might be tricky. Agreed. It introduces more context, and longer context, than a simple backslash. Plus double quotes are already used for other purposes in the larger regexp, which could introduce more ambiguities. Finally, a term "foo-bar"="foo-bar" looks, er, tautologically? at best. ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-08-26 11:53 ` Jens Schmidt @ 2023-08-26 12:00 ` Ihor Radchenko 2023-08-26 12:19 ` Jens Schmidt 0 siblings, 1 reply; 31+ messages in thread From: Ihor Radchenko @ 2023-08-26 12:00 UTC (permalink / raw) To: Jens Schmidt; +Cc: Samuel Loury, emacs-orgmode Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes: > On 2023-08-26 12:16, Ihor Radchenko wrote: >> Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes: >>> #+begin_example >>> -+-long-and-twisted-property-name-="foo" >>> +\-long\=and\\twisted\:property\.name\*="foo" >>> #+end_example >> >> This actually feels rather cubersome. > > That particular example looks awkward, but pls don't forget that > users (mostly) have been happy without any option of quoting so > far. IOW, there shouldn't be much to quote in real life. I can > provide a simpler example in the manual, but that's probably not > what you meant :-) That's partially what I meant. It would help to provide a simpler example first. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-08-26 12:00 ` Ihor Radchenko @ 2023-08-26 12:19 ` Jens Schmidt 2023-08-26 12:22 ` Ihor Radchenko 0 siblings, 1 reply; 31+ messages in thread From: Jens Schmidt @ 2023-08-26 12:19 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Samuel Loury, emacs-orgmode On 2023-08-26 14:00, Ihor Radchenko wrote: > Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes: >> On 2023-08-26 12:16, Ihor Radchenko wrote: >>> This actually feels rather cubersome. >> >> [...] I can >> provide a simpler example in the manual, but that's probably not >> what you meant :-) > > That's partially what I meant. > It would help to provide a simpler example first. How about a series of simpler examples then, like this: #+begin_example boss\-prio="C" boss\:prio="C" boss\\prio="C" #+end_example ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-08-26 12:19 ` Jens Schmidt @ 2023-08-26 12:22 ` Ihor Radchenko 2023-08-26 12:54 ` Jens Schmidt 0 siblings, 1 reply; 31+ messages in thread From: Ihor Radchenko @ 2023-08-26 12:22 UTC (permalink / raw) To: Jens Schmidt; +Cc: Samuel Loury, emacs-orgmode Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes: >> That's partially what I meant. >> It would help to provide a simpler example first. > > How about a series of simpler examples then, like this: > > #+begin_example > boss\-prio="C" > boss\:prio="C" > boss\\prio="C" > #+end_example Looks good to me. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-08-26 12:22 ` Ihor Radchenko @ 2023-08-26 12:54 ` Jens Schmidt 2023-08-27 7:11 ` Samuel Loury ` (2 more replies) 0 siblings, 3 replies; 31+ messages in thread From: Jens Schmidt @ 2023-08-26 12:54 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Samuel Loury, emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 274 bytes --] On 2023-08-26 14:22, Ihor Radchenko wrote: > Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes: >> #+begin_example >> boss\-prio="C" >> boss\:prio="C" >> boss\\prio="C" >> #+end_example > > Looks good to me. Implemented in the next version of the patch, please check. [-- Attachment #2: 0001-org-make-tags-matcher-Re-add-quoting-of-property-nam.patch --] [-- Type: text/x-patch, Size: 8930 bytes --] From 11dc3ac4ff060f1ffb9dae7b35eabe526bbbc572 Mon Sep 17 00:00:00 2001 From: Jens Schmidt <jschmidt4gnu@vodafonemail.de> Date: Thu, 24 Aug 2023 22:38:02 +0200 Subject: [PATCH] org-make-tags-matcher: Re-add quoting of property names * lisp/org.el (org-make-tags-matcher): * testing/lisp/test-org.el (test-org/map-entries): Move special cased handling of LEVEL properties. Add tests for other special cased properties TODO and CATEGORY. * lisp/org.el (org-make-tags-matcher): * doc/org-manual.org (Matching tags and properties): * testing/lisp/test-org.el (test-org/map-entries): Re-add and extend quoting of property names in search strings. Link: https://orgmode.org/list/87h6oq2nu1.fsf@gmail.com --- doc/org-manual.org | 22 ++++++++-------- lisp/org.el | 55 ++++++++++++++++++++++++---------------- testing/lisp/test-org.el | 34 +++++++++++++++++-------- 3 files changed, 68 insertions(+), 43 deletions(-) diff --git a/doc/org-manual.org b/doc/org-manual.org index 17b25fef4..249648566 100644 --- a/doc/org-manual.org +++ b/doc/org-manual.org @@ -9320,21 +9320,21 @@ With the regular =<= operator, the search would handle entries without an =EFFORT= property as having a zero effort and would include them in the result as well. -Currently, you can use only property names including alphanumeric -characters, underscores, and minus characters in search strings. In -addition, if you want to search for a property whose name starts with -a minus character, you have to "quote" that leading minus character -with an explicit positive selection plus character, like this: +You can use all characters valid in property names when matching +properties. However, you have to quote some characters in property +names with backslashes when using them in search strings, namely all +characters different from alphanumerics and underscores[fn:: If you +quote alphanumeric characters or underscores with a backslash, that +backslash is ignored.]. For example, to search for all entries having +a property =boss-prio=, =boss:prio=, or =boss\prio=, respectively, +with value =C=, use search strings #+begin_example -+-long-and-twisted-property-name-="foo" +boss\-prio="C" +boss\:prio="C" +boss\\prio="C" #+end_example -#+texinfo: @noindent -Without that extra plus character, the minus character would be taken -to indicate a negative selection on search term -=long-and-twisted-property-name-="foo"=. - You can configure Org mode to use property inheritance during a search, but beware that this can slow down searches considerably. See [[*Property Inheritance]], for details. diff --git a/lisp/org.el b/lisp/org.el index 50df1b2d9..78f8eb2e9 100644 --- a/lisp/org.el +++ b/lisp/org.el @@ -11328,15 +11328,14 @@ See also `org-scan-tags'." "\\(?2:" ;; tag regexp match "{[^}]+}\\|" - ;; LEVEL property match. For sake of consistency, - ;; recognize starred operators here as well. We do - ;; not need to process them below, however, since - ;; the LEVEL property is always present. - "LEVEL\\(?3:" opre "\\)\\*?\\(?4:[0-9]+\\)\\|" - ;; regular property match + ;; property match. Try to keep this subre generic + ;; and rather handle special properties like LEVEL + ;; and CATEGORY further below. This ensures that + ;; the same quoting mechanics can be used for all + ;; property names. "\\(?:" ;; property name [1] - "\\(?5:[[:alnum:]_-]+\\)" + "\\(?5:\\(?:[[:alnum:]_]+\\|\\\\[^[:space:]]\\)+\\)" ;; operator, optionally starred "\\(?6:" opre "\\)\\(?7:\\*\\)?" ;; operand (regexp, double-quoted string, @@ -11353,13 +11352,19 @@ See also `org-scan-tags'." (start 0) tagsmatch todomatch tagsmatcher todomatcher) - ;; [1] The minus characters in property names do *not* conflict - ;; with the exclusion operator above, since the mandatory - ;; following operator distinguishes these both cases. - ;; Accordingly, minus characters do not need any special quoting, - ;; even if https://orgmode.org/list/87jzv67k3p.fsf@localhost and - ;; commit 19b0e03f32c6032a60150fc6cb07c6f766cb3f6c suggest - ;; otherwise. + ;; [1] The history of this particular subre: + ;; - \\([[:alnum:]_]+\\) [pre-19b0e03] + ;; Does not allow for minus characters in property names. + ;; - "\\(\\(?:[[:alnum:]_]+\\(?:\\\\-\\)*\\)+\\)" [19b0e03] + ;; Incomplete fix of above issue, still resulting in, e.g., + ;; https://orgmode.org/list/87jzv67k3p.fsf@localhost. + ;; - "\\(?5:[[:alnum:]_-]+\\)" [f689eb4] + ;; Allows for unquoted minus characters in property names, but + ;; conflicts with searches like -TAG-PROP="VALUE". See + ;; https://orgmode.org/list/87h6oq2nu1.fsf@gmail.com. + ;; - current subre + ;; Like second solution, but with proper unquoting and allowing + ;; for all possible characters in property names to be quoted. ;; Expand group tags. (setq match (org-tags-expand match)) @@ -11404,22 +11409,28 @@ See also `org-scan-tags'." ;; exact tag match in [3]. (tag (match-string 2 term)) (regexp (eq (string-to-char tag) ?{)) - (levelp (match-end 4)) (propp (match-end 5)) (mm (cond (regexp ; [2] `(with-syntax-table org-mode-tags-syntax-table (org-match-any-p ,(substring tag 1 -1) tags-list))) - (levelp - `(,(org-op-to-function (match-string 3 term)) - level - ,(string-to-number (match-string 4 term)))) (propp - (let* (;; Convert property name to an Elisp + (let* (;; Determine property name. + (pn (upcase + (save-match-data + (replace-regexp-in-string + "\\\\\\(.\\)" "\\1" + (match-string 5 term) + t nil)))) + ;; Convert property name to an Elisp ;; accessor for that property (aka. as - ;; getter value). - (gv (pcase (upcase (match-string 5 term)) + ;; getter value). Symbols LEVEL and TODO + ;; referenced below get bound by the + ;; matcher that this function returns. + (gv (pcase pn + ("LEVEL" + '(number-to-string level)) ("CATEGORY" '(org-get-category (point))) ("TODO" 'todo) diff --git a/testing/lisp/test-org.el b/testing/lisp/test-org.el index e33f500a3..8355e2d77 100644 --- a/testing/lisp/test-org.el +++ b/testing/lisp/test-org.el @@ -2862,11 +2862,24 @@ test <point> (equal '(11) (org-test-with-temp-text "* Level 1\n** Level 2" (let (org-odd-levels-only) (org-map-entries #'point "LEVEL>1"))))) - ;; Level match with (ignored) starred operator. + ;; Category match. (should - (equal '(11) - (org-test-with-temp-text "* Level 1\n** Level 2" - (let (org-odd-levels-only) (org-map-entries #'point "LEVEL>*1"))))) + (equal '(59) + (org-test-with-temp-text " +#+CATEGORY: foo + +* H1 +:PROPERTIES: +:CATEGORY: bar +:END: + +* H2" + (org-map-entries #'point "CATEGORY=\"foo\"")))) + ;; Todo match. + (should + (equal '(6) + (org-test-with-temp-text "* H1\n* TODO H2\n* DONE H3" + (org-map-entries #'point "TODO=\"TODO\"")))) ;; Tag match. (should (equal '(11) @@ -2948,7 +2961,7 @@ SCHEDULED: <2014-03-04 tue.>" :END: * H3" (org-map-entries #'point "TEST!=*1")))) - ;; Property matches on names including minus characters. + ;; Property matches on names containing quoted characters. (org-test-with-temp-text " * H1 :BAR: @@ -2967,11 +2980,12 @@ SCHEDULED: <2014-03-04 tue.>" :PROPERTIES: :-FOO: 2 :END: -* H5" - (should (equal '(2) (org-map-entries #'point "TEST-FOO!=*0-FOO"))) - (should (equal '(2) (org-map-entries #'point "-FOO+TEST-FOO!=*0"))) - (should (equal '(88) (org-map-entries #'point "+-FOO!=*0-FOO"))) - (should (equal '(88) (org-map-entries #'point "-FOO+-FOO!=*0")))) +* H5 :TEST:" + (should (equal '(2) (org-map-entries #'point "TEST\\-FOO!=*0-FOO"))) + (should (equal '(2) (org-map-entries #'point "-FOO+TEST\\-FOO!=*0"))) + (should (equal '(88) (org-map-entries #'point "\\-FOO!=*0-FOO"))) + (should (equal '(88) (org-map-entries #'point "-FOO+\\-FOO!=*0"))) + (should (equal '(88) (org-map-entries #'point "-TEST-FOO-TEST\\-FOO=1")))) ;; Multiple criteria. (should (equal '(23) -- 2.30.2 ^ permalink raw reply related [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-08-26 12:54 ` Jens Schmidt @ 2023-08-27 7:11 ` Samuel Loury 2023-08-27 7:43 ` Ihor Radchenko 2023-08-30 16:28 ` [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] Jens Schmidt 2023-09-03 6:53 ` Ihor Radchenko 2 siblings, 1 reply; 31+ messages in thread From: Samuel Loury @ 2023-08-27 7:11 UTC (permalink / raw) To: Jens Schmidt, Ihor Radchenko; +Cc: emacs-orgmode [-- Attachment #1: Type: text/plain, Size: 660 bytes --] I have a dumb question. IIUC, it needs a lot of effort to deal with implicit & correctly. I initially used it because de manual said it was ok, but I would have used explicit & if the manual had said so. I wonder if we could just stop saying that & is optional and have a simpler parsing. IMHO, "-tag&-todo=TODO" is totally ok. I even imagine we could say that & and | are forbidden to say anything else than AND and OR and people would be ok with that. IOW, I wonder of the time and effort to deal with optional & is worth it. WDYT? -- Konubinix GPG Key : 7439106A Fingerprint: 5993 BE7A DA65 E2D9 06CE 5C36 75D2 3CED 7439 106A [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 487 bytes --] ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-08-27 7:11 ` Samuel Loury @ 2023-08-27 7:43 ` Ihor Radchenko 2023-09-01 16:48 ` Jens Schmidt 0 siblings, 1 reply; 31+ messages in thread From: Ihor Radchenko @ 2023-08-27 7:43 UTC (permalink / raw) To: Samuel Loury; +Cc: Jens Schmidt, emacs-orgmode Samuel Loury <konubinix@gmail.com> writes: > IMHO, "-tag&-todo=TODO" is totally ok. I even imagine we could say that > & and | are forbidden to say anything else than AND and OR and people > would be ok with that. Actually, explicit & or | might be an easier way to not worry about escaping things. Except escaping & or | themselves. It might be the preferred way to put into the manual. > IOW, I wonder of the time and effort to deal with optional & is worth > it. WDYT? We cannot remove the optionality of &, because doing so will break existing user setups. But we can certainly change our recommendations in the manual. Though pure tag matcher makes more sense with implicit &: +tag1-tag2+tag3... vs +tag1&-tag2&+tag3 Especially in the interactive agenda filters. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-08-27 7:43 ` Ihor Radchenko @ 2023-09-01 16:48 ` Jens Schmidt 2023-09-01 23:59 ` Tom Gillespie 2023-09-02 7:10 ` Ihor Radchenko 0 siblings, 2 replies; 31+ messages in thread From: Jens Schmidt @ 2023-09-01 16:48 UTC (permalink / raw) To: Ihor Radchenko, Samuel Loury; +Cc: emacs-orgmode On 2023-08-27 09:43, Ihor Radchenko wrote: > Samuel Loury <konubinix@gmail.com> writes: > >> IMHO, "-tag&-todo=TODO" is totally ok. I even imagine we could say that >> & and | are forbidden to say anything else than AND and OR and people >> would be ok with that. > > Actually, explicit & or | might be an easier way to not worry about > escaping things. Except escaping & or | themselves. > It might be the preferred way to put into the manual. > >> IOW, I wonder of the time and effort to deal with optional & is worth >> it. WDYT? > > We cannot remove the optionality of &, because doing so will break > existing user setups. But we can certainly change our recommendations in > the manual. > > Though pure tag matcher makes more sense with implicit &: > +tag1-tag2+tag3... vs +tag1&-tag2&+tag3 > Especially in the interactive agenda filters. I feel it's a bit hard to reply to this message in the right places, so I'll do a plain bottom post, sorry. TL;DR: - I think we cannot make "&" mandatory because of backward compatibility. - Even if we made "&" mandatory, it would not really solve the quoting problem, since the parser is rather hacky and other quoting and context issues would still be there. - But then I cannot see any real reason why we should recommend using "&" in the manual. - Finally, we should hope for and work towards a "real parser", as Ihor has mentioned already in a previous post: (beginning of thread) https://list.orgmode.org/orgmode/9132e58f-d89e-f7df-bbe4-43d53a2367d2@vodafonemail.de/ (mentioning of peg) https://list.orgmode.org/orgmode/87wmyendr7.fsf@localhost/ Details, as far as still required: The whole tag/property query parser seems to have a long tradition. For example, there is still that TODO-state special case that allows for queries like this (example ripped from the manual): work/WAITING Same as work+TODO="WAITING" So people *have* been worrying about a) query brevity and b) backward compatibility, and I think we should do so as well. Then the current parser is horribly ad-hoc-ish and, as a plain regexp-based parser, context insensitive. For example, a tag regexp match {regexp} is matched simply as {[^}]+} that is, you have no chance whatsoever to quote the closing curly parenthesis. Likewise for double-quoted strings. In other words, even if we make | or & mandatory, there will still remain a lot of ambiguities stemming from the rest of the parser. And I'm too lazy to think about how we could quote & in that whole picture. (| is sort of handled already, but again in a completely context-insensitive way.) This all calls for a proper parser, based on peg or bovine or whatever. Hopefully that parser would still keep backward compatibility, but that's probably wishful thinking. ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-09-01 16:48 ` Jens Schmidt @ 2023-09-01 23:59 ` Tom Gillespie 2023-09-02 0:02 ` Tom Gillespie 2023-09-02 7:10 ` Ihor Radchenko 1 sibling, 1 reply; 31+ messages in thread From: Tom Gillespie @ 2023-09-01 23:59 UTC (permalink / raw) To: Jens Schmidt; +Cc: Ihor Radchenko, Samuel Loury, emacs-orgmode Without wading too far into this, why do we need escape syntax for this? The only character that might need an escape would be colon :, but my reading of the syntax doc is that colo : will immediately terminate the property, so we would update the doc to make it clear that property names cannot contain a colon. As written, if there is an issue with the minus sign in property names then that is a bug, but I feel like I might be missing something? Tom ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-09-01 23:59 ` Tom Gillespie @ 2023-09-02 0:02 ` Tom Gillespie 0 siblings, 0 replies; 31+ messages in thread From: Tom Gillespie @ 2023-09-02 0:02 UTC (permalink / raw) To: Jens Schmidt; +Cc: Ihor Radchenko, Samuel Loury, emacs-orgmode Ignore the previous message. I see that this was about matching tags not about specifying them. Best, Tom ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-09-01 16:48 ` Jens Schmidt 2023-09-01 23:59 ` Tom Gillespie @ 2023-09-02 7:10 ` Ihor Radchenko 2023-09-02 13:14 ` Redoing the current tag/property parser in a real grammar [was: Re: [RFC] Quoting property names in tag/property matches] Jens Schmidt 2023-09-02 13:18 ` [RFC] Quoting property names in tag/property matches Jens Schmidt 1 sibling, 2 replies; 31+ messages in thread From: Ihor Radchenko @ 2023-09-02 7:10 UTC (permalink / raw) To: Jens Schmidt; +Cc: Samuel Loury, emacs-orgmode Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes: > TL;DR: > > - I think we cannot make "&" mandatory because of backward compatibility. Sorry for the confusion. I did not mean that "&" should be mandatory, just that "&" might make it easier to avoid a need to escape things. So, it could be used _instead_ of escaping. > - Even if we made "&" mandatory, it would not really solve the quoting > problem, since the parser is rather hacky and other quoting and > context issues would still be there. But at this point you are more familiar with that parser than I am, so my idea does not seem to be viable. > This all calls for a proper parser, based on peg or bovine or whatever. > Hopefully that parser would still keep backward compatibility, but > that's probably wishful thinking. Backward compatibility will be easy - just leave the current code when old query version is detected. We should better focus on the new syntax in future and leave the current syntax as compatibility layer that will be eventually deprecated. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Redoing the current tag/property parser in a real grammar [was: Re: [RFC] Quoting property names in tag/property matches] 2023-09-02 7:10 ` Ihor Radchenko @ 2023-09-02 13:14 ` Jens Schmidt 2023-09-03 7:04 ` Ihor Radchenko 2023-09-02 13:18 ` [RFC] Quoting property names in tag/property matches Jens Schmidt 1 sibling, 1 reply; 31+ messages in thread From: Jens Schmidt @ 2023-09-02 13:14 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Samuel Loury, emacs-orgmode On 2023-09-02 09:10, Ihor Radchenko wrote: > Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes: >> This all calls for a proper parser, based on peg or bovine or whatever. >> Hopefully that parser would still keep backward compatibility, but >> that's probably wishful thinking. > > Backward compatibility will be easy - just leave the current code when > old query version is detected. We should better focus on the new syntax > in future and leave the current syntax as compatibility layer that will > be eventually deprecated. Agreed except for the deprecation part. I think Org should be big enough to have two parsers: One along the lines of the current one (infix, DWIM, easy to type) and one along the lines of org-ql (sexp, better extensible, more flexible, harder to type). Ideally, it should be even possible to embed the infix-one into the sexp-one. It should also be possible to put the current infix parser onto a more stable ground as well, based on a formal grammar, providing at least parentheses for grouping and negation, and that without breaking backward compatibility. Let's rephrase that way: If I were to redo the current parser as mentioned in the previous paragraph, would these changes "eventually be deprecated"? (Which doesn't necessarily mean that I promise to do so, of course.) ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Redoing the current tag/property parser in a real grammar [was: Re: [RFC] Quoting property names in tag/property matches] 2023-09-02 13:14 ` Redoing the current tag/property parser in a real grammar [was: Re: [RFC] Quoting property names in tag/property matches] Jens Schmidt @ 2023-09-03 7:04 ` Ihor Radchenko 0 siblings, 0 replies; 31+ messages in thread From: Ihor Radchenko @ 2023-09-03 7:04 UTC (permalink / raw) To: Jens Schmidt; +Cc: Samuel Loury, emacs-orgmode Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes: >> Backward compatibility will be easy - just leave the current code when >> old query version is detected. We should better focus on the new syntax >> in future and leave the current syntax as compatibility layer that will >> be eventually deprecated. > > Agreed except for the deprecation part. I think Org should be big enough > to have two parsers: One along the lines of the current one (infix, DWIM, > easy to type) and one along the lines of org-ql (sexp, better extensible, > more flexible, harder to type). Ideally, it should be even possible to > embed the infix-one into the sexp-one. Maybe. The rough idea is to allow pluggable query syntax, so that people can implement they own, if they wish to. > It should also be possible to put the current infix parser onto a more > stable ground as well, based on a formal grammar, providing at least > parentheses for grouping and negation, and that without breaking backward > compatibility. > > Let's rephrase that way: If I were to redo the current parser as > mentioned in the previous paragraph, would these changes "eventually be > deprecated"? (Which doesn't necessarily mean that I promise to do so, > of course.) I am not sure if we need to hold to the current syntax. I am leaning towards something more similar to notmuch/"export" web search queries. In any case, deprecation is years away, and we will have plenty of time discussing the specifics. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches 2023-09-02 7:10 ` Ihor Radchenko 2023-09-02 13:14 ` Redoing the current tag/property parser in a real grammar [was: Re: [RFC] Quoting property names in tag/property matches] Jens Schmidt @ 2023-09-02 13:18 ` Jens Schmidt 1 sibling, 0 replies; 31+ messages in thread From: Jens Schmidt @ 2023-09-02 13:18 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Samuel Loury, emacs-orgmode On 2023-09-02 09:10, Ihor Radchenko wrote: > Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes: >> - Even if we made "&" mandatory, it would not really solve the quoting >> problem, since the parser is rather hacky and other quoting and >> context issues would still be there. > > But at this point you are more familiar with that parser than I am, so > my idea does not seem to be viable. I think so, yes. That would mean that my pending patch could be merged, right? (Sorry for being impertinent here, but I'd like to get that thing off my stack.) ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-08-26 12:54 ` Jens Schmidt 2023-08-27 7:11 ` Samuel Loury @ 2023-08-30 16:28 ` Jens Schmidt 2023-08-31 8:08 ` Ihor Radchenko 2023-09-03 6:53 ` Ihor Radchenko 2 siblings, 1 reply; 31+ messages in thread From: Jens Schmidt @ 2023-08-30 16:28 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Samuel Loury, emacs-orgmode On 2023-08-26 14:54, Jens Schmidt wrote: > On 2023-08-26 14:22, Ihor Radchenko wrote: >> Looks good to me. > > Implemented in the next version of the patch, please check. Gentle ping ... ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-08-30 16:28 ` [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] Jens Schmidt @ 2023-08-31 8:08 ` Ihor Radchenko 2023-08-31 10:24 ` Jens Schmidt 0 siblings, 1 reply; 31+ messages in thread From: Ihor Radchenko @ 2023-08-31 8:08 UTC (permalink / raw) To: Jens Schmidt; +Cc: Samuel Loury, emacs-orgmode Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes: >> Implemented in the next version of the patch, please check. > > Gentle ping ... I was waiting for your comment on https://list.orgmode.org/orgmode/874jklhqw2.fsf@localhost/ -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-08-31 8:08 ` Ihor Radchenko @ 2023-08-31 10:24 ` Jens Schmidt 0 siblings, 0 replies; 31+ messages in thread From: Jens Schmidt @ 2023-08-31 10:24 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Samuel Loury, emacs-orgmode On 2023-08-31 10:08, Ihor Radchenko wrote: > Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes: > >>> Implemented in the next version of the patch, please check. >> >> Gentle ping ... > > I was waiting for your comment on https://list.orgmode.org/orgmode/874jklhqw2.fsf@localhost/ Ah, ok, sorry. I will give that branch a second thought and comment soon. ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-08-26 12:54 ` Jens Schmidt 2023-08-27 7:11 ` Samuel Loury 2023-08-30 16:28 ` [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] Jens Schmidt @ 2023-09-03 6:53 ` Ihor Radchenko 2023-09-03 9:25 ` Jens Schmidt 2 siblings, 1 reply; 31+ messages in thread From: Ihor Radchenko @ 2023-09-03 6:53 UTC (permalink / raw) To: Jens Schmidt; +Cc: Samuel Loury, emacs-orgmode Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes: > Implemented in the next version of the patch, please check. > > From 11dc3ac4ff060f1ffb9dae7b35eabe526bbbc572 Mon Sep 17 00:00:00 2001 > From: Jens Schmidt <jschmidt4gnu@vodafonemail.de> > Date: Thu, 24 Aug 2023 22:38:02 +0200 > Subject: [PATCH] org-make-tags-matcher: Re-add quoting of property names Applied, onto main. https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=234650af2 Fixed. I can no longer reproduce the recipe. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] 2023-09-03 6:53 ` Ihor Radchenko @ 2023-09-03 9:25 ` Jens Schmidt 0 siblings, 0 replies; 31+ messages in thread From: Jens Schmidt @ 2023-09-03 9:25 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Samuel Loury, emacs-orgmode On 2023-09-03 08:53, Ihor Radchenko wrote: > Jens Schmidt <jschmidt4gnu@vodafonemail.de> writes: > >> Implemented in the next version of the patch, please check. >> > > Applied, onto main. Thanks! ^ permalink raw reply [flat|nested] 31+ messages in thread
end of thread, other threads:[~2023-09-03 9:27 UTC | newest] Thread overview: 31+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2023-08-23 7:57 [BUG?] Matching tags: & operator no more implicit between tags and special property Samuel Loury 2023-08-23 10:21 ` Jens Schmidt 2023-08-23 10:37 ` Ihor Radchenko 2023-08-23 10:31 ` Ihor Radchenko 2023-08-23 10:38 ` Jens Schmidt 2023-08-23 14:00 ` [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] Jens Schmidt 2023-08-23 15:55 ` Jens Schmidt 2023-08-24 7:30 ` Ihor Radchenko 2023-08-24 7:32 ` Ihor Radchenko 2023-08-24 8:52 ` Jens Schmidt 2023-08-25 18:46 ` Jens Schmidt 2023-08-26 10:16 ` Ihor Radchenko 2023-08-26 11:53 ` Jens Schmidt 2023-08-26 12:00 ` Ihor Radchenko 2023-08-26 12:19 ` Jens Schmidt 2023-08-26 12:22 ` Ihor Radchenko 2023-08-26 12:54 ` Jens Schmidt 2023-08-27 7:11 ` Samuel Loury 2023-08-27 7:43 ` Ihor Radchenko 2023-09-01 16:48 ` Jens Schmidt 2023-09-01 23:59 ` Tom Gillespie 2023-09-02 0:02 ` Tom Gillespie 2023-09-02 7:10 ` Ihor Radchenko 2023-09-02 13:14 ` Redoing the current tag/property parser in a real grammar [was: Re: [RFC] Quoting property names in tag/property matches] Jens Schmidt 2023-09-03 7:04 ` Ihor Radchenko 2023-09-02 13:18 ` [RFC] Quoting property names in tag/property matches Jens Schmidt 2023-08-30 16:28 ` [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property] Jens Schmidt 2023-08-31 8:08 ` Ihor Radchenko 2023-08-31 10:24 ` Jens Schmidt 2023-09-03 6:53 ` Ihor Radchenko 2023-09-03 9:25 ` Jens Schmidt
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).