From 6e98356dfaf3466288398ff4ecee7fd147c32a20 Mon Sep 17 00:00:00 2001 From: Jens Schmidt Date: Sun, 6 Aug 2023 16:38:04 +0200 Subject: [PATCH] org-make-tags-matcher: Add starred property operators, fix quoting * lisp/org.el (org-make-tags-matcher): Add starred property operators. Recognize additional operators "==", "!=", "/=". Clean up and document match term parsing. Remove needless and buggy unquoting of minus characters in property and tag names. (org-op-to-function): Recognize additional inequality operator "/=". * doc/org-manual.org (Matching tags and properties): Add documentation on starred and additional operators. Document allowed characters in property names and handling of minus characters in property names. * testing/lisp/test-org.el (test-org/map-entries): Add tests for starred and additional operators. Add tests for property names containing minus characters. * etc/ORG-NEWS: (~org-tags-view~ supports more property operators): Add announcement on starred and additional operators. Link: https://orgmode.org/list/9132e58f-d89e-f7df-bbe4-43d53a2367d2@vodafonemail.de --- doc/org-manual.org | 35 +++++++++++- etc/ORG-NEWS | 10 +++- lisp/org.el | 120 ++++++++++++++++++++++++++++----------- testing/lisp/test-org.el | 64 ++++++++++++++++++++- 4 files changed, 192 insertions(+), 37 deletions(-) diff --git a/doc/org-manual.org b/doc/org-manual.org index e59efc417..dc9f552e5 100644 --- a/doc/org-manual.org +++ b/doc/org-manual.org @@ -9246,16 +9246,18 @@ When matching properties, a number of different operators can be used to test the value of a property. Here is a complex example: #+begin_example -+work-boss+PRIORITY="A"+Coffee="unlimited"+Effort<2 ++work-boss+PRIORITY="A"+Coffee="unlimited"+Effort<*2 +With={Sarah\|Denny}+SCHEDULED>="<2008-10-11>" #+end_example +#+cindex: operator, for property search #+texinfo: @noindent The type of comparison depends on how the comparison value is written: - If the comparison value is a plain number, a numerical comparison is done, and the allowed operators are =<=, ===, =>=, =<==, =>==, and - =<>=. + =<>=. As a synonym for the equality operator ===, there is also + ====; =!== and =/== are synonyms of the inequality operator =<>=. - If the comparison value is enclosed in double-quotes, a string comparison is done, and the same operators are allowed. @@ -9273,6 +9275,13 @@ The type of comparison depends on how the comparison value is written: is performed, with === meaning that the regexp matches the property value, and =<>= meaning that it does not match. +- All operators may be optionally followed by an asterisk =*=, like in + =<*=, =!=*=, etc. Such /starred operators/ work like their regular, + unstarred counterparts except that they match only headlines where + the tested property is actually present. This is most useful for + search terms that logically exclude results, like the inequality + operator. + So the search string in the example finds entries tagged =work= but not =boss=, which also have a priority value =A=, a =Coffee= property with the value =unlimited=, an =EFFORT= property that is numerically @@ -9280,6 +9289,28 @@ smaller than 2, a =With= property that is matched by the regular expression =Sarah\|Denny=, and that are scheduled on or after October 11, 2008. +Note that the test on the =EFFORT= property uses operator =<*=, so +that the search result will include only entries that actually have an +=EFFORT= property defined and with numerical value smaller than 2. +With the regular =<= operator, the search would handle entries without +an =EFFORT= property as having a zero effort and would include them in +the result as well. + +Currently, you can use only property names including alphanumeric +characters, underscores, and minus characters in search strings. In +addition, if you want to search for a property whose name starts with +a minus character, you have to "quote" that leading minus character +with an explicit positive selection plus character, like this: + +#+begin_example ++-long-and-twisted-property-name-="foo" +#+end_example + +#+texinfo: @noindent +Without that extra plus character, the minus character would be taken +to indicate a negative selection on search term +=long-and-twisted-property-name-​="foo"=. + You can configure Org mode to use property inheritance during a search, but beware that this can slow down searches considerably. See [[*Property Inheritance]], for details. diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS index 4f16eda24..10c51e354 100644 --- a/etc/ORG-NEWS +++ b/etc/ORG-NEWS @@ -125,7 +125,7 @@ New functions to retrieve and set (via ~setf~) commonly used element properties: - =:contents-post-affiliated= :: ~org-element-post-affiliated~ - =:contents-post-blank= :: ~org-element-post-blank~ - =:parent= :: ~org-element-parent~ - + ***** New macro ~org-element-with-enabled-cache~ The macro arranges the element cache to be active during =BODY= execution. @@ -558,6 +558,14 @@ special repeaters ~++~ and ~.+~ are skipped. A capture template can target ~(here)~ which is the equivalent of invoking a capture template with a zero prefix. +*** ~org-tags-view~ supports more property operators + +It supports inequality operators ~!=~ and ~/=~ in addition to the less +common (BASIC? Pascal? SQL?) ~<>~. And it supports starred versions +of all relational operators (~<*~, ~=*~, ~!=*~, etc.) that work like +the regular, unstarred operators but match a headline only if the +tested property is actually present. + ** New functions and changes in function arguments *** =TYPES= argument in ~org-element-lineage~ can now be a symbol diff --git a/lisp/org.el b/lisp/org.el index ed75f3edb..c037b3ee0 100644 --- a/lisp/org.el +++ b/lisp/org.el @@ -11304,15 +11304,50 @@ See also `org-scan-tags'." "Match: " 'org-tags-completion-function nil nil nil 'org-tags-history)))) - (let ((match0 match) - (re (concat - "^&?\\([-+:]\\)?\\({[^}]+}\\|LEVEL\\([<=>]\\{1,2\\}\\)" - "\\([0-9]+\\)\\|\\(\\(?:[[:alnum:]_]+\\(?:\\\\-\\)*\\)+\\)" - "\\([<>=]\\{1,2\\}\\)" - "\\({[^}]+}\\|\"[^\"]*\"\\|-?[.0-9]+\\(?:[eE][-+]?[0-9]+\\)?\\)" - "\\|" org-tag-re "\\)")) - (start 0) - tagsmatch todomatch tagsmatcher todomatcher) + (let* ((match0 match) + (opre "[<=>]=?\\|[!/]=\\|<>") + (re (concat + "^" + ;; implicit AND operator (OR is done by global splitting) + "&?" + ;; exclusion and inclusion (the latter being implicit) + "\\(?1:[-+:]\\)?" + ;; query term + "\\(?2:" + ;; tag regexp match + "{[^}]+}\\|" + ;; LEVEL property match. For sake of consistency, + ;; recognize starred operators here as well. We do + ;; not need to process them below, however, since + ;; the LEVEL property is always present. + "LEVEL\\(?3:" opre "\\)\\*?\\(?4:[0-9]+\\)\\|" + ;; regular property match + "\\(?:" + ;; property name [1] + "\\(?5:[[:alnum:]_-]+\\)" + ;; operator, optionally starred + "\\(?6:" opre "\\)\\(?7:\\*\\)?" + ;; operand (regexp, double-quoted string, + ;; number) + "\\(?8:" + "{[^}]+}\\|" + "\"[^\"]*\"\\|" + "-?[.0-9]+\\(?:[eE][-+]?[0-9]+\\)?" + "\\)" + "\\)\\|" + ;; exact tag match + org-tag-re + "\\)")) + (start 0) + tagsmatch todomatch tagsmatcher todomatcher) + + ;; [1] The minus characters in property names do *not* conflict + ;; with the exclusion operator above, since the mandatory + ;; following operator distinguishes these both cases. + ;; Accordingly, minus characters do not need any special quoting, + ;; even if https://orgmode.org/list/87jzv67k3p.fsf@localhost and + ;; commit 19b0e03f32c6032a60150fc6cb07c6f766cb3f6c suggest + ;; otherwise. ;; Expand group tags. (setq match (org-tags-expand match)) @@ -11352,15 +11387,16 @@ See also `org-scan-tags'." (let* ((rest (substring term (match-end 0))) (minus (and (match-end 1) (equal (match-string 1 term) "-"))) - (tag (save-match-data - (replace-regexp-in-string - "\\\\-" "-" (match-string 2 term)))) + ;; Bind the whole query term to `tag' and use that + ;; variable for a tag regexp match in [2] or as an + ;; exact tag match in [3]. + (tag (match-string 2 term)) (regexp (eq (string-to-char tag) ?{)) (levelp (match-end 4)) (propp (match-end 5)) (mm (cond - (regexp + (regexp ; [2] `(with-syntax-table org-mode-tags-syntax-table (org-match-any-p ,(substring tag 1 -1) tags-list))) (levelp @@ -11368,28 +11404,46 @@ See also `org-scan-tags'." level ,(string-to-number (match-string 4 term)))) (propp - (let* ((gv (pcase (upcase (match-string 5 term)) + (let* (;; Convert property name to an Elisp + ;; accessor for that property (aka. as + ;; getter value). + (gv (pcase (upcase (match-string 5 term)) ("CATEGORY" '(org-get-category (point))) ("TODO" 'todo) (p `(org-cached-entry-get nil ,p)))) - (pv (match-string 7 term)) + ;; Determine operand (aka. property + ;; value). + (pv (match-string 8 term)) + ;; Determine type of operand. Note that + ;; these are not exclusive: Any TIMEP is + ;; also STRP. (regexp (eq (string-to-char pv) ?{)) (strp (eq (string-to-char pv) ?\")) (timep (string-match-p "^\"[[<]\\(?:[0-9]+\\|now\\|today\\|tomorrow\\|[+-][0-9]+[dmwy]\\).*[]>]\"$" pv)) + ;; Massage operand. TIMEP must come + ;; before STRP. + (pv (cond (regexp (substring pv 1 -1)) + (timep (org-matcher-time + (substring pv 1 -1))) + (strp (substring pv 1 -1)) + (t pv))) + ;; Convert operator to Elisp. (po (org-op-to-function (match-string 6 term) - (if timep 'time strp)))) - (setq pv (if (or regexp strp) (substring pv 1 -1) pv)) - (when timep (setq pv (org-matcher-time pv))) - (cond ((and regexp (eq po '/=)) - `(not (string-match ,pv (or ,gv "")))) - (regexp `(string-match ,pv (or ,gv ""))) - (strp `(,po (or ,gv "") ,pv)) - (t - `(,po - (string-to-number (or ,gv "")) - ,(string-to-number pv)))))) - (t `(member ,tag tags-list))))) + (if timep 'time strp))) + ;; Convert whole property term to Elisp. + (pt (cond ((and regexp (eq po '/=)) + `(not (string-match ,pv (or ,gv "")))) + (regexp `(string-match ,pv (or ,gv ""))) + (strp `(,po (or ,gv "") ,pv)) + (t + `(,po + (string-to-number (or ,gv "")) + ,(string-to-number pv))))) + ;; Respect the star after the operand. + (pt (if (match-end 7) `(and ,gv ,pt) pt))) + pt)) + (t `(member ,tag tags-list))))) ; [3] (push (if minus `(not ,mm) mm) tagsmatcher) (setq term rest))) (push `(and ,@tagsmatcher) orlist) @@ -11520,12 +11574,12 @@ the list of tags in this group." "Turn an operator into the appropriate function." (setq op (cond - ((equal op "<" ) '(< org-string< org-time<)) - ((equal op ">" ) '(> org-string> org-time>)) - ((member op '("<=" "=<")) '(<= org-string<= org-time<=)) - ((member op '(">=" "=>")) '(>= org-string>= org-time>=)) - ((member op '("=" "==")) '(= string= org-time=)) - ((member op '("<>" "!=")) '(/= org-string<> org-time<>)))) + ((equal op "<" ) '(< org-string< org-time<)) + ((equal op ">" ) '(> org-string> org-time>)) + ((member op '("<=" "=<" )) '(<= org-string<= org-time<=)) + ((member op '(">=" "=>" )) '(>= org-string>= org-time>=)) + ((member op '("=" "==" )) '(= string= org-time=)) + ((member op '("<>" "!=" "/=")) '(/= org-string<> org-time<>)))) (nth (if (eq stringp 'time) 2 (if stringp 1 0)) op)) (defvar org-add-colon-after-tag-completion nil) ;; dynamically scoped param diff --git a/testing/lisp/test-org.el b/testing/lisp/test-org.el index 890ea6a8c..7c85da9d5 100644 --- a/testing/lisp/test-org.el +++ b/testing/lisp/test-org.el @@ -2833,6 +2833,11 @@ test (equal '(11) (org-test-with-temp-text "* Level 1\n** Level 2" (let (org-odd-levels-only) (org-map-entries #'point "LEVEL>1"))))) + ;; Level match with (ignored) starred operator. + (should + (equal '(11) + (org-test-with-temp-text "* Level 1\n** Level 2" + (let (org-odd-levels-only) (org-map-entries #'point "LEVEL>*1"))))) ;; Tag match. (should (equal '(11) @@ -2845,12 +2850,17 @@ test (should (equal '(11 23) (org-test-with-temp-text "* H1 :no:\n* H2 :yes1:\n* H3 :yes2:" - (org-map-entries #'point "{yes?}")))) + (org-map-entries #'point "{yes.?}")))) ;; Priority match. (should (equal '(1) (org-test-with-temp-text "* [#A] H1\n* [#B] H2" (org-map-entries #'point "PRIORITY=\"A\"")))) + ;; Negative priority match. + (should + (equal '(11) + (org-test-with-temp-text "* [#A] H1\n* [#B] H2" + (org-map-entries #'point "PRIORITY/=\"A\"")))) ;; Date match. (should (equal '(36) @@ -2881,6 +2891,58 @@ SCHEDULED: <2014-03-04 tue.>" :TEST: 2 :END:" (org-map-entries #'point "TEST=1")))) + ;; Regular negative property match. + (should + (equal '(35 68) + (org-test-with-temp-text " +* H1 +:PROPERTIES: +:TEST: 1 +:END: +* H2 +:PROPERTIES: +:TEST: 2 +:END: +* H3" + (org-map-entries #'point "TEST!=1")))) + ;; Starred negative property match. + (should + (equal '(35) + (org-test-with-temp-text " +* H1 +:PROPERTIES: +:TEST: 1 +:END: +* H2 +:PROPERTIES: +:TEST: 2 +:END: +* H3" + (org-map-entries #'point "TEST!=*1")))) + ;; Property matches on names including minus characters. + (org-test-with-temp-text + " +* H1 :BAR: +:PROPERTIES: +:TEST-FOO: 1 +:END: +* H2 :FOO: +:PROPERTIES: +:TEST-FOO: 2 +:END: +* H3 :BAR: +:PROPERTIES: +:-FOO: 1 +:END: +* H4 :FOO: +:PROPERTIES: +:-FOO: 2 +:END: +* H5" + (should (equal '(2) (org-map-entries #'point "TEST-FOO!=*0-FOO"))) + (should (equal '(2) (org-map-entries #'point "-FOO+TEST-FOO!=*0"))) + (should (equal '(88) (org-map-entries #'point "+-FOO!=*0-FOO"))) + (should (equal '(88) (org-map-entries #'point "-FOO+-FOO!=*0")))) ;; Multiple criteria. (should (equal '(23) -- 2.30.2