From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ilya Shlyakhter Subject: Re: [PATCH] tags search: faster tags matcher by trusting scanner tags Date: Fri, 16 Mar 2012 00:31:13 -0400 Message-ID: References: <3953.1331867583@alphaville> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary=20cf307f32dcbb803004bb54adfc Return-path: Received: from eggs.gnu.org ([208.118.235.92]:47428) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S8Oov-0005IL-Eg for emacs-orgmode@gnu.org; Fri, 16 Mar 2012 00:31:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1S8Oot-0007AX-90 for emacs-orgmode@gnu.org; Fri, 16 Mar 2012 00:31:17 -0400 Received: from mail-vx0-f169.google.com ([209.85.220.169]:53348) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S8Oos-0007AJ-Ug for emacs-orgmode@gnu.org; Fri, 16 Mar 2012 00:31:15 -0400 Received: by vcbfk14 with SMTP id fk14so4963467vcb.0 for ; Thu, 15 Mar 2012 21:31:13 -0700 (PDT) In-Reply-To: <3953.1331867583@alphaville> List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: emacs-orgmode Cc: nicholas.dokos@hp.com --20cf307f32dcbb803004bb54adfc Content-Type: multipart/alternative; boundary=20cf307f32dcbb802b04bb54adfa --20cf307f32dcbb802b04bb54adfa Content-Type: text/plain; charset=ISO-8859-1 ,---- | If your function needs to retrieve the tags including inherited tags | at the *current* entry, 'Function' here refers to the FUNC parameter of org-map-entries, not the MATCHER parameter. The matcher is constructed by org-make-tags-matcher, so we know everything it does -- it does not move around and only asks about the current entry's tags and properties. org-scan-tags only invokes the matcher at the current entry, and sets org-scanner-tags correctly for that call. But, you're right that there is a problem: while org-scan-tags sets org-scanner-tags correctly before (eval matcher), other users of the matcher -- e.g. org-clock-get-table-data -- might not. So, org-trust-scanner-tags should be set not in the matcher, but in the function that calls the matcher. A corrected patch is attached. thanks, ilya On Thu, Mar 15, 2012 at 11:13 PM, Nick Dokos wrote: > Ilya Shlyakhter wrote: > > > The attached patch speeds up tags matching ( 50s --> 5s for my most > > common search ), by turning on org-trust-scanner-tags within the > > matcher. > > > > (When it's off, getting a non-inherited property's value causes a call > > to org-entry-properties to fetch all properties into a cache, > > including ALLTAGS; fetching ALLTAGS involves calling > > (org-get-tags-at), which is slow when org-trust-scanner-tags is off.) > > > > Can this cause problems / was this off for a reason? > > > > I haven't looked at your patch carefully enough to know if it will or > will not cause problems, but check the doc for org-map-entries: it has > some guidelines about where the technique can be used and where it > cannot: > > ,---- > | If your function needs to retrieve the tags including inherited tags > | at the *current* entry, you can use the value of the variable > | `org-scanner-tags' which will be much faster than getting the value > | with `org-get-tags-at'. If your function gets properties with > | `org-entry-properties' at the *current* entry, bind > `org-trust-scanner-tags' > | to t around the call to `org-entry-properties' to get the same speedup. > | Note that if your function moves around to retrieve tags and properties > at > | a *different* entry, you cannot use these techniques. > `---- > > > There are warnings that this variable is for internal dynamical scoping > only, > so I suspect you should not mess with the default. If your search can make > the needed guarantees, then you can just wrap it in a let to get the > speedup. > Otherwise, it probably should be left alone. > > Nick > > --20cf307f32dcbb802b04bb54adfa Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable ,----
| If your function needs to retrieve the tags including inherited tags
| at the *current* entry,

'Function' here refers to the FUN= C parameter of org-map-entries, not the MATCHER parameter.
The matcher i= s constructed by org-make-tags-matcher, so we know everything it does -- it does not move around and only asks about the current entry's tags an= d properties.
org-scan-tags only invokes the matcher at the current entr= y, and sets org-scanner-tags
correctly for that call.

But, you= 9;re right that there is a problem: while org-scan-tags sets org-scanner-ta= gs correctly
before (eval matcher), other users of the matcher -- e.g. org-clock-get-tab= le-data -- might not.
So, org-trust-scanner-tags should be set not in th= e matcher, but in the function that calls
the matcher.=A0 A corrected pa= tch is attached.

thanks,

ilya

On Thu, Mar 15, 2= 012 at 11:13 PM, Nick Dokos <nicholas.dokos@hp.com> wrot= e:
Ilya Shlyakhter <ilya_shl@alum.mit.edu> wrote:

> The attached patch speeds up tags matching ( 50s --> 5s for my most=
> common search ), by turning on org-trust-scanner-tags within the
> matcher.
>
> (When it's off, getting a non-inherited property's value cause= s a call
> to org-entry-properties to fetch all properties into a cache,
> including ALLTAGS; fetching ALLTAGS involves calling
> (org-get-tags-at), which is slow when org-trust-scanner-tags is off.)<= br> >
> Can this cause problems / was this off for a reason?
>

I haven't looked at your patch carefully enough to know if = it will or
will not cause problems, but check the doc for org-map-entries: it has
some guidelines about where the technique can be used and where it
cannot:

,----
| If your function needs to retrieve the tags including inherited tags
| at the *current* entry, you can use the value of the variable
| `org-scanner-tags' which will be much faster than getting the value | with `org-get-tags-at'. =A0If your function gets properties with
| `org-entry-properties' at the *current* entry, bind `org-trust-scanne= r-tags'
| to t around the call to `org-entry-properties' to get the same speedu= p.
| Note that if your function moves around to retrieve tags and properties a= t
| a *different* entry, you cannot use these techniques.
`----


There are warnings that this variable is for internal dynamical scoping onl= y,
so I suspect you should not mess with the default. If your search can make<= br> the needed guarantees, then you can just wrap it in a let to get the speedu= p.
Otherwise, it probably should be left alone.

Nick


--20cf307f32dcbb802b04bb54adfa-- --20cf307f32dcbb803004bb54adfc Content-Type: application/octet-stream; name="0002-Tags-properties-matcher-faster-matching-by-trusting-.patch" Content-Disposition: attachment; filename="0002-Tags-properties-matcher-faster-matching-by-trusting-.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_gzupqk6u0 RnJvbSA5NWMzOGIwNjgwM2FlYzA3ODdiYzJlYWFiM2QwMDYyMjIxMzkwMjkyIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBJbHlhIFNobHlha2h0ZXIgPGlseWFfc2hsQGFsdW0ubWl0LmVk dT4KRGF0ZTogRnJpLCAxNiBNYXIgMjAxMiAwMDoxMDoyNSAtMDQwMApTdWJqZWN0OiBbUEFUQ0gg Mi8yXSBUYWdzL3Byb3BlcnRpZXMgbWF0Y2hlcjogZmFzdGVyIG1hdGNoaW5nIGJ5IHRydXN0aW5n CiBvcmctc2Nhbm5lci10YWdzCgoqIGxpc3Avb3JnLmVsIChvcmctc2Nhbi10YWdzKTogQmluZCBv cmctdHJ1c3Qtc2Nhbm5lci10YWdzIHRvIHQKd2hpbGUgZXZhbHVhdGluZyB0aGUgbWF0Y2hlciwg c2luY2UgdGhlIG1hdGNoZXIgaXMgYWx3YXlzIGV2YWx1YXRlZAphdCB0aGUgY3VycmVudCBlbnRy eS4KClRJTllDSEFOR0UKLS0tCiBsaXNwL29yZy5lbCB8ICAgIDMgKystCiAxIGZpbGUgY2hhbmdl ZCwgMiBpbnNlcnRpb25zKCspLCAxIGRlbGV0aW9uKC0pCgpkaWZmIC0tZ2l0IGEvbGlzcC9vcmcu ZWwgYi9saXNwL29yZy5lbAppbmRleCBhZDYzMjEzLi45NTFmNjkyIDEwMDY0NAotLS0gYS9saXNw L29yZy5lbAorKysgYi9saXNwL29yZy5lbApAQCAtMTI5MDYsNyArMTI5MDYsOCBAQCBoZWFkbGlu ZXMgbWF0Y2hpbmcgdGhpcyBzdHJpbmcuIgogCiAJCSA7OyBldmFsIG1hdGNoZXIgb25seSB3aGVu IHRoZSB0b2RvIGNvbmRpdGlvbiBpcyBPSwogCQkgKGFuZCAob3IgKG5vdCB0b2RvLW9ubHkpICht ZW1iZXIgdG9kbyBvcmctbm90LWRvbmUta2V5d29yZHMpKQotCQkgICAgICAobGV0ICgoY2FzZS1m b2xkLXNlYXJjaCB0KSkgKGV2YWwgbWF0Y2hlcikpKQorCQkgICAgICAobGV0ICgoY2FzZS1mb2xk LXNlYXJjaCB0KSAob3JnLXRydXN0LXNjYW5uZXItdGFncyB0KSkKKwkJCSAgIChldmFsIG1hdGNo ZXIpKSkKIAogCQkgOzsgQ2FsbCB0aGUgc2tpcHBlciwgYnV0IHJldHVybiB0IGlmIGl0IGRvZXMg bm90IHNraXAsCiAJCSA7OyBzbyB0aGF0IHRoZSBgYW5kJyBmb3JtIGNvbnRpbnVlcyBldmFsdWF0 aW5nCi0tIAoxLjcuOS4zCgo= --20cf307f32dcbb803004bb54adfc--