From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ilya Shlyakhter Subject: Re: [PATCH] tags search: faster tags matcher by trusting scanner tags Date: Fri, 16 Mar 2012 00:34:03 -0400 Message-ID: References: <3953.1331867583@alphaville> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary=bcaec5015f3fe683dd04bb54b731 Return-path: Received: from eggs.gnu.org ([208.118.235.92]:53749) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S8Org-0006VF-0Y for emacs-orgmode@gnu.org; Fri, 16 Mar 2012 00:34:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1S8Ord-0007iP-QM for emacs-orgmode@gnu.org; Fri, 16 Mar 2012 00:34:07 -0400 Received: from mail-vx0-f169.google.com ([209.85.220.169]:41725) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S8Ord-0007iF-Hp for emacs-orgmode@gnu.org; Fri, 16 Mar 2012 00:34:05 -0400 Received: by vcbfk14 with SMTP id fk14so4965510vcb.0 for ; Thu, 15 Mar 2012 21:34:04 -0700 (PDT) In-Reply-To: List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: emacs-orgmode Cc: nicholas.dokos@hp.com --bcaec5015f3fe683dd04bb54b731 Content-Type: multipart/alternative; boundary=bcaec5015f3fe683d904bb54b72f --bcaec5015f3fe683d904bb54b72f Content-Type: text/plain; charset=ISO-8859-1 Here is a similar patch for org-clock's use of tags/properties matcher. On Fri, Mar 16, 2012 at 12:31 AM, Ilya Shlyakhter wrote: > ,---- > | If your function needs to retrieve the tags including inherited tags > | at the *current* entry, > > 'Function' here refers to the FUNC parameter of org-map-entries, not the > MATCHER parameter. > The matcher is constructed by org-make-tags-matcher, so we know everything > it does -- > it does not move around and only asks about the current entry's tags and > properties. > org-scan-tags only invokes the matcher at the current entry, and sets > org-scanner-tags > correctly for that call. > > But, you're right that there is a problem: while org-scan-tags sets > org-scanner-tags correctly > before (eval matcher), other users of the matcher -- e.g. > org-clock-get-table-data -- might not. > So, org-trust-scanner-tags should be set not in the matcher, but in the > function that calls > the matcher. A corrected patch is attached. > > thanks, > > ilya > > > On Thu, Mar 15, 2012 at 11:13 PM, Nick Dokos wrote: > >> Ilya Shlyakhter wrote: >> >> > The attached patch speeds up tags matching ( 50s --> 5s for my most >> > common search ), by turning on org-trust-scanner-tags within the >> > matcher. >> > >> > (When it's off, getting a non-inherited property's value causes a call >> > to org-entry-properties to fetch all properties into a cache, >> > including ALLTAGS; fetching ALLTAGS involves calling >> > (org-get-tags-at), which is slow when org-trust-scanner-tags is off.) >> > >> > Can this cause problems / was this off for a reason? >> > >> >> I haven't looked at your patch carefully enough to know if it will or >> will not cause problems, but check the doc for org-map-entries: it has >> some guidelines about where the technique can be used and where it >> cannot: >> >> ,---- >> | If your function needs to retrieve the tags including inherited tags >> | at the *current* entry, you can use the value of the variable >> | `org-scanner-tags' which will be much faster than getting the value >> | with `org-get-tags-at'. If your function gets properties with >> | `org-entry-properties' at the *current* entry, bind >> `org-trust-scanner-tags' >> | to t around the call to `org-entry-properties' to get the same speedup. >> | Note that if your function moves around to retrieve tags and properties >> at >> | a *different* entry, you cannot use these techniques. >> `---- >> >> >> There are warnings that this variable is for internal dynamical scoping >> only, >> so I suspect you should not mess with the default. If your search can make >> the needed guarantees, then you can just wrap it in a let to get the >> speedup. >> Otherwise, it probably should be left alone. >> >> Nick >> >> > --bcaec5015f3fe683d904bb54b72f Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Here is a similar patch for org-clock's use of tags/properties matcher.=

On Fri, Mar 16, 2012 at 12:31 AM, Ilya S= hlyakhter <il= ya_shl@alum.mit.edu> wrote:
,----
| If your function needs to retrieve the tags including inherited tags
| at the *current* entry,

'Function' here refers to t= he FUNC parameter of org-map-entries, not the MATCHER parameter.
The mat= cher is constructed by org-make-tags-matcher, so we know everything it does= --
it does not move around and only asks about the current entry's tags an= d properties.
org-scan-tags only invokes the matcher at the current entr= y, and sets org-scanner-tags
correctly for that call.

But, you= 9;re right that there is a problem: while org-scan-tags sets org-scanner-ta= gs correctly
before (eval matcher), other users of the matcher -- e.g. org-clock-get-tab= le-data -- might not.
So, org-trust-scanner-tags should be set not in th= e matcher, but in the function that calls
the matcher.=A0 A corrected pa= tch is attached.

thanks,

ilya


On Thu, Mar 15, 2012 at 11:13 PM, Nick Dokos <n= icholas.dokos@hp.com> wrote:
Ilya Shlyakhter <ilya_shl@alum.mit.edu> wrote:

> The attached patch speeds up tags matching ( 50s --> 5s for my most=
> common search ), by turning on org-trust-scanner-tags within the
> matcher.
>
> (When it's off, getting a non-inherited property's value cause= s a call
> to org-entry-properties to fetch all properties into a cache,
> including ALLTAGS; fetching ALLTAGS involves calling
> (org-get-tags-at), which is slow when org-trust-scanner-tags is off.)<= br> >
> Can this cause problems / was this off for a reason?
>

I haven't looked at your patch carefully enough to know if = it will or
will not cause problems, but check the doc for org-map-entries: it has
some guidelines about where the technique can be used and where it
cannot:

,----
| If your function needs to retrieve the tags including inherited tags
| at the *current* entry, you can use the value of the variable
| `org-scanner-tags' which will be much faster than getting the value | with `org-get-tags-at'. =A0If your function gets properties with
| `org-entry-properties' at the *current* entry, bind `org-trust-scanne= r-tags'
| to t around the call to `org-entry-properties' to get the same speedu= p.
| Note that if your function moves around to retrieve tags and properties a= t
| a *different* entry, you cannot use these techniques.
`----


There are warnings that this variable is for internal dynamical scoping onl= y,
so I suspect you should not mess with the default. If your search can make<= br> the needed guarantees, then you can just wrap it in a let to get the speedu= p.
Otherwise, it probably should be left alone.

Nick



--bcaec5015f3fe683d904bb54b72f-- --bcaec5015f3fe683dd04bb54b731 Content-Type: application/octet-stream; name="0003-Clocking-work-time-faster-filtering-of-clock-entries.patch" Content-Disposition: attachment; filename="0003-Clocking-work-time-faster-filtering-of-clock-entries.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_gzuqfxxj1 RnJvbSA0ZjdmOTFhZTYyZDQyNWY3YTg5NzM4YjI4MDA2YjE3NDNhNmJlYTRkIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBJbHlhIFNobHlha2h0ZXIgPGlseWFfc2hsQGFsdW0ubWl0LmVk dT4KRGF0ZTogRnJpLCAxNiBNYXIgMjAxMiAwMDoyNToxOCAtMDQwMApTdWJqZWN0OiBbUEFUQ0gg My8zXSBDbG9ja2luZyB3b3JrIHRpbWU6IGZhc3RlciBmaWx0ZXJpbmcgb2YgY2xvY2sgZW50cmll cyBieQogdHJ1c3RpbmcgIG9yZy1zY2FubmVyLXRhZ3MKCiogbGlzcC9vcmctY2xvY2suZWwgKG9y Zy1jbG9jay1nZXQtdGFibGUtZGF0YSk6IEJpbmQgb3JnLXNjYW5uZXItdGFncwp0byB0YWdzLWxp c3QgYW5kIG9yZy10cnVzdC1zY2FubmVyLXRhZ3MgdG8gdAp3aGlsZSBldmFsdWF0aW5nIHRoZSBt YXRjaGVyLCBzaW5jZSB0aGUgbWF0Y2hlciBpcyBhbHdheXMgZXZhbHVhdGVkCmF0IHRoZSBjdXJy ZW50IGVudHJ5LgoKVElOWUNIQU5HRQotLS0KIGxpc3Avb3JnLWNsb2NrLmVsIHwgICAgNCArKyst CiAxIGZpbGUgY2hhbmdlZCwgMyBpbnNlcnRpb25zKCspLCAxIGRlbGV0aW9uKC0pCgpkaWZmIC0t Z2l0IGEvbGlzcC9vcmctY2xvY2suZWwgYi9saXNwL29yZy1jbG9jay5lbAppbmRleCA5MjA2NjA4 Li40NmQ5YWY4IDEwMDY0NAotLS0gYS9saXNwL29yZy1jbG9jay5lbAorKysgYi9saXNwL29yZy1j bG9jay5lbApAQCAtMjQ2Myw3ICsyNDYzLDkgQEAgVElNRTogICAgICBUaGUgc3VtIG9mIGFsbCB0 aW1lIHNwZW5kIGluIHRoaXMgdHJlZSwgaW4gbWludXRlcy4gIFRoaXMgdGltZQogICAgICAgKG9y Zy1jbG9jay1zdW0gdHMgdGUKIAkJICAgICAodW5sZXNzIChudWxsIG1hdGNoZXIpCiAJCSAgICAg ICAobGFtYmRhICgpCi0JCQkgKGxldCAoKHRhZ3MtbGlzdCAob3JnLWdldC10YWdzLWF0KSkpCisJ CQkgKGxldCogKCh0YWdzLWxpc3QgKG9yZy1nZXQtdGFncy1hdCkpCisJCQkJKG9yZy1zY2FubmVy LXRhZ3MgdGFncy1saXN0KQorCQkJCShvcmctdHJ1c3Qtc2Nhbm5lci10YWdzIHQpKQogCQkJICAg KGV2YWwgbWF0Y2hlcikpKSkpCiAgICAgICAoZ290by1jaGFyIChwb2ludC1taW4pKQogICAgICAg KHNldHEgc3QgdCkKLS0gCjEuNy45LjMKCg== --bcaec5015f3fe683dd04bb54b731--