New backend for org-agenda: (Was Calendar-like view of the org-agenda)

New backend for org-agenda: (Was Calendar-like view of the org-agenda)

[SAKURAI Masashi]

Hi Bastien,

At Fri, 08 Jul 2011 10:53:08 +0200,
Bastien wrote:
> :
> > the re-design of the whole org-agenda-list algorithm
> > seems to be needed, because the key function
> > org-agenda-get-day-entries requires only one date and the subsequent
> > dependent functions also are designed by the API.
> 
> FWIW, I'm all ears -- if you have ideas on how to make org-agenda-list
> more effective please let us know, I could try to implement them.

How about caching of schedule items?
My idea is as follows.

When one query schedule items for the first time, the backend parses
the org buffer and stores all schedule items with a buffer local
variable.  From the next time of the query, the backend can use the
cache data without parsing the buffer text.

Then, because The backend has a model of schedule items, in addition
to org-agenda-get-day-entries, we can define some useful API for
schedule items, such as org-agenda-get-entries-by-range(begin end).

I illustrated this idea:
[Diagram for caching schedule items]
https://cacoo.com/diagrams/RcNaxILLHDGZlr1L#4B823

Org schedule seems to consist of two types, simple items and repeat
items. So, I think that the backend should execute two type queries.

[Query and Data Structure]
https://cacoo.com/diagrams/RcNaxILLHDGZlr1L#C071E

Of course, because I have used orgmode not so long time, this idea is
not useful or too simple to handle the org schedules. 


Here are other implementations:

The Howm schedule API (howm-reminder-search) enumerates all schedule
items. The API user can filter them easily. However, the enumeration
employs full-text searching using grep command, so it takes a long
time to be done the query.  calfw-howm.el has a cache data and the
user can refresh it explicitly.

Another schedule API is CalDAV. This is a specification for the
schedule server, such as Apple iCal server and Google Calendar.

[Calendaring Extensions to WebDAV (CalDAV) RFC 4791]
http://www.webdav.org/specs/rfc4791.html
http://en.wikipedia.org/wiki/CalDAV

This specification has many API for query. It is very heavy spec, but
I think it may be a hint for designing query API for orgmode schedule
items.


Regards
--
SAKURAI, Masashi (family, given)
m.sakurai@kiwanami.net

Re: New backend for org-agenda: (Was Calendar-like view of the org-agenda)

[Bastien]

Hi Masashi,

this is a very interesting proposal -- thanks for the diagrams and the
pointers to existing schedule API.  I will digg into this direction and
see if I can propose something useful.

The question is: would an API for the whole agenda mechanism (and not
just scheduled items) be useful?

I've never been a big fan of caching Org files information, because Org
files are often modified in impredictible ways.  But if we can update
the cache quickly, then it will certainly be useful.

I will let you know about progress about this.

Thanks again for raising this idea!

-- 
 Bastien

Re: New backend for org-agenda: (Was Calendar-like view of the org-agenda)

[SAKURAI Masashi]

Hi Bastien,

At Tue, 12 Jul 2011 09:37:21 +0200,
Bastien wrote:
> :
> The question is: would an API for the whole agenda mechanism (and not
> just scheduled items) be useful?
> 
> I've never been a big fan of caching Org files information, because Org
> files are often modified in impredictible ways.  But if we can update
> the cache quickly, then it will certainly be useful.

I'm sure that the caching mechanism is useful, but I'm not sure that
we should do it with paying the large cost of rewrite whole codes.

Because the orgmode is the center of information for the many Emacs
users, I think it is very interesting for not only elisp codes but
also the other applications to access the information of the orgmode
via emacsclient or D-Bus messaging.

If the interface for the mode of orgmode items and modification hooks
is defined, one would send todo items to one's mobile terminal
immediately, or share the items on the SNS.

Of course, it is just a trivial idea, not my request.

> I will let you know about progress about this.
> 
> Thanks again for raising this idea!

I'm glad to be able to share my idea.
It was difficult for me to discuss the abstract topic quickly in English.

Thank you
--
SAKURAI, Masashi (family, given)
m.sakurai@kiwanami.net

Re: New backend for org-agenda: (Was Calendar-like view of the org-agenda)

[Bastien]

Hi Masashi,

SAKURAI Masashi <m.sakurai@kiwanami.net> writes:

> I'm sure that the caching mechanism is useful, but I'm not sure that
> we should do it with paying the large cost of rewrite whole codes.

FWIW, this is a two separate steps process: 1) write a usable cache, 
then 2) re-implement (parts of) the agenda by using it.

I think (1) could be interesting /per-se/ and if it helps calfw, then
it's even more interesting.

2 cents,

-- 
 Bastien

Re: New backend for org-agenda: (Was Calendar-like view of the org-agenda)

[Matthew Sauer]

Bastien,

What I am hearing is that an API that allows caching could be built
that could benefit extensions and then once that is deemed stable
enough an optional or separate track would be rebuilding the Agenda as
it is today into that API?

Matt

On Mon, Jul 18, 2011 at 6:06 PM, Bastien <bzg@altern.org> wrote:
> Hi Masashi,
>
> SAKURAI Masashi <m.sakurai@kiwanami.net> writes:
>
>> I'm sure that the caching mechanism is useful, but I'm not sure that
>> we should do it with paying the large cost of rewrite whole codes.
>
> FWIW, this is a two separate steps process: 1) write a usable cache,
> then 2) re-implement (parts of) the agenda by using it.
>
> I think (1) could be interesting /per-se/ and if it helps calfw, then
> it's even more interesting.
>
> 2 cents,
>
> --
>  Bastien
>
>

Re: New backend for org-agenda: (Was Calendar-like view of the org-agenda)

[Bastien]

Hi Matthew,

Matthew Sauer <improv.philosophy@gmail.com> writes:

> What I am hearing is that an API that allows caching could be built
> that could benefit extensions and then once that is deemed stable
> enough 

Yes, stable and, moreover, *fast* enough...

> [...] an optional or separate track would be rebuilding the Agenda as
> it is today into that API?

Yes -- we can dream :)

-- 
 Bastien

Re: New backend for org-agenda: (Was Calendar-like view of the org-agenda)

[SAKURAI Masashi]

Hi Bastien,

At Tue, 19 Jul 2011 01:06:14 +0200,
Bastien wrote:
> :
> > I'm sure that the caching mechanism is useful, but I'm not sure that
> > we should do it with paying the large cost of rewrite whole codes.
> 
> FWIW, this is a two separate steps process: 1) write a usable cache, 
> then 2) re-implement (parts of) the agenda by using it.
> 
> I think (1) could be interesting /per-se/ and if it helps calfw, then
> it's even more interesting.

What does '/per-se/' mean ?
Excuse me for asking a trivial word.

I'm also think about cache in the calfw side, such as an alist:
(date . [a list of contents]). Then, uses can refresh the cache
explicitly. It is easy to implement.

Is the plan(1) the same idea?


Regards,
--
SAKURAI, Masashi (family, given)
m.sakurai@kiwanami.net

Re: New backend for org-agenda: (Was Calendar-like view of the org-agenda)

[Rasmus]

SAKURAI Masashi <m.sakurai@kiwanami.net> writes:
> What does '/per-se/' mean ?
It means 'as such'

Here is the entry from Merriam Webster Unabriged 

,----
| per se
| Function: adverb
| Etymology: Latin
| : by, of, or in itself or oneself or themselves : as such : INDEPENDENTLY, INTRINSICALLY
| <a lover of language per se -- W.T.Scott>
| <not a scientist per se and so he had none of the inhibitions of the
| scientist -- W.L.Howard> 
| <his manufactory of fireworks was per se a public nuisance -- McDade
| vs. City of Chester (Pa.)>
| <the mathematician is not interested in the truth, per se, of his
| postulates -- Harry Lass> 
| <money is evil per se and must be apologized for -- Dwight Macdonald>
| <natural environment cannot per se cause forms of culture --
| A.L.Kroeber>
`----

> Excuse me for asking a trivial word.  No worries.
No worries. 

–Rasmus

-- 
Sent from my Emacs

Re: New backend for org-agenda:

[Giovanni Ridolfi]

SAKURAI Masashi <m.sakurai@kiwanami.net> writes:

Hi Masashi,
>
> At Tue, 19 Jul 2011 01:06:14 +0200,
> Bastien wrote:
>> :
>> > I'm sure that the caching mechanism is useful, but I'm not sure that
>> > we should do it with paying the large cost of rewrite whole codes.
>> 
>> FWIW, this is a two separate steps process: 1) write a usable cache, 
>> then 2) re-implement (parts of) the agenda by using it.
>> 
>> I think (1) could be interesting /per-se/ and if it helps calfw, then
>> it's even more interesting.
>
> What does '/per-se/' mean ?

:-)
it is Latin, an ancient  *European* language
it means:
- 'In itself' 
- Also "by itself" 
or:
- Without referring to anything else, intrinsically, 
  taken without qualifications, 

> Excuse me for asking a trivial word.
You should excuse our Eurocentrism ;-) 

cheers,
Giovanni 

Re: New backend for org-agenda: (Was Calendar-like view of the org-agenda)

[SAKURAI Masashi]

Hi Rasmus and Giovanni,

At Wed, 20 Jul 2011 15:31:04 +0200,
Giovanni Ridolfi wrote:
> :
> > What does '/per-se/' mean ?
> 
> :-)
> it is Latin, an ancient  *European* language
> it means:
> - 'In itself' 
> - Also "by itself" 
> or:
> - Without referring to anything else, intrinsically, 
>   taken without qualifications, 


Thanks for your commentary.
I could understand the meaning and that '/' means italic.

Thank you,

--
SAKURAI, Masashi (family, given)
m.sakurai@kiwanami.net

Re: New backend for org-agenda: (Was Calendar-like view of the org-agenda)

[Bastien]

Hi Masashi,

SAKURAI Masashi <m.sakurai@kiwanami.net> writes:

> I'm also think about cache in the calfw side, such as an alist:
> (date . [a list of contents]). Then, uses can refresh the cache
> explicitly. It is easy to implement.
>
> Is the plan(1) the same idea?

Yes, It's the same idea, but on Org's side.

I'm focusing on bugfixes for the next release and cannot work 
through the details of the implementation, but I keep thinking
about it.

Thanks for your continuous efforts about calfw!

Best,

-- 
 Bastien

Re: New backend for org-agenda: (Was Calendar-like view of the org-agenda)

[SAKURAI Masashi]

Hi Bastien,

At Thu, 21 Jul 2011 11:52:44 +0200,
Bastien wrote:
> :
> > I'm also think about cache in the calfw side, such as an alist:
> > (date . [a list of contents]). Then, uses can refresh the cache
> > explicitly. It is easy to implement.
> >
> > Is the plan(1) the same idea?
> 
> Yes, It's the same idea, but on Org's side.

I see. It sounds great.


> I'm focusing on bugfixes for the next release and cannot work 
> through the details of the implementation, but I keep thinking
> about it.

Ok. I also need a time to do other things.
Indeed, I have to write the document of calfw.

Regards,
--
SAKURAI, Masashi (family, given)
m.sakurai@kiwanami.net