Re: Remembrance Agents

[Jean Louis <bugs@gnu.support>]

* hpgisleropen@bluewin.ch <hpgisleropen@bluewin.ch> [2020-12-02 12:57]:
> > The more information user
> > enters into the database and the more tags and relations have been
> > created the better the relevance.
> 
> Isn't the real problem-to-solve finding the actual semantic context and
> then to relate it matching information?

That problem may easily be solved by using external tools such as
PostgreSQL as it has a built-in relevance search and indexing
functions.

To index files into PostgreSQL database is not hard task. If you
happen to have a need, let me know and I will help.

Yesterday I have implemented relevance searć in relation to website
pages in the database. As often I wish to reference URLs from my
websites to our clients. Among 4000+ URLs one has to find it by
thinking and not by browsing hierarchy.

Until now I was using 2 inputs approach like:

1. Write one word or part of word that I remember being part of the
   page.

2. When faced with completion candidates narrow it down with second or
   third word.

Yesterday I have implemented this search by using relevance matching
feature and it works fast:

(format "SELECT pages_id || ' ' || pages_title || ' ' || ts_rank_cd(to_tsvector(pages_title || pages_description),%s,32 /* rank/(rank+1) */) AS rank FROM pages, to_tsquery(%s) query WHERE query @@ to_tsvector(pages_title || pages_description) ORDER BY rank DESC LIMIT 30;" query query)) ;; TODO this cannot order by rank

The query is prepared with this:

(defun rcd-sql-prepare-text-query (query)
  (let* ((query (string-trim query))
	 (query (if (and (string-match " " query)
			 (not (string-match "&" query)))
		    (string-replace " " " & " query))))
    query))

because I normally need WORD AND WORD. But I could make it any how, I
could search for (WORD AND WORD) OR WORD OR (WORD AND WORD) all is
possible. The function converts "my term words" into "my & term &
words" where & has the meaning of logical AND.

This shortens my search within Emacs to just one query and not having
two steps. It finds relevant results with accuracy and precision. My
searches locate the hyperlink within 10 first results, so I have
limited it to 30 results.

> There is als After all, words (TAGS, ...) have different meaning in
> different semantic contexts.

In their own meanings yes. But for the user's mind they may be
related and useful to be searched together. If tag is there it is
related to the object, search without tag and with tag may give quite
different results.

> I suspect, 'just' relating to tags (words) and showing the attached
> document (part) to her is not very useful, yes?

Thinking of many other systems, in many of other systems tags are main
method of locating things. But it all comes from well organized user's
mind. If mind is not organized user will not know how to tag to make
future locating features work well. If it is tagged well locating
things is very fast.

- todo family son joe bicycle
- todo family son kim drivers-license
- todo business appointment
- todo health kidneys

When tags are used that way then locating things related to one son
among two sons become easy. Locating things for family is larger group
and locating business appointments becomes easy. When something is
done, the TODO should be changed to something else.

If user tags well then searches can be very useful.

But what if user does not tag well but still has some attributes
attached such as titles, body text, etc. then the relevance search may
help.

Chapter 12. Full Text Search
https://www.postgresql.org/docs/current/textsearch-intro.html#TEXTSEARCH-MATCHING

emacs-libpq @ Github
https://github.com/anse1/emacs-libpq