org-links for Gnus in imap with Gmail (for a convenient todo.org list)

org-links for Gnus in imap with Gmail (for a convenient todo.org list)

[Joseph Vidal-Rosset]

Hello,

Documentation to create a todo.org list  that provides links to email as
tasks   is  not   seldom  on   the  web.   I  recommend   for  example
[[http://pragmaticemacs.com/]]. 

But for  gnus-imap-gmail users, it is  more difficult to get  the same
thing. 

Here is a the lines of code  in my setup (user.el in Kitchin's scimax)
that work for gnus in imap:

 #+BEGIN_SRC elisp
 
;;capture todo items using C-c c t
(define-key global-map (kbd "C-c c") 'org-capture)
(setq org-capture-templates
      '(("t" "todo" entry (file+headline "~/Dropbox/Orgzly/todo.org" "Tasks")
         "* TODO [#A] %?\n [[~/Dropbox/Orgzly/links.org::%(org-insert-time-stamp (org-read-date nil t \"%:date\"))]] \n* %(org-insert-time-stamp (org-read-date nil t \"%:date\")) %a ")))


(defun hs/replace ()
   (interactive)
   (goto-char 1)
   (let ((search-invisible t)) (replace-string "gnus:INBOX#" "gnus:%5BGmail%5D/Tous les messages#")))
(define-key global-map (kbd "C-c r") 'hs/replace)
(add-hook 'org-capture-prepare-finalize-hook 'hs/replace)
(defun my/refile (file headline &optional arg)
  (let ((pos (save-excursion
               (find-file file)
               (org-find-exact-headline-in-buffer headline))))
    (org-refile arg nil (list headline file nil pos)))
  (switch-to-buffer (current-buffer)))

(defhydra jo/org-refile-hydra (:foreign-keys run) 
  "Refile"
  ("l" (my/refile "~/Dropbox/Orgzly/links.org" "Links") "Links")
  ("j" org-refile-goto-last-stored "Jump to last refile")
  ("q" nil "cancel"))
;; Or whatever you want your keybinding to be
(global-set-key (kbd "<f9> r") 'jo/org-refile-hydra/body)

(global-set-key (kbd "<f9> a") 'org-archive-subtree)

;;refile  https://blog.aaronbieber.com/2017/03/19/organizing-notes-with-refile.html
(setq org-refile-targets '((org-agenda-files :maxlevel . 2)))
(setq org-refile-use-outline-path 'file)
(setq org-outline-path-complete-in-steps nil)
(setq org-refile-allow-creating-parent-nodes 'confirm)

(setq org-refile-targets '((nil :maxlevel . 2)
                                (org-agenda-files :maxlevel . 2)))
(setq org-outline-path-complete-in-steps nil)         ; Refile in a single go
(setq org-refile-use-outline-path t)                  ; Show full paths for refiling

(setq org-refile-targets (quote (("todo.org" :maxlevel . 2)
                                ("links.org" :level . 2)
                                 )))
#+END_SRC  

with any  file open  in emacs, like  for example this  email, C-c  c t
gives the following result that I paste below: 

 #+BEGIN_SRC elisp
 
** TODO [#A] 
  [[~/Dropbox/Orgzly/links.org::<2019-07-25 jeu.>]] 
** <2019-07-25 jeu.> [[file:~/News/drafts/drafts/129]]

#+END_SRC

Because I  like to  see my  todo list  with conky  on my  wallpaper, I
refile  the second  link into  links.org  and it  is now  easy to  get
immediately from todo.org the correct  entry that have the appropriate
link in links.org. 

In Gnus, it is the same thing. For example C-c c t in reading the last
email that I received from John Kitchin provides:

 #+BEGIN_SRC elisp
 
** TODO [#A] 
  [[~/Dropbox/Orgzly/links.org::<2019-07-25 jeu.>]] 
** <2019-07-25 jeu.> [[gnus:INBOX#CAJ51ETqLcFTeZvPYg0AuzZL2HGwVGCOewKem8OkZnLL-ES59-g@mail.gmail.com][Email from John Kitchin: Re: scimax question]] 
#+END_SRC

But  I need  to change  the mention  of INBOX  in the  link to  John's
email. Hence the function hs/replace via  C-c r with the cursor on the
link that gives the wanted result: 

 #+BEGIN_SRC elisp   
** TODO [#A] 
  [[~/Dropbox/Orgzly/links.org::<2019-07-25 jeu.>]] 
** <2019-07-25         jeu.>         [[gnus:%5BGmail%5D/Tous         les
messages#CAJ51ETqLcFTeZvPYg0AuzZL2HGwVGCOewKem8OkZnLL-ES59-g@mail.gmail.com][Email
from John Kitchin: Re: scimax question]]

#+END_SRC 

and then  the email can  be archived in  Tous les messages,  Gnus will
find it, via imap.gmail.com . 

That's it.  I hope  some of  you will find  this useful.  Let me know. Of
course the code  must be adapted according to the  language of the gmail
user, it goes without saying. 

(In refiling,  I do not  succeed to get other  level that level  one for
sections, but it is a detail. )

 Best wishes,
-- 
Jo.

Re: org-links for Gnus in imap with Gmail (for a convenient todo.org list)

[Fraga, Eric]

On Thursday, 25 Jul 2019 at 18:24, Joseph Vidal-Rosset wrote:
> But for  gnus-imap-gmail users, it is  more difficult to get  the same
> thing. 

[...]

> But  I need  to change  the mention  of INBOX  in the  link to  John's
> email. 

Why is that?  If I create a link to an email from gmail using %a, this
seems to work (having just tested it).  Am I missing something?  I don't
often link to gmail messages as they are mostly junk emails (in my
case).  I know that gmail does not really have imap folders but uses
tags (of a sort) to make it look like emails are in different folders
(or in more than one folder simultaneously) but INBOX seems to work for
me.

Just curious.
-- 
Eric S Fraga via Emacs 27.0.50, Org release_9.2.4-399-g4e6222

Re: org-links for Gnus in imap with Gmail (for a convenient todo.org list)

[Joseph Vidal-Rosset]

Le   jeu.   07/25/19   juil.   2019   à   04:47:14   ,   "Fraga,   Eric"
<e.fraga@ucl.ac.uk> a envoyé ce message:
> On Thursday, 25 Jul 2019 at 18:24, Joseph Vidal-Rosset wrote:
>> But for  gnus-imap-gmail users, it is  more difficult to get  the same
>> thing. 
>
> [...]
>
>> But  I need  to change  the mention  of INBOX  in the  link to  John's
>> email. 
>
> Why is that?  If I create a link to an email from gmail using %a, this
> seems to work (having just tested it).  Am I missing something?  I don't
> often link to gmail messages as they are mostly junk emails (in my
> case).  I know that gmail does not really have imap folders but uses
> tags (of a sort) to make it look like emails are in different folders
> (or in more than one folder simultaneously) but INBOX seems to work for
> me.
>
> Just curious.

Thanks for your question Eric. :)

Of course, INBOX works  for you if you do not archive  the message in All
emails folder or elsewhere. If you move it from INBOX, in case where you
like to have the cleanest INBOX that  is the empty one, you won't get no
more the email via this link.

I wanted to have in IMAP the  shortest list to see only three folders in
my Gnus 
INBOX
[Gmail]/Tous les messages
[Gmail]/Corbeille

I have defined shortucts: key "Inser" archives email into
[Gmail]/Tous    les   messages    and   "Suppr"    sends   email    into
[Gmail]/Corbeille. 

The most  tricky is to  get the Summary  [Gmail]/Tous les messages  in a
reasonable time, because with  a lot of emails, Gnus in  imap can be too
slow. My solution is in this line: 

(setq gnus-newsgroup-maximum-articles 70000)

and it works. 

For  people who  are  reluctant to  use offlineimap  or  mbsync and  who
therefore prefer  Gnus over  mu4e ,  it makes almost  the same  job, via
imap.

I can share  of course all my  setup and I am also  curious of improving
it.

 Best wishes,
-- 
Jo.

Re: org-links for Gnus in imap with Gmail (for a convenient todo.org list)

[Fraga, Eric]

On Thursday, 25 Jul 2019 at 19:24, Joseph Vidal-Rosset wrote:
> Of course, INBOX works  for you if you do not archive  the message in All
> emails folder or elsewhere. If you move it from INBOX, in case where you
> like to have the cleanest INBOX that  is the empty one, you won't get no
> more the email via this link.

Okay, yes, if you move it, the link needs to be updated.  I wonder if
the gnus registery could help with this?  But that's well beyond my
gnus-fu.

-- 
: Professor Eric S Fraga, http://www.homepages.ucl.ac.uk/~ucecesf
: PGP/GPG key: 8F5C 279D 3907 E14A 5C29  570D C891 93D8 FFFC F67D
: Use plain text email when possible: https://useplaintext.email/

Re: org-links for Gnus in imap with Gmail (for a convenient todo.org list)

[Joseph Vidal-Rosset]

Le   jeu.   07/25/19   juil.   2019   à   05:33:57   ,   "Fraga,   Eric"
<e.fraga@ucl.ac.uk> a envoyé ce message:

> Okay, yes, if you move it, the link needs to be updated.  I wonder if
> the gnus registery could help with this?  But that's well beyond my
> gnus-fu.

The only mention of gnus registry in my setup is the following one:

(setq gnus-registry-max-entries 50000)

and, by the way, it is maybe inconsistent with

(setq gnus-newsgroup-maximum-articles 70000)

But  without the  trick that  I just  gave, I  had no  solution, and  my
knowledge of gnus is too weak to reply about gnus-registry.


 Best wishes,
-- 
Jo.

Re: org-links for Gnus in imap with Gmail (for a convenient todo.org list)

[Eric Abrahamsen]

Joseph Vidal-Rosset <joseph.vidal.rosset@gmail.com> writes:

> Le   jeu.   07/25/19   juil.   2019   à   05:33:57   ,   "Fraga,   Eric"
> <e.fraga@ucl.ac.uk> a envoyé ce message:
>
>> Okay, yes, if you move it, the link needs to be updated.  I wonder if
>> the gnus registery could help with this?  But that's well beyond my
>> gnus-fu.
>
> The only mention of gnus registry in my setup is the following one:
>
> (setq gnus-registry-max-entries 50000)
>
> and, by the way, it is maybe inconsistent with
>
> (setq gnus-newsgroup-maximum-articles 70000)

You'd have to put (gnus-registry-initialize) in your gnus.el init file
to make use of the registry. And no, I don't think the above settings
have anything to do with one another.

Eric

Re: org-links for Gnus in imap with Gmail (for a convenient todo.org list)

[Ken Mankoff]

On 2019-07-25 at 12:24 -04, Joseph Vidal-Rosset <joseph.vidal.rosset@gmail.com> wrote...
> Documentation to create a todo.org list  that provides links to email as
> tasks   is  not   seldom  on   the  web.   I  recommend   for  example
> [[http://pragmaticemacs.com/]]. 
>
> But for gnus-imap-gmail users, it is more difficult to get the same
> thing.

FYI, here is the mu4e implementation defined so that the standard org-store-link properly work based on msgid:

https://github.com/djcb/mu/blob/master/mu4e/org-mu4e.el#L117

When viewing email I use the same keybindings as elswhere, C-l to store link, and C-c C-l to paste it and I get:

[[mu4e:msgid:325fooetc@mx.google.com][SUBJECT OF EMAIL]]

  -k.

Re: org-links for Gnus in imap with Gmail (for a convenient todo.org list)

[Joseph Vidal-Rosset]

Le jeu. 07/25/19 juil. 2019 à 03:15:49 , Ken Mankoff <mankoff@gmail.com>
a envoyé ce message:
> On 2019-07-25 at 12:24 -04, Joseph Vidal-Rosset <joseph.vidal.rosset@gmail.com> wrote...
>> Documentation to create a todo.org list  that provides links to email as
>> tasks   is  not   seldom  on   the  web.   I  recommend   for  example
>> [[http://pragmaticemacs.com/]]. 
>>
>> But for gnus-imap-gmail users, it is more difficult to get the same
>> thing.
>
> FYI, here is the mu4e implementation defined so that the standard org-store-link properly work based on msgid:
>
> https://github.com/djcb/mu/blob/master/mu4e/org-mu4e.el#L117
>
> When viewing email I use the same keybindings as elswhere, C-l to store link, and C-c C-l to paste it and I get:
>
> [[mu4e:msgid:325fooetc@mx.google.com][SUBJECT OF EMAIL]]
>
>   -k.

Many thanks for this information Ken. Did you want to suggest that it is
not difficult to adapt such a function for gnus by replacing the mention
of mu4e by gnus' in the code?

I would be a good thing indeed  to avoid this need of updating link that
is presently necessary with gnus working with imap.gmail.com .


 Best wishes,
-- 
Jo.

Re: org-links for Gnus in imap with Gmail (for a convenient todo.org list)

[Ken Mankoff]

On 2019-07-26 at 03:14 -04, Joseph Vidal-Rosset <joseph.vidal.rosset@gmail.com> wrote...
> Did you want to suggest that it is not difficult to adapt such a
> function for gnus by replacing the mention of mu4e by gnus' in the
> code?

I wasn't suggesting anything, just letting you/others know that the functionality you described exists elsewhere, and hoping that that information would be useful and not noise.

  -k.

Re: org-links for Gnus in imap with Gmail (for a convenient todo.org list)

[Joseph Vidal-Rosset]

Le ven. 07/26/19 juil. 2019 à 03:24:52 , Ken Mankoff <mankoff@gmail.com>
a envoyé ce message:
> On 2019-07-26 at 03:14 -04, Joseph Vidal-Rosset <joseph.vidal.rosset@gmail.com> wrote...
>> Did you want to suggest that it is not difficult to adapt such a
>> function for gnus by replacing the mention of mu4e by gnus' in the
>> code?
>
> I wasn't suggesting anything, just letting you/others know that the
> functionality you described exists elsewhere, and hoping that that
> information would be useful and not noise.

Thanks again.  I just tried  to replace in this  part of code  "mu4e" by
"gnus", but it  would have been too  much simple. Of course  it does not
work. Maybe  someone will find a  far better solution than  that is mine
presently. I hope so.


 Best wishes,
-- 
Jo.

Re: org-links for Gnus in imap with Gmail (for a convenient todo.org list)

[Joseph Vidal-Rosset]

[-- Attachment #1: Type: text/plain, Size: 1974 bytes --]

Le   jeu.   07/25/19   juil.   2019   à   04:47:14   ,   "Fraga,   Eric"
<e.fraga@ucl.ac.uk> a envoyé ce message:
> On Thursday, 25 Jul 2019 at 18:24, Joseph Vidal-Rosset wrote:
>> But for  gnus-imap-gmail users, it is  more difficult to get  the same
>> thing. 
>
> [...]
>
>> But  I need  to change  the mention  of INBOX  in the  link to  John's
>> email. 
>
> Why is that?  If I create a link to an email from gmail using %a, this
> seems to work (having just tested it).  Am I missing something?

Hi again Eric and everybody,

Considering the code:

#+BEGIN_SRC 
;;capture todo items using C-c c t
(define-key global-map (kbd "C-c c") 'org-capture)
(setq org-capture-templates
      '(
	("t" "todo" entry (file+headline "~/Dropbox/Orgzly/todo.org" "Tasks")
         "* TODO [#A] %?\n [[~/Dropbox/Orgzly/links.org::%(org-insert-time-stamp (org-read-date nil t \"%:date\"))]] \n* %(org-insert-time-stamp (org-read-date nil t \"%:date\")) %a "
	 )
	)
      )

(defun hs/replace ()
   (interactive)
   (goto-char 1)
   (let ((search-invisible t)) (replace-string "gnus:INBOX#" "gnus:%5BGmail%5D/Tous les messages#")))
(define-key global-map (kbd "C-c r") 'hs/replace)
(add-hook 'org-capture-prepare-finalize-hook 'hs/replace)
(defun my/refile (file headline &optional arg)
  (let ((pos (save-excursion
               (find-file file)
               (org-find-exact-headline-in-buffer headline))))
    (org-refile arg nil (list headline file nil pos)))
  (switch-to-buffer (current-buffer)))
#+END_SRC

a better option  would be to get in only one function org-capture-template
and hs/replace :  if string "gnus/INBOX" exists,  hs/replace works and
replaces the  string, if not,  the link  is given without  change. The
idea is simple, but  I do not succeed to realize it,  because I do not
know enough  the syntax.  But I  would be surprised  if this  would be
impossible to make it. 


Best wishes,
 

-- 
Joseph 

[-- Attachment #2.1: Type: text/html, Size: 2841 bytes --]

Re: org-links for Gnus in imap with Gmail (for a convenient todo.org list)

[Bob Newell]

Aloha,

I too came across this long ago. I like a 'zero inbox' and so I archive
(or delete!) my gmail after reading. And indeed, when you move from
INBOX to "All Mail" an org-link to INBOX will no longer function.

I use Gnus and do something rather simple and easy:

(defun changegroupinemaillink (args)
  (setcar args
	  (replace-regexp-in-string
	   "gmail.com:INBOX"
	   "gmail.com:[Gmail]/All Mail" (car args)))
  args
)
(advice-add 'org-gnus-article-link :filter-args #'changegroupinemaillink)

After this, links point to All Mail and the mail will always be found.

-- 
Bob Newell
Honolulu, Hawai`i
* Via Gnus/BBDB/Org/Emacs/Linux *

Re: org-links for Gnus in imap with Gmail (for a convenient todo.org list)

[Joseph Vidal-Rosset]

Le ven. 26 juil. 2019  à 07:30:38 , Bob Newell <bobnewell@bobnewell.net>
a envoyé ce message:
> Aloha,
>
> I too came across this long ago. I like a 'zero inbox' and so I archive
> (or delete!) my gmail after reading. And indeed, when you move from
> INBOX to "All Mail" an org-link to INBOX will no longer function.
>
> I use Gnus and do something rather simple and easy:
>
> (defun changegroupinemaillink (args)
>   (setcar args
> 	  (replace-regexp-in-string
> 	   "gmail.com:INBOX"
> 	   "gmail.com:[Gmail]/All Mail" (car args)))
>   args
> )
> (advice-add 'org-gnus-article-link :filter-args #'changegroupinemaillink)
>
> After this, links point to All Mail and the mail will always be found.

Many thanks for your message Bob (I know your impressive website that is
helpful for org-mode, thanks also for that).

My problem is how can I change the following code:

(setq org-capture-templates
      '(
	("t" "todo" entry (file+headline "~/Dropbox/Orgzly/todo.org" "Tasks")
         "* TODO [#A] %?\n [[~/Dropbox/Orgzly/links.org::%(org-insert-time-stamp (org-read-date nil t \"%:date\"))]] \n* %(org-insert-time-stamp (org-read-date nil t \"%:date\")) %a "
	 )
	)
      )

in order to get  All Mail (or Tous les messages in  my case) directly in
the produced link once "t" key is entered?

I have tried to do it with  the function hs/replace that I have defined,
without success. I  guess that it is  of course possible, but  it is not
easy for me.

Your help will be again very welcome.

 Best wishes,
-- 
Jo.

Re: org-links for Gnus in imap with Gmail (for a convenient todo.org list)

[Joseph Vidal-Rosset]

Le sam. 27 juil. 2019  à 11:32:01 , Bob Newell <bobnewell@bobnewell.net>
a envoyé ce message:
> Bonjour Josef,
>
> Je ne suis pas assez doué avec org-mode pour donner des conseils
> expertes. Mais j'ai quand même une idée. Peut-être vous pouviez utiliser
> 'advice' au lieu d'un 'hook' sur org-capture-finalize
>
> Par exemple (attention! pas verifié!)
>
> (defun jr/fix-the-link (&rest args)
>  (interactive)
>  (save-excursion
>    (find-file "~/Dropbox/Orgzly/links.org")
>    (goto-char (point-min))
>    (while (search-forward "INBOX" nil t)
>      (replace-match "Tous les messages"))
>    (save-buffer)))
> (advice-add 'org-capture-finalize :after #'jr/fix-the-link)
>
> Bonne chance ... !

Merci vivement Bob,  c'est vraiment très gentil de me  répondre et de le
faire en français!

I feel free  now to speak in English  and to cc. the email  to the list,
because it is interesting.

If I understand the code above, Bob suggested to me to define a function
that replace a definite string in a definite file just with saving it. I
do  not doubt  that it  is possible  for  an expert.  Even if  I am  not
desperate, it's  like a  message in  a bottle thrown  into the  sea: can
someone help me to do that?

(It would be also possible to change the string at refiling, because the
link in todo.org must be refiled to links.org)

Best wishes, 

-- 
Joseph 

Re: org-links for Gnus in imap with Gmail (for a convenient todo.org list)

[Joseph Vidal-Rosset]

Le sam. 27 juil. 2019  à 11:32:01 , Bob Newell <bobnewell@bobnewell.net>
a envoyé ce message:
> Bonjour Josef,
>
> Je ne suis pas assez doué avec org-mode pour donner des conseils
> expertes. Mais j'ai quand même une idée. Peut-être vous pouviez utiliser
> 'advice' au lieu d'un 'hook' sur org-capture-finalize
>
> Par exemple (attention! pas verifié!)
>
> (defun jr/fix-the-link (&rest args)
>  (interactive)
>  (save-excursion
>    (find-file "~/Dropbox/Orgzly/links.org")
>    (goto-char (point-min))
>    (while (search-forward "INBOX" nil t)
>      (replace-match "Tous les messages"))
>    (save-buffer)))
> (advice-add 'org-capture-finalize :after #'jr/fix-the-link)
>
> Bonne chance ... !

Dear Bob,

In fact, I just realized that this code works as I wanted:

  (defun jr/fix-the-link (&rest args)
 (interactive)
 (save-excursion
   (find-file "~/Dropbox/Orgzly/todo.org")
   (goto-char (point-min))
   (while (search-forward "gnus:INBOX#" nil t)
     (replace-match "gnus:%5BGmail%5D/Tous les messages#"))
   (save-buffer)))
(advice-add 'org-capture-finalize :after #'jr/fix-the-link)

after C-c c t  the link is in todo.org, and with C-c C-w , once refiled
in links.org, the email link is  correctly changed! I do not understand
this magic trick, but many many  thanks for your kind and very efficient
help !


Best wishes, 
-- 
Joseph