Re: Email -> Org-mode: charset problem

[Richard Lawrence <richard.lawrence@berkeley.edu>]

> At Mon, 27 Jun 2011 10:30:06 +0200,
> Karl Voit wrote:
>> Is there somebody who managed to develop an email to Org-mode bridge
>> without having charset problems? (No, I do not use Emacs as a
>> MUA[4])

David Maus <dmaus@ictsoc.de> writes:
> I do use an Emacs base mua (Wanderlust) and recently started to file
> reference notes for certain messages. Because in cases I want to keept
> the message (e.g. an interesting usenet post) I save the raw message
> to a temporary file and add it as an attachment. This works nice,
> although I'd really like to have an automated way of save+attache when
> capturing.
>
> So, what about org-capture? You could use a script that serializes the
> raw message to disk and -- somehow -- calls capture with a reference
> to this file. Emacs opens the mail file and extracts information
> required for the template using the build-in MIME libaries (mml-*
> IIRC) which are capable of correctly parsing and if necessary decoding
> (e.g. quoted printable encoded letters in the subject). After
> extracting the information Emacs creates the appropriate capture
> entry, somehow adds the message file as attachment and finishes the
> capture process.

I do something like this, though I also use Emacs as an MUA (Gnus).  

My approach was to write a little parser that binds the various parts of
a message in a pre-determined format to variables that I grab in an Org
capture template, and dynamically scope those variables into a call to
org-capture.  I have included my code below.

But to address your original question:  unfortunately, I do not have a
solution to the encoding problem.  Despite my "pre-determined format",
the emails I got (student paper submissions) were often in strange
encodings that I ended up fixing by hand during the capture process.

However, as David points out, I believe that it would be possible to use
Emacs' MIME libraries to do the dirty work.  I just haven't had time (or
desire, honestly) to figure out how.  If you figure it out, though, I'd
love to hear about it!

My (quite simplistic) code is below.  I hope you find it useful if you
decide to go this route!

Best,
Richard


;;;; simple working example: Gnus side
; handling paper submissions
(defun capture-paper-part ()
  "Call parse-paper and capture the result using org-capture"
  (interactive)
  (let ((paper (parse-paper *phil100-paper-header-re*)))
    (if paper
	  (org-capture nil "ap") ; "ap" is Org-capture key combo for template shown below
      (message "No paper found in this part"))))

(if (not (boundp 'gnus-mime-action-alist))
    (setq gnus-mime-action-alist '()))     
(add-to-list 'gnus-mime-action-alist '("capture paper" . capture-paper-part))

(defconst *phil100-paper-header-re*
  "^\\(?1:.*\\)\n\\(?2:.*\\)\n\\(?3:[Pp]aper *\\(?31:[0-9]+\\).*\\)")

(defun parse-paper (header-re)
  "Return an alist representing a paper, by parsing the buffer using header-re"
  (save-excursion
    (goto-char (point-min))
    (if (re-search-forward header-re nil t)
	(list `(header . ,(or (match-string 0) ""))
	      `(author . ,(or (match-string 2) ""))
	      `(title . ,(or (match-string 1) ""))
	      `(date . ,(or (match-string 3) ""))
	      `(num . ,(or (match-string 31) ""))
	      `(body . ,(buffer-substring (match-end 0) (point-max))))
      nil)))

(defun paper-property (key)
  ; paper is dynamically scoped in by capture-paper-part
  (or (cdr (assoc key paper)) ""))
  
(defun string-replace-downcase (from to in)
  "Replace FROM with TO in string IN, and also downcase the result"
  (with-temp-buffer
    (insert in)
    (goto-char (point-min))
    (while (search-forward from nil t)
      (replace-match to nil t))
    (downcase-region (point-min) (point-max))
    (buffer-substring (point-min) (point-max))))

#### simple working example: Org capture template
** %(paper-property 'author)
   :PROPERTIES:
   :EXPORT_TITLE: %(paper-property 'title)
   :EXPORT_AUTHOR: %(paper-property 'author)
   :EXPORT_DATE: Paper %(paper-property 'num)
   :EXPORT_FILE_NAME: %(concat (paper-property 'num) "-" (string-replace-downcase " " "-" (paper-property 'author)) ".tex")
   :END:

%(paper-property 'body)