Add the capture feature "%(sexp)" to org-feed

Add the capture feature "%(sexp)" to org-feed

[Michael Brand]

Hi all

I am almost done with adding the capture feature "%(sexp)" to
org-feed.el. Because for this I would like to use
org-capture-escaped-% from org-capture.el in org-feed.el I renamed
this function to org-template-escaped-% and call it in org-feed.el.
The only thing what is missing is that unless I do some manual
evaluations before, there is still "Symbol's function definition is
void: org-template-escaped-%". For this I would like to ask for some
design and implementation guidance, if available also where I should
read in the elisp reference manual or wherever.

Michael

Re: Add the capture feature "%(sexp)" to org-feed

[Nick Dokos]

Michael Brand <michael.ch.brand@gmail.com> wrote:

> I am almost done with adding the capture feature "%(sexp)" to
> org-feed.el. Because for this I would like to use
> org-capture-escaped-% from org-capture.el in org-feed.el I renamed
> this function to org-template-escaped-% and call it in org-feed.el.
> The only thing what is missing is that unless I do some manual
> evaluations before, there is still "Symbol's function definition is
> void: org-template-escaped-%". For this I would like to ask for some
> design and implementation guidance, if available also where I should
> read in the elisp reference manual or wherever.
> 

Either require org-capture in org-feed or move the function to org.el, which
both org-capture and org-feed require. 

Nick

Re: Add the capture feature "%(sexp)" to org-feed

[Michael Brand]

[-- Attachment #1: Type: text/plain, Size: 1227 bytes --]

Hi all

On Sun, Jun 17, 2012 at 4:07 AM, Nick Dokos <nicholas.dokos@hp.com> wrote:
> Either require org-capture in org-feed or move the function to org.el, which
> both org-capture and org-feed require.

Thank you for the help. I have chosen the solution with require org-capture.

The patch is now finished and attached:

Add the capture feature "%(sexp)" to org-feed

* lisp/org-capture.el (org-capture-fill-template): Fold the code for
embedded elisp $(sexp) into the following new function and constant.
(org-capture-template-embedded-elisp-re): New constant, contains the
already existing common regexp.
(org-capture-eval-and-replace-embedded-elisp): New function, contains the
already existing common code.
(org-capture-inside-embedded-elisp-p): New function to tell whether the
point is inside embedded elisp %(sexp).
* lisp/org-feed.el (org-feed-default-template): Add the new functionality
to the docstring.
(org-feed-format-entry): Add require org-capture to access the new common
function.  Add escaping of `"' to "simple %-escapes" if necessary for
embedded lisp %(sexp).  Add evaluation and replacement of embedded lisp
%(sexp).

Support for example %(capitalize \\\"%h\\\")" in the template of org-feed.

Michael

[-- Attachment #2: 0001-Add-the-capture-feature-sexp-to-org-feed.patch.txt --]
[-- Type: text/plain, Size: 7262 bytes --]

From a0932d9aedfd00fdeb2446b441b4572e39f8111a Mon Sep 17 00:00:00 2001
From: Michael Brand <michael.ch.brand@gmail.com>
Date: Sun, 24 Jun 2012 19:46:17 +0200
Subject: [PATCH] Add the capture feature "%(sexp)" to org-feed

* lisp/org-capture.el (org-capture-fill-template): Fold the code for
embedded elisp $(sexp) into the following new function and constant.
(org-capture-template-embedded-elisp-re): New constant, contains the
already existing common regexp.
(org-capture-eval-and-replace-embedded-elisp): New function, contains the
already existing common code.
(org-capture-inside-embedded-elisp-p): New function to tell whether the
point is inside embedded elisp %(sexp).
* lisp/org-feed.el (org-feed-default-template): Add the new functionality
to the docstring.
(org-feed-format-entry): Add require org-capture to access the new common
function.  Add escaping of `"' to "simple %-escapes" if necessary for
embedded lisp %(sexp).  Add evaluation and replacement of embedded lisp
%(sexp).

Support for example %(capitalize \\\"%h\\\")" in the template of org-feed.
---
 lisp/org-capture.el |   51 ++++++++++++++++++++++++++++++++++++++++++---------
 lisp/org-feed.el    |   51 +++++++++++++++++++++++++++++++++++++--------------
 2 files changed, 79 insertions(+), 23 deletions(-)

diff --git a/lisp/org-capture.el b/lisp/org-capture.el
index 0e6ab2c..9a0e9c2 100644
--- a/lisp/org-capture.el
+++ b/lisp/org-capture.el
@@ -1369,15 +1369,7 @@ The template may still contain \"%?\" for cursor positioning."
 	      (error (insert (format "%%![Couldn't insert %s: %s]"
 				     filename error)))))))
       ;; %() embedded elisp
-      (goto-char (point-min))
-      (while (re-search-forward "%\\((.+)\\)" nil t)
-	(unless (org-capture-escaped-%)
-	  (goto-char (match-beginning 0))
-	  (let ((template-start (point)))
-	    (forward-char 1)
-	    (let ((result (org-eval (read (current-buffer)))))
-	      (delete-region template-start (point))
-	      (insert result)))))
+      (org-capture-eval-and-replace-embedded-elisp)
 
       ;; The current time
       (goto-char (point-min))
@@ -1511,6 +1503,43 @@ The template may still contain \"%?\" for cursor positioning."
 	t)
     nil))
 
+(defconst org-capture-template-embedded-elisp-re
+  ;; embedded elisp %(sexp) in `org-feed-format-entry' supports simple
+  ;; %-escapes as arguments, therefore allow here \n coming from the
+  ;; result of evaluated simple %-escapes inside embedded elisp %(sexp)
+  "%\\((\\(.\\|\n\\)*\\)")
+
+(defun org-capture-eval-and-replace-embedded-elisp ()
+  "Evaluate embedded elisp %(sexp) and replace with the result."
+  (goto-char (point-min))
+  (while (re-search-forward org-capture-template-embedded-elisp-re nil t)
+    (unless (org-capture-escaped-%)
+      (goto-char (match-beginning 0))
+      (let ((template-start (point)))
+	(forward-char 1)
+	(let ((result (org-eval (read (current-buffer)))))
+	  (delete-region template-start (point))
+	  (insert result))))))
+
+(defun org-capture-inside-embedded-elisp-p ()
+  "Return non-nil if point is inside of embedded elisp %(sexp)."
+  (let ((p-orig (point)) p-temp embedded)
+    (save-excursion
+      (save-match-data
+	(when (re-search-backward org-capture-template-embedded-elisp-re
+				  nil t)
+	  ;; match again forward, backward did not match beyond its start
+	  (re-search-forward org-capture-template-embedded-elisp-re nil t)
+	  ;; convert original point to corresponding point in temp buffer
+	  (setq p-temp (+ p-orig (- (match-beginning 1)) 1))
+	  (setq embedded (match-string 1))
+	  (with-temp-buffer  ; don't interfere with original major mode
+	    (insert embedded)
+	    (beginning-of-buffer)
+	    (emacs-lisp-mode)  ; to deal with for example %(length ")")
+	    (when (ignore-errors (forward-sexp) t)
+	      (< p-temp (point)))))))))
+
 ;;;###autoload
 (defun org-capture-import-remember-templates ()
   "Set org-capture-templates to be similar to `org-remember-templates'."
@@ -1552,4 +1581,8 @@ The template may still contain \"%?\" for cursor positioning."
 
 (provide 'org-capture)
 
+;; Local variables:
+;; sentence-end-double-space: t
+;; End:
+
 ;;; org-capture.el ends here
diff --git a/lisp/org-feed.el b/lisp/org-feed.el
index 6901ffa..46dc6b7 100644
--- a/lisp/org-feed.el
+++ b/lisp/org-feed.el
@@ -230,7 +230,10 @@ following special escapes are valid as well:
         the current date.
 %T      date and time
 %u,%U   like %t,%T, but inactive time stamps
-%a      A link, from <guid> if that is a permalink, else from <link>"
+%a      A link, from <guid> if that is a permalink, else from <link>
+%(sexp) evaluate elisp `(sexp)' and replace with the result,
+        the above simple %-escapes can be used as arguments,
+        for example %(capitalize \\\"%h\\\")"
   :group 'org-feed
   :type '(string :tag "Template"))
 
@@ -506,9 +509,10 @@ This will find DRAWER and extract the alist."
 ENTRY is a property list.  This function adds a `:formatted-for-org' property
 and returns the full property list.
 If that property is already present, nothing changes."
+  (require 'org-capture)
   (if formatter
       (funcall formatter entry)
-    (let (dlines fmt tmp indent time name
+    (let (dlines time escape name tmp
 		 v-h v-t v-T v-u v-U v-a)
       (setq dlines (org-split-string (or (plist-get entry :description) "???")
 				     "\n")
@@ -527,20 +531,35 @@ If that property is already present, nothing changes."
 		  ""))
       (with-temp-buffer
 	(insert template)
+
+	;; Simple %-escapes
+	;; before embedded elisp to support simple %-escapes as
+	;; arguments for embedded elisp
 	(goto-char (point-min))
 	(while (re-search-forward "%\\([a-zA-Z]+\\)" nil t)
-	  (setq name (match-string 1))
-	  (cond
-	   ((member name '("h" "t" "T" "u" "U" "a"))
-	    (replace-match (symbol-value (intern (concat "v-" name))) t t))
-	   ((setq tmp (plist-get entry (intern (concat ":" name))))
-	    (save-excursion
-	      (save-match-data
-		(beginning-of-line 1)
-		(when (looking-at (concat "^\\([ \t]*\\)%" name "[ \t]*$"))
-		  (setq tmp (org-feed-make-indented-block
-			     tmp (org-get-indentation))))))
-	    (replace-match tmp t t))))
+	  (unless (org-capture-escaped-%)
+	    (setq escape (org-capture-inside-embedded-elisp-p)
+		  name (match-string 1))
+	    (cond
+	     ((member name '("h" "t" "T" "u" "U" "a"))
+	      (setq tmp (symbol-value (intern (concat "v-" name)))))
+	     ((setq tmp (plist-get entry (intern (concat ":" name))))
+	      (save-excursion
+		(save-match-data
+		  (beginning-of-line 1)
+		  (when (looking-at
+			 (concat "^\\([ \t]*\\)%" name "[ \t]*$"))
+		    (setq tmp (org-feed-make-indented-block
+			       tmp (org-get-indentation))))))))
+	    (when tmp
+	      ;; escape string delimiters `"' when inside %() embedded lisp
+	      (when escape
+		(setq tmp (replace-regexp-in-string "\"" "\\\\\"" tmp)))
+	      (replace-match tmp t t))))
+
+	;; %() embedded elisp
+	(org-capture-eval-and-replace-embedded-elisp)
+
 	(decode-coding-string
 	 (buffer-string) (detect-coding-region (point-min) (point-max) t))))))
 
@@ -673,4 +692,8 @@ formatted as a string, not the original XML data."
 
 (provide 'org-feed)
 
+;; Local variables:
+;; sentence-end-double-space: t
+;; End:
+
 ;;; org-feed.el ends here
-- 
1.7.4.2

Re: Add the capture feature "%(sexp)" to org-feed

[Bastien]

[-- Attachment #1: Type: text/plain, Size: 373 bytes --]

Hi Michael,

Michael Brand <michael.ch.brand@gmail.com> writes:

> Thank you for the help. I have chosen the solution with require org-capture.
>
> The patch is now finished and attached:

I've reworked some parts of your patch, specifically, I used
`org-at-regexp-p' instead of `org-capture-inside-embedded-elisp-p'.

Can you check this is working okay for you?

Thanks!


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-org-feed.el-Add-the-capture-feature-sexp.patch --]
[-- Type: text/x-patch, Size: 6125 bytes --]

From 35e5c3d0b098dc65f321cc32626172e984cf2c2f Mon Sep 17 00:00:00 2001
From: Bastien Guerry <bzg@altern.org>
Date: Wed, 8 Aug 2012 01:01:29 +0200
Subject: [PATCH] org-feed.el: Add the capture feature "%(sexp)".

* org-feed.el (org-feed-format-entry): Support %(sexp).
(org-feed-default-template): Update docstring to document the
support of %(sexp).

* org-capture.el (org-capture-template-embedded-elisp-re): New
constant.
(org-capture-eval-and-replace-embedded-elisp): New function.
(org-capture-fill-template): Use the new function.
---
 lisp/org-capture.el |   25 ++++++++++++++---------
 lisp/org-feed.el    |   56 ++++++++++++++++++++++++++++++++++-----------------
 2 files changed, 53 insertions(+), 28 deletions(-)

diff --git a/lisp/org-capture.el b/lisp/org-capture.el
index 05e3a0c..352e655 100644
--- a/lisp/org-capture.el
+++ b/lisp/org-capture.el
@@ -74,6 +74,9 @@
 ;; to indicate that the link properties have already been stored
 (defvar org-capture-link-is-already-stored nil)
 
+(defconst org-capture-template-embedded-elisp-re
+  "%\\((\\(.\\|\n\\)*)\\)")
+
 (defgroup org-capture nil
   "Options concerning capturing new entries."
   :tag "Org Capture"
@@ -1277,6 +1280,18 @@ Lisp programs can force the template by setting KEYS to a string."
 	       '(("C" "Customize org-capture-templates")
 		 ("q" "Abort"))))))
 
+(defun org-capture-eval-and-replace-embedded-elisp ()
+  "Evaluate embedded elisp %(sexp) and replace with the result."
+  (goto-char (point-min))
+  (while (re-search-forward org-capture-template-embedded-elisp-re nil t)
+    (unless (org-capture-escaped-%)
+      (goto-char (match-beginning 0))
+      (let ((template-start (point)))
+	(forward-char 1)
+	(let ((result (org-eval (read (current-buffer)))))
+	  (delete-region template-start (point))
+	  (insert result))))))
+
 (defun org-capture-fill-template (&optional template initial annotation)
   "Fill a template and return the filled template as a string.
 The template may still contain \"%?\" for cursor positioning."
@@ -1371,15 +1386,7 @@ The template may still contain \"%?\" for cursor positioning."
 	      (error (insert (format "%%![Couldn't insert %s: %s]"
 				     filename error)))))))
       ;; %() embedded elisp
-      (goto-char (point-min))
-      (while (re-search-forward "%\\((.+)\\)" nil t)
-	(unless (org-capture-escaped-%)
-	  (goto-char (match-beginning 0))
-	  (let ((template-start (point)))
-	    (forward-char 1)
-	    (let ((result (org-eval (read (current-buffer)))))
-	      (delete-region template-start (point))
-	      (insert result)))))
+      (org-capture-eval-and-replace-embedded-elisp)
 
       ;; The current time
       (goto-char (point-min))
diff --git a/lisp/org-feed.el b/lisp/org-feed.el
index 6901ffa..912dd6e 100644
--- a/lisp/org-feed.el
+++ b/lisp/org-feed.el
@@ -225,12 +225,15 @@ Any fields from the feed item can be interpolated into the template with
 %name, for example %title, %description, %pubDate etc.  In addition, the
 following special escapes are valid as well:
 
-%h      the title, or the first line of the description
-%t      the date as a stamp, either from <pubDate> (if present), or
-        the current date.
-%T      date and time
-%u,%U   like %t,%T, but inactive time stamps
-%a      A link, from <guid> if that is a permalink, else from <link>"
+%h      The title, or the first line of the description
+%t      The date as a stamp, either from <pubDate> (if present), or
+        The current date.
+%T      Date and time
+%u,%U   Like %t,%T, but inactive time stamps
+%a      A link, from <guid> if that is a permalink, else from <link>
+%(sexp) Evaluate the Elisp sexp `(sexp)' and replace with the result.
+        The above simple %-escapes can be used as arguments,
+        for example %(capitalize \\\"%h\\\")"
   :group 'org-feed
   :type '(string :tag "Template"))
 
@@ -506,9 +509,10 @@ This will find DRAWER and extract the alist."
 ENTRY is a property list.  This function adds a `:formatted-for-org' property
 and returns the full property list.
 If that property is already present, nothing changes."
+  (require 'org-capture)
   (if formatter
       (funcall formatter entry)
-    (let (dlines fmt tmp indent time name
+    (let (dlines time escape name tmp
 		 v-h v-t v-T v-u v-U v-a)
       (setq dlines (org-split-string (or (plist-get entry :description) "???")
 				     "\n")
@@ -527,20 +531,34 @@ If that property is already present, nothing changes."
 		  ""))
       (with-temp-buffer
 	(insert template)
+
+	;; Simple %-escapes before embedded elisp to support simple
+	;; %-escapes as arguments for embedded elisp
 	(goto-char (point-min))
 	(while (re-search-forward "%\\([a-zA-Z]+\\)" nil t)
-	  (setq name (match-string 1))
-	  (cond
-	   ((member name '("h" "t" "T" "u" "U" "a"))
-	    (replace-match (symbol-value (intern (concat "v-" name))) t t))
-	   ((setq tmp (plist-get entry (intern (concat ":" name))))
-	    (save-excursion
-	      (save-match-data
-		(beginning-of-line 1)
-		(when (looking-at (concat "^\\([ \t]*\\)%" name "[ \t]*$"))
-		  (setq tmp (org-feed-make-indented-block
-			     tmp (org-get-indentation))))))
-	    (replace-match tmp t t))))
+	  (unless (org-capture-escaped-%)
+	    (setq name (match-string 1)
+		  escape (org-at-regexp-p org-capture-template-embedded-elisp-re))
+	    (cond
+	     ((member name '("h" "t" "T" "u" "U" "a"))
+	      (setq tmp (symbol-value (intern (concat "v-" name)))))
+	     ((setq tmp (plist-get entry (intern (concat ":" name))))
+	      (save-excursion
+		(save-match-data
+		  (beginning-of-line 1)
+		  (when (looking-at
+			 (concat "^\\([ \t]*\\)%" name "[ \t]*$"))
+		    (setq tmp (org-feed-make-indented-block
+			       tmp (org-get-indentation))))))))
+	    (when tmp
+	      ;; escape string delimiters `"' when inside %() embedded lisp
+	      (when escape
+		(setq tmp (replace-regexp-in-string "\"" "\\\\\"" tmp)))
+	      (replace-match tmp t t))))
+
+	;; %() embedded elisp
+	(org-capture-eval-and-replace-embedded-elisp)
+
 	(decode-coding-string
 	 (buffer-string) (detect-coding-region (point-min) (point-max) t))))))
 
-- 
1.7.10.2


[-- Attachment #3: Type: text/plain, Size: 14 bytes --]


-- 
 Bastien

Re: Add the capture feature "%(sexp)" to org-feed

[Michael Brand]

Hi Bastien

Thank you for reviewing my patch.

On Wed, Aug 8, 2012 at 1:05 AM, Bastien <bzg@gnu.org> wrote:

> I've reworked some parts of your patch,

You omitted the Local variables sentence-end-double-space: t. Isn't
this a good idea for all the users like me that have set this to nil
in their config? I wanted to add it to more files as soon as I will
change them.

“The current date.” is probably meant to remain “the current date.”.

> specifically, I used `org-at-regexp-p' instead of
> `org-capture-inside-embedded-elisp-p'.

With `org-capture-inside-embedded-elisp-p' I wanted to be quite more
waterproof than `org-at-regexp-p' with the new additional closing
parenthesis in `org-capture-template-embedded-elisp-re'. See its
comment “to deal with for example %(length ")")”. Isn't it a good idea
to use `forward-sexp' in Emacs Lisp Mode for this? What can I improve?

> Can you check this is working okay for you?

I'll do with the final version of the patch if there will be changes.

Michael

Re: Add the capture feature "%(sexp)" to org-feed

[Bastien]

Hi Michael,

Michael Brand <michael.ch.brand@gmail.com> writes:

> You omitted the Local variables sentence-end-double-space: t. Isn't
> this a good idea for all the users like me that have set this to nil
> in their config? I wanted to add it to more files as soon as I will
> change them.

I've check against current Emacs trunk, and only three files/packages
uses this: rst.el, filesets.el and mh-e.

So I'd rather not pollute the hundred files of Org with this.

> “The current date.” is probably meant to remain “the current date.”.

Yes.

>> specifically, I used `org-at-regexp-p' instead of
>> `org-capture-inside-embedded-elisp-p'.
>
> With `org-capture-inside-embedded-elisp-p' I wanted to be quite more
> waterproof than `org-at-regexp-p' with the new additional closing
> parenthesis in `org-capture-template-embedded-elisp-re'. See its
> comment “to deal with for example %(length ")")”. Isn't it a good idea
> to use `forward-sexp' in Emacs Lisp Mode for this? What can I improve?

(org-at-regexp-p org-capture-template-embedded-elisp-re) already 
returns the correct value for %(length ")") -- but you're right there 
is a problem with my patch: `org-at-regexp-p' does not match over 
multiple lines.  Maybe you can play with `org-in-regexp':

  (org-in-regexp org-capture-template-embedded-elisp-re 3)

If using `forward-sexp' is necessary let's do so -- but I thought it 
was too complex first.

Thanks,

-- 
 Bastien

Re: Add the capture feature "%(sexp)" to org-feed

[Michael Brand]

Hi Bastien

On Thu, Aug 9, 2012 at 4:53 PM, Bastien <bzg@gnu.org> wrote:

> I've check against current Emacs trunk, and only three files/packages
> uses this: rst.el, filesets.el and mh-e.
>
> So I'd rather not pollute the hundred files of Org with this.

That's all right.

> (org-at-regexp-p org-capture-template-embedded-elisp-re) already
> returns the correct value for %(length ")")

For the above example yes, but not with more than one %():
“- %(capitalize "simple percent-escape")\n- %(capitalize "one more")”

Also non-greedy regexp is not enough here:
“%(capitalize "(some) text")”

> -- but you're right there
> is a problem with my patch: `org-at-regexp-p' does not match over
> multiple lines.  Maybe you can play with `org-in-regexp':
>
>   (org-in-regexp org-capture-template-embedded-elisp-re 3)
>
> If using `forward-sexp' is necessary let's do so -- but I thought it
> was too complex first.

When I wrote the patch my conclusion was that regexp alone is not
sufficient. For real feed templates used by me, not only for
artificial examples. I must admit that my solution with `forward-sexp'
is complicated but it makes available the power of the Emacs Lisp Mode
parser. Maybe there are still some simplifications?

Michael

Re: Add the capture feature "%(sexp)" to org-feed

[Bastien]

[-- Attachment #1: Type: text/plain, Size: 225 bytes --]

Hi Michael,

Michael Brand <michael.ch.brand@gmail.com> writes:

> Maybe there are still some simplifications?

Please try the attached patch and let me know if it works.

If you're okay, I'll then apply it under your name.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-the-capture-feature-sexp-to-org-feed.patch --]
[-- Type: text/x-patch, Size: 6045 bytes --]

From 757d415af6247ea85f260daaeeb9f143f41e6103 Mon Sep 17 00:00:00 2001
From: Bastien Guerry <bzg@altern.org>
Date: Fri, 10 Aug 2012 10:52:13 +0200
Subject: [PATCH] Add the capture feature sexp to org feed.

* org-feed.el (org-feed-format-entry): Require `org-capture'.
Expand Elisp %(...) templates.
(org-feed-default-template): Update docstring.

* org-capture.el (org-capture-expand-embedded-elisp): New
function.
(org-capture-fill-template): Use it.
(org-capture-inside-embedded-elisp-p): New function to tell if
we are within an Elisp %(...) template.
---
 lisp/org-capture.el |   34 ++++++++++++++++++++++---------
 lisp/org-feed.el    |   56 ++++++++++++++++++++++++++++++++++-----------------
 2 files changed, 62 insertions(+), 28 deletions(-)

diff --git a/lisp/org-capture.el b/lisp/org-capture.el
index 05e3a0c..7119c2d 100644
--- a/lisp/org-capture.el
+++ b/lisp/org-capture.el
@@ -1371,15 +1371,7 @@ The template may still contain \"%?\" for cursor positioning."
 	      (error (insert (format "%%![Couldn't insert %s: %s]"
 				     filename error)))))))
       ;; %() embedded elisp
-      (goto-char (point-min))
-      (while (re-search-forward "%\\((.+)\\)" nil t)
-	(unless (org-capture-escaped-%)
-	  (goto-char (match-beginning 0))
-	  (let ((template-start (point)))
-	    (forward-char 1)
-	    (let ((result (org-eval (read (current-buffer)))))
-	      (delete-region template-start (point))
-	      (insert result)))))
+      (org-capture-expand-embedded-elisp)
 
       ;; The current time
       (goto-char (point-min))
@@ -1513,6 +1505,30 @@ The template may still contain \"%?\" for cursor positioning."
 	t)
     nil))
 
+(defun org-capture-expand-embedded-elisp ()
+  "Evaluate embedded elisp %(sexp) and replace with the result."
+  (goto-char (point-min))
+  (while (re-search-forward "%(" nil t)
+    (unless (org-capture-escaped-%)
+      (goto-char (match-beginning 0))
+      (let ((template-start (point)))
+	(forward-char 1)
+	(let ((result (org-eval (read (current-buffer)))))
+	  (delete-region template-start (point))
+	  (insert result))))))
+
+(defun org-capture-inside-embedded-elisp-p ()
+  "Return non-nil if point is inside of embedded elisp %(sexp)."
+  (let (beg end)
+    (save-excursion
+      (save-match-data
+	(when (or (looking-at "%(")
+		  (and (search-backward "%" nil t) (looking-at "%(")))
+	  (setq beg (point))
+	  (setq end (progn (forward-char) (forward-sexp) (1- (point)))))))
+    (when (and beg end)
+      (and (<= (point) end) (>= (point) beg)))))
+
 ;;;###autoload
 (defun org-capture-import-remember-templates ()
   "Set org-capture-templates to be similar to `org-remember-templates'."
diff --git a/lisp/org-feed.el b/lisp/org-feed.el
index 6901ffa..8b3414b 100644
--- a/lisp/org-feed.el
+++ b/lisp/org-feed.el
@@ -225,12 +225,14 @@ Any fields from the feed item can be interpolated into the template with
 %name, for example %title, %description, %pubDate etc.  In addition, the
 following special escapes are valid as well:
 
-%h      the title, or the first line of the description
-%t      the date as a stamp, either from <pubDate> (if present), or
-        the current date.
-%T      date and time
-%u,%U   like %t,%T, but inactive time stamps
-%a      A link, from <guid> if that is a permalink, else from <link>"
+%h      The title, or the first line of the description
+%t      The date as a stamp, either from <pubDate> (if present), or
+        the current date
+%T      Date and time
+%u,%U   Like %t,%T, but inactive time stamps
+%a      A link, from <guid> if that is a permalink, else from <link>
+%(sexp) Evaluate elisp `(sexp)' and replace with the result, the simple
+        %-escapes above can be used as arguments, e.g. %(capitalize \\\"%h\\\")"
   :group 'org-feed
   :type '(string :tag "Template"))
 
@@ -506,9 +508,10 @@ This will find DRAWER and extract the alist."
 ENTRY is a property list.  This function adds a `:formatted-for-org' property
 and returns the full property list.
 If that property is already present, nothing changes."
+  (require 'org-capture)
   (if formatter
       (funcall formatter entry)
-    (let (dlines fmt tmp indent time name
+    (let (dlines time escape name tmp
 		 v-h v-t v-T v-u v-U v-a)
       (setq dlines (org-split-string (or (plist-get entry :description) "???")
 				     "\n")
@@ -527,20 +530,35 @@ If that property is already present, nothing changes."
 		  ""))
       (with-temp-buffer
 	(insert template)
+
+	;; Simple %-escapes
+	;; before embedded elisp to support simple %-escapes as
+	;; arguments for embedded elisp
 	(goto-char (point-min))
 	(while (re-search-forward "%\\([a-zA-Z]+\\)" nil t)
-	  (setq name (match-string 1))
-	  (cond
-	   ((member name '("h" "t" "T" "u" "U" "a"))
-	    (replace-match (symbol-value (intern (concat "v-" name))) t t))
-	   ((setq tmp (plist-get entry (intern (concat ":" name))))
-	    (save-excursion
-	      (save-match-data
-		(beginning-of-line 1)
-		(when (looking-at (concat "^\\([ \t]*\\)%" name "[ \t]*$"))
-		  (setq tmp (org-feed-make-indented-block
-			     tmp (org-get-indentation))))))
-	    (replace-match tmp t t))))
+	  (unless (org-capture-escaped-%)
+	    (setq name (match-string 1)
+		  escape (org-capture-inside-embedded-elisp-p))
+	    (cond
+	     ((member name '("h" "t" "T" "u" "U" "a"))
+	      (setq tmp (symbol-value (intern (concat "v-" name)))))
+	     ((setq tmp (plist-get entry (intern (concat ":" name))))
+	      (save-excursion
+		(save-match-data
+		  (beginning-of-line 1)
+		  (when (looking-at
+			 (concat "^\\([ \t]*\\)%" name "[ \t]*$"))
+		    (setq tmp (org-feed-make-indented-block
+			       tmp (org-get-indentation))))))))
+	    (when tmp
+	      ;; escape string delimiters `"' when inside %() embedded lisp
+	      (when escape
+		(setq tmp (replace-regexp-in-string "\"" "\\\\\"" tmp)))
+	      (replace-match tmp t t))))
+
+	;; %() embedded elisp
+	(org-capture-expand-embedded-elisp)
+
 	(decode-coding-string
 	 (buffer-string) (detect-coding-region (point-min) (point-max) t))))))
 
-- 
1.7.10.2


[-- Attachment #3: Type: text/plain, Size: 14 bytes --]


-- 
 Bastien

Re: Add the capture feature "%(sexp)" to org-feed

[Michael Brand]

[-- Attachment #1: Type: text/plain, Size: 807 bytes --]

Hi Bastien

On Fri, Aug 10, 2012 at 10:53 AM, Bastien <bzg@gnu.org> wrote:
> Please try the attached patch and let me know if it works.

Thank you for making `org-capture-inside-embedded-elisp-p' much
clearer. I was not sure if you really wanted to omit switching to
Emacs Lisp Mode although it is useful for example for “%(length ")")”.
So I attach a new patch where I restored this part, now cleaner too.
Also a small change against your latest patch to not stop at “%”
without following “(” and possibly miss a “%(” before.

I temporarily emptied “:FEEDSTATUS:” in all my feeds and updated them
and also tried some odd artificial examples. All works the same as
with my first patch. Can you please review this new patch compared
against your latest and apply when appropriate?

Michael

[-- Attachment #2: 0001-Add-the-capture-feature-sexp-to-org-feed.patch.txt --]
[-- Type: text/plain, Size: 6390 bytes --]

From 142625bad45ca07a185d95caa0d810225e389b95 Mon Sep 17 00:00:00 2001
From: Michael Brand <michael.ch.brand@gmail.com>
Date: Sat, 11 Aug 2012 17:26:57 +0200
Subject: [PATCH] Add the capture feature sexp to org feed

* org-feed.el (org-feed-format-entry): Require `org-capture'.
Expand Elisp %(...) templates.
(org-feed-default-template): Update docstring.

* org-capture.el (org-capture-expand-embedded-elisp): New
function.
(org-capture-fill-template): Use it.
(org-capture-inside-embedded-elisp-p): New function to tell if
we are within an Elisp %(...) template.
---
 lisp/org-capture.el |   40 ++++++++++++++++++++++++++++--------
 lisp/org-feed.el    |   56 +++++++++++++++++++++++++++++++++-----------------
 2 files changed, 68 insertions(+), 28 deletions(-)

diff --git a/lisp/org-capture.el b/lisp/org-capture.el
index 84e5ff6..4a11462 100644
--- a/lisp/org-capture.el
+++ b/lisp/org-capture.el
@@ -1388,15 +1388,7 @@ The template may still contain \"%?\" for cursor positioning."
 	      (error (insert (format "%%![Couldn't insert %s: %s]"
 				     filename error)))))))
       ;; %() embedded elisp
-      (goto-char (point-min))
-      (while (re-search-forward "%\\((.+)\\)" nil t)
-	(unless (org-capture-escaped-%)
-	  (goto-char (match-beginning 0))
-	  (let ((template-start (point)))
-	    (forward-char 1)
-	    (let ((result (org-eval (read (current-buffer)))))
-	      (delete-region template-start (point))
-	      (insert result)))))
+      (org-capture-expand-embedded-elisp)
 
       ;; The current time
       (goto-char (point-min))
@@ -1530,6 +1522,36 @@ The template may still contain \"%?\" for cursor positioning."
 	t)
     nil))
 
+(defun org-capture-expand-embedded-elisp ()
+  "Evaluate embedded elisp %(sexp) and replace with the result."
+  (goto-char (point-min))
+  (while (re-search-forward "%(" nil t)
+    (unless (org-capture-escaped-%)
+      (goto-char (match-beginning 0))
+      (let ((template-start (point)))
+	(forward-char 1)
+	(let ((result (org-eval (read (current-buffer)))))
+	  (delete-region template-start (point))
+	  (insert result))))))
+
+(defun org-capture-inside-embedded-elisp-p ()
+  "Return non-nil if point is inside of embedded elisp %(sexp)."
+  (let ((buf (buffer-string)) (p (point)) beg end)
+    (with-temp-buffer   ; to keep major mode and font-lock in original buffer
+      (emacs-lisp-mode) ; to deal with for example %(length ")")
+      (insert buf)
+      (goto-char p)
+      ;; `looking-at' and `search-backward' below do not match the "%(" if
+      ;; point is in its middle
+      (when (equal (char-before) ?%)
+        (backward-char))
+      (save-match-data
+	(when (or (looking-at "%(") (search-backward "%(" nil t))
+	  (setq beg (point))
+	  (setq end (progn (forward-char) (forward-sexp) (1- (point)))))))
+    (when (and beg end)
+      (and (<= p end) (>= p beg)))))
+
 ;;;###autoload
 (defun org-capture-import-remember-templates ()
   "Set org-capture-templates to be similar to `org-remember-templates'."
diff --git a/lisp/org-feed.el b/lisp/org-feed.el
index 6901ffa..8b3414b 100644
--- a/lisp/org-feed.el
+++ b/lisp/org-feed.el
@@ -225,12 +225,14 @@ Any fields from the feed item can be interpolated into the template with
 %name, for example %title, %description, %pubDate etc.  In addition, the
 following special escapes are valid as well:
 
-%h      the title, or the first line of the description
-%t      the date as a stamp, either from <pubDate> (if present), or
-        the current date.
-%T      date and time
-%u,%U   like %t,%T, but inactive time stamps
-%a      A link, from <guid> if that is a permalink, else from <link>"
+%h      The title, or the first line of the description
+%t      The date as a stamp, either from <pubDate> (if present), or
+        the current date
+%T      Date and time
+%u,%U   Like %t,%T, but inactive time stamps
+%a      A link, from <guid> if that is a permalink, else from <link>
+%(sexp) Evaluate elisp `(sexp)' and replace with the result, the simple
+        %-escapes above can be used as arguments, e.g. %(capitalize \\\"%h\\\")"
   :group 'org-feed
   :type '(string :tag "Template"))
 
@@ -506,9 +508,10 @@ This will find DRAWER and extract the alist."
 ENTRY is a property list.  This function adds a `:formatted-for-org' property
 and returns the full property list.
 If that property is already present, nothing changes."
+  (require 'org-capture)
   (if formatter
       (funcall formatter entry)
-    (let (dlines fmt tmp indent time name
+    (let (dlines time escape name tmp
 		 v-h v-t v-T v-u v-U v-a)
       (setq dlines (org-split-string (or (plist-get entry :description) "???")
 				     "\n")
@@ -527,20 +530,35 @@ If that property is already present, nothing changes."
 		  ""))
       (with-temp-buffer
 	(insert template)
+
+	;; Simple %-escapes
+	;; before embedded elisp to support simple %-escapes as
+	;; arguments for embedded elisp
 	(goto-char (point-min))
 	(while (re-search-forward "%\\([a-zA-Z]+\\)" nil t)
-	  (setq name (match-string 1))
-	  (cond
-	   ((member name '("h" "t" "T" "u" "U" "a"))
-	    (replace-match (symbol-value (intern (concat "v-" name))) t t))
-	   ((setq tmp (plist-get entry (intern (concat ":" name))))
-	    (save-excursion
-	      (save-match-data
-		(beginning-of-line 1)
-		(when (looking-at (concat "^\\([ \t]*\\)%" name "[ \t]*$"))
-		  (setq tmp (org-feed-make-indented-block
-			     tmp (org-get-indentation))))))
-	    (replace-match tmp t t))))
+	  (unless (org-capture-escaped-%)
+	    (setq name (match-string 1)
+		  escape (org-capture-inside-embedded-elisp-p))
+	    (cond
+	     ((member name '("h" "t" "T" "u" "U" "a"))
+	      (setq tmp (symbol-value (intern (concat "v-" name)))))
+	     ((setq tmp (plist-get entry (intern (concat ":" name))))
+	      (save-excursion
+		(save-match-data
+		  (beginning-of-line 1)
+		  (when (looking-at
+			 (concat "^\\([ \t]*\\)%" name "[ \t]*$"))
+		    (setq tmp (org-feed-make-indented-block
+			       tmp (org-get-indentation))))))))
+	    (when tmp
+	      ;; escape string delimiters `"' when inside %() embedded lisp
+	      (when escape
+		(setq tmp (replace-regexp-in-string "\"" "\\\\\"" tmp)))
+	      (replace-match tmp t t))))
+
+	;; %() embedded elisp
+	(org-capture-expand-embedded-elisp)
+
 	(decode-coding-string
 	 (buffer-string) (detect-coding-region (point-min) (point-max) t))))))
 
-- 
1.7.4.2

Re: Add the capture feature "%(sexp)" to org-feed

[Bastien]

Hi Michael,

Michael Brand <michael.ch.brand@gmail.com> writes:

> Thank you for making `org-capture-inside-embedded-elisp-p' much
> clearer. I was not sure if you really wanted to omit switching to
> Emacs Lisp Mode although it is useful for example for “%(length ")")”.
> So I attach a new patch where I restored this part, now cleaner too.
> Also a small change against your latest patch to not stop at “%”
> without following “(” and possibly miss a “%(” before.

Thanks.  Actually we don't need the whole emacs-lisp-mode so that
forward-sexp do something sensible.  This is enough:

  (with-syntax-table emacs-lisp-mode-syntax-table ...)

See the commit I just pushed:

  http://orgmode.org/w/?p=org-mode.git;a=commit;h=042db3

> I temporarily emptied “:FEEDSTATUS:” in all my feeds and updated them
> and also tried some odd artificial examples. All works the same as
> with my first patch. Can you please review this new patch compared
> against your latest and apply when appropriate?

Done.  Thanks again for your patience,

-- 
 Bastien

Re: Add the capture feature "%(sexp)" to org-feed

[Michael Brand]

[-- Attachment #1: Type: text/plain, Size: 703 bytes --]

Hi Bastien

On Sat, Aug 11, 2012 at 6:55 PM, Bastien <bzg@gnu.org> wrote:
> Michael Brand <michael.ch.brand@gmail.com> writes:
>> [...]
>> Also a small change against your latest patch to not stop at “%”
>> without following “(” and possibly miss a “%(” before.

This part that I changed between
http://patchwork.newartisans.com/patch/1405
and
http://patchwork.newartisans.com/patch/1408
got lost, the attached patch restores it.

> Actually we don't need the whole emacs-lisp-mode so that
> forward-sexp do something sensible.  This is enough:
>
>   (with-syntax-table emacs-lisp-mode-syntax-table ...)

Thank you for the improvement, this is a very interesting `with-'.

Michael

[-- Attachment #2: 0001-Improve-parsing-of-org-capture-inside-embedded-elisp.patch.txt --]
[-- Type: text/plain, Size: 1174 bytes --]

From 10888cbd40e3f546b9e577ff59bfa152679f9628 Mon Sep 17 00:00:00 2001
From: Michael Brand <michael.ch.brand@gmail.com>
Date: Sat, 11 Aug 2012 20:08:35 +0200
Subject: [PATCH] Improve parsing of org-capture-inside-embedded-elisp-p

* org-capture.el (org-capture-inside-embedded-elisp-p): Improve parsing.
---
 lisp/org-capture.el |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/lisp/org-capture.el b/lisp/org-capture.el
index 6882613..4d6e91d 100644
--- a/lisp/org-capture.el
+++ b/lisp/org-capture.el
@@ -1539,9 +1539,12 @@ The template may still contain \"%?\" for cursor positioning."
   (let (beg end)
     (with-syntax-table emacs-lisp-mode-syntax-table
       (save-excursion
+	;; `looking-at' and `search-backward' below do not match the "%(" if
+	;; point is in its middle
+	(when (equal (char-before) ?%)
+	  (backward-char))
 	(save-match-data
-	  (when (or (looking-at "%(")
-		    (and (search-backward "%" nil t) (looking-at "%(")))
+	  (when (or (looking-at "%(") (search-backward "%(" nil t))
 	    (setq beg (point))
 	    (setq end (progn (forward-char) (forward-sexp) (1- (point)))))))
       (when (and beg end)
-- 
1.7.4.2

Re: Add the capture feature "%(sexp)" to org-feed

[Bastien]

Hi Michael,

Michael Brand <michael.ch.brand@gmail.com> writes:

> This part that I changed between
> http://patchwork.newartisans.com/patch/1405
> and
> http://patchwork.newartisans.com/patch/1408
> got lost, the attached patch restores it.

Unless I miss something obvious, the current version returns exactly 
the same thing than the one you propose -- can you double-check or let 
me know what is wrong with the current solution?

Thanks!

-- 
 Bastien

Re: Add the capture feature "%(sexp)" to org-feed

[Michael Brand]

Hi Bastien

On Sat, Aug 11, 2012 at 11:59 PM, Bastien <bzg@gnu.org> wrote:
> Unless I miss something obvious, the current version returns exactly
> the same thing than the one you propose -- can you double-check or let
> me know what is wrong with the current solution?

Your solution is wrong when there is a “%” not immediately followed by “(”:

On Sat, Aug 11, 2012 at 8:11 PM, Michael Brand
<michael.ch.brand@gmail.com> wrote:
> On Sat, Aug 11, 2012 at 6:55 PM, Bastien <bzg@gnu.org> wrote:
>> Michael Brand <michael.ch.brand@gmail.com> writes:
>>> [...]
>>> Also a small change against your latest patch to not stop at “%”
>>> without following “(” and possibly miss a “%(” before.

A realistic example, with point somewhere between “%” and the last “)”:
“- %(capitalize "5 % less (see item \"3)\" below)\n")\n”

This can happen with a feed template containing
"- %(capitalize \"%description\")\n"
and %description expanding to
“5 % less (see item "3)" below)\n”

All problems of the example are solved:
- escaping of the quotes that surround the “3)” in string
- seemingly unbalanced parenthesis in string coming from “3)”
- newline within %(...)
- “%” not immediately followed by “(”

Michael

Re: Add the capture feature "%(sexp)" to org-feed

[Bastien]

Hi Michael,

Michael Brand <michael.ch.brand@gmail.com> writes:

> Your solution is wrong when there is a “%” not immediately followed by
> “(”:

Argh, right.  I applied your patch, thanks again for the detailed
examples. 

-- 
 Bastien

Re: Add the capture feature "%(sexp)" to org-feed

[Ivan Andrus]

On Aug 9, 2012, at 4:53 PM, Bastien wrote:

> Hi Michael,
> 
> Michael Brand <michael.ch.brand@gmail.com> writes:
> 
>> You omitted the Local variables sentence-end-double-space: t. Isn't
>> this a good idea for all the users like me that have set this to nil
>> in their config? I wanted to add it to more files as soon as I will
>> change them.
> 
> I've check against current Emacs trunk, and only three files/packages
> uses this: rst.el, filesets.el and mh-e.
> 
> So I'd rather not pollute the hundred files of Org with this.

I'm probably missing something, but isn't this what directory local variables are for?

-Ivan

Re: Add the capture feature "%(sexp)" to org-feed

[Michael Brand]

[-- Attachment #1: Type: text/plain, Size: 338 bytes --]

Hi Ivan

On Thu, Aug 9, 2012 at 11:09 PM, Ivan Andrus <darthandrus@gmail.com> wrote:
> I'm probably missing something, but isn't this what directory local variables are for?

Yes. This is useful in general for me, thank you for the hint. I
propose the attached patch for the already existing .dir-locals.el and
.dir-settings.el.

Michael

[-- Attachment #2: 0001-Directory-local-variables-Set-sentence-end-double-sp.patch.txt --]
[-- Type: text/plain, Size: 1538 bytes --]

From 345419858e85d75a96f73b7d7f6e0ddbde0aae81 Mon Sep 17 00:00:00 2001
From: Michael Brand <michael.ch.brand@gmail.com>
Date: Fri, 10 Aug 2012 06:40:10 +0200
Subject: [PATCH] Directory local variables: Set sentence-end-double-space to t

* .dir-locals.el: Add comments and set `sentence-end-double-space' to t.

* .dir-settings.el: Add comments and set `sentence-end-double-space' to t.
---
 .dir-locals.el   |    9 +++++++--
 .dir-settings.el |    9 +++++++--
 2 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/.dir-locals.el b/.dir-locals.el
index 7094cb9..05d50f6 100644
--- a/.dir-locals.el
+++ b/.dir-locals.el
@@ -1,4 +1,9 @@
+;; - keep in sync dir-locals.el (for Emacs >= 24) with .dir-settings.el
+;;   (for Emacs < 24)
+;; - don't use a symbolic link to prevent problems on cygwin
+;;   distributions (commit 971b9eeacd38959439ddaa7c650430cc2dcb673e)
+
 ((nil . ((indent-tabs-mode . t)
 	 (tab-width . 8)
-	 (fill-column . 70))))
-
+	 (fill-column . 70)
+	 (sentence-end-double-space . t))))
diff --git a/.dir-settings.el b/.dir-settings.el
index 7094cb9..05d50f6 100644
--- a/.dir-settings.el
+++ b/.dir-settings.el
@@ -1,4 +1,9 @@
+;; - keep in sync dir-locals.el (for Emacs >= 24) with .dir-settings.el
+;;   (for Emacs < 24)
+;; - don't use a symbolic link to prevent problems on cygwin
+;;   distributions (commit 971b9eeacd38959439ddaa7c650430cc2dcb673e)
+
 ((nil . ((indent-tabs-mode . t)
 	 (tab-width . 8)
-	 (fill-column . 70))))
-
+	 (fill-column . 70)
+	 (sentence-end-double-space . t))))
-- 
1.7.4.2

Re: Add the capture feature "%(sexp)" to org-feed

[Bastien]

Michael Brand <michael.ch.brand@gmail.com> writes:

> Yes. This is useful in general for me, thank you for the hint. I
> propose the attached patch for the already existing .dir-locals.el and
> .dir-settings.el.

Applied, thanks to you and Ivan.

-- 
 Bastien