Weird behavior in `org-agenda-redo' and/or `org-mobile-push'

Weird behavior in `org-agenda-redo' and/or `org-mobile-push'

[Konstantin Kliakhandler]

[-- Attachment #1: Type: text/plain, Size: 3454 bytes --]

Hello,

[This behavior appears in a previous and the following org versions:
Org-mode version 8.2.6 (8.2.6-14-g063c8b-elpa @
/home/kosta/.emacs.d/elpa/org-20140505/)]

I recently began seeing weird behavior when trying to run `org-mobile-push'
in an agenda buffer; It appears the files are indeed pushed, but at the end
I get an error: "(wrong-type-argument arrayp nil)"

This only happens when I attempt to push from an agenda buffer - if I push
in a regular org buffer, everything works.

The stack trace is:
org-agenda-filter-apply(nil category)
org-agenda-redo()
org-mobile-push()

where the `org-agenda-filter-apply' call is from this sexp in
`org-agenda-redo':
(and (or cat-filter cat-preset) (org-agenda-filter-apply cat-filter
'category))

And cat-filter, cat-preset are set in a let* form higher up in
org-agenda-redo as (their values are included):

(cat-filter org-agenda-category-filter)
nil
(cat-preset (get 'org-agenda-category-filter :preset-filter))
("+Academic")

When I start emacs or even look at the agenda,
(get 'org-agenda-category-filter :preset-filter) always evaluates to nil
(up to the point where I eval `org-mobile-push'). It gets set in
`org-mobile-create-sumo-agenda', which is called at some point from
`org-mobile-push' before `org-agenda-redo'.

Apparently this started happening after I added the following custom agenda
command:

("z" "All academic agenda items" alltodo ""
 ((org-agenda-overriding-header "Academic todos.")
  (org-agenda-category-filter-preset
   (quote
("+Academic")))
(org-agenda-todo-list-sublevels nil)))

Now, I have two questions/comments:
1. It appears that `org-agenda-filter-apply' expects an array, and an array
always evaluates to togical t (As far as I remember, have just checked). If
this is the case, in the following sexp (from `org-agenda-filter-apply'):
(and (or cat-filter cat-preset) (org-agenda-filter-apply cat-filter
'category)) why is the (or cat-filter cat-preset)? It seems that either
`cat-filter' is an array, in which case it is (possibly) valid to pass it
to `org-agenda-filter-apply', or is nil, in which case it isn't. What role
does cat-preset play here?
2. It appears to me that `org-mobile-create-sumo-agenda' does not work
entirely correctly, as it leaves some properties set that were not set
before it ran?

Ok, so after writing the last two questions, I looked into
`org-agenda-filter-apply', and it appears that it expects a list of strings
of the form "+cat" or "-cat", which is what `cat-preset' holds at this
point.

Changing
(and (or cat-filter cat-preset) (org-agenda-filter-apply cat-filter
'category))
to
(and (or cat-filter cat-preset) (org-agenda-filter-apply (or cat-filter
cat-preset) 'category))
as a check and reevaling the defun makes org-mobile-push complete with no
errors even when ran from an agenda buffer. Note however:
1. There are several more similar lines in org-agenda-redo, which I suspect
also need to be checked.
2. After org-mobile-push, a category filter is applied to the agenda buffer
("+Academic"), despite the original agenda buffer having no such filter, so
it seems that something is broken in `org-mobile-create-sumo-agenda'
regardless.

I found a (somewhat) relevant post here:
http://thread.gmane.org/gmane.emacs.orgmode/57481
If what I wrote makes sense, let me know and I will submit a patch for
`org-agenda-filter-apply', and if I figure out how to fix
`org-mobile-create-sumo-agenda', for that as well.

Thanks,
Kosta

[-- Attachment #2: Type: text/html, Size: 4468 bytes --]

Re: Weird behavior in `org-agenda-redo' and/or `org-mobile-push'

[Konstantin Kliakhandler]

[-- Attachment #1: Type: text/plain, Size: 4392 bytes --]

While investigating the previous issue, I came upon this code:

;;;###autoload
(defun org-store-agenda-views (&rest parameters)
  "Store agenda views."
  (interactive)
  (eval (list 'org-batch-store-agenda-views)))

I guess there is some advantage over the following, simpler version?

;;;###autoload
(defun org-store-agenda-views (&rest parameters)
  "Store agenda views."
  (interactive)
  (org-batch-store-agenda-views))

If so, what is it?

Thanks in advance for furthering my education,
Kosta

--
Konstantin Kliakhandler
    http://slumpy.org
          )°) )°( (°(


On Tue, May 6, 2014 at 2:00 PM, Konstantin Kliakhandler <kosta@slumpy.org>wrote:

> Hello,
>
> [This behavior appears in a previous and the following org versions:
> Org-mode version 8.2.6 (8.2.6-14-g063c8b-elpa @
> /home/kosta/.emacs.d/elpa/org-20140505/)]
>
> I recently began seeing weird behavior when trying to run
> `org-mobile-push' in an agenda buffer; It appears the files are indeed
> pushed, but at the end I get an error: "(wrong-type-argument arrayp nil)"
>
> This only happens when I attempt to push from an agenda buffer - if I push
> in a regular org buffer, everything works.
>
> The stack trace is:
> org-agenda-filter-apply(nil category)
> org-agenda-redo()
> org-mobile-push()
>
> where the `org-agenda-filter-apply' call is from this sexp in
> `org-agenda-redo':
> (and (or cat-filter cat-preset) (org-agenda-filter-apply cat-filter
> 'category))
>
> And cat-filter, cat-preset are set in a let* form higher up in
> org-agenda-redo as (their values are included):
>
> (cat-filter org-agenda-category-filter)
> nil
> (cat-preset (get 'org-agenda-category-filter :preset-filter))
> ("+Academic")
>
> When I start emacs or even look at the agenda,
> (get 'org-agenda-category-filter :preset-filter) always evaluates to nil
> (up to the point where I eval `org-mobile-push'). It gets set in
> `org-mobile-create-sumo-agenda', which is called at some point from
> `org-mobile-push' before `org-agenda-redo'.
>
> Apparently this started happening after I added the following custom
> agenda command:
>
> ("z" "All academic agenda items" alltodo ""
>  ((org-agenda-overriding-header "Academic todos.")
>   (org-agenda-category-filter-preset
>    (quote
> ("+Academic")))
> (org-agenda-todo-list-sublevels nil)))
>
> Now, I have two questions/comments:
> 1. It appears that `org-agenda-filter-apply' expects an array, and an
> array always evaluates to togical t (As far as I remember, have just
> checked). If this is the case, in the following sexp (from
> `org-agenda-filter-apply'): (and (or cat-filter cat-preset)
> (org-agenda-filter-apply cat-filter 'category)) why is the (or cat-filter
> cat-preset)? It seems that either `cat-filter' is an array, in which case
> it is (possibly) valid to pass it to `org-agenda-filter-apply', or is nil,
> in which case it isn't. What role does cat-preset play here?
> 2. It appears to me that `org-mobile-create-sumo-agenda' does not work
> entirely correctly, as it leaves some properties set that were not set
> before it ran?
>
> Ok, so after writing the last two questions, I looked into
> `org-agenda-filter-apply', and it appears that it expects a list of strings
> of the form "+cat" or "-cat", which is what `cat-preset' holds at this
> point.
>
> Changing
> (and (or cat-filter cat-preset) (org-agenda-filter-apply cat-filter
> 'category))
> to
> (and (or cat-filter cat-preset) (org-agenda-filter-apply (or cat-filter
> cat-preset) 'category))
> as a check and reevaling the defun makes org-mobile-push complete with no
> errors even when ran from an agenda buffer. Note however:
> 1. There are several more similar lines in org-agenda-redo, which I
> suspect also need to be checked.
> 2. After org-mobile-push, a category filter is applied to the agenda
> buffer ("+Academic"), despite the original agenda buffer having no such
> filter, so it seems that something is broken in
> `org-mobile-create-sumo-agenda' regardless.
>
> I found a (somewhat) relevant post here:
> http://thread.gmane.org/gmane.emacs.orgmode/57481
> If what I wrote makes sense, let me know and I will submit a patch for
> `org-agenda-filter-apply', and if I figure out how to fix
> `org-mobile-create-sumo-agenda', for that as well.
>
> Thanks,
> Kosta
>

[-- Attachment #2: Type: text/html, Size: 5874 bytes --]

Re: Weird behavior in `org-agenda-redo' and/or `org-mobile-push'

[Konstantin Kliakhandler]

[-- Attachment #1.1: Type: text/plain, Size: 5088 bytes --]

Ok, I don't know specifically why the code in the last message was
necessary, but I realized the difference between the two cases (in the
first case, the macro is expanded every time the function is called,
whereas in the second it is expanded once, when the function is defined).

I also fixed the problems in both org-agenda.el and am including the patch.

Thanks,
Kosta


--
Konstantin Kliakhandler
    http://slumpy.org
          )°) )°( (°(


On Tue, May 6, 2014 at 6:54 PM, Konstantin Kliakhandler <kosta@slumpy.org>wrote:

> While investigating the previous issue, I came upon this code:
>
> ;;;###autoload
> (defun org-store-agenda-views (&rest parameters)
>   "Store agenda views."
>   (interactive)
>   (eval (list 'org-batch-store-agenda-views)))
>
> I guess there is some advantage over the following, simpler version?
>
> ;;;###autoload
> (defun org-store-agenda-views (&rest parameters)
>   "Store agenda views."
>   (interactive)
>   (org-batch-store-agenda-views))
>
> If so, what is it?
>
> Thanks in advance for furthering my education,
> Kosta
>
> --
> Konstantin Kliakhandler
>     http://slumpy.org
>           )°) )°( (°(
>
>
> On Tue, May 6, 2014 at 2:00 PM, Konstantin Kliakhandler <kosta@slumpy.org>wrote:
>
>> Hello,
>>
>> [This behavior appears in a previous and the following org versions:
>> Org-mode version 8.2.6 (8.2.6-14-g063c8b-elpa @
>> /home/kosta/.emacs.d/elpa/org-20140505/)]
>>
>> I recently began seeing weird behavior when trying to run
>> `org-mobile-push' in an agenda buffer; It appears the files are indeed
>> pushed, but at the end I get an error: "(wrong-type-argument arrayp nil)"
>>
>> This only happens when I attempt to push from an agenda buffer - if I
>> push in a regular org buffer, everything works.
>>
>> The stack trace is:
>> org-agenda-filter-apply(nil category)
>> org-agenda-redo()
>> org-mobile-push()
>>
>> where the `org-agenda-filter-apply' call is from this sexp in
>> `org-agenda-redo':
>> (and (or cat-filter cat-preset) (org-agenda-filter-apply cat-filter
>> 'category))
>>
>> And cat-filter, cat-preset are set in a let* form higher up in
>> org-agenda-redo as (their values are included):
>>
>> (cat-filter org-agenda-category-filter)
>> nil
>> (cat-preset (get 'org-agenda-category-filter :preset-filter))
>> ("+Academic")
>>
>> When I start emacs or even look at the agenda,
>> (get 'org-agenda-category-filter :preset-filter) always evaluates to nil
>> (up to the point where I eval `org-mobile-push'). It gets set in
>> `org-mobile-create-sumo-agenda', which is called at some point from
>> `org-mobile-push' before `org-agenda-redo'.
>>
>> Apparently this started happening after I added the following custom
>> agenda command:
>>
>> ("z" "All academic agenda items" alltodo ""
>>  ((org-agenda-overriding-header "Academic todos.")
>>   (org-agenda-category-filter-preset
>>    (quote
>> ("+Academic")))
>> (org-agenda-todo-list-sublevels nil)))
>>
>> Now, I have two questions/comments:
>> 1. It appears that `org-agenda-filter-apply' expects an array, and an
>> array always evaluates to togical t (As far as I remember, have just
>> checked). If this is the case, in the following sexp (from
>> `org-agenda-filter-apply'): (and (or cat-filter cat-preset)
>> (org-agenda-filter-apply cat-filter 'category)) why is the (or cat-filter
>> cat-preset)? It seems that either `cat-filter' is an array, in which case
>> it is (possibly) valid to pass it to `org-agenda-filter-apply', or is nil,
>> in which case it isn't. What role does cat-preset play here?
>> 2. It appears to me that `org-mobile-create-sumo-agenda' does not work
>> entirely correctly, as it leaves some properties set that were not set
>> before it ran?
>>
>> Ok, so after writing the last two questions, I looked into
>> `org-agenda-filter-apply', and it appears that it expects a list of strings
>> of the form "+cat" or "-cat", which is what `cat-preset' holds at this
>> point.
>>
>> Changing
>> (and (or cat-filter cat-preset) (org-agenda-filter-apply cat-filter
>> 'category))
>> to
>> (and (or cat-filter cat-preset) (org-agenda-filter-apply (or cat-filter
>> cat-preset) 'category))
>> as a check and reevaling the defun makes org-mobile-push complete with no
>> errors even when ran from an agenda buffer. Note however:
>> 1. There are several more similar lines in org-agenda-redo, which I
>> suspect also need to be checked.
>> 2. After org-mobile-push, a category filter is applied to the agenda
>> buffer ("+Academic"), despite the original agenda buffer having no such
>> filter, so it seems that something is broken in
>> `org-mobile-create-sumo-agenda' regardless.
>>
>> I found a (somewhat) relevant post here:
>> http://thread.gmane.org/gmane.emacs.orgmode/57481
>> If what I wrote makes sense, let me know and I will submit a patch for
>> `org-agenda-filter-apply', and if I figure out how to fix
>> `org-mobile-create-sumo-agenda', for that as well.
>>
>> Thanks,
>> Kosta
>>
>
>

[-- Attachment #1.2: Type: text/html, Size: 6974 bytes --]

[-- Attachment #2: org-agenda.patch --]
[-- Type: text/x-patch, Size: 3330 bytes --]

From d03aa078723a1e30b7238ab76aee673032e8d453 Mon Sep 17 00:00:00 2001
From: Konstantin Kliakhandler <kosta@slumpy.org>
Date: Tue, 6 May 2014 21:04:20 +0300
Subject: [PATCH] Fixed bad behavior of org-agenda-redo and
 org-batch-store-agenda-views.

---
 lisp/ChangeLog     |  5 +++++
 lisp/org-agenda.el | 34 ++++++++++++++++++++--------------
 2 files changed, 25 insertions(+), 14 deletions(-)
 create mode 100644 lisp/ChangeLog

diff --git a/lisp/ChangeLog b/lisp/ChangeLog
new file mode 100644
index 0000000..9534e13
--- /dev/null
+++ b/lisp/ChangeLog
@@ -0,0 +1,5 @@
+2014-05-06  Konstantin Kliakhandler  <kosta@slumpy.org>
+
+	* org-agenda.el
+	(org-agenda-redo): Making sure to feed `org-agenda-filter-apply' with the appropriate value.
+	(org-batch-store-agenda-views): Restoring the original `org-agenda-category-filter' value after constructing the exported agendas.
diff --git a/lisp/org-agenda.el b/lisp/org-agenda.el
index 576d5b9..134af00 100644
--- a/lisp/org-agenda.el
+++ b/lisp/org-agenda.el
@@ -3282,17 +3282,20 @@ This ensures the export commands can easily use it."
 	      opts (nth (if (listp cmd-or-set) 3 4) cmd)
 	      files (nth (if (listp cmd-or-set) 4 5) cmd))
 	(if (stringp files) (setq files (list files)))
-	(when files
-	  (org-eval-in-environment (append org-agenda-exporter-settings
-					   opts pars)
-	    (org-agenda nil thiscmdkey))
-	  (set-buffer bufname)
-	  (while files
-	    (org-eval-in-environment (append org-agenda-exporter-settings
-					     opts pars)
-	      (org-agenda-write (expand-file-name (pop files) dir) nil t bufname)))
-	  (and (get-buffer bufname)
-	       (kill-buffer bufname)))))))
+	(let ((org-agenda-category-filter-plist (copy-sequence (symbol-plist 'org-agenda-category-filter))))
+	  (unwind-protect
+	      (when files
+		(org-eval-in-environment (append org-agenda-exporter-settings
+						 opts pars)
+		  (org-agenda nil thiscmdkey))
+		(set-buffer bufname)
+		(while files
+		  (org-eval-in-environment (append org-agenda-exporter-settings
+						   opts pars)
+		    (org-agenda-write (expand-file-name (pop files) dir) nil t bufname)))
+		(and (get-buffer bufname)
+		     (kill-buffer bufname)))
+	    (setplist 'org-agenda-category-filter org-agenda-category-filter-plist)))))))
 
 (defvar org-agenda-current-span nil
   "The current span used in the agenda view.") ; local variable in the agenda buffer
@@ -7325,9 +7328,12 @@ in the agenda."
     (put 'org-agenda-tag-filter :preset-filter tag-preset)
     (put 'org-agenda-category-filter :preset-filter cat-preset)
     (put 'org-agenda-regexp-filter :preset-filter re-preset)
-    (and (or tag-filter tag-preset) (org-agenda-filter-apply tag-filter 'tag))
-    (and (or cat-filter cat-preset) (org-agenda-filter-apply cat-filter 'category))
-    (and (or re-filter re-preset) (org-agenda-filter-apply re-filter 'regexp))
+    (let ((tag (or tag-filter tag-preset))
+	  (cat (or cat-filter cat-preset))
+	  (re  (or re-filter re-preset)))
+      (when tag (org-agenda-filter-apply tag 'tag))
+      (when cat (org-agenda-filter-apply cat 'category))
+      (when re  (org-agenda-filter-apply re 'regexp)))
     (and top-hl-filter (org-agenda-filter-top-headline-apply top-hl-filter))
     (and cols (org-called-interactively-p 'any) (org-agenda-columns))
     (org-goto-line line)
-- 
1.8.5.3

Re: Weird behavior in `org-agenda-redo' and/or `org-mobile-push'

[Bastien]

Hi Konstantin,

Konstantin Kliakhandler <kosta@slumpy.org> writes:

> I also fixed the problems in both org-agenda.el and am including the
> patch.

Thanks for the patch -- I applied the part that I understand:
http://orgmode.org/cgit.cgi/org-mode.git/commit/?id=445a8ec6

As for the other part, can you restate what bug it fixes in very
simple words?

Also, we cannot accept it unless you signed the FSF copyright
assignment, so prepare to go this route if you want to submit
a consequent patch.  Otherwise simply describe the problem and
a possible fix and we'll implement it.

Thanks,

-- 
 Bastien

Re: Weird behavior in `org-agenda-redo' and/or `org-mobile-push'

[Konstantin Kliakhandler]

[-- Attachment #1: Type: text/plain, Size: 2015 bytes --]

Hi Bastien,

I'm willing to sign the FSF copyright papers, provided it isn't too big of
a hassle :-). What do I need to do?

The problem that the other part of the patch solves is as follows:

org-batch-store-agenda-views writes all agendas (according to some
criteria) to disk.
To do this, it first generates all those agendas in a temporary buffer,
which is killed at the end.
This is done in a special environment, so as to not change the currently
open agenda buffer (if there is one).
The process is not hermetic - the plist of org-agenda-category-filter can
get modified while generating the temporary agendas (when generating
agendas with a category filter).
If indeed modified, then at the end of the process, a user sees a filtered
agenda buffer even when starting out from an unfiltered one.

The patch addresses this by first saving the plist of the
org-agenda-category-filter variable, then executing the regular
functionality of org-batch-store-agenda-views, and finally restoring the
plist of org-agenda-category-filter variable.

In case something falls through with the FSF papers, I think this should be
enough to implement a similar fix.

Thanks,
Kosta


-- 
Konstantin Kliakhandler
    http://slumpy.org
          )°) )°( (°(


On Thu, May 22, 2014 at 1:59 PM, Bastien <bzg@gnu.org> wrote:

> Hi Konstantin,
>
> Konstantin Kliakhandler <kosta@slumpy.org> writes:
>
> > I also fixed the problems in both org-agenda.el and am including the
> > patch.
>
> Thanks for the patch -- I applied the part that I understand:
> http://orgmode.org/cgit.cgi/org-mode.git/commit/?id=445a8ec6
>
> As for the other part, can you restate what bug it fixes in very
> simple words?
>
> Also, we cannot accept it unless you signed the FSF copyright
> assignment, so prepare to go this route if you want to submit
> a consequent patch.  Otherwise simply describe the problem and
> a possible fix and we'll implement it.
>
> Thanks,
>
> --
>  Bastien
>

[-- Attachment #2: Type: text/html, Size: 2895 bytes --]

Re: Weird behavior in `org-agenda-redo' and/or `org-mobile-push'

[Konstantin Kliakhandler]

[-- Attachment #1: Type: text/plain, Size: 2611 bytes --]

Dear Bastien,

Would you please consider incorporating my patch (or reimplementing it)? As
it stands, the problem I described earlier (agenda buffer gets screwed up
after org mobile push) persists and I need to repatch on every new version
I install. I will also look into the FSF papers myself.

Thanks in advance,
Kosta

-- 
Konstantin Kliakhandler
    http://slumpy.org
          )°) )°( (°(

On Thu, May 22, 2014 at 7:08 PM, Konstantin Kliakhandler <kosta@slumpy.org>
wrote:

> Hi Bastien,
>
> I'm willing to sign the FSF copyright papers, provided it isn't too big of
> a hassle :-). What do I need to do?
>
> The problem that the other part of the patch solves is as follows:
>
> org-batch-store-agenda-views writes all agendas (according to some
> criteria) to disk.
> To do this, it first generates all those agendas in a temporary buffer,
> which is killed at the end.
> This is done in a special environment, so as to not change the currently
> open agenda buffer (if there is one).
> The process is not hermetic - the plist of org-agenda-category-filter can
> get modified while generating the temporary agendas (when generating
> agendas with a category filter).
> If indeed modified, then at the end of the process, a user sees a filtered
> agenda buffer even when starting out from an unfiltered one.
>
> The patch addresses this by first saving the plist of the
> org-agenda-category-filter variable, then executing the regular
> functionality of org-batch-store-agenda-views, and finally restoring the
> plist of org-agenda-category-filter variable.
>
> In case something falls through with the FSF papers, I think this should
> be enough to implement a similar fix.
>
> Thanks,
> Kosta
>
>
> --
> Konstantin Kliakhandler
>     http://slumpy.org
>           )°) )°( (°(
>
>
> On Thu, May 22, 2014 at 1:59 PM, Bastien <bzg@gnu.org> wrote:
>
>> Hi Konstantin,
>>
>> Konstantin Kliakhandler <kosta@slumpy.org> writes:
>>
>> > I also fixed the problems in both org-agenda.el and am including the
>> > patch.
>>
>> Thanks for the patch -- I applied the part that I understand:
>> http://orgmode.org/cgit.cgi/org-mode.git/commit/?id=445a8ec6
>>
>> As for the other part, can you restate what bug it fixes in very
>> simple words?
>>
>> Also, we cannot accept it unless you signed the FSF copyright
>> assignment, so prepare to go this route if you want to submit
>> a consequent patch.  Otherwise simply describe the problem and
>> a possible fix and we'll implement it.
>>
>> Thanks,
>>
>> --
>>  Bastien
>>
>
>

[-- Attachment #2: Type: text/html, Size: 3911 bytes --]