org-depend: TRIGGER XYZ(KEYWORD) not working

org-depend: TRIGGER XYZ(KEYWORD) not working

[Karl Voit]

Hi,

I am playing around with org-depend[1] again[2].

I noticed that certain TRIGGER events do not get executed.

> # -*- mode: org; coding: utf-8; -*-
> * heading 1
> :PROPERTIES:
> :ID:       foo
> :TRIGGER: bar(NEXT) baz(NEXT)
> :END:
> 
> * Heading 2
> :PROPERTIES:
> :ID:       bar
> :END:
> 
> * Heading 3
> :PROPERTIES:
> :ID:      baz
> :END:

First observation:

When I set the heading 1 to DONE (without assigning it any other
keyword), the TRIGGER events are ignored totally.  I guess this is
an edge-case that may be considered as a bug.

Second observation:

However, what is bugging me even more is that even when "heading 1"
has a TODO keyword assigned, in *some* cases, the TRIGGER event does
not get executed when I do it in my large Org-mode file.

Copying the corresponding headings (the one with the TRIGGER prop
and the headings containing the "target" IDs) from my real-world
Org-mode file to my small test.org from above and marking the
heading that contains the TRIGGER properties to DONE, it then works
as expected. 

I then took the simple example from above, added a TODO keyword to
the heading 1, copied it to my real-world Org-mode file and even
this did not work: Heading 2 and 3 don't get their "NEXT" keyword
assigned.

So the behavior changes within different Org-file contexts.
Therefore, I do have an issue creating a minimal example to
demonstrate the bug.

Can somebody give me an advice how to debug this behavior?

[1] http://orgmode.org/worg/org-contrib/org-depend.html
[2] http://karl-voit.at/2016/12/18/org-depend/

-- 
get mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML into Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <
Personal Information Management > http://Karl-Voit.at/tags/pim/
Emacs-related > http://Karl-Voit.at/tags/emacs/

Re: org-depend: TRIGGER XYZ(KEYWORD) not working

[Karl Voit]

* Karl Voit <devnull@Karl-Voit.at> wrote:
> Hi,
>
> I am playing around with org-depend[1] again[2].

I forgot to mention: I am using maint from the git from today.

-- 
get mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML into Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <
Personal Information Management > http://Karl-Voit.at/tags/pim/
Emacs-related > http://Karl-Voit.at/tags/emacs/

Re: org-depend: TRIGGER XYZ(KEYWORD) not working

[Adrian Bradd]

[-- Attachment #1: Type: text/plain, Size: 2979 bytes --]

Hi,

First observation:
>
> When I set the heading 1 to DONE (without assigning it any other
> keyword), the TRIGGER events are ignored totally.  I guess this is
> an edge-case that may be considered as a bug.
>

​This is the result of Line 219 in org-depend.el:

(unless (and (member from org-not-done-keywords)
            (member to org-done-keywords))

In this instance 'from' is set to nil so the 'and' returns nil and
org-depend doesn't process the change of TODO to look for dependents. I
don't know why it was defined this way as org-todo.el accepts headings with
no TODO entry. I can't think of any reason other than explicitly blocking a
user from processing a heading without a TODO entry, but it seems that if
the user went to the effort of populating a TRIGGER property that they
probably want this to be acted upon by org-depend.el.

If there are no concerns raised in this thread I'll push a patch to all
have org-depend process the example you listed.


> Second observation:
>
> However, what is bugging me even more is that even when "heading 1"
> has a TODO keyword assigned, in *some* cases, the TRIGGER event does
> not get executed when I do it in my large Org-mode file.
>
> Copying the corresponding headings (the one with the TRIGGER prop
> and the headings containing the "target" IDs) from my real-world
> Org-mode file to my small test.org from above and marking the
> heading that contains the TRIGGER properties to DONE, it then works
> as expected.
>
> I then took the simple example from above, added a TODO keyword to
> the heading 1, copied it to my real-world Org-mode file and even
> this did not work: Heading 2 and 3 don't get their "NEXT" keyword
> assigned.
>
> So the behavior changes within different Org-file contexts.
> Therefore, I do have an issue creating a minimal example to
> demonstrate the bug.
>
> Can somebody give me an advice how to debug this behavior?
>

​My only thought is that there is an ID clashing with your sample ID and it
is being updated instead of the heading you are interested in.
org-depend.el looks at the local file for an ID and then looks globally for
an ID match.

You could instrument 'org-depend-trigger-todo' to trace the behaviour while
processing the TODO update.

Otherwise, you could use a process of elimination in a copy of your
existing file to find the MWE that causes the issue and post it here. You
could successively delete headings or other entries until the issue stops
occurring.​

I have read your post on org-depend and found it insightful so thanks for
that. When I was looking around at patching some new functionality into
org-depend I was put onto org-edna[1] which was built as an alternative to
org-depend. The syntax isn't compatible with org-depend, but org-edna is
more powerful than org-depend. I am now using org-edna and am quite happy
with it.

[1] https://elpa.gnu.org/packages/org-edna.html

[-- Attachment #2: Type: text/html, Size: 4696 bytes --]

Re: org-depend: TRIGGER XYZ(KEYWORD) not working

[Karl Voit]

Hi Adrian,

Glad that you replied - I was worrying that this thread is going to
die before any improvement can be developed ...

* Adrian Bradd <adrian.bradd@gmail.com> wrote:
>
> First observation:
>>
>> When I set the heading 1 to DONE (without assigning it any other
>> keyword), the TRIGGER events are ignored totally.  I guess this is
>> an edge-case that may be considered as a bug.
>>
>
> This is the result of Line 219 in org-depend.el:
>
> (unless (and (member from org-not-done-keywords)
>             (member to org-done-keywords))
>
> In this instance 'from' is set to nil so the 'and' returns nil and
> org-depend doesn't process the change of TODO to look for dependents. I
> don't know why it was defined this way as org-todo.el accepts headings with
> no TODO entry. I can't think of any reason other than explicitly blocking a
> user from processing a heading without a TODO entry, but it seems that if
> the user went to the effort of populating a TRIGGER property that they
> probably want this to be acted upon by org-depend.el.

I totally agree: there should not be any reason to accept missing
TODO keywords in this case as well.

> If there are no concerns raised in this thread I'll push a patch to all
> have org-depend process the example you listed.

Cool ;-)

>> Second observation:
>>
>> However, what is bugging me even more is that even when "heading 1"
>> has a TODO keyword assigned, in *some* cases, the TRIGGER event does
>> not get executed when I do it in my large Org-mode file.
>>
>> Copying the corresponding headings (the one with the TRIGGER prop
>> and the headings containing the "target" IDs) from my real-world
>> Org-mode file to my small test.org from above and marking the
>> heading that contains the TRIGGER properties to DONE, it then works
>> as expected.
>>
>> I then took the simple example from above, added a TODO keyword to
>> the heading 1, copied it to my real-world Org-mode file and even
>> this did not work: Heading 2 and 3 don't get their "NEXT" keyword
>> assigned.
>>
>> So the behavior changes within different Org-file contexts.
>> Therefore, I do have an issue creating a minimal example to
>> demonstrate the bug.
>>
>> Can somebody give me an advice how to debug this behavior?
>
> My only thought is that there is an ID clashing with your sample I=
> D and it
> is being updated instead of the heading you are interested in.
> org-depend.el looks at the local file for an ID and then looks globally for
> an ID match.

Well, the IDs I was using were unique over all of my files.
Valid guess but not the case at my setup.

> You could instrument 'org-depend-trigger-todo' to trace the behaviour while
> processing the TODO update.

I found out that following part does not return any TRIGGER line:

: ;; OK, we just switched from a TODO state to a DONE state
: ;; Lets see if this entry has a TRIGGER property.
: ;; If yes, split it up on whitespace.
: (setq trigger (org-entry-get pos "TRIGGER")    ;; <-- no result here!
:   triggers (and trigger (split-string trigger)))

So the whole procedure of looking for the IDs to trigger is not
reached at all. 

After that I edebug-defun org-entry-get and the difference between
working and non-working TRIGGER (non-nil list of property values
versus nil) can be found at:

: (let* ((local (org--property-local-values property literal-nil))

... within org.el (git maint 946f76d70)

Another edebug-defun of org--property-local-values shows that this
code does not return the list of values in the big file where it is
working with my working minimal file:

: (let ((v (and (re-search-forward
:            (org-re-property property nil t) end t)
:           (match-string-no-properties 3))))

I could not jump into re-search-forward (built-in function) but I
found out something fishy: The value for "end" held a position which
is *not* the correct one.

Instead of holding the end position of the property drawer of the
heading I invoked org-todo on, it held the end of the properties
drawer of the next higher heading which has a progress indicator in
it.

Examples:

     * Top-Heading
     
     ** TODO Here I invoke org-todo to DONE
     :PROPERTIES:
     :TRIGGER: 2017-12-03-target(TODO)
     :END:
     
     ** This should be changed to TODO
     :PROPERTIES:
     :ID: 2017-12-03-target
     :END:

... this is working (i.e., "This should be changed to TODO" gets its
TODO keyword).

     * Top-Heading with process indicator [0/2]
     
     ** TODO Here I invoke org-todo to DONE
     :PROPERTIES:
     :TRIGGER: 2017-12-03-target(TODO)
     :END:
     
     ** This should be changed to TODO
     :PROPERTIES:
     :ID: 2017-12-03-target
     :END:

... this is *not* working.

However, anticipating the wrong position:

     * Top-Heading with process indicator [0/2]
     :PROPERTIES:
     :TRIGGER: 2017-12-03-target(TODO)
     :END:
     
     ** TODO Here I invoke org-todo to DONE
     :PROPERTIES:
     :TRIGGER: 2017-12-03-target(TODO)
     :END:
     
     ** This should be changed to TODO
     :PROPERTIES:
     :ID: 2017-12-03-target
     :END:

... is working. And according to this:

     * Top-Heading with process indicator [0/2]
     :PROPERTIES:
     :TRIGGER: 2017-12-03-target(TODO)
     :END:
     
     ** TODO Here I invoke org-todo to DONE
     
     ** This should be changed to TODO
     :PROPERTIES:
     :ID: 2017-12-03-target
     :END:

... this is also working. So the update of the process indicator
causes the wrong property drawer to be parsed for the relevant
TRIGGER property. 

Contrary to my previous assumption, this is issue is *not* related
to big and small files. This is purely related to the existing or
missing process indicator of the upper-level heading.

I guess I have found the origin of the bug.

Can you please do me the favor and fix it for me. I feel
extraordinary proud of me having dig into elisp (which I don't know
how to code mostly) and found the bug ;-)

However, I can't fix it on my own :-(

> I have read your post on org-depend and found it insightful so thanks for
> that. 

You're welcome.

> When I was looking around at patching some new functionality into
> org-depend I was put onto org-edna[1] which was built as an alternative to
> org-depend. The syntax isn't compatible with org-depend, but org-edna is
> more powerful than org-depend. I am now using org-edna and am quite happy
> with it.
>
> [1] https://elpa.gnu.org/packages/org-edna.html

WOT!

Why did nobody tell me about org-edna yet? ;-)

Some of my proposed org-depend-improvements[2] found its way to
org-edna, as it seems. However, I do have to invest quite some time
to migrate to org-edna. Meanwhile, I'd love to see org-depend
working (again).

Thanks for the pointer to org-edna - this is going to be great I
think.

[2] http://karl-voit.at/2016/12/18/org-depend/

-- 
get mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML into Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <
Personal Information Management > http://Karl-Voit.at/tags/pim/
Emacs-related > http://Karl-Voit.at/tags/emacs/

Re: org-depend: TRIGGER XYZ(KEYWORD) not working

[Adrian Bradd]

[-- Attachment #1: Type: text/plain, Size: 2950 bytes --]

Hi Karl,


> Examples:
>
>      * Top-Heading
>
>      ** TODO Here I invoke org-todo to DONE
>      :PROPERTIES:
>      :TRIGGER: 2017-12-03-target(TODO)
>      :END:
>
>      ** This should be changed to TODO
>      :PROPERTIES:
>      :ID: 2017-12-03-target
>      :END:
>
> ... this is working (i.e., "This should be changed to TODO" gets its
> TODO keyword).
>
>      * Top-Heading with process indicator [0/2]
>
>      ** TODO Here I invoke org-todo to DONE
>      :PROPERTIES:
>      :TRIGGER: 2017-12-03-target(TODO)
>      :END:
>
>      ** This should be changed to TODO
>      :PROPERTIES:
>      :ID: 2017-12-03-target
>      :END:
>
> ... this is *not* working.
>
> However, anticipating the wrong position:
>
>      * Top-Heading with process indicator [0/2]
>      :PROPERTIES:
>      :TRIGGER: 2017-12-03-target(TODO)
>      :END:
>
>      ** TODO Here I invoke org-todo to DONE
>      :PROPERTIES:
>      :TRIGGER: 2017-12-03-target(TODO)
>      :END:
>
>      ** This should be changed to TODO
>      :PROPERTIES:
>      :ID: 2017-12-03-target
>      :END:
>
> ... is working. And according to this:
>
>      * Top-Heading with process indicator [0/2]
>      :PROPERTIES:
>      :TRIGGER: 2017-12-03-target(TODO)
>      :END:
>
>      ** TODO Here I invoke org-todo to DONE
>
>      ** This should be changed to TODO
>      :PROPERTIES:
>      :ID: 2017-12-03-target
>      :END:
>
> ... this is also working. So the update of the process indicator
> causes the wrong property drawer to be parsed for the relevant
> TRIGGER property.
>
> Contrary to my previous assumption, this is issue is *not* related
> to big and small files. This is purely related to the existing or
> missing process indicator of the upper-level heading.
>
> I guess I have found the origin of the bug.


​I'm not able to reproduce what you are seeing above.​ Even instances with
a progress indicator correctly triggered if the TRIGGER property was set.
Your final example updates the "Here I invoke org-todo to DONE" entry to
DONE and doesn't touch any other heading for me.

I did see an issue with triggered headings not honouring note logging when
the todo was set to DONE, but I haven't looked any closer at it just yet.

What version of org-mode are you running? If you are using the git version
can you pull the latest on maint. I performed my testing on maint.


> Can you please do me the favor and fix it for me. I feel

extraordinary proud of me having dig into elisp (which I don't know
> how to code mostly) and found the bug ;-)
>
> However, I can't fix it on my own :-(
>

​Even just being able to dig through some code and debug can be a major
benefit. :-)


> Why did nobody tell me about org-edna yet? ;-)
>

​I found out on the mailing list as well. Doubt I would have even stumbled
upon it otherwise.
​
Cheers,

Adrian

[-- Attachment #2: Type: text/html, Size: 4615 bytes --]

Re: org-depend: TRIGGER XYZ(KEYWORD) not working

[Karl Voit]

Hi Adrian,

* Adrian Bradd <adrian.bradd@gmail.com> wrote:
>
>> Examples:
>>
>>      * Top-Heading
>>
>>      ** TODO Here I invoke org-todo to DONE
>>      :PROPERTIES:
>>      :TRIGGER: 2017-12-03-target(TODO)
>>      :END:
>>
>>      ** This should be changed to TODO
>>      :PROPERTIES:
>>      :ID: 2017-12-03-target
>>      :END:
>>
>> ... this is working (i.e., "This should be changed to TODO" gets its
>> TODO keyword).
>>
>>      * Top-Heading with process indicator [0/2]
>>
>>      ** TODO Here I invoke org-todo to DONE
>>      :PROPERTIES:
>>      :TRIGGER: 2017-12-03-target(TODO)
>>      :END:
>>
>>      ** This should be changed to TODO
>>      :PROPERTIES:
>>      :ID: 2017-12-03-target
>>      :END:
>>
>> ... this is *not* working.
>>
>> However, anticipating the wrong position:
>>
>>      * Top-Heading with process indicator [0/2]
>>      :PROPERTIES:
>>      :TRIGGER: 2017-12-03-target(TODO)
>>      :END:
>>
>>      ** TODO Here I invoke org-todo to DONE
>>      :PROPERTIES:
>>      :TRIGGER: 2017-12-03-target(TODO)
>>      :END:
>>
>>      ** This should be changed to TODO
>>      :PROPERTIES:
>>      :ID: 2017-12-03-target
>>      :END:
>>
>> ... is working. And according to this:
>>
>>      * Top-Heading with process indicator [0/2]
>>      :PROPERTIES:
>>      :TRIGGER: 2017-12-03-target(TODO)
>>      :END:
>>
>>      ** TODO Here I invoke org-todo to DONE
>>
>>      ** This should be changed to TODO
>>      :PROPERTIES:
>>      :ID: 2017-12-03-target
>>      :END:
>>
>> ... this is also working. So the update of the process indicator
>> causes the wrong property drawer to be parsed for the relevant
>> TRIGGER property.
>>
>> Contrary to my previous assumption, this is issue is *not* related
>> to big and small files. This is purely related to the existing or
>> missing process indicator of the upper-level heading.
>>
>> I guess I have found the origin of the bug.
>
>
> I'm not able to reproduce what you are seeing above.

Damnit.

This is because I removed the wrong progress indicator from my
examples before posting. You have to change all "[0/2]" with "[%]"
and then you see the issue I described. It seems to be the case that
[/] do work whereas [%] do cause the described malfunctional
behaviour. I was using both and did remove the wrong one.

> I did see an issue with triggered headings not honouring note logging when
> the todo was set to DONE, but I haven't looked any closer at it just yet.

Oh.

> What version of org-mode are you running? If you are using the git version
> can you pull the latest on maint. I performed my testing on maint.

I was using git maint from a couple of days ago and I just updated
the repository.

> =E2=80=8BEven just being able to dig through some code and debug can be a m=
> ajor
> benefit. :-)

And I got the feeling that I learned more about how the elisp code
is working. I should do more debugging even for working code ;-)

-- 
get mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML into Org-mode:
       > get Memacs from https://github.com/novoid/Memacs <
Personal Information Management > http://Karl-Voit.at/tags/pim/
Emacs-related > http://Karl-Voit.at/tags/emacs/

Re: org-depend: TRIGGER XYZ(KEYWORD) not working

[Adrian Bradd]

[-- Attachment #1: Type: text/plain, Size: 1001 bytes --]

​Hi Karl,​

This is because I removed the wrong progress indicator from my
> examples before posting. You have to change all "[0/2]" with "[%]"
> and then you see the issue I described. It seems to be the case that
> [/] do work whereas [%] do cause the described malfunctional
> behaviour. I was using both and did remove the wrong one.


​I have found the issue, but haven't had time to patch it. It actually does
happen with both [%] and [/].

org-todo is using a point to pass the beginning of the heading to
org-depend instead of a marker. As such, when characters are added to the
statistics in the parent heading the point shifts up to the line above the
heading and when (org-entry-get pos "TRIGGER") runs it searches the heading
above it, which is why the trigger works correctly if you populate the
parent heading property.

change-plist looks llike it is used in a few other functions as well. I'll
have to see if using a marker breaks them.

Cheers,

Adrian

[-- Attachment #2: Type: text/html, Size: 1846 bytes --]