Bug: Bug: org-clock commands spawn drawer outside of narrowing [9.2.1 (9.2.1-2-gc6d37c-elpaplus @ /home/zaeph/.emacs.d/elpa/org-plus-contrib-20190204/)]

Bug: Bug: org-clock commands spawn drawer outside of narrowing [9.2.1 (9.2.1-2-gc6d37c-elpaplus @ /home/zaeph/.emacs.d/elpa/org-plus-contrib-20190204/)]

[Leo Vivier]

Hello,

Version info:
- Emacs : GNU Emacs 26.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version
  3.22.30), of 2018-07-05
- Package: Org mode version 9.2.1 (9.2.1-2-gc6d37c-elpaplus)

The bug only happens in narrowed org-mode buffers when the tree at
point (or targeted by the resolving) is a single line not followed by a
blank line.

I was able to replicate the problem with `emacs -q`.

MWE:
--------------------------------[START]--------------------------------
* Tree 1
* Tree 2
---------------------------------[END]---------------------------------

- Narrow to ‘Tree 1’l
- Clock in.

Observations:
- No clock drawer visible in the narrowed buffer.
- Feedback in the minibuffer that the clock was started.
- Widening the buffer confirms the presence of the buffer where it
  should be.

Whilst the observations would lead one to think that everything ‘Just
Works™’, it causes a slew of problems.  Two examples:
- After clocking in, adding a new heading ‘Subtree’ bellow ‘Tree 1’
  would make the drawer belong to ‘Subtree’ instead of ‘Tree 1’
- `org-clock-out-when-done` isn’t respected since the drawer is not
  visible

It seems to be part of a larger set of bugs related to single-line
trees, such as the one I’d reported before and which was addressed in
503ede74bc0a1db59fd2fb7bac0bf1ba7352d15b.

I tried to fix it on my own by tracking down the problem with edebug,
and that led me to the `save-restriction` used in `org-clock-in`,
`org-clock-out`, and `org-clock-resolve-clock`.  Those calls to
`save-restriction` are sometimes embedded into macros, such as
`org-with-wide-buffer` or `org-with-point-at`.

I initially thought that replacing those calls in favour of a `widen`
followed by a `org-narrow-to-subtree` would refresh the bounds of the
narrowing.  This proved to be a lot more finicky than I anticipated, and
I’d hate to break anything.

Would you be able to look into it?

Thank you for your time,
-- 
Leo Vivier
English Studies & General Linguistics
Master Student, English Department
Université Rennes 2

Re: Bug: Bug: org-clock commands spawn drawer outside of narrowing [9.2.1 (9.2.1-2-gc6d37c-elpaplus @ /home/zaeph/.emacs.d/elpa/org-plus-contrib-20190204/)]

[Nicolas Goaziou]

Hello,

Leo Vivier <leo.vivier@gmail.com> writes:

> The bug only happens in narrowed org-mode buffers when the tree at
> point (or targeted by the resolving) is a single line not followed by a
> blank line.

[...]

> MWE:
>
> --------------------------------[START]--------------------------------
> * Tree 1
> * Tree 2
> ---------------------------------[END]---------------------------------
>
> - Narrow to ‘Tree 1’l
> - Clock in.
>
> Observations:
> - No clock drawer visible in the narrowed buffer.
> - Feedback in the minibuffer that the clock was started.
> - Widening the buffer confirms the presence of the buffer where it
>   should be.

This looks correct, indeed.

> Whilst the observations would lead one to think that everything ‘Just
> Works™’, it causes a slew of problems.  Two examples:
> - After clocking in, adding a new heading ‘Subtree’ bellow ‘Tree 1’
>   would make the drawer belong to ‘Subtree’ instead of ‘Tree 1’

This is to be expected. The same would happen in a widened buffer, if
you insert a new headline (M-RET) at the end of the one being clocked.

The difference here is that you cannot see the running clock, but this
is to be expected when you narrow the document to a single line which is
not meaningful syntactically. Try narrowing the buffer to a single
character: all weird things may happen.

I suggest to narrow only to meaningful parts of a document, e.g.,
a paragraph, a subtree…

> - `org-clock-out-when-done` isn’t respected since the drawer is not
>   visible

This is a bug. I fixed it.

Regards,

-- 
Nicolas Goaziou

Re: Bug: Bug: org-clock commands spawn drawer outside of narrowing [9.2.1 (9.2.1-2-gc6d37c-elpaplus @ /home/zaeph/.emacs.d/elpa/org-plus-contrib-20190204/)]

[Leo Vivier]

Hello,

I’d forgotten to CC the list in my previous message, so I’ve included
all the context from our two previous emails.

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Leo Vivier <leo.vivier@gmail.com> writes:
>
>> You’re right, that’s the behaviour we would expect from M-RET.  However,
>> with C-RET, the new heading should respect the content of the tree at
>> point.
>
> I disagree. C-RET is expected to move point, so it should obviously
> operate on the accessible part of the buffer. 
>
> The only other option, AFAICT, would be to automatically widen the
> buffer, which would be most surprising.
>
>> Even though this is an expected behaviour with narrowed buffers, maybe
>> we could find a way to work around this limitation.  The reason I’m
>> suggesting this is because I often find myself dealing with 1-line tree
>> which are meaningful parts of my documents.
>
> Then you ought to include the final newline character in the narrowed
> part of the buffer. It mitigate some issues you are encountering.

Including the final newline mitigates most of the problems I’m
encountering, you’re right, but I have two issues with this solution:
- ‘org-narrow-to-subtree’ does not do that by default (although it’d
  probably be easy to patch in a conditional behaviour).
- The included newline isn’t protected, meaning that the user can delete
  it without warning.

MWE:
--------------------------------[START]--------------------------------
* Inbox
** Captured task
* Tree
---------------------------------[END]---------------------------------

- Narrow to ‘* Captured task^J’ (i.e. including the new-line at eol).
  This is the state we’d have with a patched org-narrow-to-subtree.
- ‘(end-of-buffer)’
- ‘(delete-char 1)’
- ‘(widen)’

Result:
--------------------------------[START]--------------------------------
* Inbox
** Captured task*Tree
---------------------------------[END]---------------------------------

Since the line is visible in the buffer, it follows that the user is
able to delete it, but I wonder if that’s something s/he’d ever want to
do.  This goes back to the tentative solution I’ve mentioned in my
previous email.

>> I’m aware that this problem only affects people who do not have
>> empty-lines between their trees.  However, 90% of the time, those 1-line
>> trees are the result of simple org-capture templates on which no work
>> has been done.  When the time comes, I access them from the agenda with
>> ‘org-agenda-tree-to-indirect-buffer’.  I have no way to know that the
>> buffer is narrowed char-wise rather than line-wise.  So, when
>> clocking-in on that tree, it doesn’t feel right that the clock-table
>> should be spawned outside of the view-port.
>>
>
> I'm not sure about "this problem" you're talking about. You are
> encountering different "problems". Some are certainly genuine bugs, but
> not all of them, per above. In any case, please report them precisely so
> we can see if there is something to fix, piece-wise.

I’ll make sure to specify those behaviours, then.

>>  Since the problem only happens with 1-line trees, a tentative solution
>>  could be the following:
>>  - When evaluating ‘org-narrow-to-subtree’ or
>>    ‘org-agenda-tree-to-indirect-buffer’
>>    1. Check whether the tree being considered is a 1-line tree.
>>    2. If t: Add a newline at the end of the heading.
>>       (This bypasses the narrowing limitation.)
>>    3. Store the result of the check in a local variable.
>>  - When evaluating ‘widen’ inside commands like ‘org-capture-finalize’ 
>>    1. Check whether the local variable mentioned above is set and true.
>>    2. If t: Remove newline at the end of the narrowed buffer if it still
>>       exists.
>>
>>  If this solution seems sound, I could work on it and submit a patch.
>>
>>>> - `org-clock-out-when-done` isn’t respected since the drawer is not
>>>>   visible
>>>
>>> This is a bug. I fixed it.
>>
>>Thank you for the fix.

Thanks for your help.

Best,
-- 
Leo Vivier
English Studies & General Linguistics
Master Student, English Department
Université Rennes 2