Emacs bug 37890; killing capture buffer

Emacs bug 37890; killing capture buffer

[Michael Heerdegen]

Hi,

I want to speak about my Emacs bug report 37890 about org-capture.
Seems my main point:

| I want to capture an APPT with `org-capture'.  I the pop-up buffer to
| edit the item I move the date to the second line and add text after the
| date (personal preference).  That loses the final newline in
| CAPTURE-todo.org.  As a result, the headline of the item following the
| item to be inserted gets appended to the last line of the text:
|
| ** APPT Abc
|    <2019-10-23 Mi>
| text... ** APPT 8:30 Important Appointment
|
| breaking the whole item.  The user should somehow be prevented from that
| happening.

has been resolved with the latest merge into Emacs master - is that
correct?  Then I will close that report.

The report also included a feature request which I now want to tell
here: I often kill the capture buffer instead of hitting C-c C-k.  Just
by habit.  It's wrong and I now it but I guess it happens to others.

I guess the capture buffer is just a narrowed indirect buffer copy of
the buffer visiting the according org file.  Because of this, when you
kill the capture buffer, the original buffer reflects the partial and
uncomplete entry one tried to add.  This may damage your file.  But such
internals are not known to all, so it would be good if killing the
capture buffer, or even just closing the window, would warn the user and
offer to undo the partial changes.  Or (really better IMHO) consider a
different implementation where the original buffer is not modified until
the user explicitly confirms the stuff to capture with C-c C-c.

TIA,

Michael.

Re: Emacs bug 37890; killing capture buffer

[Adam Porter]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Or (really better IMHO) consider a different implementation where the
> original buffer is not modified until the user explicitly confirms the
> stuff to capture with C-c C-c.

That would be helpful in some ways, but harmful in others.  For example,
consider a capture that is started while in a meeting, on a phone call,
away from one's desk, etc., with some notes in it, clock start time,
etc.  (You can find examples of this workflow in, e.g. Bernt Hansen's
Org config.)  If Emacs were interrupted (crash, power failure, reboot,
etc), the un-finalized capture would still be present in the auto-save
file and could be recovered when restarting Emacs and finding the file
again.

But if the original buffer were not modified until the capture is
finalized, where would the unfinalized data be, and how would it be
recovered into the desired capture location?

The way Org uses indirect, narrowed buffers for capturing is an elegant
use of Emacs features that helps protect user data from accidental loss.

Re: Emacs bug 37890; killing capture buffer

[Michael Heerdegen]

Adam Porter <adam@alphapapa.net> writes:

> Michael Heerdegen <michael_heerdegen@web.de> writes:
>
> > Or (really better IMHO) consider a different implementation where the
> > original buffer is not modified until the user explicitly confirms the
> > stuff to capture with C-c C-c.
>
> That would be helpful in some ways, but harmful in others.  For example,
> consider a capture that is started while in a meeting, on a phone call,
> away from one's desk, etc., with some notes in it, clock start time,
> etc.  (You can find examples of this workflow in, e.g. Bernt Hansen's
> Org config.)  If Emacs were interrupted (crash, power failure, reboot,
> etc), the un-finalized capture would still be present in the auto-save
> file and could be recovered when restarting Emacs and finding the file
> again.

A legitimate objection.

> The way Org uses indirect, narrowed buffers for capturing is an
> elegant use of Emacs features that helps protect user data from
> accidental loss.

Let me rethink from the other side: how could the issue I described
(globbered org file) be prevented?  If you happen to kill the capture
buffer or forget about it, is there any indication left that there is a
problem?  A modified buffer visiting the org file is left, but as soon
as you successfully capture something else Org happily saves both edits
to the file.

So maybe we could prevent the user from doing something wrong?  Maybe
like this:

 - kill-buffer-hook in the capture buffer could be used to prevent the
 user from killing such a buffer by accident.  Or it could be made
 configurable what to do (e.g. undo the change with or without user
 prompting, ask for what to do, etc.)

 - kill-emacs-hook could be used to register a function that warns when
 any capture buffers are left when Emacs is to be killed.  That would
 make a difference if you have captured more stuff after having
 forgotten about a former capture buffer.  The user would be guided to
 finish what he would otherwise have forgotten.

That would improve security even further without getting in the way in
the normal workflow.


Michael.

Re: Emacs bug 37890; killing capture buffer

[Michael Heerdegen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

>  - kill-buffer-hook in the capture buffer could be used to prevent the
>  user from killing such a buffer by accident.  Or it could be made
>  configurable what to do (e.g. undo the change with or without user
>  prompting, ask for what to do, etc.)
>
>  - kill-emacs-hook could be used to register a function that warns when
>  any capture buffers are left when Emacs is to be killed.  That would
>  make a difference if you have captured more stuff after having
>  forgotten about a former capture buffer.  The user would be guided to
>  finish what he would otherwise have forgotten.

That doesn't seem to be hard to implement:

#+begin_src emacs-lisp
(add-hook
 'org-capture-mode-hook
 (defun my-org-capture-mode-hook-fun ()
   (add-hook 'kill-buffer-hook
             (defun my-org-capture-inhibit-accidental-kill ()
               (user-error "Please don't just kill org capture buffers"))
             nil 'local)
   (add-hook 'kill-emacs-hook
             (defun my-org-capture-barf-for-left-capture-buffers ()
               (dolist (b (buffer-list))
                 (when (with-current-buffer b
                         (bound-and-true-p org-capture-mode))
                   (user-error "Please care about org capture buffer %s"
                               (buffer-name b))))))))

(add-hook 'org-capture-prepare-finalize-hook
          (defun my-org-capture-prepare-finalize-hook-fun ()
            (remove-hook 'kill-buffer-hook
                         'my-org-capture-inhibit-accidental-kill
                         'local)))
#+end_src

Would you consider to do something like this by default?

BTW, what about my question whether my original bug report can be
closed?

TIA,

Michael.

Re: Emacs bug 37890; killing capture buffer

[Adam Porter]

Hi Michael,

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Would you consider to do something like this by default?

I guess you're asking me, since I'm the only other person in this
thread--but I'm not an Org maintainer, so my opinion isn't very
important.  IMO, the hooks are worth considering, however they should be
done very, very carefully, because bad things can happen when functions
called in kill hooks don't work as expected (e.g. they can make it very
difficult to kill a buffer or the Emacs process, especially for an
average user).

> BTW, what about my question whether my original bug report can be
> closed?

I haven't seen your bug report.  Was there discussion about it
previously?

Adam

Re: Emacs bug 37890; killing capture buffer

[Michael Heerdegen]

Adam Porter <adam@alphapapa.net> writes:

> I guess you're asking me, since I'm the only other person in this
> thread--but I'm not an Org maintainer, so my opinion isn't very
> important.  IMO, the hooks are worth considering, however they should be
> done very, very carefully, because bad things can happen when functions
> called in kill hooks don't work as expected (e.g. they can make it very
> difficult to kill a buffer or the Emacs process, especially for an
> average user).

Yes, all true.  I didn't even check if the code I posted breaks any org
commands that kill the buffer.  So far it's only meant for demonstration
(and my personal usage).

Will my request be noticed here, or do you think I should report it to
some other place?

> > BTW, what about my question whether my original bug report can be
> > closed?
>
> I haven't seen your bug report.  Was there discussion about it
> previously?

No, no discussion at all.  As I said, it is Emacs bug #37890, this was
my issue:

| I want to capture an APPT with `org-capture'.  I the pop-up buffer to
| edit the item I move the date to the second line and add text after the
| date (personal preference).  That loses the final newline in
| CAPTURE-todo.org.  As a result, the headline of the item following the
| item to be inserted gets appended to the last line of the text:
|
| ** APPT Abc
|    <2019-10-23 Mi>
| text... ** APPT 8:30 Important Appointment
|
| breaking the whole item.  The user should somehow be prevented from that
| happening.

It seems it has been resolved, just without noticing my bug report.  If
you can confirm that the cited issue has been cared about, I'll close it
as done.

BTW, what is the canonical place to report org-mode bugs?  Emacs bug
reports are not (takes a long time until someone even notices) -- I
thought this list would be good...or is there a better place?   @Adam
it's ok if you answer, though I'm a bit disappointed that no one else
seems to care so far...


Thanks,

Michael.

Re: Emacs bug 37890; killing capture buffer

[Adam Porter]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Adam Porter <adam@alphapapa.net> writes:
>
>> I guess you're asking me, since I'm the only other person in this
>> thread--but I'm not an Org maintainer, so my opinion isn't very
>> important.  IMO, the hooks are worth considering, however they should
>> be done very, very carefully, because bad things can happen when
>> functions called in kill hooks don't work as expected (e.g. they can
>> make it very difficult to kill a buffer or the Emacs process,
>> especially for an average user).
>
> Yes, all true.  I didn't even check if the code I posted breaks any
> org commands that kill the buffer.  So far it's only meant for
> demonstration (and my personal usage).
>
> Will my request be noticed here, or do you think I should report it to
> some other place?

I can't speak for what the maintainers are reading.  They are active on
this list, as you can see, but I doubt they have time to read every
message.  If you don't get the response you're looking for, you might
post the proposal in a separate thread, perhaps prefixed with [RFC] or
[PATCH] as appropriate.  They're definitely more likely to notice and
respond to actual patches, but that's not a requirement; if you ask for
specific feedback about a specific idea or code, that usually gets
noticed.

>> > BTW, what about my question whether my original bug report can be
>> > closed?
>>
>> I haven't seen your bug report.  Was there discussion about it
>> previously?
>
> No, no discussion at all.  As I said, it is Emacs bug #37890, this was
> my issue:
>
> | I want to capture an APPT with `org-capture'.  I the pop-up buffer to
> | edit the item I move the date to the second line and add text after the
> | date (personal preference).  That loses the final newline in
> | CAPTURE-todo.org.  As a result, the headline of the item following the
> | item to be inserted gets appended to the last line of the text:
> |
> | ** APPT Abc
> |    <2019-10-23 Mi>
> | text... ** APPT 8:30 Important Appointment
> |
> | breaking the whole item.  The user should somehow be prevented from that
> | happening.
>
> It seems it has been resolved, just without noticing my bug report.
> If you can confirm that the cited issue has been cared about, I'll
> close it as done.

I can neither confirm nor deny.  ;)  I do seem to recall discussion of
that issue here recently (the problem, not necessarily your official bug
report of it), and maybe seeing that a fix was made.  If you search the
list archive for this month or November, you should find something--I
think that's the right timeframe.  You might also search the git log for
capture-related commits.  Shouldn't be hard to find if there was
something.

If it helps, none of my capture templates seem to end with newlines, and
I don't see anything in the docstring for org-capture-templates that
suggests one is required, so I don't think one is.  If you're still
having the problem, I'd suggest trying to reproduce it in a clean Emacs
configuration with the latest version of Org.  You may find this script
helpful for that:

https://github.com/alphapapa/alpha-org/blob/master/emacs-sandbox.sh

> BTW, what is the canonical place to report org-mode bugs?  Emacs bug
> reports are not (takes a long time until someone even notices) -- I
> thought this list would be good...or is there a better place?

I think this list is a canonical place.  Some people do report Org bugs
as official bug reports on the Emacs bug tracker.  I don't know whether
the Org maintainers read them all.  Since Org is officially part of
Emacs, sometimes the Emacs maintainers Cc this list about such reports.
Anyway, here is a fine place.

If you're not sure that something is a bug, you might consider
mentioning it on https://old.reddit.com/r/orgmode before posting it
here, since /r/orgmode has a wider audience.

> @Adam it's ok if you answer, though I'm a bit disappointed that no one
> else seems to care so far...

People do care!  But everyone here works on Org in their spare time, and
Org is a big project, and things slip through the cracks.

Re: Emacs bug 37890; killing capture buffer

[Michael Heerdegen]

Adam Porter <adam@alphapapa.net> writes:

> People do care!  But everyone here works on Org in their spare time,
> and Org is a big project, and things slip through the cracks.

I didn't meant to criticize anyone (I have the same problem with my own
stuff).  Anyway, we got some attention now, thanks for your help!


Regards,

Michael.

Re: Emacs bug 37890; killing capture buffer

[Kyle Meyer]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Adam Porter <adam@alphapapa.net> writes:
[...]
>> I haven't seen your bug report.  Was there discussion about it
>> previously?
>
> No, no discussion at all.  As I said, it is Emacs bug #37890, this was
> my issue:
>
> | I want to capture an APPT with `org-capture'.  I the pop-up buffer to
> | edit the item I move the date to the second line and add text after the
> | date (personal preference).  That loses the final newline in
> | CAPTURE-todo.org.  As a result, the headline of the item following the
> | item to be inserted gets appended to the last line of the text:
> |
> | ** APPT Abc
> |    <2019-10-23 Mi>
> | text... ** APPT 8:30 Important Appointment
> |
> | breaking the whole item.  The user should somehow be prevented from that
> | happening.
>
> It seems it has been resolved, just without noticing my bug report.  If
> you can confirm that the cited issue has been cared about, I'll close it
> as done.

This issue was resolved by 0fa8bb4c5 (org-capture: Inserting an entry
doesn't break structure, 2018-10-22), which part of the 9.2 release and
prompted by <http://lists.gnu.org/r/emacs-orgmode/2018-10/msg00256.html>.

Re: Emacs bug 37890; killing capture buffer

[Ihor Radchenko]

Dear Michael,

> BTW, what is the canonical place to report org-mode bugs?  Emacs bug
> reports are not (takes a long time until someone even notices) -- I
> thought this list would be good...or is there a better place?   @Adam
> it's ok if you answer, though I'm a bit disappointed that no one else
> seems to care so far...

You can try M-x org-submit-bug-report ;)
Then it becomes clear that you are in the right place already. 

Best,
Ihor


Michael Heerdegen <michael_heerdegen@web.de> writes:

> Adam Porter <adam@alphapapa.net> writes:
>
>> I guess you're asking me, since I'm the only other person in this
>> thread--but I'm not an Org maintainer, so my opinion isn't very
>> important.  IMO, the hooks are worth considering, however they should be
>> done very, very carefully, because bad things can happen when functions
>> called in kill hooks don't work as expected (e.g. they can make it very
>> difficult to kill a buffer or the Emacs process, especially for an
>> average user).
>
> Yes, all true.  I didn't even check if the code I posted breaks any org
> commands that kill the buffer.  So far it's only meant for demonstration
> (and my personal usage).
>
> Will my request be noticed here, or do you think I should report it to
> some other place?
>
>> > BTW, what about my question whether my original bug report can be
>> > closed?
>>
>> I haven't seen your bug report.  Was there discussion about it
>> previously?
>
> No, no discussion at all.  As I said, it is Emacs bug #37890, this was
> my issue:
>
> | I want to capture an APPT with `org-capture'.  I the pop-up buffer to
> | edit the item I move the date to the second line and add text after the
> | date (personal preference).  That loses the final newline in
> | CAPTURE-todo.org.  As a result, the headline of the item following the
> | item to be inserted gets appended to the last line of the text:
> |
> | ** APPT Abc
> |    <2019-10-23 Mi>
> | text... ** APPT 8:30 Important Appointment
> |
> | breaking the whole item.  The user should somehow be prevented from that
> | happening.
>
> It seems it has been resolved, just without noticing my bug report.  If
> you can confirm that the cited issue has been cared about, I'll close it
> as done.
>
> BTW, what is the canonical place to report org-mode bugs?  Emacs bug
> reports are not (takes a long time until someone even notices) -- I
> thought this list would be good...or is there a better place?   @Adam
> it's ok if you answer, though I'm a bit disappointed that no one else
> seems to care so far...
>
>
> Thanks,
>
> Michael.
>

Re: Emacs bug 37890; killing capture buffer

[Michael Heerdegen]

Ihor Radchenko <yantar92@gmail.com> writes:

> You can try M-x org-submit-bug-report ;)
> Then it becomes clear that you are in the right place already.

Thanks!  I didn't know about this command - actually I had searched for
it but didn't find it because I had searched for something named
"report-org-bug" (analogue to `report-emacs-bug').  I think this may
happen to other org users not following org-mode development as well -
so I just used the command to suggest this name as alias.

Regards,

Michael.

Re: Emacs bug 37890; killing capture buffer

[Samuel Wales]

i might be completely off on this, but it seems the problem is that
there is a corrupted buffer.

in particular, there is a missing newline at the end of the narrowed
region in the capture buffer.  this causes the next header to join the
last line in the captured buffer.

i encountered this problem today.  i added a task and duplicated it.
this caused the corruption.  it also screwed up the stars level, but
never mind that.

so if i'm not completely off, which i might be, it's not so much
killing the buffer as the fact that the state is corrupted in the
first place.

if the state is corrupted by the lack of a newline at the end of the
capture buffer, that can be fixed by adding one there.  but the user
can screw that up by deleting it.

if the state is corrupted by the lack of a newline just after the
capture buffer, then that can be fixed -- maybe partly -- by adding a
newline there.

this seems to me to be the right thing to do.

i don't think this will create any blank lines.

but *if* it does, it is much better to have the computer crash with
the only corruption being the addition of a blank line, than to have a
header corrupted.

i am the designer of the indirect buffer capture idea.  i did not
implement it, however.

the indirect buffer capture mechanism was to be an improvement on
remember.el, and replaced it.  you might still be able to find
remember.el if you prefer the separate file idea.

not sure if my brain is working properly right now, but that's the basic idea.


<michael_heerdegen@web.de> wrote:
> Hi,
>
> I want to speak about my Emacs bug report 37890 about org-capture.
> Seems my main point:
>
> | I want to capture an APPT with `org-capture'.  I the pop-up buffer to
> | edit the item I move the date to the second line and add text after the
> | date (personal preference).  That loses the final newline in
> | CAPTURE-todo.org.  As a result, the headline of the item following the
> | item to be inserted gets appended to the last line of the text:
> |
> | ** APPT Abc
> |    <2019-10-23 Mi>
> | text... ** APPT 8:30 Important Appointment
> |
> | breaking the whole item.  The user should somehow be prevented from that
> | happening.
>
> has been resolved with the latest merge into Emacs master - is that
> correct?  Then I will close that report.
>
> The report also included a feature request which I now want to tell
> here: I often kill the capture buffer instead of hitting C-c C-k.  Just
> by habit.  It's wrong and I now it but I guess it happens to others.
>
> I guess the capture buffer is just a narrowed indirect buffer copy of
> the buffer visiting the according org file.  Because of this, when you
> kill the capture buffer, the original buffer reflects the partial and
> uncomplete entry one tried to add.  This may damage your file.  But such
> internals are not known to all, so it would be good if killing the
> capture buffer, or even just closing the window, would warn the user and
> offer to undo the partial changes.  Or (really better IMHO) consider a
> different implementation where the original buffer is not modified until
> the user explicitly confirms the stuff to capture with C-c C-c.
>
> TIA,
>
> Michael.
>
>


-- 
The Kafka Pandemic

What is misopathy?
https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html

The disease DOES progress. MANY people have died from it. And ANYBODY
can get it at any time.

Re: Emacs bug 37890; killing capture buffer

[Fraga, Eric]

On Tuesday, 17 Dec 2019 at 17:07, Samuel Wales wrote:
> i encountered this problem today.  i added a task and duplicated it.
> this caused the corruption.  it also screwed up the stars level, but
> never mind that.

I've encountered the problem in the past week or two when I was tweaking
one of my capture templates.  I had to add a \n to the end of the
template string to avoid the problem but didn't explore more than that.

-- 
Eric S Fraga via Emacs 27.0.50, Org release_9.3-34-g2eee3c

Re: Emacs bug 37890; killing capture buffer

[Michael Heerdegen]

Samuel Wales <samologist@gmail.com> writes:

> the indirect buffer capture mechanism was to be an improvement on
> remember.el, and replaced it.  you might still be able to find
> remember.el if you prefer the separate file idea.

I find it acceptable once it's been made more secure (e.g. by using
hooks like I mentioned).

But I wonder now why you said there are still problems with final
newlines (and you already got a confirmation), while I and other
developers say it has been fixed.  Do you use the org mode version built
in Emacs, and you don't use Emacs master?

Thanks,

Michael.