Refiling list items

Refiling list items

[Jeff Horn]

Should org-refile be able to refile list items? I suspect it would be
non-trivial to add this functionality if it isn't already there, but I
feel a bit overwhelmed keeping notes as headlines, particularly
because I like to leave soft wrapping off and have a hard wrap at 80
columns.

,----[ Sample Outline ]
| * Headline 1
| - List level 1
|   - List level 2
| * Headline 2
| Some text.
`----

If I attempt to refile the item "- List level 1" to the second headline, I get

,----[ Mangled Outline ]
| * Headline 2
| Some text.
| ** Headline 1
| - List level 1
|   - List level 2
`----

when the following is expected

,----[ Expected Outline ]
| * Headline 1
| * Headline 2
| - List level 1
|  - List level 2
|  Some text.
`----

Where the list items appear before or after "Some text." depending on
refile variables.

-- 
Jeffrey Horn
http://www.failuretorefrain.com/jeff/

Re: Refiling list items

[Jeff Horn]

Would someone throw me a bone? I couldn't find anything on gmane, but
I my gmane-fu isn't the strongest. :D

On Sat, Aug 6, 2011 at 14:54, Jeff Horn <jrhorn424@gmail.com> wrote:
> Should org-refile be able to refile list items? I suspect it would be
> non-trivial to add this functionality if it isn't already there, but I
> feel a bit overwhelmed keeping notes as headlines, particularly
> because I like to leave soft wrapping off and have a hard wrap at 80
> columns.
>
> ,----[ Sample Outline ]
> | * Headline 1
> | - List level 1
> |   - List level 2
> | * Headline 2
> | Some text.
> `----
>
> If I attempt to refile the item "- List level 1" to the second headline, I get
>
> ,----[ Mangled Outline ]
> | * Headline 2
> | Some text.
> | ** Headline 1
> | - List level 1
> |   - List level 2
> `----
>
> when the following is expected
>
> ,----[ Expected Outline ]
> | * Headline 1
> | * Headline 2
> | - List level 1
> |  - List level 2
> |  Some text.
> `----
>
> Where the list items appear before or after "Some text." depending on
> refile variables.
>
> --
> Jeffrey Horn
> http://www.failuretorefrain.com/jeff/
>



-- 
Jeffrey Horn
http://www.failuretorefrain.com/jeff/

Re: Refiling list items

[Nicolas Goaziou]

Hello,

Jeff Horn <jrhorn424@gmail.com> writes:

> Should org-refile be able to refile list items?

Extending `org-refile', or creating an equivalent function for list
items would be overkill, in my opinion.

Just kill the item, repair the list, move point to an appropriate place,
and paste the item there.

Regards,

-- 
Nicolas Goaziou

Re: Refiling list items

[Florian Beck]

Jeff Horn <jrhorn424@gmail.com> writes:

> Would someone throw me a bone? I couldn't find anything on gmane, but
> I my gmane-fu isn't the strongest. :D
>
> On Sat, Aug 6, 2011 at 14:54, Jeff Horn <jrhorn424@gmail.com> wrote:
>> Should org-refile be able to refile list items? I suspect it would be
>> non-trivial to add this functionality if it isn't already there, but I
>> feel a bit overwhelmed keeping notes as headlines, particularly
>> because I like to leave soft wrapping off and have a hard wrap at 80
>> columns.

Here is a hack I use. It doesn't do any bookkeeping, doesn't check for
errors, (temporarily) moves the point during refiling and you might want
to set `org-refile-targets' to your liking. On the other hand, it might
just do the trick:

(defun org-copy-item (&optional kill)
  "Copy item at point to another location.
With prefix argument, move the item."
  (interactive "P")
  (org-get-item kill)
  (let ((org-refile-targets
	 '((org-default-notes-file :maxlevel . 4))))
    (save-window-excursion
      (org-refile t)
      (outline-next-visible-heading 1)
      (skip-chars-backward " \t\n")
      (insert "\n")
      (yank))))

(defun org-get-item (&optional kill)
  "Copy the item at point to the kill ring.
Optionally, kill it."
  (save-excursion
    (let ((beg (org-in-item-p)))
      (org-end-of-item)
      (funcall (if kill
		   'kill-region
		 'copy-region-as-kill)
	       beg (point)))))


-- 
Florian Beck

Re: Refiling list items

[Nicolas Goaziou]

Hello,

Florian Beck <abstraktion@t-online.de> writes:

> (defun org-get-item (&optional kill)
>   "Copy the item at point to the kill ring.
> Optionally, kill it."
>   (save-excursion
>     (let ((beg (org-in-item-p)))
>       (org-end-of-item)
>       (funcall (if kill
> 		   'kill-region
> 		 'copy-region-as-kill)
> 	       beg (point)))))

The latest Org introduced the function `org-list-send-item', which may
be useful here. I'd also suggest to repair list and, eventually, update
check-boxes when killing it.

Regards,

-- 
Nicolas Goaziou

Re: Refiling list items

[Jeff Horn]

Thanks to Florian for sharing code. Nicolas, I still think this is a
bug. I don't doubt extending org-refile would be messy. It's above my
head, and I realize I'm asking a lot from anyone that would tackle
this for me, but I still think its a bug. Your workaround sounds
exactly like what org-refile is designed to avoid. Quoting the manual:

,----[ http://orgmode.org/manual/Refiling-notes.html#Refiling-notes ]
| When reviewing the captured data, you may want to refile some of the
| entries into a different list, for example into a project. Cutting,
| finding the right location, and then pasting the note is
| cumbersome. To simplify this process, you can use the following
| special command:
`----

Of course, I can keep working for now, and I'll try Florian's code.
Maybe it might be adaptable to a robust function called
'org-refile-list-item'? Thanks for listening!

Jeff

On Mon, Aug 8, 2011 at 05:48, Nicolas Goaziou <n.goaziou@gmail.com> wrote:
> Hello,
>
> Jeff Horn <jrhorn424@gmail.com> writes:
>
>> Should org-refile be able to refile list items?
>
> Extending `org-refile', or creating an equivalent function for list
> items would be overkill, in my opinion.
>
> Just kill the item, repair the list, move point to an appropriate place,
> and paste the item there.
>
> Regards,
>
> --
> Nicolas Goaziou
>



-- 
Jeffrey Horn
http://www.failuretorefrain.com/jeff/

Re: Refiling list items

[Nicolas Goaziou]

Jeff Horn <jrhorn424@gmail.com> writes:

> Thanks to Florian for sharing code. Nicolas, I still think this is a
> bug. I don't doubt extending org-refile would be messy. It's above my
> head, and I realize I'm asking a lot from anyone that would tackle
> this for me, but I still think its a bug.

Where is the bug? As far as I can see, you're expecting a function
designed for headlines only to operate on list items. Am I missing the
point?

> Your workaround sounds exactly like what org-refile is designed to
> avoid. Quoting the manual:

Except that I was suggesting to turn these instructions into a function,
not doing them manually.

> Of course, I can keep working for now, and I'll try Florian's code.
> Maybe it might be adaptable to a robust function called
> 'org-refile-list-item'? Thanks for listening!

What would be the specifications of that function? Would it only send
the item at point to the end of the headline specified through the
refile interface?

Regards,

-- 
Nicolas Goaziou

Re: Refiling list items

[Jeff Horn]

On Mon, Aug 8, 2011 at 19:03, Nicolas Goaziou <n.goaziou@gmail.com> wrote:
> Jeff Horn <jrhorn424@gmail.com> writes:
>
> Where is the bug? As far as I can see, you're expecting a function
> designed for headlines only to operate on list items. Am I missing the
> point?

I'm approaching this as a user. I've made one contribution to this
project, and that was 3 or 4 lines of documentation. I don't know the
code base, I don't know the design principles, and I don't have the
time to teach it to myself.

So, as a user, I was expecting something to happen that didn't. That's
a bug. That may be a misuse of the term, and I apologize for using it
loosely.

Now perhaps I should have restated it as a "feature request" in the
follow up e-mail. That's probably more accurate, and that's what I'll
do now. The function isn't designed to work the way I thought it
would, so it would be nice if it worked that way. Really, list items
seem (to me) to be a perfectly natural extension. Maybe I'm just Doing
It Wrong (tm). :D

> Except that I was suggesting to turn these instructions into a function,
> not doing them manually.

A technique of which I'm woefully ignorant. I've made half-hearted
attempts once or twice in the past year to write my own functions, but
it isn't the best use of my time to learn elisp at the moment. A long
term goal, but nothing I can fix overnight. That's why Florian's code
was so welcome!

>> Of course, I can keep working for now, and I'll try Florian's code.
>> Maybe it might be adaptable to a robust function called
>> 'org-refile-list-item'? Thanks for listening!
>
> What would be the specifications of that function? Would it only send
> the item at point to the end of the headline specified through the
> refile interface?

I hope its clear that this is all above my head. I know enough to make
suggestions, but not contribute to implementing them. That makes me a
free-rider, but a free-rider that recognizes he's at the mercy of
others' talents.

Thanks for your patience, Nicolas.

-- 
Jeffrey Horn
http://www.failuretorefrain.com/jeff/

Re: Refiling list items

[Nick Dokos]

Jeff Horn <jrhorn424@gmail.com> wrote:

> So, as a user, I was expecting something to happen that didn't. That's
> a bug. That may be a misuse of the term, and I apologize for using it
> loosely.
> 

It's a bug all right: the question is whether the bug is in the code,
in the docs or in the user's head ;-) 99.9% of the bugs I find are
in my head (most recently the find-file wildcard thing...)

Nick

Re: Refiling list items

[Nick Dokos]

Jeff Horn <jrhorn424@gmail.com> wrote:

> > What would be the specifications of that function? Would it only send
> > the item at point to the end of the headline specified through the
> > refile interface?
> 
> I hope its clear that this is all above my head. I know enough to make
> suggestions, but not contribute to implementing them. That makes me a
> free-rider, but a free-rider that recognizes he's at the mercy of
> others' talents.
> 

Ah, no - you don't get off that easily! This is not "implementing"
anything.  He is asking about your expectations:

"If I have a list item here and I do an org-refile with such and such
arguments in a file that looks like so and so, I expect it to do such
and such. Instead it did this and that, which was rather surprising.
OTOH, if the list item is *there*..." etc. etc.

I haven't read the thread (apologies) but ISTR you provided such a
description to begin with. What Nicolas is asking is: what should happen
in other cases of interest? You may want to cover just that one special
case, but an implementation has to worry about *all* cases[fn:1]:
otherwise, there *will* be bug reports in the very near future and guess
who their target will be (hint: it won't be you :-) )

Hope-your-sense-of-humor-is-working-today-ly yours,
Nick

Footnotes:

[fn:1] It might punt of course: cover the special case only and raise an
error in all other cases e.g., but that's not particularly appealing.

Re: Refiling list items

[Jeff Horn]

On Mon, Aug 8, 2011 at 20:20, Nick Dokos <nicholas.dokos@hp.com> wrote:
> Jeff Horn <jrhorn424@gmail.com> wrote:
>
>> > What would be the specifications of that function? Would it only send
>> > the item at point to the end of the headline specified through the
>> > refile interface?
>>
>> I hope its clear that this is all above my head. I know enough to make
>> suggestions, but not contribute to implementing them. That makes me a
>> free-rider, but a free-rider that recognizes he's at the mercy of
>> others' talents.
>>
>
> Ah, no - you don't get off that easily! This is not "implementing"
> anything.  He is asking about your expectations:
>
> I haven't read the thread (apologies) but ISTR you provided such a
> description to begin with. What Nicolas is asking is: what should happen
> in other cases of interest? You may want to cover just that one special
> case, but an implementation has to worry about *all* cases[fn:1]:
> otherwise, there *will* be bug reports in the very near future and guess
> who their target will be (hint: it won't be you :-) )

I see, thanks Nicholas. As a start, in a subjectively ideal world,
org-refile-list-item would work on list items:

1) and their children to arbitrary depth
2) in the current buffer, or any agenda file
3) using either path-like headline specification or IDO completion

But I'd settle for an in-buffer restriction like Florian used. I just
noticed org-goto makes the manual workaround a bit easier.

I guess this really amounts to the first time I've ever been convinced
that "treating list items like headlines" would be useful to me. At
least, it feels like list items might benefit from being a proper
subset of headlines. I don't remember what the disadvantages of such a
move would be, but I suspect there's a lot of tacit knowledge in the
codebase already.

> Hope-your-sense-of-humor-is-working-today-ly yours,

It is. :)

-- 
Jeffrey Horn
http://www.failuretorefrain.com/jeff/

Re: Refiling list items

[Jeff Horn]

On Mon, Aug 8, 2011 at 20:40, Jeff Horn <jrhorn424@gmail.com> wrote:
> I see, thanks Nicholas. As a start, in a subjectively ideal world,
> org-refile-list-item would work on list items:
>
> 1) and their children to arbitrary depth
> 2) in the current buffer, or any agenda file
> 3) using either path-like headline specification or IDO completion
>
> But I'd settle for an in-buffer restriction like Florian used. I just
> noticed org-goto makes the manual workaround a bit easier.

I should also note that having the only valid refile *target* as a
headline would be OK. More specifically, a list item refiled to a
target headline should be placed at the highest (list) level possible
within the headline, as my OP might have suggested. It would be nice
to have child hierarchy maintained in the move, however.

I don't expect a list item to be refiled to another arbitrary list
item. As a user, I wouldn't want to navigate that completion list.

-- 
Jeffrey Horn
http://www.failuretorefrain.com/jeff/

Re: Refiling list items

[Florian Beck]

Jeff Horn <jrhorn424@gmail.com> writes:


>> What would be the specifications of that function? Would it only send
>> the item at point to the end of the headline specified through the
>> refile interface?
>
> I hope its clear that this is all above my head. I know enough to make
> suggestions, but not contribute to implementing them. That makes me a
> free-rider, but a free-rider that recognizes he's at the mercy of
> others' talents.

The thing is, the code I sent you is just that: a shortcut for killing
an item and pasting it at a refile location. Orgs refiling interface
goes to a lot of trouble to do the right thing. Nicolas already
mentioned reparing the list and updating checkboxes, both of which never
was an issue for me.

FWIW, refiling items seems like a natural extension to me. In any case,
it is not about you being able to specify or implement a function. But I
found it very helpful to pick up just enough elisp to automate tasks.

-- 
Florian Beck

Re: Refiling list items

[suvayu ali]

On Tue, Aug 9, 2011 at 2:02 AM, Nick Dokos <nicholas.dokos@hp.com> wrote:
> 99.9% of the bugs I find are
> in my head (most recently the find-file wildcard thing...)
>

I think you can let that go now :D

> Nick



-- 
Suvayu

Open source is the future. It sets us free.

Re: Refiling list items

[Nicolas Goaziou]

Hello,

> I guess this really amounts to the first time I've ever been convinced
> that "treating list items like headlines" would be useful to me. At
> least, it feels like list items might benefit from being a proper
> subset of headlines. I don't remember what the disadvantages of such a
> move would be, but I suspect there's a lot of tacit knowledge in the
> codebase already.

From my point of view, lists cannot be a subset of headlines. Indeed,
headlines are global structural elements, whereas items are local
structural elements. In other words, moving an item outside of its
scope, which is the list it belongs, will remove any structural meaning
it has.

For example, what is the point of moving an un-ordered item into an
ordered list, or, worse, an un-ordered item into a description list?
Sure, the item being moved and the destination list may share the same
structure, but it's only a part of the equation.

Also, we can imagine the following situation, where a section holds
a description list and an un-ordered one. If one wants to move an
un-ordered item there, should it be moved into the logical, but
mismatched, first description list, or into the un-ordered one?

My point is that outside of its list, an item is just plain text.

Thus, why not take that into account? Instead of creating a magical
function to refile items anywhere, let's just extend `org-refile' to
work on a region of text which is not a sub-tree.

At the moment, org-refile understands the concept of region, but checks
if that region holds a sub-tree. What about removing that check, and
adapt the code to text without trees? It will then be the user's problem
if he wants to match apples and oranges. Furthermore, as a side effect,
refiling an item would simply mean selecting it and using refile
interface.

Regards,

-- 
Nicolas Goaziou

Re: Refiling list items

[Jeff Horn]

On Wed, Aug 10, 2011 at 06:05, Nicolas Goaziou <n.goaziou@gmail.com> wrote:
> From my point of view, lists cannot be a subset of headlines. Indeed,
> headlines are global structural elements, whereas items are local
> structural elements. In other words, moving an item outside of its
> scope, which is the list it belongs, will remove any structural meaning
> it has.

I don't disagree.

> For example, what is the point of moving an un-ordered item into an
> ordered list, or, worse, an un-ordered item into a description list?
> Sure, the item being moved and the destination list may share the same
> structure, but it's only a part of the equation.

I frequently use description lists as a quick way to get something
like "bold inline headers" for lists of test answers and questions.
Often, extended answers will be a description list item of several
paragraphs, or several paragraphs with sublists thrown in. The latter
doesn't work well. I agree I should probably figure out a better way
to organize this information, but it just feels natural at the time
and doesn't seem worth the trouble to fix after the fact.

> My point is that outside of its list, an item is just plain text.
>
> Thus, why not take that into account? Instead of creating a magical
> function to refile items anywhere, let's just extend `org-refile' to
> work on a region of text which is not a sub-tree.
>
> At the moment, org-refile understands the concept of region, but checks
> if that region holds a sub-tree. What about removing that check, and
> adapt the code to text without trees? It will then be the user's problem
> if he wants to match apples and oranges. Furthermore, as a side effect,
> refiling an item would simply mean selecting it and using refile
> interface.

Brilliant!

-- 
Jeffrey Horn
http://www.failuretorefrain.com/jeff/

Re: Refiling list items

[Bastien]

Hi Jeff,

thanks for bringing up this issue, and thanks all for the input.

I pushed three small changes to the org-refile interface:

1. before prompting the user for a refile target, put the point at the
   beginning of the region/subtree to refile.  This will make sure
   everyone understands we are refiling headlines, not text.

2. Improve the prompt itself, explicitely saying whether we are refiling
   a subtree or a region (containing subtree(s)).

3. Add a new option `org-refile-active-region-within-subtree' which,
   when turned on, allows the user to refile the active region, turning
   the first line into a headline using `org-toggle-heading'.

I think these are improvements going in the right direction -- let me
know what you think.  On the overall, I'm with Nicolas in thinking that
we need to be extra careful when we try to extend a functionality to
heterogeneous elements.

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> My point is that outside of its list, an item is just plain text.

Yes!

« Outside of the hive, a bee is just a fly. » (René Descartes)

:)

> Thus, why not take that into account? Instead of creating a magical
> function to refile items anywhere, let's just extend `org-refile' to
> work on a region of text which is not a sub-tree.

This is what I tried to achieve with the new option.

> At the moment, org-refile understands the concept of region, but checks
> if that region holds a sub-tree. What about removing that check, and
> adapt the code to text without trees? It will then be the user's problem
> if he wants to match apples and oranges. Furthermore, as a side effect,
> refiling an item would simply mean selecting it and using refile
> interface.

Thanks for this idea!

Best,

-- 
 Bastien

Re: Refiling list items

[Jeff Horn]

On Wed, Aug 17, 2011 at 05:32, Bastien <bzg@altern.org> wrote:
> I think these are improvements going in the right direction -- let me
> know what you think.  On the overall, I'm with Nicolas in thinking that
> we need to be extra careful when we try to extend a functionality to
> heterogeneous elements.

Bastien, thanks so much for adding this functionality. I've recently
moved and it has taken the better part of a week to get settled. I'll
test soon and share my comments!

-- 
Jeffrey Horn
http://www.failuretorefrain.com/jeff/

Re: Refiling list items

[Jeff Horn]

On Wed, Aug 17, 2011 at 05:32, Bastien <bzg@altern.org> wrote:
> I pushed three small changes to the org-refile interface:
>
> 1. before prompting the user for a refile target, put the point at the
>   beginning of the region/subtree to refile.  This will make sure
>   everyone understands we are refiling headlines, not text.
>
> 2. Improve the prompt itself, explicitely saying whether we are refiling
>   a subtree or a region (containing subtree(s)).
>
> 3. Add a new option `org-refile-active-region-within-subtree' which,
>   when turned on, allows the user to refile the active region, turning
>   the first line into a headline using `org-toggle-heading'.
>
> I think these are improvements going in the right direction -- let me
> know what you think.  On the overall, I'm with Nicolas in thinking that
> we need to be extra careful when we try to extend a functionality to
> heterogeneous elements.

I've just come across an instance when this functionality was useful
again. I like the solution you've implemented, Bastien. There's just a
small problem I see right now, which pops up in particular with the
example outline from my OP.

,----[ Sample Outline ]
| * Headline 1
| - List level 1
|   - List level 2
| * Headline 2
| Some text.
`----

Selecting the region with the second line and third line, inclusive
(the plain list under Heading 1), refiling with
`org-refile-active-region-within-subtree' toggled on produces the
following outline:

,----[ Results ]
| * Headline 1
| 2
| * Headline 2
| Some text.
| ** List level 1
|   - List level
`----

As you can see, the digit at the end of the list gets orphaned.

-- 
Jeffrey Horn
http://www.failuretorefrain.com/jeff/

Re: Refiling list items

[Samuel Wales]

I confirm the following bug in git master.

P.S.  One possibility for the OP's question would be to allow a
special prefix arg to select the list item and its children as a
region then call refile region.

On 1/17/12, Jeff Horn <jrhorn424@gmail.com> wrote:
> On Wed, Aug 17, 2011 at 05:32, Bastien <bzg@altern.org> wrote:
>> I pushed three small changes to the org-refile interface:
>>
>> 1. before prompting the user for a refile target, put the point at the
>>   beginning of the region/subtree to refile.  This will make sure
>>   everyone understands we are refiling headlines, not text.
>>
>> 2. Improve the prompt itself, explicitely saying whether we are refiling
>>   a subtree or a region (containing subtree(s)).
>>
>> 3. Add a new option `org-refile-active-region-within-subtree' which,
>>   when turned on, allows the user to refile the active region, turning
>>   the first line into a headline using `org-toggle-heading'.
>>
>> I think these are improvements going in the right direction -- let me
>> know what you think.  On the overall, I'm with Nicolas in thinking that
>> we need to be extra careful when we try to extend a functionality to
>> heterogeneous elements.
>
> I've just come across an instance when this functionality was useful
> again. I like the solution you've implemented, Bastien. There's just a
> small problem I see right now, which pops up in particular with the
> example outline from my OP.
>
> ,----[ Sample Outline ]
> | * Headline 1
> | - List level 1
> |   - List level 2
> | * Headline 2
> | Some text.
> `----
>
> Selecting the region with the second line and third line, inclusive
> (the plain list under Heading 1), refiling with
> `org-refile-active-region-within-subtree' toggled on produces the
> following outline:
>
> ,----[ Results ]
> | * Headline 1
> | 2
> | * Headline 2
> | Some text.
> | ** List level 1
> |   - List level
> `----
>
> As you can see, the digit at the end of the list gets orphaned.
>
> --
> Jeffrey Horn
> http://www.failuretorefrain.com/jeff/
>
>


-- 
The Kafka Pandemic: http://thekafkapandemic.blogspot.com

The disease DOES progress.  MANY people have died from it.  ANYBODY can get it.

Denmark: free Karina Hansen NOW.

Re: Refiling list items

[Bastien]

Hi Samuel,

Samuel Wales <samologist@gmail.com> writes:

> I confirm the following bug in git master.

Fixed, thanks.

-- 
 Bastien

Re: Refiling list items

[Samuel Wales]

On 6/30/13, Bastien <bzg@gnu.org> wrote:
>> I confirm the following bug in git master.
>
> Fixed, thanks.

Thanks.  Does this work for the OP?

Now there is a new bug.  When the region is not active, I expect an
ordinary refile.

What happens instead is that the entire entry gets copied to the
target location and the line point is on gets deleted.

Samuel

-- 
The Kafka Pandemic: http://thekafkapandemic.blogspot.com

The disease DOES progress.  MANY people have died from it.  ANYBODY can get it.

Denmark: free Karina Hansen NOW.

Re: Refiling list items

[Bastien]

Samuel Wales <samologist@gmail.com> writes:

> Now there is a new bug.  When the region is not active, I expect an
> ordinary refile.
>
> What happens instead is that the entire entry gets copied to the
> target location and the line point is on gets deleted.

Argh, should be fixed now.  Thanks for reporting!

-- 
 Bastien