Smart archiving of subtrees with parent headlines

Smart archiving of subtrees with parent headlines

[Mark Edgington]

Hello all,

I have looked at a few threads related to the archiving of subtrees,
but haven't found anything that matches what I think would be a very
sensible archiving behavior.  I already posted this as a question on
the emacs stack-exchange site
(https://emacs.stackexchange.com/questions/38530/how-to-archive-an-org-mode-subtree-along-with-its-parent-headlines),
but realize that the mailing list is probably more likely to get some
feedback from people.  So here's what I'm trying to figure out how to
do:

Say I start with an org-file that looks like:

    #+ARCHIVE: ::* Archived

    * Foo
      * Tasks
        * Task1
      * Thoughts
        * Thought1
        * Thought2

    * Archived

Now I put the point (i.e. cursor) on Thought1 and run
org-super-archive (the magical command I'm hoping to find). The result
should be:

    #+ARCHIVE: ::* Archived

    * Foo
      * Tasks
        * Task1
      * Thoughts
        * Thought2

    * Archived
      * Foo
        * Thoughts
          * Thought1

Now I move the point to Thought2 and again run org-super-archive,
which should give me:

    #+ARCHIVE: ::* Archived

    * Foo
      * Tasks
        * Task1
      * Thoughts

    * Archived
      * Foo
        * Thoughts
          * Thought1
          * Thought2

So the basic operation I'm seeking is the ability to archive a subtree
to whatever target location is specified with #+ARCHIVE (or
org-archive-location) where (1) the full path of the archived subtree
is mirrored beneath that target location, and (2) the subtree will be
merged into an existing path under the target if an appropriate path
already exists (e.g. a path consisting of all of the subtree's parent
headlines, regardless of the content of the bodies of these
headlines).

As an added bonus, it would be nice if it were possible to choose
whether or not the "full path" of a subtree to be archived will
include the org-file name as the root of the path. This would be
useful in cases where you archive from multiple org-files to a single
archive.org file.

Any thoughts on this -- has it already been done, or would it be easy to do?

Re: Smart archiving of subtrees with parent headlines

[numbchild]

[-- Attachment #1: Type: text/plain, Size: 2599 bytes --]

I like this idea.

[stardiviner]           <Hack this world!>      GPG key ID: 47C32433
IRC(freeenode): stardiviner                     Twitter:  @numbchild
Key fingerprint = 9BAA 92BC CDDD B9EF 3B36  CB99 B8C4 B8E5 47C3 2433
Blog: http://stardiviner.github.io/

On Sat, Feb 10, 2018 at 12:42 AM, Mark Edgington <edgimar@gmail.com> wrote:

> Hello all,
>
> I have looked at a few threads related to the archiving of subtrees,
> but haven't found anything that matches what I think would be a very
> sensible archiving behavior.  I already posted this as a question on
> the emacs stack-exchange site
> (https://emacs.stackexchange.com/questions/38530/how-to-
> archive-an-org-mode-subtree-along-with-its-parent-headlines),
> but realize that the mailing list is probably more likely to get some
> feedback from people.  So here's what I'm trying to figure out how to
> do:
>
> Say I start with an org-file that looks like:
>
>     #+ARCHIVE: ::* Archived
>
>     * Foo
>       * Tasks
>         * Task1
>       * Thoughts
>         * Thought1
>         * Thought2
>
>     * Archived
>
> Now I put the point (i.e. cursor) on Thought1 and run
> org-super-archive (the magical command I'm hoping to find). The result
> should be:
>
>     #+ARCHIVE: ::* Archived
>
>     * Foo
>       * Tasks
>         * Task1
>       * Thoughts
>         * Thought2
>
>     * Archived
>       * Foo
>         * Thoughts
>           * Thought1
>
> Now I move the point to Thought2 and again run org-super-archive,
> which should give me:
>
>     #+ARCHIVE: ::* Archived
>
>     * Foo
>       * Tasks
>         * Task1
>       * Thoughts
>
>     * Archived
>       * Foo
>         * Thoughts
>           * Thought1
>           * Thought2
>
> So the basic operation I'm seeking is the ability to archive a subtree
> to whatever target location is specified with #+ARCHIVE (or
> org-archive-location) where (1) the full path of the archived subtree
> is mirrored beneath that target location, and (2) the subtree will be
> merged into an existing path under the target if an appropriate path
> already exists (e.g. a path consisting of all of the subtree's parent
> headlines, regardless of the content of the bodies of these
> headlines).
>
> As an added bonus, it would be nice if it were possible to choose
> whether or not the "full path" of a subtree to be archived will
> include the org-file name as the root of the path. This would be
> useful in cases where you archive from multiple org-files to a single
> archive.org file.
>
> Any thoughts on this -- has it already been done, or would it be easy to
> do?
>
>

[-- Attachment #2: Type: text/html, Size: 3844 bytes --]

Re: Smart archiving of subtrees with parent headlines

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 704 bytes --]

On Friday,  9 Feb 2018 at 11:42, Mark Edgington wrote:

[...]

> So the basic operation I'm seeking is the ability to archive a subtree
> to whatever target location is specified with #+ARCHIVE (or
> org-archive-location) where (1) the full path of the archived subtree
> is mirrored beneath that target location, and (2) the subtree will be
> merged into an existing path under the target if an appropriate path
> already exists (e.g. a path consisting of all of the subtree's parent
> headlines, regardless of the content of the bodies of these
> headlines).

+1

I've wanted this (specifically (1)) like forever...

-- 
Eric S Fraga via Emacs 27.0.50, Org release_9.1.6-118-g134243

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: Smart archiving of subtrees with parent headlines

[Ken Mankoff]

On 2018-02-09 at 16:42, Mark Edgington <edgimar@gmail.com> wrote:
> I have looked at a few threads related to the archiving of subtrees,
> but haven't found anything that matches what I think would be a very
> sensible archiving behavior. [...]

Does the attached file here work for you? I use it and it seems to do what you describe.

https://lists.gnu.org/archive/html/emacs-orgmode/2014-08/msg00109.html


My full archive setup from my emacs.org follows. It is based on the above code but tweaked in ways by past-me that present-me doesn't fully understand, but it works for me.


(setq org-archive-location (concat org-directory "/archive/%s_archive::"))

(define-key org-mode-map (kbd "C-c C-x C-s") nil)
(setq org-archive-default-command 'kdm/org-archive-local-or-hierarchical) ;; C-c C-x C-a

;; only do hierarchical archiving if default var used. If archiving into
;; local file, then just use default org-archive-subtree command
(defun kdm/org-archive-local-or-hierarchical ()
  "Archive locally if location set to local file; Otherwise use org-archive-subtree-hierarchical"
  (interactive)
  (if (let ((arch-file (org-extract-archive-file))
            (this-file (buffer-file-name)))
           (equal arch-file this-file))
  (org-archive-subtree)
  (org-archive-subtree-hierarchical)))

(require 'org-archive)

(defun line-content-as-string ()
  "Returns the content of the current line as a string"
  (save-excursion
    (beginning-of-line)
    (buffer-substring-no-properties
     (line-beginning-position) (line-end-position))))

(defun org-child-list ()
  "This function returns all children of a heading as a list. "
  (interactive)
  (save-excursion
    ;; this only works with org-version > 8.0, since in previous
    ;; org-mode versions the function (org-outline-level) returns
    ;; gargabe when the point is not on a heading.
    (if (= (org-outline-level) 0)
	(outline-next-visible-heading 1)
      (org-goto-first-child))
    (let ((child-list (list (line-content-as-string))))
      (while (org-goto-sibling)
        (setq child-list (cons (line-content-as-string) child-list)))
      child-list)))

(defun fa/org-struct-subtree ()
  "This function returns the tree structure in which a subtree
belongs as a list."
  (interactive)
  (let ((archive-tree nil))
    (save-excursion
      (while (org-up-heading-safe)
        (let ((heading
               (buffer-substring-no-properties
                (line-beginning-position) (line-end-position))))
          (if (eq archive-tree nil)
              (setq archive-tree (list heading))
            (setq archive-tree (cons heading archive-tree))))))
    archive-tree))

(defun org-archive-subtree-hierarchical ()
  "This function archives a subtree hierarchical"
  (interactive)
  (let ((org-tree (fa/org-struct-subtree))
        (this-buffer (current-buffer))
        (file (abbreviate-file-name
		   (or (buffer-file-name (buffer-base-buffer))
		       (error "No file associated to buffer")))))
    (save-excursion
      (setq location (org-get-local-archive-location)
            afile (org-extract-archive-file location)
            heading (org-extract-archive-heading location)
            infile-p (equal file (abbreviate-file-name (or afile ""))))
      (unless afile
        (error "Invalid `org-archive-location'"))
      (if (> (length afile) 0)
          (setq newfile-p (not (file-exists-p afile))
                visiting (find-buffer-visiting afile)
                buffer (or visiting (find-file-noselect afile)))
        (setq buffer (current-buffer)))
      (unless buffer
        (error "Cannot access file \"%s\"" afile))
      (org-cut-subtree)
      (set-buffer buffer)
      (org-mode)
      (goto-char (point-min))
      (while (not (equal org-tree nil))
        (let ((child-list (org-child-list)))
          (if (member (car org-tree) child-list)
              (progn
                (search-forward (car org-tree) nil t)
                (setq org-tree (cdr org-tree)))
            (progn
              (newline)
              (org-insert-struct org-tree)
              (setq org-tree nil)))))
      (newline)
      (org-yank)
      ;; Save and kill the buffer, if it is not the same buffer.
      (when (not (eq this-buffer buffer))
        (save-buffer)
        (kill-buffer))
      (message "Subtree archived %s"
               (concat "in file: " (abbreviate-file-name afile))))))

(defun org-insert-struct (struct)
  "TODO"
  (interactive)
  (when struct
    (insert (car struct))
    (newline)
    (org-insert-struct (cdr struct))))

  -k.
  



> So here's what I'm trying to figure out how to do:
>
> Say I start with an org-file that looks like:
>
>     #+ARCHIVE: ::* Archived
>
>     * Foo
>       * Tasks
>         * Task1
>       * Thoughts
>         * Thought1
>         * Thought2
>
>     * Archived
>
> Now I put the point (i.e. cursor) on Thought1 and run
> org-super-archive (the magical command I'm hoping to find). The result
> should be:
>
>     #+ARCHIVE: ::* Archived
>
>     * Foo
>       * Tasks
>         * Task1
>       * Thoughts
>         * Thought2
>
>     * Archived
>       * Foo
>         * Thoughts
>           * Thought1
>
> Now I move the point to Thought2 and again run org-super-archive,
> which should give me:
>
>     #+ARCHIVE: ::* Archived
>
>     * Foo
>       * Tasks
>         * Task1
>       * Thoughts
>
>     * Archived
>       * Foo
>         * Thoughts
>           * Thought1
>           * Thought2
>
> So the basic operation I'm seeking is the ability to archive a subtree
> to whatever target location is specified with #+ARCHIVE (or
> org-archive-location) where (1) the full path of the archived subtree
> is mirrored beneath that target location, and (2) the subtree will be
> merged into an existing path under the target if an appropriate path
> already exists (e.g. a path consisting of all of the subtree's parent
> headlines, regardless of the content of the bodies of these
> headlines).
>
> As an added bonus, it would be nice if it were possible to choose
> whether or not the "full path" of a subtree to be archived will
> include the org-file name as the root of the path. This would be
> useful in cases where you archive from multiple org-files to a single
> archive.org file.

Re: Smart archiving of subtrees with parent headlines

[Mark Edgington]

On Mon, Feb 12, 2018 at 1:54 AM, Ken Mankoff <mankoff@gmail.com> wrote:
>
> Does the attached file here work for you? I use it and it seems to do what you describe.
>
> https://lists.gnu.org/archive/html/emacs-orgmode/2014-08/msg00109.html
>

Ken, I tried the code you included from your config file, and while it does
satisfy my requirement 2 (the subtree will be merged into an existing path
under the target if an appropriate path already exists), the first requirement
of it being moved to be "beneath" a specified target location seems not to be
working correctly.

I've modified your code so that it can at least handle archiving subtrees
beneath a specified target headline.  The new code assumes that the specified
target headline is at level 1 (has a single asterisk), but it would be nice if
this could be made to work with a target headline having a larger depth.  Note
that I changed the behavior from what you had so that hierarchical archiving is
used whether or not the target is in the current buffer.

Although archiving is a bit less painful for me now with this new code, there
are still a few things it would be nice to have:

 - arbitrary-depth target-headline
 - option to prefix target-headline with source filename (this probably won't take too much work)
 - option to add archival properties to each archived item (e.g. date archived)


Here's a diff from the code you posted:

--- old 2018-02-12 10:14:07.646226775 -0500
+++ new 2018-02-12 14:51:20.676703024 -0500
@@ -1,18 +1,11 @@
-(setq org-archive-location (concat org-directory "/archive/%s_archive::"))
+; (setq org-archive-location (concat org-directory "/archive/%s_archive::"))
+(setq org-archive-location "archive/archived_%s::")
 
+; unmap org-archive-subtree
 (define-key org-mode-map (kbd "C-c C-x C-s") nil)
-(setq org-archive-default-command 'kdm/org-archive-local-or-hierarchical) ;; C-c C-x C-a
 
-;; only do hierarchical archiving if default var used. If archiving into
-;; local file, then just use default org-archive-subtree command
-(defun kdm/org-archive-local-or-hierarchical ()
-  "Archive locally if location set to local file; Otherwise use org-archive-subtree-hierarchical"
-  (interactive)
-  (if (let ((arch-file (org-extract-archive-file))
-            (this-file (buffer-file-name)))
-           (equal arch-file this-file))
-  (org-archive-subtree)
-  (org-archive-subtree-hierarchical)))
+; select command to execute via org-archive-subtree-default (C-c C-x C-a)
+(setq org-archive-default-command 'org-archive-subtree-hierarchical)
 
 (require 'org-archive)
 
@@ -23,16 +16,17 @@
     (buffer-substring-no-properties
      (line-beginning-position) (line-end-position))))
 
-(defun org-child-list ()
+(defun org-child-list (&optional top-level)
   "This function returns all children of a heading as a list. "
   (interactive)
   (save-excursion
     ;; this only works with org-version > 8.0, since in previous
     ;; org-mode versions the function (org-outline-level) returns
     ;; gargabe when the point is not on a heading.
-    (if (= (org-outline-level) 0)
-        (outline-next-visible-heading 1)
-      (org-goto-first-child))
+    (unless top-level
+        (if (= (org-outline-level) 0)
+            (outline-next-visible-heading 1)
+        (org-goto-first-child)))
     (let ((child-list (list (line-content-as-string))))
       (while (org-goto-sibling)
         (setq child-list (cons (line-content-as-string) child-list)))
@@ -68,6 +62,11 @@
             infile-p (equal file (abbreviate-file-name (or afile ""))))
       (unless afile
         (error "Invalid `org-archive-location'"))
+      (if (not (equal heading ""))
+          (progn
+            (setq org-tree (cons heading
+                               (mapcar (lambda (s) (concat "*" s)) org-tree)))
+            (org-demote-subtree)))
       (if (> (length afile) 0)
           (setq newfile-p (not (file-exists-p afile))
                 visiting (find-buffer-visiting afile)
@@ -79,16 +78,18 @@
       (set-buffer buffer)
       (org-mode)
       (goto-char (point-min))
+      (setq top-level-p t)
       (while (not (equal org-tree nil))
-        (let ((child-list (org-child-list)))
+        (let ((child-list (org-child-list top-level-p)))
           (if (member (car org-tree) child-list)
               (progn
-                (search-forward (car org-tree) nil t)
+                (re-search-forward (concat "^" (regexp-quote (car org-tree))) nil t)
                 (setq org-tree (cdr org-tree)))
             (progn
-              (newline)
+              (if (not top-level-p) (newline))
               (org-insert-struct org-tree)
-              (setq org-tree nil)))))
+              (setq org-tree nil))))
+        (setq top-level-p nil))
       (newline)
       (org-yank)
       ;; Save and kill the buffer, if it is not the same buffer.
@@ -103,5 +104,6 @@
   (interactive)
   (when struct
     (insert (car struct))
-    (newline)
+    (if  (not (equal (length struct) 1))
+        (newline))
     (org-insert-struct (cdr struct))))

Re: Smart archiving of subtrees with parent headlines

[Mark Edgington]

While further evaluating my code, I realized that it wasn't working
when target headlines contained tags.  I've updated the code to handle
this case -- the result is posted as a gist since it may change later
on, and all of you esteemed elisp hackers can more easily contribute
to it that way, until it's worthy to be considered for inclusion in
contrib.

Here's the gist URL:

  https://gist.github.com/edgimar/072d99d8650abe81a9fe7c8687c0c993

Re: Smart archiving of subtrees with parent headlines

[Bastien]

Hi Mark,

Mark Edgington <edgimar@gmail.com> writes:

> Here's the gist URL:
>
>   https://gist.github.com/edgimar/072d99d8650abe81a9fe7c8687c0c993

I have added a FAQ in Worg for this:

"Can I archive a subtree while keeping its hiearchy above?"

I'd be interested in integrating such a functionality in
org-archive.el.

Do you think that is feasible?

Thanks,

-- 
 Bastien

Re: Smart archiving of subtrees with parent headlines

[Mark Edgington]

[-- Attachment #1: Type: text/plain, Size: 431 bytes --]

Hi Bastien,

On Thu, Apr 26, 2018 at 7:34 PM, Bastien <bzg@gnu.org> wrote:

> I'd be interested in integrating such a functionality in
> org-archive.el.
>
> Do you think that is feasible?


I don't know why not -- I'm OK with the code being used in org-archive.el.
As far as I'm concerned, you can use it however you wish (it is based on
Ken's code though, so he would also need to sign off on its use, I suppose).

Regards,

Mark

[-- Attachment #2: Type: text/html, Size: 828 bytes --]

Re: Smart archiving of subtrees with parent headlines

[Bastien]

Hi Mark,

Mark Edgington <edgimar@gmail.com> writes:

> I don't know why not -- I'm OK with the code being used in
> org-archive.el.  As far as I'm concerned, you can use it however you
> wish (it is based on Ken's code though, so he would also need to sign
> off on its use, I suppose).

Would you like to contribute by submitting a patch against master?

For this you would need to sign FSF copyright assignment.

See http://orgmode.org/request-assign-future.txt

Thanks!

-- 
 Bastien

Re: Smart archiving of subtrees with parent headlines

[Ken Mankoff]

[-- Attachment #1: Type: text/plain, Size: 1358 bytes --]

On Tue, May 1, 2018 at 11:01 AM Bastien <bzg@gnu.org> wrote:

> Hi Mark,
>
> Mark Edgington <edgimar@gmail.com> writes:
>
> > I don't know why not -- I'm OK with the code being used in
> > org-archive.el.  As far as I'm concerned, you can use it however you
> > wish (it is based on Ken's code though, so he would also need to sign
> > off on its use, I suppose).
>
> Would you like to contribute by submitting a patch against master?
>
> For this you would need to sign FSF copyright assignment.
>
> See http://orgmode.org/request-assign-future.txt
>
>
Replying to an email almost 1 year old. I've lost archive-to-subtree
functionality recently and don't know how to get it back w/ the recent Org
code changes. While searching I found this old thread where Mark & Bastien
suggest I'd need to sign off on someone using some code I pasted. I got
that code from elsewhere, and only did minor customizations. In my original
email I link to where I got the code from (
https://lists.gnu.org/archive/html/emacs-orgmode/2014-08/msg00109.html ). I
don't think I have the rights to sign off on this, but sure, you can use
what I pasted and my customizations. I've signed the FSF documents already,
a long time ago. It seems like archive-to-subtree should be a core
functionality, and if someone were to get that working again I'd be happy
to test and use it.

  -k.

[-- Attachment #2: Type: text/html, Size: 2046 bytes --]

Re: Smart archiving of subtrees with parent headlines

[Mark Edgington]

[-- Attachment #1: Type: text/plain, Size: 2007 bytes --]

Ken, I believe Bastien's signing comment was also directed towards me, and
I am fine with signing a FSF assignment document, but this had simply
gotten buried in my emails.

I do still have it on my list to take a look at this at some point, but if
one of you is inclined to merge it, by all means do!  I don't think I've
needed to change the function I've been using for this (works with a
relatively recent org-mode release).  But it sounds like you're saying that
it won't work with the latest git versions?

Regards,

Mark


On Thu, Mar 28, 2019, 2:14 PM Ken Mankoff <mankoff@gmail.com> wrote:

>
> On Tue, May 1, 2018 at 11:01 AM Bastien <bzg@gnu.org> wrote:
>
>> Hi Mark,
>>
>> Mark Edgington <edgimar@gmail.com> writes:
>>
>> > I don't know why not -- I'm OK with the code being used in
>> > org-archive.el.  As far as I'm concerned, you can use it however you
>> > wish (it is based on Ken's code though, so he would also need to sign
>> > off on its use, I suppose).
>>
>> Would you like to contribute by submitting a patch against master?
>>
>> For this you would need to sign FSF copyright assignment.
>>
>> See http://orgmode.org/request-assign-future.txt
>>
>>
> Replying to an email almost 1 year old. I've lost archive-to-subtree
> functionality recently and don't know how to get it back w/ the recent Org
> code changes. While searching I found this old thread where Mark & Bastien
> suggest I'd need to sign off on someone using some code I pasted. I got
> that code from elsewhere, and only did minor customizations. In my original
> email I link to where I got the code from (
> https://lists.gnu.org/archive/html/emacs-orgmode/2014-08/msg00109.html ).
> I don't think I have the rights to sign off on this, but sure, you can use
> what I pasted and my customizations. I've signed the FSF documents already,
> a long time ago. It seems like archive-to-subtree should be a core
> functionality, and if someone were to get that working again I'd be happy
> to test and use it.
>
>   -k.
>

[-- Attachment #2: Type: text/html, Size: 3191 bytes --]

Re: Smart archiving of subtrees with parent headlines

[Ken Mankoff]

On 2019-03-29 at 03:23 +0100, Mark Edgington <edgimar@gmail.com> wrote...
> But it sounds like you're saying that it won't work with the latest
> git versions?

elpa, not git. But yes, this code is expecting and calling functions no longer in the core code.

  -k.

Re: Smart archiving of subtrees with parent headlines

[Ken Mankoff]

[-- Attachment #1: Type: text/plain, Size: 994 bytes --]

I'm not an advanced lisper so I hate to say "this seems like it should be
easy" - it isn't for me. But while playing around with the default archive
behavior (C-c C-x C-a OR C-c C-x C-s?), I notice that ":ARCHIVE_OLPATH:
Testing/SomeTest Archive Project" is include in the archived item. So Org
appears to know where the item should go (if not at the top level). It
seems like it should be easy to either a) put it in the right place when
archiving, b) put it in the right place at some later point in time or c)
perform a search on the archive file that at least limits the results to a
tree or subtree. I can do the last one!

  -k.


On Fri, Mar 29, 2019 at 8:18 AM Ken Mankoff <mankoff@gmail.com> wrote:

>
> On 2019-03-29 at 03:23 +0100, Mark Edgington <edgimar@gmail.com> wrote...
> > But it sounds like you're saying that it won't work with the latest
> > git versions?
>
> elpa, not git. But yes, this code is expecting and calling functions no
> longer in the core code.
>
>   -k.
>
>

[-- Attachment #2: Type: text/html, Size: 1493 bytes --]