Bug: clocktable doesn't preserve formulas with :scope file-with-archives

Bug: clocktable doesn't preserve formulas with :scope file-with-archives

[Dale]

[-- Attachment #1: Type: text/plain, Size: 2050 bytes --]

Hi!  I've found that the "#+TBLFM:" in a clocktable can get changed or
deleted when used together with ":scope file-with-archives".  Here's a
minimal org file to reproduce with:

------8<------
* Test
:LOGBOOK:
CLOCK: [2016-10-20 Thu 17:42]--[2016-10-20 Thu 18:03] =>  0:21
:END:

#+BEGIN: clocktable :scope file-with-archives
#+TBLFM: $3=string("foo")
#+END:
------8<------

Steps to reproduce:

1. emacs -Q, load above file with org-mode from Git

2. Update clocktable dblock (move to "#+BEGIN" and C-c C-c)

Expected result: a third column is added with value "foo" in every
row; #+TBLFM line is preserved

Observed result: table has two columns, the second of which contains
"foo" in every row; #+TBLFM line changes from $3=string("foo") to
$2=string("foo")

If you keep updating the block, the formula's "$2" then becomes "$1".
Do it one more time and the "#+TBLFM:" is preserved but now the
formula is gone entirely.

Emacs  : GNU Emacs 25.1.1 (x86_64-apple-darwin15.6.0)
 of 2016-09-23
Package: Org-mode version 8.2.10 (release_8.2.10 @ /opt/local/share/emacs/25.1/\
lisp/org/)

I have also reproduced this with org-mode from Git as of an hour or so ago.

My hunch is that the problem is in org-clocktable-write-default.  It
writes the table (the dblock's contents have already been deleted),
restores any #+TBLFM: line that used to be there pre-update, and then,
if you're using :scope file-with-archives, it deletes the file column,
which is the first column.  The order here is the problem:
org-table-delete-column updates the formula in #+TBLFM, decrementing
the column reference to account for the deleted column.

If this sounds right then I'd suggest that the solution may be as
simple as just updating org-clocktable-write-default to insert table
formulas *after* deleting the file column, along with some

I'm attaching a patch but I'm not sure whether you'll be able to use
it because my FSF assignment hasn't been updated for my new employer.
Even if you can't use it, hopefully my description above is still
useful.

Thanks!

Dale

[-- Attachment #2: clocktable-formulas.patch --]
[-- Type: application/octet-stream, Size: 2139 bytes --]

 lisp/org-clock.el | 34 ++++++++++++++++++----------------
 1 file changed, 18 insertions(+), 16 deletions(-)

diff --git a/lisp/org-clock.el b/lisp/org-clock.el
index 3448d84..de7fcab 100644
--- a/lisp/org-clock.el
+++ b/lisp/org-clock.el
@@ -2619,6 +2619,24 @@ from the dynamic block definition."
     ;; When exporting subtrees or regions the region might be
     ;; activated, so let's disable ̀delete-active-region'
     (let ((delete-active-region nil)) (backward-delete-char 1))
+    ;; Back to beginning, align the table, recalculate if necessary
+    (goto-char ipos)
+    (skip-chars-forward "^|")
+    (org-table-align)
+    (when org-hide-emphasis-markers
+      ;; we need to align a second time
+      (org-table-align))
+    (when sort
+      (save-excursion
+	(org-table-goto-line 3)
+	(org-table-goto-column (car sort))
+	(org-table-sort-lines nil (cdr sort))))
+    (when rm-file-column
+      ;; The file column is actually not wanted
+      (forward-char 1)
+      (org-table-delete-column))
+    (goto-char (org-table-end))
+    (backward-char 1)
     (if (setq formula (plist-get params :formula))
 	(cond
 	 ((eq formula '%)
@@ -2654,18 +2672,6 @@ from the dynamic block definition."
 	  (setq recalc t)
 	  (insert "\n" (match-string 1 (plist-get params :content)))
 	  (beginning-of-line 0))))
-    ;; Back to beginning, align the table, recalculate if necessary
-    (goto-char ipos)
-    (skip-chars-forward "^|")
-    (org-table-align)
-    (when org-hide-emphasis-markers
-      ;; we need to align a second time
-      (org-table-align))
-    (when sort
-      (save-excursion
-	(org-table-goto-line 3)
-	(org-table-goto-column (car sort))
-	(org-table-sort-lines nil (cdr sort))))
     (when recalc
       (if (eq formula '%)
 	  (save-excursion
@@ -2673,10 +2679,6 @@ from the dynamic block definition."
 	    (org-table-goto-column pcol nil 'force)
 	    (insert "%")))
       (org-table-recalculate 'all))
-    (when rm-file-column
-      ;; The file column is actually not wanted
-      (forward-char 1)
-      (org-table-delete-column))
     total-time))
 
 (defun org-clocktable-indent-string (level)

Re: Bug: clocktable doesn't preserve formulas with :scope file-with-archives

[Nicolas Goaziou]

Hello,

Dale <dale@codefu.org> writes:

> Hi!  I've found that the "#+TBLFM:" in a clocktable can get changed or
> deleted when used together with ":scope file-with-archives".  Here's a
> minimal org file to reproduce with:
>
> ------8<------
> * Test
> :LOGBOOK:
> CLOCK: [2016-10-20 Thu 17:42]--[2016-10-20 Thu 18:03] =>  0:21
> :END:
>
> #+BEGIN: clocktable :scope file-with-archives
> #+TBLFM: $3=string("foo")
>
> #+END:
> ------8<------
>
> Steps to reproduce:
>
> 1. emacs -Q, load above file with org-mode from Git
>
> 2. Update clocktable dblock (move to "#+BEGIN" and C-c C-c)
>
> Expected result: a third column is added with value "foo" in every
> row; #+TBLFM line is preserved
>
> Observed result: table has two columns, the second of which contains
> "foo" in every row; #+TBLFM line changes from $3=string("foo") to
> $2=string("foo")
>
> If you keep updating the block, the formula's "$2" then becomes "$1".
> Do it one more time and the "#+TBLFM:" is preserved but now the
> formula is gone entirely.
>
> Emacs  : GNU Emacs 25.1.1 (x86_64-apple-darwin15.6.0)
>  of 2016-09-23
> Package: Org-mode version 8.2.10 (release_8.2.10 @ /opt/local/share/emacs/25.1/\
> lisp/org/)
>
> I have also reproduced this with org-mode from Git as of an hour or so ago.
>
> My hunch is that the problem is in org-clocktable-write-default.  It
> writes the table (the dblock's contents have already been deleted),
> restores any #+TBLFM: line that used to be there pre-update, and then,
> if you're using :scope file-with-archives, it deletes the file column,
> which is the first column.  The order here is the problem:
> org-table-delete-column updates the formula in #+TBLFM, decrementing
> the column reference to account for the deleted column.
>
> If this sounds right then I'd suggest that the solution may be as
> simple as just updating org-clocktable-write-default to insert table
> formulas *after* deleting the file column

Thank you for the report and the analysis. This is now fixed.

Regards,

-- 
Nicolas Goaziou