Evaluate all org tables in file before exporting?

Evaluate all org tables in file before exporting?

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 458 bytes --]

Hi folks,

I am looking if anyone already has a solution (a function) that evaluates
all the tables in the file/buffer. I should be then simply able to add that
to local value of before-save-hook in org-mode, right?

Currently, I need to remember to C-u C-c C-c on all the tables before
exporting to ensure that all the calculated values are correct.

Or am I missing some org variable, setting which should take care of this?

Thanks.
-- 

-- 
Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 620 bytes --]

Buffer local value lost when exporting? (was Evaluate all org tables in file before exporting)

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 3388 bytes --]

Hi all,

I almost have a squeaky clean solution to this thanks to the hint provided
by someone on emacs.SE ( http://emacs.stackexchange.com/a/22215/115 ).

But in the process, I seem to have stumbled upon a bug.. the buffer local
value of variables are not respected at the time of exports.

To demonstrate that, please evaluate the elisp code at the end of this
email and do C-c C-e h o on the below MWE of org file:

As the buffer local value of modi/org-table-enable-buffer-wide-recalculation
var is nil, that table does not get recalculated when saving. But it always
gets recalculated when exporting (e.g. using C-c C-e h o).

===== sample.org =====
| N |
|---|
|   |
|   |
|   |
|   |
#+TBLFM: $1=@#-1

# Local Variables:
# modi/org-table-enable-buffer-wide-recalculation: nil
# End:
=====

===== auto-recalculate-org-tables.el =====
;; Recalculate all org tables in the buffer when saving.
(defvar-local modi/org-table-enable-buffer-wide-recalculation t
  "When non-nil, all the org tables in the buffer will be recalculated when
saving the file or when exporting.
This variable is buffer local.")
;; Mark `modi/org-table-enable-buffer-wide-recalculation' as a safe local
;; variable as long as its value is t or nil. That way you are not prompted
;; to add that to `safe-local-variable-values' in custom.el.
(put 'modi/org-table-enable-buffer-wide-recalculation
     'safe-local-variable (lambda (val) (or (equal val nil) (equal val t))))

(defun modi/org-table-recalculate-buffer-tables (&rest args)
  "Wrapper function for `org-table-recalculate-buffer-tables' that runs
that function only if `modi/org-table-enable-buffer-wide-recalculation' is
non-nil.

Also, this function has ARGS as optional arguments that are needed for any
function that is added to the `org-export-before-processing-hook'."
  (message "modi/org-table-enable-buffer-wide-recalculation: %S"
           modi/org-table-enable-buffer-wide-recalculation)
  (when modi/org-table-enable-buffer-wide-recalculation
    (org-table-recalculate-buffer-tables)))

(defun modi/org-table-recalculate-before-save ()
  "Recalculate all org tables in the buffer before saving."
  (add-hook 'before-save-hook #'modi/org-table-recalculate-buffer-tables
nil :local))
(add-hook 'org-mode-hook #'modi/org-table-recalculate-before-save)
;; FIXME: The buffer local value of
`modi/org-table-enable-buffer-wide-recalculation'
;; does not seem to be respected at the moment at the time of running
;; `org-export-before-processing-hook'. Investigating this ..
;; For now, as the default value of that variable is t, all org tables
;; in the buffer will always be recalculated at the time of export even
;; if the buffer local value of that var is nil.
(add-hook 'org-export-before-processing-hook
#'modi/org-table-recalculate-buffer-tables)
=====

On Fri, May 6, 2016 at 10:58 AM Kaushal Modi <kaushal.modi@gmail.com> wrote:

> Hi folks,
>
> I am looking if anyone already has a solution (a function) that evaluates
> all the tables in the file/buffer. I should be then simply able to add that
> to local value of before-save-hook in org-mode, right?
>
> Currently, I need to remember to C-u C-c C-c on all the tables before
> exporting to ensure that all the calculated values are correct.
>
> Or am I missing some org variable, setting which should take care of this?
>
> Thanks.
> --
>
> --
> Kaushal Modi
>
-- 

-- 
Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 4820 bytes --]

Re: Buffer local value lost when exporting? (was Evaluate all org tables in file before exporting)

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 292 bytes --]

Forgot to mention my emacs and org versions; here they are:

GNU Emacs 25.0.93.5 (x86_64-unknown-linux-gnu, GTK+ Version 2.24.23) of
2016-05-05

Org-mode version 8.3.4 (release_8.3.4-778-g8127b3 @
/home/kmodi/usr_local/apps/6/emacs/emacs-25/share/emacs/site-lisp/org/)

-- 

-- 
Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 431 bytes --]

Re: Buffer local value lost when exporting? (was Evaluate all org tables in file before exporting)

[Eric S Fraga]

On Thursday, 12 May 2016 at 15:54, Kaushal Modi wrote:
> Hi all,
>
> I almost have a squeaky clean solution to this thanks to the hint
> provided by someone on emacs.SE (
> http://emacs.stackexchange.com/a/22215/115 ).
>
> But in the process, I seem to have stumbled upon a bug.. the buffer
> local value of variables are not respected at the time of exports.

It's not a bug but a known design feature.  A new buffer is created as
part of the export process and this buffer does not inherit local
variable values.  What you need to do is tell org to set specific
variables upon export.  You can use the #+BIND: directive to do this.

I don't know where this is documented but there is mention of whether to
allow this to happen in the org info manual:

   If ‘org-export-allow-bind-keywords’ is non-‘nil’, Emacs variables can
   become buffer-local during export by using the BIND keyword.  Its
   syntax is ‘#+BIND: variable value’.  This is particularly useful for
   in-buffer settings that cannot be changed using specific keywords.

Give this a try.
-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.0.92.1, Org release_8.3.4-775-g3308a5

Re: Buffer local value lost when exporting? (was Evaluate all org tables in file before exporting)

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 1614 bytes --]

On Fri, May 13, 2016 at 3:04 AM Eric S Fraga <e.fraga@ucl.ac.uk> wrote:

> It's not a bug but a known design feature.  A new buffer is created as
> part of the export process and this buffer does not inherit local
> variable values.


Hi Eric, I think this is a bug because the comments in the code say that
the buffer local values should be preserved.

The org-export-before-processing-hook is called in ox.el in the
org-export-with-buffer-copy macro form (
http://orgmode.org/cgit.cgi/org-mode.git/tree/lisp/ox.el?id=1c923b9ae734304b0dc6688ee056864facb7c3b8#n3002
)
 and org-export-with-buffer-copy macro docstring says:

Apply BODY in a copy of the current buffer.
The copy preserves local variables, visibility and contents of
the original buffer.  Point is at the beginning of the buffer
when BODY is applied.

So I was led to believe that the local variables will be preserved.. Or are
they lost at a later stage?


> What you need to do is tell org to set specific
> variables upon export.  You can use the #+BIND: directive to do this.
>
> I don't know where this is documented but there is mention of whether to
> allow this to happen in the org info manual:
>
>    If ‘org-export-allow-bind-keywords’ is non-‘nil’, Emacs variables can
>    become buffer-local during export by using the BIND keyword.  Its
>    syntax is ‘#+BIND: variable value’.  This is particularly useful for
>    in-buffer settings that cannot be changed using specific keywords.
>
> Give this a try.


I'll dig into this and get back to you.

Thanks.
-- 

-- 
Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 2346 bytes --]

Re: Buffer local value lost when exporting? (was Evaluate all org tables in file before exporting)

[Nicolas Goaziou]

Hello,

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> On Thursday, 12 May 2016 at 15:54, Kaushal Modi wrote:
>> Hi all,
>>
>> I almost have a squeaky clean solution to this thanks to the hint
>> provided by someone on emacs.SE (
>> http://emacs.stackexchange.com/a/22215/115 ).
>>
>> But in the process, I seem to have stumbled upon a bug.. the buffer
>> local value of variables are not respected at the time of exports.
>
> It's not a bug but a known design feature.  A new buffer is created as
> part of the export process and this buffer does not inherit local
> variable values.  What you need to do is tell org to set specific
> variables upon export.  You can use the #+BIND: directive to do this.
>
> I don't know where this is documented but there is mention of whether to
> allow this to happen in the org info manual:
>
>    If ‘org-export-allow-bind-keywords’ is non-‘nil’, Emacs variables can
>    become buffer-local during export by using the BIND keyword.  Its
>    syntax is ‘#+BIND: variable value’.  This is particularly useful for
>    in-buffer settings that cannot be changed using specific keywords.
>
> Give this a try.

For completeness, export process actually inherits some buffer local
values. See `org-export--generate-copy-script', in particular

  ;; Copy specific buffer local variables and variables set
  ;; through BIND keywords.
  ,@(let ((bound-variables (org-export--list-bound-variables))
          vars)
      (dolist (entry (buffer-local-variables (buffer-base-buffer)) vars)
        (when (consp entry)
          (let ((var (car entry))
                (val (cdr entry)))
            (and (not (memq var org-export-ignored-local-variables))
                 (or (memq var
                           '(default-directory
                              buffer-file-name
                              buffer-file-coding-system))
                     (assq var bound-variables)
                     (string-match "^\\(org-\\|orgtbl-\\)"
                                   (symbol-name var)))
                 ;; Skip unreadable values, as they cannot be
                 ;; sent to external process.
                 (or (not val) (ignore-errors (read (format "%S" val))))
                 (push `(set (make-local-variable (quote ,var))
                             (quote ,val))
                       vars))))))

So basically, it copies all Org related variables, default directory,
buffer-file-name buffer-file-coding-system and any variable defined as
a BIND keyword, provided their value is `read'-able (e.g., not a hash
table).


Regards,

-- 
Nicolas Goaziou

Re: Buffer local value lost when exporting? (was Evaluate all org tables in file before exporting)

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 2177 bytes --]

On Fri, May 13, 2016 at 12:25 PM Nicolas Goaziou <mail@nicolasgoaziou.fr>
wrote:

> For completeness, export process actually inherits some buffer local
> values. See `org-export--generate-copy-script', in particular
>
>   ;; Copy specific buffer local variables and variables set
>   ;; through BIND keywords.
>   ,@(let ((bound-variables (org-export--list-bound-variables))
>           vars)
>       (dolist (entry (buffer-local-variables (buffer-base-buffer)) vars)
>         (when (consp entry)
>           (let ((var (car entry))
>                 (val (cdr entry)))
>             (and (not (memq var org-export-ignored-local-variables))
>                  (or (memq var
>                            '(default-directory
>                               buffer-file-name
>                               buffer-file-coding-system))
>                      (assq var bound-variables)
>                      (string-match "^\\(org-\\|orgtbl-\\)"
>                                    (symbol-name var)))
>                  ;; Skip unreadable values, as they cannot be
>                  ;; sent to external process.
>                  (or (not val) (ignore-errors (read (format "%S" val))))
>                  (push `(set (make-local-variable (quote ,var))
>                              (quote ,val))
>                        vars))))))
>
> So basically, it copies all Org related variables, default directory,
> buffer-file-name buffer-file-coding-system and any variable defined as
> a BIND keyword, provided their value is `read'-able (e.g., not a hash
> table).
>

Thanks for that info guys!

It got it working after setting org-export-allow-bind-keywords to t BUT now
I need to set the buffer local value twice: (1) Using the Local Variables
footer style so that that local value is effective when before-save-hook is
run (2) Using #+BIND so that that value is effective when
 org-export-before-processing-hook is run.

Below is the MWE now:

=====
| N |
|---|
|   |
|   |
|   |
|   |
#+TBLFM: $1=@#-1

#+BIND: modi/org-table-enable-buffer-wide-recalculation nil
# Local Variables:
# modi/org-table-enable-buffer-wide-recalculation: nil
# End:
=====

-- 

-- 
Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 3033 bytes --]

Re: Buffer local value lost when exporting? (was Evaluate all org tables in file before exporting)

[Nicolas Goaziou]

Hello,

Kaushal Modi <kaushal.modi@gmail.com> writes:

> It got it working after setting org-export-allow-bind-keywords to t BUT now
> I need to set the buffer local value twice: (1) Using the Local Variables
> footer style so that that local value is effective when before-save-hook is
> run (2) Using #+BIND so that that value is effective when
>  org-export-before-processing-hook is run.
>
> Below is the MWE now:
>
> =====
> | N |
> |---|
> |   |
> |   |
> |   |
> |   |
> #+TBLFM: $1=@#-1
>
> #+BIND: modi/org-table-enable-buffer-wide-recalculation nil
> # Local Variables:
> # modi/org-table-enable-buffer-wide-recalculation: nil
> # End:
> =====

You could cheat and prefix your variable with "org-", e.g.,
"org-modi/...".

You could also try removing the limitation in the code I quoted earlier
so that it really copies all local variables before the export process,
and test if there are some drawbacks to it.

Regards,

-- 
Nicolas Goaziou