How to add content from a file to a table automatically

How to add content from a file to a table automatically

[Darlan Cavalcante Moreira]

Is there a way to detect when a file changes and then add the content from
it (if any, the file could be empty) to a specified org-mode table? Each
line in that file is already constructed in a way that it could be copied
and pasted to the table directly (but I can change that if it makes things
easier).

For instance, suppose I have the table below

--8<---------------cut here---------------start------------->8---
#+TBLNAME: FoodJanuary2013
|   | *Place*   | *Date*           | *Debit (R$)* | *Credit (R$)* |
|---+-----------+------------------+--------------+---------------|
|   | Some name | [2013-01-07 Seg] |         0.00 |         10.00 |
|---+-----------+------------------+--------------+---------------|
| # |           | *Total*          |         0.00 |         10.00 |
#+TBLFM: @3$4=vsum(@2..@-1);%.2f::@3$5=vsum(@2..@-1);%.2f
--8<---------------cut here---------------end--------------->8---

and a file "food.txt" that was modified externally (Dropbox sync) and now
has the content below

--8<---------------cut here---------------start------------->8---
| # | place name | [2013-01-08 Ter] | 0.00 | 12.50 |
| # | other place | [2013-01-09 Qua] | 9.30 | 0.00 |
--8<---------------cut here---------------end--------------->8---

I'd like the table to be automatically updated to include these two new
lines so that it should now corresponds to

--8<---------------cut here---------------start------------->8---
#+TBLNAME: FoodJanuary2013
|   | *Place*     | *Date*           | *Debit (R$)* | *Credit (R$)* |
|---+-------------+------------------+--------------+---------------|
|   | Some name   | [2013-01-09 Qua] |         0.00 |         10.00 |
| # | place name  | [2013-01-08 Ter] |         0.00 |         12.50 |
| # | other place | [2013-01-09 Qua] |         9.30 |          0.00 |
|---+-------------+------------------+--------------+---------------|
| # |             | *Total*          |         9.30 |         22.50 |
#+TBLFM: @5$4=vsum(@2..@-1);%.2f::@5$5=vsum(@2..@-1);%.2f
--8<---------------cut here---------------end--------------->8---

Adapting the formula as org-mode would automatically do if I typed enter
twice before the last hline is important, but updating the total value is
not (although nice), since I can do that with "C-u C-u C-c *" later when I
actually see the table.

After that the content of the file should be erased to prevent future
additions of the same information.

Any hints on how to do the different parts for this are welcome.

--
Darlan

Ps: I use the latest org-mode and Emacs versions (from git and bzr,
respectively) in a Linux box.


The motivation for all this is that I use org-mode tables to control my
expenses, but I want to be able to add new expenses from my android
phone. Using the "tasker" app (REALLY good) I could create a simple UI to
add the expense to a file that will be synced using Dropbox (one file for
each category, which corresponds to a different table). All of this is
working and now I the next step is the one described in this e-mail.

Usecase for inotify? Re: How to add content from a file to a table automatically

[Jambunathan K]

There is support for inotify in Emacs.  Last I heard, people were
interested in hearing use-cases.

I am not sure it qualifies as a minimal use-case.  But it is something
that comes out of real life.  So here it goes to emacs-devel...

Darlan Cavalcante Moreira <darcamo@gmail.com> writes:

> Is there a way to detect when a file changes and then add the content from
> it (if any, the file could be empty) to a specified org-mode table? Each
> line in that file is already constructed in a way that it could be copied
> and pasted to the table directly (but I can change that if it makes things
> easier).
>
> For instance, suppose I have the table below
>
>
> #+TBLNAME: FoodJanuary2013
> |   | *Place*   | *Date*           | *Debit (R$)* | *Credit (R$)* |
> |---+-----------+------------------+--------------+---------------|
> |   | Some name | [2013-01-07 Seg] |         0.00 |         10.00 |
> |---+-----------+------------------+--------------+---------------|
> | # |           | *Total*          |         0.00 |         10.00 |
> #+TBLFM: @3$4=vsum(@2..@-1);%.2f::@3$5=vsum(@2..@-1);%.2f
>
>
> and a file "food.txt" that was modified externally (Dropbox sync) and now
> has the content below
>
>
> | # | place name | [2013-01-08 Ter] | 0.00 | 12.50 |
> | # | other place | [2013-01-09 Qua] | 9.30 | 0.00 |
>
>
> I'd like the table to be automatically updated to include these two new
> lines so that it should now corresponds to
>
>
> #+TBLNAME: FoodJanuary2013
> |   | *Place*     | *Date*           | *Debit (R$)* | *Credit (R$)* |
> |---+-------------+------------------+--------------+---------------|
> |   | Some name   | [2013-01-09 Qua] |         0.00 |         10.00 |
> | # | place name  | [2013-01-08 Ter] |         0.00 |         12.50 |
> | # | other place | [2013-01-09 Qua] |         9.30 |          0.00 |
> |---+-------------+------------------+--------------+---------------|
> | # |             | *Total*          |         9.30 |         22.50 |
> #+TBLFM: @5$4=vsum(@2..@-1);%.2f::@5$5=vsum(@2..@-1);%.2f
>
>
> Adapting the formula as org-mode would automatically do if I typed enter
> twice before the last hline is important, but updating the total value is
> not (although nice), since I can do that with "C-u C-u C-c *" later when I
> actually see the table.
>
> After that the content of the file should be erased to prevent future
> additions of the same information.
>
> Any hints on how to do the different parts for this are welcome.
>
> --
> Darlan
>
> Ps: I use the latest org-mode and Emacs versions (from git and bzr,
> respectively) in a Linux box.
>
>
> The motivation for all this is that I use org-mode tables to control my
> expenses, but I want to be able to add new expenses from my android
> phone. Using the "tasker" app (REALLY good) I could create a simple UI to
> add the expense to a file that will be synced using Dropbox (one file for
> each category, which corresponds to a different table). All of this is
> working and now I the next step is the one described in this e-mail.
>
>

-- 

Re: Usecase for inotify? Re: How to add content from a file to a table automatically

[Eli Zaretskii]

> From: Jambunathan K <kjambunathan@gmail.com>
> Date: Thu, 10 Jan 2013 10:37:19 +0530
> Cc: emacs-orgmode <emacs-orgmode@gnu.org>, emacs-devel@gnu.org
> 
> 
> There is support for inotify in Emacs.  Last I heard, people were
> interested in hearing use-cases.
> 
> I am not sure it qualifies as a minimal use-case.  But it is something
> that comes out of real life.  So here it goes to emacs-devel...

Thanks.

However, at least from my perspective, a useful use case should
include a description of how a feature was actually implemented using
the file notification, how the provided APIs were used to implement
it, and what experience and suggestions for API modifications were
gained as result.

Re: How to add content from a file to a table automatically

[Eric Schulte]

[-- Attachment #1: Type: text/plain, Size: 928 bytes --]

Darlan Cavalcante Moreira <darcamo@gmail.com> writes:

> Is there a way to detect when a file changes and then add the content from
> it (if any, the file could be empty) to a specified org-mode table? Each
> line in that file is already constructed in a way that it could be copied
> and pasted to the table directly (but I can change that if it makes things
> easier).
>

Interesting question.  Here is a partial answer.

If you can write a code block which reads the external file, and then
outputs the desired table, then the attached Org-mode file should work
for you.

The following defines a function (`continually-run') which re-runs a
named code block every couple of seconds.  Code blocks already handle
the updating of their results every time they are run.  See the file
itself for more information.  After testing this I will warn that it
caused some weirdness with my cursor in the mini-buffer.

Hope this helps,


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: code-block-rerunner.org --]
[-- Type: text/x-org, Size: 1508 bytes --]

#+Title: Periodically Rerun a Code Block

Here's the lisp code to define a function to continually re-run a code
block.  Evaluate the following code block, then =M-x continually-run=,
press ENTER and then type in the name of the code block to continually
run (in this case "date").
#+begin_src emacs-lisp :results silent
  (defvar continual-runners nil
    "Holds running block timers (so they may be canceled).")
  
  (defun run-block-in-buffer (name buffer)
    (save-match-data
      (with-current-buffer buffer
        (save-excursion
          (org-babel-goto-named-src-block name)
          (with-local-quit
            (undo-boundary)
            (with-temp-message (format "re-running %s" name)
              (org-babel-execute-src-block))
            (undo-boundary))))))
  
  (defun continually-run (name)
    "Continually run the supplied code block name.
  The code block is assumed to be in the current buffer."
    (interactive "scode block to continually run: ")
    (let ((run-buffer (current-buffer)))
      (add-to-list 'continual-runners
                   (run-at-time nil 5 'run-block-in-buffer name run-buffer))))
#+end_src

Here's the code block to be continually re-run.
#+Name: date
#+begin_src sh
  date
#+end_src

And here are the results which will be continually updated.
#+RESULTS: date
: Wed Jan  9 22:04:08 MST 2013

Execute the following to stop all continually updating code blocks.
#+begin_src emacs-lisp :results silent
  (mapc #'cancel-timer continual-runners)
#+end_src

[-- Attachment #3: Type: text/plain, Size: 46 bytes --]


-- 
Eric Schulte
http://cs.unm.edu/~eschulte

Re: How to add content from a file to a table automatically

[Darlan Cavalcante Moreira]

Thank you Eric and Jambunathan for your answers.

If I process the whole table with babel to manually add the lines from the
external file than I would have to take care of updating the formula for
the "Total" value as well as making sure the new content was added before
the "Total" line, right?. The later is simple, but updating the formula
could be complicated.

Meanwhile, I'm investigating using org-capture for this.

For instance, if I use a template such as
#+begin_src emacs-lisp
  ("f" "test" table-line
   (file+headline "test.org" "Test table")
   (file "path/food.org")
   :table-line-pos "II-1"
   :immediate-finish t)
#+end_src
where the food.org file has a single line then I can capture with
#+begin_src emacs-lisp
  (org-capture nil "f")'
#+end_src
and org-capture will place the content from the "food.org" file in a table
in the "Test Table" headline of the test.org file.

The advantage of using org-capture instead of pure babel is that it will
already put the content in the correct place and update the
formulas. However, the formulas are updated correctly only if I capture a
single line each time.

I think a combination of babel with org-capture is my best option.
Once that is working I will use your timer approach for automating this
(while I investigate inotify suggested by Jambunathan).

--
Darlan


At Wed, 09 Jan 2013 22:10:37 -0700,
Eric Schulte wrote:
> 
> #+Title: Periodically Rerun a Code Block
> 
> Here's the lisp code to define a function to continually re-run a code
> block.  Evaluate the following code block, then =M-x continually-run=,
> press ENTER and then type in the name of the code block to continually
> run (in this case "date").
> #+begin_src emacs-lisp :results silent
>   (defvar continual-runners nil
>     "Holds running block timers (so they may be canceled).")
>   
>   (defun run-block-in-buffer (name buffer)
>     (save-match-data
>       (with-current-buffer buffer
>         (save-excursion
>           (org-babel-goto-named-src-block name)
>           (with-local-quit
>             (undo-boundary)
>             (with-temp-message (format "re-running %s" name)
>               (org-babel-execute-src-block))
>             (undo-boundary))))))
>   
>   (defun continually-run (name)
>     "Continually run the supplied code block name.
>   The code block is assumed to be in the current buffer."
>     (interactive "scode block to continually run: ")
>     (let ((run-buffer (current-buffer)))
>       (add-to-list 'continual-runners
>                    (run-at-time nil 5 'run-block-in-buffer name run-buffer))))
> #+end_src
> 
> Here's the code block to be continually re-run.
> #+Name: date
> #+begin_src sh
>   date
> #+end_src
> 
> And here are the results which will be continually updated.
> #+RESULTS: date
> : Wed Jan  9 22:04:08 MST 2013
> 
> Execute the following to stop all continually updating code blocks.
> #+begin_src emacs-lisp :results silent
>   (mapc #'cancel-timer continual-runners)
> #+end_src

Re: How to add content from a file to a table automatically

[Eric Schulte]

Darlan Cavalcante Moreira <darcamo@gmail.com> writes:

> Thank you Eric and Jambunathan for your answers.
>
> If I process the whole table with babel to manually add the lines from the
> external file than I would have to take care of updating the formula for
> the "Total" value as well as making sure the new content was added before
> the "Total" line, right?. The later is simple, but updating the formula
> could be complicated.
>

Alternately, since the table is simply an array you could pass it to
another code block in your preferred language, and then perform the
total calculation on the array.  Then run `continually-run' on this
second code block.

>
> Meanwhile, I'm investigating using org-capture for this.
>

I'm not familiar with Org-capture myself, but the below sounds like a
good plan, I hope it works out.

If you do find a natural way to combine org-capture and Babel code
blocks I'm sure such a system would be of use to others on the list, so
please do share.

Best,

>
> For instance, if I use a template such as
> #+begin_src emacs-lisp
>   ("f" "test" table-line
>    (file+headline "test.org" "Test table")
>    (file "path/food.org")
>    :table-line-pos "II-1"
>    :immediate-finish t)
> #+end_src
> where the food.org file has a single line then I can capture with
> #+begin_src emacs-lisp
>   (org-capture nil "f")'
> #+end_src
> and org-capture will place the content from the "food.org" file in a table
> in the "Test Table" headline of the test.org file.
>
> The advantage of using org-capture instead of pure babel is that it will
> already put the content in the correct place and update the
> formulas. However, the formulas are updated correctly only if I capture a
> single line each time.
>
> I think a combination of babel with org-capture is my best option.
> Once that is working I will use your timer approach for automating this
> (while I investigate inotify suggested by Jambunathan).
>
> --
> Darlan
>
>
> At Wed, 09 Jan 2013 22:10:37 -0700,
> Eric Schulte wrote:
>> 
>> #+Title: Periodically Rerun a Code Block
>> 
>> Here's the lisp code to define a function to continually re-run a code
>> block.  Evaluate the following code block, then =M-x continually-run=,
>> press ENTER and then type in the name of the code block to continually
>> run (in this case "date").
>> #+begin_src emacs-lisp :results silent
>>   (defvar continual-runners nil
>>     "Holds running block timers (so they may be canceled).")
>>   
>>   (defun run-block-in-buffer (name buffer)
>>     (save-match-data
>>       (with-current-buffer buffer
>>         (save-excursion
>>           (org-babel-goto-named-src-block name)
>>           (with-local-quit
>>             (undo-boundary)
>>             (with-temp-message (format "re-running %s" name)
>>               (org-babel-execute-src-block))
>>             (undo-boundary))))))
>>   
>>   (defun continually-run (name)
>>     "Continually run the supplied code block name.
>>   The code block is assumed to be in the current buffer."
>>     (interactive "scode block to continually run: ")
>>     (let ((run-buffer (current-buffer)))
>>       (add-to-list 'continual-runners
>>                    (run-at-time nil 5 'run-block-in-buffer name run-buffer))))
>> #+end_src
>> 
>> Here's the code block to be continually re-run.
>> #+Name: date
>> #+begin_src sh
>>   date
>> #+end_src
>> 
>> And here are the results which will be continually updated.
>> #+RESULTS: date
>> : Wed Jan  9 22:04:08 MST 2013
>> 
>> Execute the following to stop all continually updating code blocks.
>> #+begin_src emacs-lisp :results silent
>>   (mapc #'cancel-timer continual-runners)
>> #+end_src

-- 
Eric Schulte
http://cs.unm.edu/~eschulte