* [BABEL] ob-rec.el and some questions
@ 2011-02-14 21:07 Jose E. Marchesi
2011-02-15 18:35 ` Eric Schulte
0 siblings, 1 reply; 5+ messages in thread
From: Jose E. Marchesi @ 2011-02-14 21:07 UTC (permalink / raw)
To: emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 1851 bytes --]
Hi.
I have been playing a bit with org babel, the goal being to be able to
query a recfile[1] and insert the result as a rec table. Thanks to the
superb design of org-babel and the existing examples, I quickly wrote
the attached little hack.
The hack allows to query a rec file 'foo.rec' as follows:
#+begin_src rec :data hackers.rec :fields Name,Email :type Hacker
Papers = 'requested' && CreatedAt << '01 January 2011'
#+end_src
Where :data points to the recfile (or list of files), :fields is a
comma-separated list of fields (allowing subscripts) and :type selects
the record set where to make the query. :cmdline can also be used to
specify any other command line option to recsel. All parameters but
:data are optional.
The selection expression can be any expression allowed by recsel in the
-e command line option.
Of course you have to install the recutils for it to work.
It roughly works and I want to add more functionality, but due to my
lack of experience with org-babel I would like to ask some questions
before continuing working on it.
- Is there a way to execute empty code blocks? It would be quite common
to require all the records stored in a record set, e.g.
#+begin_src rec :data inventory.rec
#+end_src
C-cC-c in that block echoes "Local setup has been refreshed" and
nothing happens. A workaround would be to use a selection expression
that always evaluates to "true", such as:
#+begin_src rec :data inventory.rec
1
#+end_src
but it would be nice to avoid it.
- Is there a way to dinamically change the value of the :results
parameter in the org-babel-execute:rec function? I would like to add
a :template parameter containing a template for recfmt, and in that
case the default value "raw" would not be appropriate.
Thanks in advance :)
[1] http://www.gnu.org/software/recutils
[-- Attachment #2: ob-rec.el --]
[-- Type: application/emacs-lisp, Size: 2025 bytes --]
[-- Attachment #3: Type: text/plain, Size: 80 bytes --]
--
Jose E. Marchesi jemarch@gnu.org
GNU Project http://www.gnu.org
[-- Attachment #4: Type: text/plain, Size: 201 bytes --]
_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [BABEL] ob-rec.el and some questions
2011-02-14 21:07 [BABEL] ob-rec.el and some questions Jose E. Marchesi
@ 2011-02-15 18:35 ` Eric Schulte
2011-02-15 19:29 ` Jose E. Marchesi
0 siblings, 1 reply; 5+ messages in thread
From: Eric Schulte @ 2011-02-15 18:35 UTC (permalink / raw)
To: Jose E. Marchesi; +Cc: emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 3172 bytes --]
jemarch@gnu.org (Jose E. Marchesi) writes:
> Hi.
>
> I have been playing a bit with org babel, the goal being to be able to
> query a recfile[1] and insert the result as a rec table. Thanks to the
> superb design of org-babel and the existing examples, I quickly wrote
> the attached little hack.
>
> The hack allows to query a rec file 'foo.rec' as follows:
>
> #+begin_src rec :data hackers.rec :fields Name,Email :type Hacker
> Papers = 'requested' && CreatedAt << '01 January 2011'
> #+end_src
>
Very cool! Now is the time when I ask, would you consider adding this
to the Org-mode core, which requires filling out the FSF copyright
assignment form? http://orgmode.org/worg/org-contribute.html#sec-2
>
> Where :data points to the recfile (or list of files), :fields is a
> comma-separated list of fields (allowing subscripts) and :type selects
> the record set where to make the query. :cmdline can also be used to
> specify any other command line option to recsel. All parameters but
> :data are optional.
>
> The selection expression can be any expression allowed by recsel in the
> -e command line option.
>
> Of course you have to install the recutils for it to work.
>
> It roughly works and I want to add more functionality, but due to my
> lack of experience with org-babel I would like to ask some questions
> before continuing working on it.
>
> - Is there a way to execute empty code blocks? It would be quite common
> to require all the records stored in a record set, e.g.
>
> #+begin_src rec :data inventory.rec
> #+end_src
>
> C-cC-c in that block echoes "Local setup has been refreshed" and
> nothing happens. A workaround would be to use a selection expression
> that always evaluates to "true", such as:
>
With my attached version of ob-rec.el it is possible to have a mostly
empty body as follows, however, currently `org-babel-get-src-block-info'
*does* require that there be at least 1 character as well as a newline
in the body. This could be changed, however I'd want to do some local
testing before pushing up such a change, as I fear it may break existing
code blocks.
#+begin_src rec :data book.rec :fields Title,Author
#+end_src
#+results:
| Title...
| GNU Emacs Manual...
>
> #+begin_src rec :data inventory.rec
> 1
> #+end_src
>
> but it would be nice to avoid it.
>
> - Is there a way to dinamically change the value of the :results
> parameter in the org-babel-execute:rec function? I would like to add
> a :template parameter containing a template for recfmt, and in that
> case the default value "raw" would not be appropriate.
>
Please see my attached revision of ob-rec.el. I do not believe that the
default value of raw is appropriate currently as Babel should know when
the returned results are a table. I've slightly modified your existing
code so that raw is no longer the default value, and so that tables are
passed back as tables. Does this change subsume your format question?
>
Also, it sounds like there is currently only support for selecting
records *from* a .rec file, do you plan on adding support for inserting
records *into* a .rec file?
Thanks for sharing! -- Eric
[-- Attachment #2: ob-rec.el --]
[-- Type: application/emacs-lisp, Size: 2319 bytes --]
[-- Attachment #3: Type: text/plain, Size: 71 bytes --]
>
> Thanks in advance :)
>
> [1] http://www.gnu.org/software/recutils
[-- Attachment #4: Type: text/plain, Size: 201 bytes --]
_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [BABEL] ob-rec.el and some questions
2011-02-15 18:35 ` Eric Schulte
@ 2011-02-15 19:29 ` Jose E. Marchesi
2011-02-15 20:35 ` Eric Schulte
0 siblings, 1 reply; 5+ messages in thread
From: Jose E. Marchesi @ 2011-02-15 19:29 UTC (permalink / raw)
To: Eric Schulte; +Cc: emacs-orgmode
> #+begin_src rec :data hackers.rec :fields Name,Email :type Hacker
> Papers = 'requested' && CreatedAt << '01 January 2011'
> #+end_src
>
Very cool! Now is the time when I ask, would you consider adding this
to the Org-mode core, which requires filling out the FSF copyright
assignment form? http://orgmode.org/worg/org-contribute.html#sec-2
Sure. I assigned future changes to Emacs some years ago.
> C-cC-c in that block echoes "Local setup has been refreshed" and
> nothing happens. A workaround would be to use a selection expression
> that always evaluates to "true", such as:
>
With my attached version of ob-rec.el it is possible to have a
mostly empty body as follows, however, currently
`org-babel-get-src-block-info' *does* require that there be at least
1 character as well as a newline in the body. This could be
changed, however I'd want to do some local testing before pushing up
such a change, as I fear it may break existing code blocks.
That looks like a reasonable workaround.
> - Is there a way to dinamically change the value of the :results
> parameter in the org-babel-execute:rec function? I would like to add
> a :template parameter containing a template for recfmt, and in that
> case the default value "raw" would not be appropriate.
>
Please see my attached revision of ob-rec.el. I do not believe that the
default value of raw is appropriate currently as Babel should know when
the returned results are a table. I've slightly modified your existing
code so that raw is no longer the default value, and so that tables are
passed back as tables. Does this change subsume your format
question?
Yes, now it is clear: I have to adapt the returned data to the
result-params settings, and not the other way around.
Thanks for the fixes :)
Also, it sounds like there is currently only support for selecting
records *from* a .rec file, do you plan on adding support for inserting
records *into* a .rec file?
Extracting the records from org tables or raw rec data, you mean? It
would be easy to use csv2rec and recins to implement a recins block:
#+begin_src recins :data foo.rec
| Title | Author |
| GNU Emacs Manual | Richard M. Stallman |
...
#+end_src
or
#+begin_src recins :data foo.rec
%rec: Book
...
Title: GNU Emacs Manual
Author: Richard M. Stallman
...
#+end_src
By evaluating all the code blocks in the file in a sequential way,
additional "rec" blocks could then be used to make (and publish)
selections of the table inserted above:
#+begin_src rec :data foo.rec
some selection expression
#+end_src
#+results
| Title | Author |
| ... | ... |
That sounds like fun! org-mode could then be used as a GUI for recutils
:D
--
Jose E. Marchesi jemarch@gnu.org
GNU Project http://www.gnu.org
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [BABEL] ob-rec.el and some questions
2011-02-15 19:29 ` Jose E. Marchesi
@ 2011-02-15 20:35 ` Eric Schulte
2011-02-15 21:05 ` Jose E. Marchesi
0 siblings, 1 reply; 5+ messages in thread
From: Eric Schulte @ 2011-02-15 20:35 UTC (permalink / raw)
To: Jose E. Marchesi; +Cc: emacs-orgmode
jemarch@gnu.org (Jose E. Marchesi) writes:
> > #+begin_src rec :data hackers.rec :fields Name,Email :type Hacker
> > Papers = 'requested' && CreatedAt << '01 January 2011'
> > #+end_src
> >
>
> Very cool! Now is the time when I ask, would you consider adding this
> to the Org-mode core, which requires filling out the FSF copyright
> assignment form? http://orgmode.org/worg/org-contribute.html#sec-2
>
> Sure. I assigned future changes to Emacs some years ago.
>
Ah, I should have noticed the @gnu in your email address. That's great,
once this file settles down I'll add it to the repository.
>
> > C-cC-c in that block echoes "Local setup has been refreshed" and
> > nothing happens. A workaround would be to use a selection expression
> > that always evaluates to "true", such as:
> >
>
> With my attached version of ob-rec.el it is possible to have a
> mostly empty body as follows, however, currently
> `org-babel-get-src-block-info' *does* require that there be at least
> 1 character as well as a newline in the body. This could be
> changed, however I'd want to do some local testing before pushing up
> such a change, as I fear it may break existing code blocks.
>
> That looks like a reasonable workaround.
>
> > - Is there a way to dinamically change the value of the :results
> > parameter in the org-babel-execute:rec function? I would like to add
> > a :template parameter containing a template for recfmt, and in that
> > case the default value "raw" would not be appropriate.
> >
>
> Please see my attached revision of ob-rec.el. I do not believe that the
> default value of raw is appropriate currently as Babel should know when
> the returned results are a table. I've slightly modified your existing
> code so that raw is no longer the default value, and so that tables are
> passed back as tables. Does this change subsume your format
> question?
>
> Yes, now it is clear: I have to adapt the returned data to the
> result-params settings, and not the other way around.
>
well put
>
> Thanks for the fixes :)
>
> Also, it sounds like there is currently only support for selecting
> records *from* a .rec file, do you plan on adding support for inserting
> records *into* a .rec file?
>
> Extracting the records from org tables or raw rec data, you mean? It
> would be easy to use csv2rec and recins to implement a recins block:
>
Yes, I was thinking of something like the following. Rather than create
a new block type, simply add a header argument (or body content) that
(somehow) indicates that we are writing not reading. So for example
writing an Org-mode table into a fresh .rec file would look like the
following.
#+tblname: org-data
| Foo | Bar |
|-----+-----|
| 1 | a |
| 2 | b |
| 3 | c |
| 4 | d |
#+begin_src rec :data something.rec :var data=org-data :write
some rec code to dump data into the rec file...
#+end_src
[...]
>
> By evaluating all the code blocks in the file in a sequential way,
> additional "rec" blocks could then be used to make (and publish)
> selections of the table inserted above:
>
Yes, and once two-way movement of data is possible, then other languages
could be used to update the contents of column in rec-files from within
an Org-mode document. Running an arbitrary python function over the
"Foo" column of a .rec file and saving the results should be as easy as
evaluating the write-foos code block below.
1. Select Foo for all records
#+source: all-foos
#+begin_src rec :data something.rec :fields Foo
1
#+end_src
2. Run all foos through some arbitrary processing in any Babel language
#+source: process-foos
#+begin_src python :var foos=all-foos
do something to foos
#+end_src
3. Save the processed values back into the foos column of the original
rec file
#+source: write-foos
#+begin_src rec :data something.rec :fields Foo :write :var foos=process-foos
insert foos
#+end_src
Hope this makes sense. Best -- Eric
>
> #+begin_src rec :data foo.rec
> some selection expression
> #+end_src
>
> #+results
> | Title | Author |
> | ... | ... |
>
>
> That sounds like fun! org-mode could then be used as a GUI for recutils
> :D
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [BABEL] ob-rec.el and some questions
2011-02-15 20:35 ` Eric Schulte
@ 2011-02-15 21:05 ` Jose E. Marchesi
0 siblings, 0 replies; 5+ messages in thread
From: Jose E. Marchesi @ 2011-02-15 21:05 UTC (permalink / raw)
To: Eric Schulte; +Cc: emacs-orgmode
> Extracting the records from org tables or raw rec data, you mean? It
> would be easy to use csv2rec and recins to implement a recins block:
>
Yes, I was thinking of something like the following. Rather than create
a new block type, simply add a header argument (or body content) that
(somehow) indicates that we are writing not reading. So for example
writing an Org-mode table into a fresh .rec file would look like the
following.
#+tblname: org-data
| Foo | Bar |
|-----+-----|
| 1 | a |
| 2 | b |
| 3 | c |
| 4 | d |
#+begin_src rec :data something.rec :var data=org-data :write
some rec code to dump data into the rec file...
#+end_src
I like that approach, but I would put the raw rec code into the src
block:
#+begin_src rec :data something.rec :write
Foo: 1
Bar: a
Foo: 2
Bar: b
...
#+end_src
Then use a :table parameter in case we want to insert the contents of a
table:
#+begin_src rec :data something.rec :write :table org-data
#+end_src
I will work on this and will be back as soon as I have something
working. Many thanks for the feedback :)
--
Jose E. Marchesi jemarch@gnu.org
GNU Project http://www.gnu.org
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2011-02-15 21:10 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-02-14 21:07 [BABEL] ob-rec.el and some questions Jose E. Marchesi
2011-02-15 18:35 ` Eric Schulte
2011-02-15 19:29 ` Jose E. Marchesi
2011-02-15 20:35 ` Eric Schulte
2011-02-15 21:05 ` Jose E. Marchesi
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).