From mboxrd@z Thu Jan 1 00:00:00 1970 From: Marc-Oliver Ihm Subject: [babel] Code for simple set-operations on two tables. Asking for some input. Date: Mon, 26 Dec 2011 12:00:49 +0100 Message-ID: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------070409010702090309020005" Return-path: Received: from eggs.gnu.org ([140.186.70.92]:56575) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Rf8Il-0003oK-Bj for emacs-orgmode@gnu.org; Mon, 26 Dec 2011 06:01:08 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Rf8Ij-0001f5-Kr for emacs-orgmode@gnu.org; Mon, 26 Dec 2011 06:01:07 -0500 Received: from lo.gmane.org ([80.91.229.12]:41220) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Rf8Ij-0001f0-6N for emacs-orgmode@gnu.org; Mon, 26 Dec 2011 06:01:05 -0500 Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Rf8Ih-0002uT-JU for emacs-orgmode@gnu.org; Mon, 26 Dec 2011 12:01:03 +0100 Received: from p54a88662.dip0.t-ipconnect.de ([84.168.134.98]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 26 Dec 2011 12:01:03 +0100 Received: from marc-oliver.ihm by p54a88662.dip0.t-ipconnect.de with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 26 Dec 2011 12:01:03 +0100 List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: emacs-orgmode@gnu.org This is a multi-part message in MIME format. --------------070409010702090309020005 Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit Hello, Please find attached the elisp-file and a tutorial (as an org-file) for org-babel-table-proc. It provides some simple set-operations (mostly merge and intersect), which treat org-mode tables as sets. An example for merging two tables would be: > #+name: lower > | 2 | b | > | 4 | d | > | 5 | e | > | 6 | h | > > #+name: upper > | 1 | A | > | 3 | C | > | 4 | D | > | 10 | J | > | 2 | B | > > #+begin_src emacs-lisp :var t1=lower :var t2=upper > (babel-table-proc-merge t1 t2) > #+end_src > > #+results: > | 1 | | A | > | 2 | b | B | > | 3 | | C | > | 4 | d | D | > | 5 | e | | > | 6 | h | | > | 10 | | J | which merges the two input tables lower and upper into a single table, even, if they do not have all keys (from the first column) in common. In general this merges information from two different sources without loosing anything. The package org-babel-table-proc provides these and a few other operations, but definitely still needs some debugging and polishing. However, before finishing this work, I would like to ask, if something like this is already present in babel or the library of babel ? Also, I am not sure, which prefix to use for packages and functions; is org-babel-table-proc the right name or can it be shortened to ob-table-proc. Any input would be very welcome ! With kind regards, Marc-Oliver Ihm --------------070409010702090309020005 Content-Type: text/plain; name="org-babel-table-proc.el" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="org-babel-table-proc.el" ;;; org-babel-table-proc.el --- Common operations on tables for use with orgmode and lob ;; Copyright (C) 2010-2011 ;; Free Software Foundation, Inc. ;; Author: Marc-Oliver Ihm ;; Keywords: tables library of babel orgmode ;; Version: 0.01 ;;; License: ;; This program is free software; you can redistribute it and/or modify ;; it under the terms of the GNU General Public License as published by ;; the Free Software Foundation; either version 3, or (at your option) ;; any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; ;; You should have received a copy of the GNU General Public License ;; along with GNU Emacs; see the file COPYING. If not, write to the ;; Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, ;; Boston, MA 02110-1301, USA. ;;; Commentary: ;; Purpose: ;; ;; Common operations on tables for use with orgmode and lob: ;; - simple set operations (intersection and merge) ;; - filtering one table according to another ;; ;; ;; Setup: ;; ;; (require 'org-babel-table-proc) ;; ;; Further reading: ;; ;; See the file org-babel-table-proc.org for complete examples. ;; ;;; Code: (defun babel-table-proc-keep (t1 t2) "Keep only those keys from the second table, that appear within the first" (lob-tbl-filter 'keep t1 t2) ) (defun babel-table-proc-remove (t1 t2) "Remove those keys from the second table, that appear within the first" (lob-tbl-filter 'remove t1 t2) ) (defun lob-tbl-filter (what t1 t2) "Internal function to do the work of babel-table-proc-keep and -remove" (let (keys result) (setq keys (mapcar 'car t1)) (dolist (line t2) (if (equal (member (car line) keys) (equal what 'keep)) (setq result (cons line result)) ) ) (nreverse result) ) ) (defun babel-table-proc-merge (&rest tables) "Merge two tables by first column; sort the result" (babel-table-proc-two-tables-to-one 'merge tables)) (defun babel-table-proc-intersect (&rest tables) "Intersect two tables by first column; sort the result" (babel-table-proc-two-tables-to-one 'intersect tables)) (defun babel-table-proc-two-tables-to-one (what tables) "Internal function to do the work of babel-table-proc-merge and -intersect" (let (is-all-numbers less-than-function equal-function conversion-function format-specifier rests-of-tables rest-of-rests-of-tables rest-of-table widths-of-tables current-key current-key-in-intersection result-table result-line i) ;; Find out, if all keys in all tables are numbers or if there are strings among them (setq is-all-numbers (catch 'not-a-number (dolist (table tables) (dolist (line table) (unless (numberp (car line)) (throw 'not-a-number 'nil)))) 't)) ;; prepare functions to treat table contents in a unified way (setq format-specifier (if is-all-numbers "%g" "%s")) (setq conversion-function (if is-all-numbers (lambda (x) x) (lambda (x) (if (numberp x) (number-to-string x) x)) )) (setq less-than-function (lambda (x y) (if is-all-numbers (< x y) (string< (funcall conversion-function x) (funcall conversion-function y))))) (setq equal-function (lambda (x y) (if is-all-numbers (= x y) (string= (funcall conversion-function x) (funcall conversion-function y))))) ;; sort tables (setq tables (mapcar (lambda (table) (sort table (lambda (x y) (funcall less-than-function (car x) (car y))))) tables)) ;; compute and remember table widths (setq widths-of-tables (mapcar (lambda (x) (length (car x))) tables)) (setq rests-of-tables (copy-list tables)) ;; loop as long as the rest of table still contains lines (while (progn ;; find lowest key among all tables, which is the key for the next line of the result (setq current-key nil) (setq current-key-in-intersection 't) (dolist (rest-of-table rests-of-tables) (when (and rest-of-table (or (null current-key) (funcall less-than-function (caar rest-of-table) current-key))) (setq current-key (caar rest-of-table)))) current-key) (progn (setq result-line (list current-key)) ;; go through all tables and collect one line for the result table ... (setq i 0) ; table-count ;; cannot use dolist like above, because we need to modify the cons-cells (setq rest-of-rests-of-tables rests-of-tables) (while (progn (setq rest-of-table (car rest-of-rests-of-tables)) (setq i (1+ i)) ;; if table contains current key (if (and rest-of-table (funcall equal-function current-key (caar rest-of-table))) ;; then copy rest of line (progn (nconc result-line (cdar rest-of-table)) ;; and shorten rest (setcar rest-of-rests-of-tables (cdar rest-of-rests-of-tables)) ;; and check, if current-key appears again (when (and (caadr rest-of-table) (funcall equal-function current-key (caadr rest-of-table)) ) (error (concat "Key '" format-specifier "'appears twice within input table %i") (funcall conversion-function current-key) i) ) ) ;; otherwise fill with nil and do not shorten rest of table (progn (setq current-key-in-intersection nil) (nconc result-line (make-list (1- (elt widths-of-tables (1- i))) "")) ) ) (setq rest-of-rests-of-tables (cdr rest-of-rests-of-tables)) ;; condition for while-loop rest-of-rests-of-tables ) ) (if (or (eq what 'merge) current-key-in-intersection) (setq result-table (cons result-line result-table)) ; store away line ) ) ) (nreverse result-table) ) ) (provide 'org-babel-table-proc) --------------070409010702090309020005 Content-Type: text/plain; name="org-babel-table-proc.org" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="org-babel-table-proc.org" * Introduction This file (org-babel-table-proc.org) contains the documentation and living examples for org-babel-table-proc.el, which is a libary of functions, which in turn makes it easier to process tables with babel, which is part of orgmode. The functions within this library take two tables and apply simple filter or set operations on them. The first column within each table plays the role of a key, which determines how the table-lines is treated; the remaining columns are payload. * Installation To install the library org-babel-table-proc.el within emacs you need to do two things: First: Move the file into any directory within your emacs load-path (see [[id:b3ec5718-8a32-4b1e-8203-f620aa819d82][Finding out, what your load-path is]] for details) Second: Put the following line (require 'org-babel-table-proc) into your emacs-startup file (".emacs" in most cases). For now, it is even enough to execute the following source-code-block (see [[id:170aa925-e8e1-4937-9ed7-c889b83e8419][Executing code blocks]]) #+begin_src emacs-lisp (require 'org-babel-table-proc) #+end_src * Some input tables, that are used within the examples For the examples in the rest of this document, we will use two input tables: "lower" and "upper" (see below). Each of these tables associates a position within the alphabet with a letter (either lower or upper case). For example the table line "| 2 | b |" below simply expresses the fact, that the letter "b" comes at position 2 within the alphabet. #+name: lower | 2 | b | | 4 | d | | 5 | e | | 6 | h | #+name: upper | 1 | A | | 3 | C | | 4 | D | | 10 | J | | 2 | B | Please note further, that those tables have names (given by the line "#+name: lower"). A third table, named "keys", has only one column, which will be used as keys to select rows from the other tables. #+name: keys | 1 | | 2 | | 4 | * Set operations #+begin_src emacs-lisp :var t1=lower :var t2=upper (babel-table-proc-merge t1 t2) #+end_src #+results: | 1 | | A | | 2 | b | B | | 3 | | C | | 4 | d | D | | 5 | e | | | 6 | h | | | 10 | | J | #+begin_src emacs-lisp :var t1=keys :var t2=upper (babel-table-proc-remove t1 t2) #+end_src #+results: | 3 | C | | 10 | J | * Appendices ** Executing code blocks :PROPERTIES: :ID: 170aa925-e8e1-4937-9ed7-c889b83e8419 :END: Throughout this document you will be urged to execute code blocks. This refers to orgmode's feature of beeing able to run code that is embedded within the document; such code blocks begin with a line "#+begin_src" and end with "#+end_src". Throughout this document you will find many such code blocks, just see below for an example. For a complete description of the many great things, that can be done with such code blocks, please refer to the orgmode manual, chapter 14 ("Working with source code"). For the moment however, it is enough to know, how to execute those blocks: Just move the cursor to the line below (the one starting with "#+begin_src") and press C-c C-c (this means: Hold down the control-key an press the key "c" twice). #+begin_src emacs-lisp (current-time-string) #+end_src The result of executing this block of code can be seen above, following the line starting with "#+results:". It is simply the current time. If you execute this code block again, the old result is overwritten with a new timestamp. If you want to remove the whole result to restore the "clean" state of this buffer, please check out [[id:224b0c4a-6b2c-495c-afc1-8a52c9dea778][Cleaning up results]]. For the purpose of this document, you do not need to know much more about executing code blocks and you may read on. ** Finding out, what your load-path is :PROPERTIES: :ID: b3ec5718-8a32-4b1e-8203-f620aa819d82 :END: #+begin_src emacs-lisp (mapcar 'list load-path) #+end_src ** Cleaning up results :PROPERTIES: :ID: 224b0c4a-6b2c-495c-afc1-8a52c9dea778 :END: Execute the code block below (see [[id:170aa925-e8e1-4937-9ed7-c889b83e8419][Executing code blocks]]) to clear any results that have been generated previously from executing other code blocks within this document. However, as each execution of a code block overwrites any results of previous executions, there is no real need to clean up anything. If at all, than only to tidy up the file, before passing it along. #+begin_src emacs-lisp (save-excursion (beginning-of-buffer) (while (re-search-forward "^#\\+results:\n\\(^\|\\|:.*\n\\)*\n" nil t) (replace-match "")) ) #+end_src #+results: Note, that the only remaining result (introduced by the line "#+results:") ist the one immediately above. --------------070409010702090309020005--