[Babel] Handling of errors when using Ledger

[Babel] Handling of errors when using Ledger

[Sébastien Vauban]

#+TITLE:     Handling of errors when using Ledger
#+LANGUAGE:  en_US

* Journal data

#+srcname: data
#+begin_src ledger :tangle journal.dat
2008/01/03 * (SCORPIOS ) SEB VAUBAN
	Assets:Bank:Checking:799997400530	                        550.00 EUR
	Assets:Bank:Transferred

2008/01/01 * ( ) UNKNOWN-PAYEE
	Assets:Bank:Checking:799997400530	                         21.91 EUR
	Expenses:Unknown
#+end_src

* Testing

** Default case

Here the results of the standards =registry= command:

#+srcname: registry-cmd
#+begin_src ledger :cmdline reg :noweb yes :session
<<data>>
#+end_src

#+results: registry-cmd
: 08-Jan-03 SEB VAUBAN            As:Ba:Che:799997400530   550.00 EUR   550.00 EUR
:                                 Asset:Bank:Transferred  -550.00 EUR            0
: 08-Jan-01 UNKNOWN-PAYEE         As:Ba:Che:799997400530    21.91 EUR    21.91 EUR
:                                 Expenses:Unknown         -21.91 EUR            0

This is a perfectly acceptable output. Ideally, it could be converted to a
real Org table, but that's not the purpose of this posting.

** Other default case

Here, I would like to take a look at the transactions which involve the
unknown account:

#+srcname: just-show-unknown
#+begin_src ledger :cmdline reg unknown :noweb yes :session
<<data>>
#+end_src

#+results: just-show-unknown
: 08-Jan-01 UNKNOWN-PAYEE         Expenses:Unknown         -21.91 EUR   -21.91 EUR

Perfect result. That will serve as "demo".

** No output
   :PROPERTIES:
   :ID:       52aa2449-0c3d-4dee-ad57-8b2e916ed501
   :END:

#+srcname: no-output
#+begin_src ledger :cmdline reg unkXXXnown :noweb yes :session
<<data>>
#+end_src

#+results: no-output

** Error neither trapped nor shown

Let's imagine I thought (which was the case at some point) I needed to enclose
the parameters between quotes:

#+srcname: quoted-params
#+begin_src ledger :cmdline "reg unknown" :noweb yes :session
<<data>>
#+end_src

#+results: quoted-params

Nothing is returned. In fact, I would expect an error to be thrown, the same
way it should when run in a shell:

#+begin_src sh
ledger -f journal.dat "reg unknown"
#+end_src

that is:

: ~>ledger -f journal.dat "reg unknown"
: Error: Unrecognized command 'reg unknown'
: 
: ~>echo $?
: 1

Here, the shown results is exactly the same as in the [[id:52aa2449-0c3d-4dee-ad57-8b2e916ed501][No output]] case. As if
the command ended successfully, with an empty results set...

* Observations and suggestions

I don't know if this is a common problem (to Org-Babel) or only to the Ledger
part of it, but I think *we* should somehow improve the handling of errors.

- Maybe displaying a =#+results-err= block which would be what's shown on
  =/dev/stderr=, when not void?

- And having a way to display the error code would be a plus.

Best regards,
  Seb

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: [Babel] Handling of errors when using Ledger

[Dan Davison]

Sébastien Vauban <wxhgmqzgwmuf-geNee64TY+gS+FvcfC7Uqw@public.gmane.org>
writes:
[...]
> Let's imagine I thought (which was the case at some point) I needed to enclose
> the parameters between quotes:
>
> #+srcname: quoted-params
> #+begin_src ledger :cmdline "reg unknown" :noweb yes :session
> <<data>>
> #+end_src
>
> #+results: quoted-params
>
> Nothing is returned. In fact, I would expect an error to be thrown, the same
[...]
> I don't know if this is a common problem (to Org-Babel) or only to the Ledger
> part of it, but I think *we* should somehow improve the handling of errors.

Hi Seb,

Babel has a standard mechanism for evaluating shell commands and
displaying errors if any. It is the function `org-babel-eval' in
ob-eval.el. The problem is that ob-ledger is not using this
mechanism. Would you be interested in fixing this? Basically what is
required is to re-implement `org-babel-execute:ledger' using
`org-babel-eval'. (There are plenty of examples in the other langauges
to follow.) Please don't worry if you are too busy though.

> - Maybe displaying a =#+results-err= block which would be what's shown on
>   =/dev/stderr=, when not void?

I've vaguely wondered about this sort of thing in the past. The thing is
that that's getting close to the idea of proper exception handling in
Org-babel. That would certainly be interesting, and I'm sure we would
welcome well thought-through proposals on the topic. It would need to
deal with errors occurring in a block anywhere in the `call tree'
(e.g. what happens when block A is evaluated and A references blocks B
and C, and B references D and an error occurs in D)

> - And having a way to display the error code would be a plus.

That will happen automatically when ledger is converted to use
`org-babel-eval'.

Dan

>
> Best regards,
>   Seb

Re: [Babel] Handling of errors when using Ledger

[Sébastien Vauban]

Hi Dan,

Dan Davison wrote:
> Sébastien Vauban writes:
> [...]
>> Let's imagine I thought (which was the case at some point) I needed to
>> enclose the parameters between quotes:
>>
>> #+srcname: quoted-params
>> #+begin_src ledger :cmdline "reg unknown" :noweb yes :session
>> <<data>>
>> #+end_src
>>
>> #+results: quoted-params
>>
>> Nothing is returned. In fact, I would expect an error to be thrown [...] I
>> don't know if this is a common problem (to Org-Babel) or only to the Ledger
>> part of it, but I think *we* should somehow improve the handling of errors.
>
> Babel has a standard mechanism for evaluating shell commands and displaying
> errors if any. It is the function `org-babel-eval' in ob-eval.el. The
> problem is that ob-ledger is not using this mechanism. Would you be
> interested in fixing this? Basically what is required is to re-implement
> `org-babel-execute:ledger' using `org-babel-eval'. (There are plenty of
> examples in the other langauges to follow.) Please don't worry if you are
> too busy though.

I'd be interested to try and fix it. I am busy, yes: I am just recovering 3
days of lost time due to my 5 years old PC which crashed on Friday morning. I
now temporarily am on a Windows; but I've had to tweak my Emacs and Gnus a
bit, to recover all the backups done through Unison, and experience all the
problems due to differences in filenames (characters allowed, case
insensitiveness, etc.). Apart from that, we're rebuilding my house, and that
demands a lot of time from me. So, yes, time is a scarce resource, but I'm
sure everybody can say so.

So, I'd propose to fight for being the first one to fix that... And let's see
who will win... ;-)


>> - Maybe displaying a =#+results-err= block which would be what's shown on
>>   =/dev/stderr=, when not void?
>
> I've vaguely wondered about this sort of thing in the past. The thing is
> that that's getting close to the idea of proper exception handling in
> Org-babel. That would certainly be interesting, and I'm sure we would
> welcome well thought-through proposals on the topic.

Quite sad nobody else reacts on this.


> It would need to deal with errors occurring in a block anywhere in the `call
> tree' (e.g. what happens when block A is evaluated and A references blocks B
> and C, and B references D and an error occurs in D)

I don't this we need to go in such a complicated path. After all, if I launch
a script with calls to external commands, I only will see that my parent
script has gotten an error (if checked, even). I never will see exactly which
subcommand can have failed (or only through side-effects: messages with a
clear indication of which program is outputting something on the console).

Plus, I could use D in both A and K. Maybe D succeeds when called from K and
not when called from A. So, when getting an error, I think it is sufficient to
see it associated to the parent block. It's inside it that the error occurs,
after all.


>> - And having a way to display the error code would be a plus.
>
> That will happen automatically when ledger is converted to use
> `org-babel-eval'.

OK. Thanks for your explanations.

Best regards,
  Seb

PS- BTW, I had a very bad habit of letting the important mails as "unread".
Now that I've crashed my PC, I've lost all my mail marks as well... Should I
have used links from Gnus to the important mails...

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: [Babel] Handling of errors when using Ledger

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 1664 bytes --]

On Tue, 12 Oct 2010 21:58:46 +0200, Sébastien Vauban <wxhgmqzgwmuf@spammotel.com> wrote:
> 
> Hi Dan,
> 
> Dan Davison wrote:
> > Sébastien Vauban writes:
> > [...]
> >> Let's imagine I thought (which was the case at some point) I needed to
> >> enclose the parameters between quotes:
> >>
> >> #+srcname: quoted-params
> >> #+begin_src ledger :cmdline "reg unknown" :noweb yes :session
> >> <<data>>
> >> #+end_src
> >>
> >> #+results: quoted-params
> >>
> >> Nothing is returned. In fact, I would expect an error to be thrown [...] I
> >> don't know if this is a common problem (to Org-Babel) or only to the Ledger
> >> part of it, but I think *we* should somehow improve the handling of errors.
> >
> > Babel has a standard mechanism for evaluating shell commands and displaying
> > errors if any. It is the function `org-babel-eval' in ob-eval.el. The
> > problem is that ob-ledger is not using this mechanism. Would you be
> > interested in fixing this? Basically what is required is to re-implement
> > `org-babel-execute:ledger' using `org-babel-eval'. (There are plenty of
> > examples in the other langauges to follow.) Please don't worry if you are
> > too busy though.
> 
> I'd be interested to try and fix it. I am busy, yes: I am just recovering 3

[...]

> So, I'd propose to fight for being the first one to fix that... And let's see
> who will win... ;-)

Please do!!  As the original author (basing my code on other babel
codes, mind you), I am happy to lose this fight ;-)  I really don't
know enough babel to make the changes required quickly and I also am
struggling with "free" time at the moment.


[-- Attachment #2: Type: text/plain, Size: 75 bytes --]

-- 
Eric S Fraga
GnuPG: 8F5C 279D 3907 E14A 5C29  570D C891 93D8 FFFC F67D

[-- Attachment #3: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: [Babel] Handling of errors when using Ledger

[Dan]

Eric S Fraga <ucecesf <at> ucl.ac.uk> writes:

> 
> On Tue, 12 Oct 2010 21:58:46 +0200, Sébastien Vauban <wxhgmqzgwmuf <at>
spammotel.com> wrote:
> > 
> > Hi Dan,
> > 
> > Dan Davison wrote:
> > > Sébastien Vauban writes:
> > > [...]
> > >> Let's imagine I thought (which was the case at some point) I needed to
> > >> enclose the parameters between quotes:
> > >>
> > >> #+srcname: quoted-params
> > >> #+begin_src ledger :cmdline "reg unknown" :noweb yes :session
> > >> <<data>>
> > >> #+end_src
> > >>
> > >> #+results: quoted-params
> > >>
> > >> Nothing is returned. In fact, I would expect an error to be thrown [...] 

[...]

> > > Babel has a standard mechanism for evaluating shell commands and displaying
> > > errors if any. It is the function `org-babel-eval' in ob-eval.el. The
> > > problem is that ob-ledger is not using this mechanism.

[...]

This patch should make ob-ledger use the common org-babel error mechanism. It is
in branch ledger of the main repo. However, I'm not yet a ledger user. Could
someone test it?

Incidentally, can any gnus users tell me -- is there a non-painful way of
retrieving an old message from nntp in order to create a reply in the correct
thread? Or do you hack the headers to make it appear in the right thread? I'm
using the gmane web interface to send this.

Thanks,

Dan

----------------------------------------------------------
diff --git a/lisp/ob-ledger.el b/lisp/ob-ledger.el
index 33ec9d3..f06ea5d 100644
--- a/lisp/ob-ledger.el
+++ b/lisp/ob-ledger.el
@@ -43,24 +43,15 @@
   '((:results . "output") (:cmdline . "bal"))
   "Default arguments to use when evaluating a ledger source block.")
 
+(defvar org-babel-ledger-command "ledger"
+  "Command to invoke ledger")
+
 (defun org-babel-execute:ledger (body params)
   "Execute a block of Ledger entries with org-babel.  This function is
 called by `org-babel-execute-src-block'."
-  (message "executing Ledger source code block")
-  (let ((result-params (split-string (or (cdr (assoc :results params)) "")))
-       (cmdline (cdr (assoc :cmdline params)))
-        (in-file (org-babel-temp-file "ledger-"))
-       (out-file (org-babel-temp-file "ledger-output-")))
-    (with-temp-file in-file (insert body))
-    (message (concat "ledger"
-                    " -f " (org-babel-process-file-name in-file)
-                    " " cmdline))
-    (with-output-to-string
-      (shell-command (concat "ledger"
-                            " -f " (org-babel-process-file-name in-file)
-                            " " cmdline
-                            " > " (org-babel-process-file-name out-file))))
-    (with-temp-buffer (insert-file-contents out-file) (buffer-string))))
+  (org-babel-eval
+   (concat org-babel-ledger-command " " (cdr (assoc :cmdline params)))
+   body))
 
 (defun org-babel-prep-session:ledger (session params)
   (error "Ledger does not support sessions"))
----------------------------------------------------------

> 
> Please do!!  As the original author (basing my code on other babel
> codes, mind you), I am happy to lose this fight   I really don't
> know enough babel to make the changes required quickly and I also am
> struggling with "free" time at the moment.
> 
> 
> 

Re: [Babel] Handling of errors when using Ledger

[Bernt Hansen]

Dan <dandavison7@gmail.com> writes:

> Incidentally, can any gnus users tell me -- is there a non-painful way of
> retrieving an old message from nntp in order to create a reply in the correct
> thread? Or do you hack the headers to make it appear in the right thread? I'm
> using the gmane web interface to send this.

I read the list through gnus and gmane.  When I'm in the summary buffer
for the org-mode list I can retrieve old messages with

   / O 500 RET

to get the last 500 messages from the list.
Then you can limit the displayed messages by some filter (say subject) with

  / / <filter text> RET

On a message ^ will retrieve the previous message in the thread and you
can do that repeatedly to get the entire thread displayed.

Then I just reply normally to a specific message with
  S W

and It Just Works(tm)

HTH,
Bernt

Re: [Babel] Handling of errors when using Ledger

[Sébastien Vauban]

Hi Dan,

Dan wrote:
> Eric S Fraga <ucecesf <at> ucl.ac.uk> writes:
>> On Tue, 12 Oct 2010 21:58:46 +0200, Sébastien Vauban wrote:
>> > Dan Davison wrote:
>> > > Sébastien Vauban writes:
>> > >> Let's imagine I thought (which was the case at some point) I needed to
>> > >> enclose the parameters between quotes:
>> > >>
>> > >> #+srcname: quoted-params
>> > >> #+begin_src ledger :cmdline "reg unknown" :noweb yes :session
>> > >> <<data>>
>> > >> #+end_src
>> > >>
>> > >> #+results: quoted-params
>> > >>
>> > >> Nothing is returned. In fact, I would expect an error to be thrown [...] 
>
> [...]
>
>> > > Babel has a standard mechanism for evaluating shell commands and displaying
>> > > errors if any. It is the function `org-babel-eval' in ob-eval.el. The
>> > > problem is that ob-ledger is not using this mechanism.
>
> [...]
>
> This patch should make ob-ledger use the common org-babel error mechanism. It is
> in branch ledger of the main repo.

So you won (being the first to implement it)... Thanks a lot.


> However, I'm not yet a ledger user. Could someone test it?

I'll try, but I first need to ensure Ledger works here -- as told, I've now
inherited a Windows PC, after my laptop (Ubuntu) crash, and must redo some
painful configuration.

However, I still plan on really using Ledger, hence on testing your patch
(very) soon.


> Incidentally, can any gnus users tell me -- is there a non-painful way of
> retrieving an old message from nntp in order to create a reply in the correct
> thread? Or do you hack the headers to make it appear in the right thread? I'm
> using the gmane web interface to send this.

I don't know how you did it, but it appeared in the correct thread to me.

What I do, with Gnus, is C-u 1000 RET for entering the group, and fetching the
last 1000 posts (1000 is chosen depending on the date, and the estimated
amount of post over that period of time). Then, search on title...

Maybe (euh, no, certainly) some better methods do exist. Like keeping a link
to the post, and C-c C-o on the link.

Others?

Best regards,
  Seb

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

[OT] retrieving old messages [WAS] Re: [Babel] Handling of errors when using Ledger

[Dan Davison]

Bernt Hansen <bernt@norang.ca> writes:

> Dan <dandavison7@gmail.com> writes:
>
>> Incidentally, can any gnus users tell me -- is there a non-painful way of
>> retrieving an old message from nntp in order to create a reply in the correct
>> thread? Or do you hack the headers to make it appear in the right thread? I'm
>> using the gmane web interface to send this.
>
> I read the list through gnus and gmane.  When I'm in the summary buffer
> for the org-mode list I can retrieve old messages with
>
>    / O 500 RET
>
> to get the last 500 messages from the list.

Thanks Bernt and Seb, but that was the route I was referring to when I
said "painful" :) The list volume is quite high, and retrieving
thousands of messages takes a while. And of course you go for 1500 and
it turns out to be message 1728...

It seems a rather primitive practice, can we really not do better? Is it
not possible to use a unique identifier to pull down a specific message?
(I was not reading usenet or whatever it was called in the 1980s; I
don't really know anything about these technologies.)

Dan


> Then you can limit the displayed messages by some filter (say subject) with
>
>   / / <filter text> RET
>
> On a message ^ will retrieve the previous message in the thread and you
> can do that repeatedly to get the entire thread displayed.
>
> Then I just reply normally to a specific message with
>   S W
>
> and It Just Works(tm)
>
> HTH,
> Bernt
>
> _______________________________________________
> Emacs-orgmode mailing list
> Please use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: [Babel] Handling of errors when using Ledger

[Sébastien Vauban]

Hi Bernt,

Bernt Hansen wrote:
> Dan <dandavison7-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>> Incidentally, can any gnus users tell me -- is there a non-painful way of
>> retrieving an old message from nntp in order to create a reply in the correct
>> thread? Or do you hack the headers to make it appear in the right thread? I'm
>> using the gmane web interface to send this.
>
> I read the list through gnus and gmane.  When I'm in the summary buffer
> for the org-mode list I can retrieve old messages with
>
>    / O 500 RET
>
> to get the last 500 messages from the list.
> Then you can limit the displayed messages by some filter (say subject) with
>
>   / / <filter text> RET
>
> On a message ^ will retrieve the previous message in the thread and you
> can do that repeatedly to get the entire thread displayed.

Or A T to directly get the whole thread...

> Then I just reply normally to a specific message with
>   S W
>
> and It Just Works(tm)

Not that different from my procedure, it seems...

Best regards,
  Seb

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: [OT] retrieving old messages

[Bernt Hansen]

Dan Davison <dandavison7@gmail.com> writes:

> Bernt Hansen <bernt@norang.ca> writes:
>
>> Dan <dandavison7@gmail.com> writes:
>>
>>> Incidentally, can any gnus users tell me -- is there a non-painful way of
>>> retrieving an old message from nntp in order to create a reply in the correct
>>> thread? Or do you hack the headers to make it appear in the right thread? I'm
>>> using the gmane web interface to send this.
>>
>> I read the list through gnus and gmane.  When I'm in the summary buffer
>> for the org-mode list I can retrieve old messages with
>>
>>    / O 500 RET
>>
>> to get the last 500 messages from the list.
>
> Thanks Bernt and Seb, but that was the route I was referring to when I
> said "painful" :) The list volume is quite high, and retrieving
> thousands of messages takes a while. And of course you go for 1500 and
> it turns out to be message 1728...
>
> It seems a rather primitive practice, can we really not do better? Is it
> not possible to use a unique identifier to pull down a specific message?
> (I was not reading usenet or whatever it was called in the 1980s; I
> don't really know anything about these technologies.)

Hi Dan,

I also make org-capture mode links to interesting messages when I read
them if I intend to follow up on it at a later time.  So on a
gnus-message I just do C-M-r and create a task that links to the mail.
This means I can open the linked message directly from the captured
org-mode task with C-c C-o and just reply to that - and the reply is
threaded properly.  This is my preferred way to do this type of thing.

My email links to org-mode articles on gmane look like this:
[[gnus:nntp%2Bnews.gmane.org:gmane.emacs.orgmode#4CEE66D5.2030909@gmail.com][Email from Rainer M. Krug: OT: Question concerning vc-wor]]

When retrieving old messages with / O 500 RET it gets the next 500 older
messages you haven't already retrieved -- so if you're at 1500 and you
need to get to 1728 ... you just need to / O 228 RET to get the rest of
the way.  I rarely search though the entire list of messages for old
stuff - but I find this method of searching in Gnus much better than any
web based search -- since the message id isn't available in the gmane
web archives as far as I know.

-Bernt

Re: Re: [Babel] Handling of errors when using Ledger

[Eric S Fraga]

Dan <dandavison7@gmail.com> writes:

[...]

> This patch should make ob-ledger use the common org-babel error mechanism. It is
> in branch ledger of the main repo. However, I'm not yet a ledger user. Could
> someone test it?

Unfortunately, it doesn't work at all [1] for a very simple reason: you
have removed the -f option which specifies the file that ledger should
read.  Ledger does not interpret a file argument directly but needs the
-f option, as in:

   ledger -f file.ledger <rest of command line>

Note the second line in the concatenated list of strings below:

[...]

> -    (with-output-to-string
> -      (shell-command (concat "ledger"
> -                            " -f " (org-babel-process-file-name in-file)
> -                            " " cmdline
> -                            " > " (org-babel-process-file-name out-file))))

I am not sure how you pass this option with the much shorter code you
have written:

> +  (org-babel-eval
> +   (concat org-babel-ledger-command " " (cdr (assoc :cmdline params)))
> +   body))

I hope this makes sense?

Footnotes: 
[1]  actually, what does work very nicely indeed is that error messages
     are reported perfectly!!  So thank you very much for this.

-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 23.2.1
: using Org-mode version 7.3 (release_7.3.99.g9db0.dirty)

Re: [Babel] Handling of errors when using Ledger

[Dan Davison]

Eric S Fraga <ucecesf@ucl.ac.uk> writes:

> Dan <dandavison7@gmail.com> writes:
>
> [...]
>
>> This patch should make ob-ledger use the common org-babel error mechanism. It is
>> in branch ledger of the main repo. However, I'm not yet a ledger user. Could
>> someone test it?
>
> Unfortunately, it doesn't work at all [1] for a very simple reason: you
> have removed the -f option which specifies the file that ledger should
> read.

Hi Eric,

Yes, ob-eval passes the src block body to the interpreter on standard
input. I assumed that ledger would be able to read its transaction data
from standard input. Is that not the case?  What about with "--file -"
or "-f -"?

Incidentally, I have for a long time wondered whether we should permit
src blocks to read *input data* from standard input. This would require
altering ob-eval such that the src block body is read from file. Then we
could do things like

#+source: output-some-text
#+begin_src sh :results output
  # print stuff
#+end_src

#+begin_src perl :stdin output-some-text
  while ( <> ) {
      # process the text
  }
#+end_src

If ledger can't read the src blocks body from stdin (and if it can't, I
expect there's a good reason why not), then maybe this is a motivation
for changing ob-eval so that the block body is read from file.

Dan

>  Ledger does not interpret a file argument directly but needs the
> -f option, as in:
>
>    ledger -f file.ledger <rest of command line>
>
> Note the second line in the concatenated list of strings below:
>
> [...]
>
>> -    (with-output-to-string
>> -      (shell-command (concat "ledger"
>> -                            " -f " (org-babel-process-file-name in-file)
>> -                            " " cmdline
>> -                            " > " (org-babel-process-file-name out-file))))
>
> I am not sure how you pass this option with the much shorter code you
> have written:
>
>> +  (org-babel-eval
>> +   (concat org-babel-ledger-command " " (cdr (assoc :cmdline params)))
>> +   body))
>
> I hope this makes sense?
>
> Footnotes: 
> [1]  actually, what does work very nicely indeed is that error messages
>      are reported perfectly!!  So thank you very much for this.

Re: [Babel] Handling of errors when using Ledger

[Eric S Fraga]

Dan Davison <dandavison7@gmail.com> writes:

> Eric S Fraga <ucecesf@ucl.ac.uk> writes:
>
>> Dan <dandavison7@gmail.com> writes:
>>
>> [...]
>>
>>> This patch should make ob-ledger use the common org-babel error mechanism. It is
>>> in branch ledger of the main repo. However, I'm not yet a ledger user. Could
>>> someone test it?
>>
>> Unfortunately, it doesn't work at all [1] for a very simple reason: you
>> have removed the -f option which specifies the file that ledger should
>> read.
>
> Hi Eric,
>
> Yes, ob-eval passes the src block body to the interpreter on standard
> input. I assumed that ledger would be able to read its transaction data
> from standard input. Is that not the case?  What about with "--file -"
> or "-f -"?

> [...]

> If ledger can't read the src blocks body from stdin (and if it can't, I
> expect there's a good reason why not), then maybe this is a motivation
> for changing ob-eval so that the block body is read from file.

Indeed, ledger cannot read from standard input.  From the man page:

,----
|  All commands require a Ledger data file which can be specified with -f
|  filename or via the LEDGER_FILE environment variable.
`----

and later:

,----
|  -f, --file FILE
| 
|     Reads FILE as a Ledger file. This option may be specified multiple
|     times.  FILE may also be a list of file names separated by colons.
|     Typically, the environment variable LEDGER_FILE is set rather than
|     using this command-line option.
`----

Using an environment variable is not a good or viable solution for
babel, as far as I can figure, and there does not seem to be any concept
of "-" for standard input in lieu of a file.  I cannot see any reason
for this and, to be honest, I find it annoying [1] as a long time Unix user
wishing to join tools together with pipes etc.  But maybe John Wiegley
had his reasons for taking this decision.  Maybe he can comment on
this... I think he reads this list?

> Incidentally, I have for a long time wondered whether we should permit
> src blocks to read *input data* from standard input. This would require
> altering ob-eval such that the src block body is read from file. Then we
> could do things like
>
> #+source: output-some-text
> #+begin_src sh :results output
>   # print stuff
> #+end_src
> #+begin_src perl :stdin output-some-text
>   while ( <> ) {
>       # process the text
>   }
> #+end_src

This could be a solution.  In thinking about all of my babel uses, this
would have no impact on me but others may have different opinions?  It
would seem to be a safer route in any case?

Thanks,
eric


Footnotes: 
[1]   but not annoying enough to stop using ledger, of course, as it's a
      brilliant tool otherwise!

-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 23.2.1
: using Org-mode version 7.3 (release_7.3.99.g9db0.dirty)

Re: [Babel] Piping between code blocks Was: Handling of errors when using Ledger

[Eric Schulte]

Dan Davison <dandavison7@gmail.com> writes:

>
> Incidentally, I have for a long time wondered whether we should permit
> src blocks to read *input data* from standard input. This would require
> altering ob-eval such that the src block body is read from file. Then we
> could do things like
>
> #+source: output-some-text
> #+begin_src sh :results output
>   # print stuff
> #+end_src
>
> #+begin_src perl :stdin output-some-text
>   while ( <> ) {
>       # process the text
>   }
> #+end_src
>

This is interesting, it would involve starting all such pipe-chained
blocks at the same time, and coordinating their input and output through
Unix pipes (note: I don't know how pipes would work on a Windows
machine).

One question is whether we allow data to pass /directly/ between blocks,
or if we insert babel in between to do its usual parsing (i.e. arrays
into elisp lists, etc...).  I'd lean against inserting babel as I
believe this would be of use mainly for performance, and Emacs/Babel
parsing would blow any performance gains.  Also it could be difficult to
implement.

I think that in addition to the :stdin option above we would need a
=:stdout= option would imply both =:results output= and =:results
scalar= and would allow the process to write intermediate results
directly to a supplied pipe.  Perhaps such a :stdout option would never
be supplied directly by the user, but would rather be used by babel to
organize piping when that code block has referenced by another code
block using :stdin.

Does the above make sense or sound appealing?

Best -- Eric

Re: [OT] retrieving old messages [WAS] Re: [Babel] Handling of errors when using Ledger

[Matt Lundin]

Dan Davison <dandavison7@gmail.com> writes:

> Bernt Hansen <bernt@norang.ca> writes:
>
>> Dan <dandavison7@gmail.com> writes:
>>
>>> Incidentally, can any gnus users tell me -- is there a non-painful way of
>>> retrieving an old message from nntp in order to create a reply in the correct
>>> thread? Or do you hack the headers to make it appear in the right thread? I'm
>>> using the gmane web interface to send this.
>>
>> I read the list through gnus and gmane.  When I'm in the summary buffer
>> for the org-mode list I can retrieve old messages with
>>
>>    / O 500 RET
>>
>> to get the last 500 messages from the list.
>
> Thanks Bernt and Seb, but that was the route I was referring to when I
> said "painful" :) The list volume is quite high, and retrieving
> thousands of messages takes a while. And of course you go for 1500 and
> it turns out to be message 1728...
>
> It seems a rather primitive practice, can we really not do better? Is it
> not possible to use a unique identifier to pull down a specific message?
> (I was not reading usenet or whatever it was called in the 1980s; I
> don't really know anything about these technologies.)

FWIW, recent changes nnir (in nognus and development Emacs) allow for
easy searching of individual Gmane groups. Simply type "G G" on the
org-mode group in the in the Group buffer or call
gnus-group-make-nnir-group from within the summary buffer or an article.
I typed M-x gnus-group-make-nnir-group while editing this message,
entered "gnus painful" and received the following summary buffer
(limited to the org-mode list) in less than a second:

--8<---------------cut here---------------start------------->8---
 . 27-Sep: Bernt Hansen     - [85: gmane.emacs.orgmode/3394] Wish list: Storing links and enabling MIME actions on files
 . 27-Sep: Carsten Dominik  \-> [83: gmane.emacs.orgmode/3400] Re: Wish list: Storing links and enabling MIME actions on files
 . 03-Oct: Bernt Hansen     - [80: gmane.emacs.orgmode/3559] Intermittent Font Face problem
 . 25-Apr: Chao Lu          - [89: gmane.emacs.orgmode/24512] Edit Worg, Tracking changes, ect.
 . 29-Apr: David Frascone   - [91: gmane.emacs.orgmode/24740] Re: Re: Linking Mail ?
 . 29-Apr: Matt Lundin      +-> [94: gmane.emacs.orgmode/24746] Re: Linking Mail ?
 . 29-Apr: David Frascone   | \-> [91: gmane.emacs.orgmode/24749] Re: Linking Mail ?
 . 29-Apr: Rémi Vanicat     |   +-> [87: gmane.emacs.orgmode/24777] Re: Linking Mail ?
 . 30-Apr: David Maus       |   \-> [89: gmane.emacs.orgmode/24793] Re: Re: Linking Mail ?
 . 30-Apr: Uday S Reddy     \-> [82: gmane.emacs.orgmode/24826] Re: Linking Mail ?
 . 29-Sep: Matt Lundin      - [54: gmane.emacs.orgmode/31143] Re: Having (too) many files in org-agenda-files
 . 25-Nov: Dan              - [60: gmane.emacs.orgmode/34047] Re: [Babel] Handling of errors when using Ledger
 . 25-Nov: Bernt Hansen     +-> [88: gmane.emacs.orgmode/34055] Re: [Babel] Handling of errors when using Ledger
 . 25-Nov: Dan Davison      | +-> [91: gmane.emacs.orgmode/34057] [OT] retrieving old messages [WAS] Re: [Babel] Handling of errors when using Ledger
 . 25-Nov: Bernt Hansen     | | \-> [100: gmane.emacs.orgmode/34065] Re: [OT] retrieving old messages
 . 25-Nov: Sébastien Vauba  | \-> [86: gmane.emacs.orgmode/34059] Re: [Babel] Handling of errors when using Ledger
 . 25-Nov: Sébastien Vauba  \-> [88: gmane.emacs.orgmode/34056] Re: [Babel] Handling of errors when using Ledger
--8<---------------cut here---------------end--------------->8---

You can reply directly to any of these articles or type A T to recover
the full thread in the Gmane group.

If your search is too broad (i.e.,., hundreds of results) and you choose
to display all of the results at once, nnir can take a bit of time. I'm
not sure it's any more painful than wading through web search results on
gmane, though. ;)

Best,
Matt

Re: [OT] retrieving old messages

[Sébastien Vauban]

Hi Matt,

Matt Lundin wrote:
> FWIW, recent changes nnir (in nognus and development Emacs) allow for
> easy searching of individual Gmane groups. Simply type "G G" on the
> org-mode group in the in the Group buffer or call
> gnus-group-make-nnir-group from within the summary buffer or an article.
> I typed M-x gnus-group-make-nnir-group while editing this message,
> entered "gnus painful" and received the following summary buffer
> (limited to the org-mode list) in less than a second:

Do we have to activate something special?

Pressing G G on the Org group, I get:

--8<---------------cut here---------------start------------->8---
Debugger entered--Lisp error: (error "Couldn't request group: Search produced empty results.")
  signal(error ("Couldn't request group: Search produced empty results."))
  error("Couldn't request group: %s" "Search produced empty results.")
  gnus-group-read-ephemeral-group("nnir:((query . \"gnus painful\") (unique-id . \"80ipzgijf2.fsf\"))" (nnir "") t (#<buffer *Group*> . group) nil)
  (let ((parms nil)) (if extra-parms (setq parms ...) (setq parms ...)) (add-to-list (quote parms) (cons ... ...) t) (gnus-group-read-ephemeral-group (concat "nnir:" ...) (quote ...) t (cons ... gnus-current-window-configuration) nil))
  gnus-group-make-nnir-group(nil "gnus painful")
  call-interactively(gnus-group-make-nnir-group nil nil)
--8<---------------cut here---------------end--------------->8---

Any idea?

Best regards,
  Seb

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: [OT] retrieving old messages

[Matt Lundin]

Sébastien Vauban <wxhgmqzgwmuf-geNee64TY+gS+FvcfC7Uqw@public.gmane.org>
writes:

> Hi Matt,
>
> Matt Lundin wrote:
>> FWIW, recent changes nnir (in nognus and development Emacs) allow for
>> easy searching of individual Gmane groups. Simply type "G G" on the
>> org-mode group in the in the Group buffer or call
>> gnus-group-make-nnir-group from within the summary buffer or an article.
>> I typed M-x gnus-group-make-nnir-group while editing this message,
>> entered "gnus painful" and received the following summary buffer
>> (limited to the org-mode list) in less than a second:
>
> Do we have to activate something special?
>
> Pressing G G on the Org group, I get:
>
> Debugger entered--Lisp error: (error "Couldn't request group: Search produced empty results.")
>   signal(error ("Couldn't request group: Search produced empty results."))
>   error("Couldn't request group: %s" "Search produced empty results.")
>   gnus-group-read-ephemeral-group("nnir:((query . \"gnus painful\") (unique-id . \"80ipzgijf2.fsf\"))" (nnir "") t (#<buffer *Group*> . group) nil)
>   (let ((parms nil)) (if extra-parms (setq parms ...) (setq parms ...)) (add-to-list (quote parms) (cons ... ...) t) (gnus-group-read-ephemeral-group (concat "nnir:" ...) (quote ...) t (cons ... gnus-current-window-configuration) nil))
>   gnus-group-make-nnir-group(nil "gnus painful")
>   call-interactively(gnus-group-make-nnir-group nil nil)
>

Alas, at this point, you have to run the development version of Gnus
(nognus):

http://www.gnus.org/distribution.html

Gmane searching was added with the following (Oct 31) commit:

http://git.gnus.org/cgit/gnus.git/commit/?id=a14d856fc1432f862e075b7ee792fe5e381a38ff

Building development emacs will give you essentially the same bleeding
edge as nognus.

Otherwise, all I have in my setup is:

(require 'nnir)

And I'm not even sure that is necessary anymore.

Best,
Matt

Re: [Babel] Handling of errors when using Ledger

[Dan Davison]

Eric S Fraga <ucecesf@ucl.ac.uk> writes:

> Dan Davison <dandavison7@gmail.com> writes:
>
>> Eric S Fraga <ucecesf@ucl.ac.uk> writes:
>>
>>> Dan <dandavison7@gmail.com> writes:
>>>
>>> [...]
>>>
>>>> This patch should make ob-ledger use the common org-babel error mechanism. It is
>>>> in branch ledger of the main repo. However, I'm not yet a ledger user. Could
>>>> someone test it?
>>>
>>> Unfortunately, it doesn't work at all [1] for a very simple reason: you
>>> have removed the -f option which specifies the file that ledger should
>>> read.
>>
>> Hi Eric,
>>
>> Yes, ob-eval passes the src block body to the interpreter on standard
>> input. I assumed that ledger would be able to read its transaction data
>> from standard input. Is that not the case?  What about with "--file -"
>> or "-f -"?
>
>> [...]
>
>> If ledger can't read the src blocks body from stdin (and if it can't, I
>> expect there's a good reason why not), then maybe this is a motivation
>> for changing ob-eval so that the block body is read from file.
>
> Indeed, ledger cannot read from standard input.

Hi Eric,

I've asked on the ledger list and the 3.0 (and perhaps some older)
versions of ledger do support -f - to read from stdin. Could you try
this version of the patch?

Thanks,

Dan

diff --git a/lisp/ob-ledger.el b/lisp/ob-ledger.el
index a02eb6f..ef0f52e 100644
--- a/lisp/ob-ledger.el
+++ b/lisp/ob-ledger.el
@@ -43,24 +43,15 @@
   '((:results . "output") (:cmdline . "bal"))
   "Default arguments to use when evaluating a ledger source block.")
 
+(defvar org-babel-ledger-command "ledger"
+  "Command to invoke ledger")
+
 (defun org-babel-execute:ledger (body params)
   "Execute a block of Ledger entries with org-babel.  This function is
 called by `org-babel-execute-src-block'."
-  (message "executing Ledger source code block")
-  (let ((result-params (split-string (or (cdr (assoc :results params)) "")))
-	(cmdline (cdr (assoc :cmdline params)))
-        (in-file (org-babel-temp-file "ledger-"))
-	(out-file (org-babel-temp-file "ledger-output-")))
-    (with-temp-file in-file (insert body))
-    (message (concat "ledger"
-		     " -f " (org-babel-process-file-name in-file)
-		     " " cmdline))
-    (with-output-to-string
-      (shell-command (concat "ledger"
-			     " -f " (org-babel-process-file-name in-file)
-			     " " cmdline
-			     " > " (org-babel-process-file-name out-file))))
-    (with-temp-buffer (insert-file-contents out-file) (buffer-string))))
+  (org-babel-eval
+   (concat org-babel-ledger-command " -f - " (cdr (assoc :cmdline params)))
+   body))



>  From the man page:
>
> ,----
> |  All commands require a Ledger data file which can be specified with -f
> |  filename or via the LEDGER_FILE environment variable.
> `----
>
> and later:
>
> ,----
> |  -f, --file FILE
> | 
> |     Reads FILE as a Ledger file. This option may be specified multiple
> |     times.  FILE may also be a list of file names separated by colons.
> |     Typically, the environment variable LEDGER_FILE is set rather than
> |     using this command-line option.
> `----
>
> Using an environment variable is not a good or viable solution for
> babel, as far as I can figure, and there does not seem to be any concept
> of "-" for standard input in lieu of a file.  I cannot see any reason
> for this and, to be honest, I find it annoying [1] as a long time Unix user
> wishing to join tools together with pipes etc.  But maybe John Wiegley
> had his reasons for taking this decision.  Maybe he can comment on
> this... I think he reads this list?
>
>> Incidentally, I have for a long time wondered whether we should permit
>> src blocks to read *input data* from standard input. This would require
>> altering ob-eval such that the src block body is read from file. Then we
>> could do things like
>>
>> #+source: output-some-text
>> #+begin_src sh :results output
>>   # print stuff
>> #+end_src
>> #+begin_src perl :stdin output-some-text
>>   while ( <> ) {
>>       # process the text
>>   }
>> #+end_src
>
> This could be a solution.  In thinking about all of my babel uses, this
> would have no impact on me but others may have different opinions?  It
> would seem to be a safer route in any case?
>
> Thanks,
> eric
>
>
> Footnotes: 
> [1]   but not annoying enough to stop using ledger, of course, as it's a
>       brilliant tool otherwise!

Re: [Babel] Handling of errors when using Ledger

[Sébastien Vauban]

Hi Dan and Eric,

Dan Davison wrote:
> Eric S Fraga <ucecesf-hclig2XLE9Zaa/9Udqfwiw@public.gmane.org> writes:
>> Indeed, ledger cannot read from standard input.
>
> I've asked on the ledger list and the 3.0 (and perhaps some older)
> versions of ledger do support -f - to read from stdin. Could you try
> this version of the patch?

Unluckily for me, ledger 2.6.1 (the latest precompiled binary, on
http://www.assembla.com/spaces/Goldcoast/documents) does not work like that:

#+begin_src sh
  Seb@MEDIACENTER:...ness/Accounting 0$ ledger --version
  Ledger 2.6.1, the command-line accounting tool

  Copyright (c) 2003-2008, John Wiegley.  All rights reserved.

  This program is made available under the terms of the BSD Public License.
  See LICENSE file included with the distribution for details and disclaimer.

  (modules: gmp, pcre, xml)

  Seb@MEDIACENTER:...ness/Accounting 0$ cat 00307599427-telechargement.dat | ledger -f - bal

  Seb@MEDIACENTER:...ness/Accounting 0$ ledger -f 00307599427-telechargement.dat bal
           -181.52 EUR  Assets
            181.52 EUR  Expenses

  Seb@MEDIACENTER:...ness/Accounting 0$
#+end_src

Everything is that easier under Ubuntu...

Best regards,
  Seb

-- 
Sébastien Vauban


_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode-mXXj517/zsQ@public.gmane.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: [Babel] Handling of errors when using Ledger

[Eric S Fraga]

Dan Davison <dandavison7@gmail.com> writes:

[...]

> Hi Eric,
>
> I've asked on the ledger list and the 3.0 (and perhaps some older)
> versions of ledger do support -f - to read from stdin. Could you try
> this version of the patch?

I was about to say that I could not because the most up to date version
on Debian (testing) is 2.6.2 and this version does not support the "-f
-" option.  However, I have just noticed that version 3.0.0 is now in
Debian experimental.  I will have a look to see what kind of
repercussions there may be from installing that version on my systems
and I will get back to you (but probably not until after xmas...).

-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 23.2.1
: using Org-mode version 7.4 (release_7.4.79.g9c4b9)

Re: [Babel] Handling of errors when using Ledger

[Dan Davison]

Hi Eric, Seb and any other ledger users,

Is anyone using a version of ledger that does not support reading from
stdin with -f - ?

Thanks,

Dan

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> Dan Davison <dandavison7@gmail.com> writes:
>
> [...]
>
>> Hi Eric,
>>
>> I've asked on the ledger list and the 3.0 (and perhaps some older)
>> versions of ledger do support -f - to read from stdin. Could you try
>> this version of the patch?
>
> I was about to say that I could not because the most up to date version
> on Debian (testing) is 2.6.2 and this version does not support the "-f
> -" option.  However, I have just noticed that version 3.0.0 is now in
> Debian experimental.  I will have a look to see what kind of
> repercussions there may be from installing that version on my systems
> and I will get back to you (but probably not until after xmas...).

Re: [Babel] Handling of errors when using Ledger

[Eric S Fraga]

Dan Davison <dandavison7@gmail.com> writes:

> Hi Eric, Seb and any other ledger users,
>
> Is anyone using a version of ledger that does not support reading from
> stdin with -f - ?
>
> Thanks,
>
> Dan

Thanks for the reminder!  I had forgotten about the new version of
ledger.  Apologies for not checking it out as I had promised.  It's been
a very hectic time since Christmas...  :(

In any case, it turns out that ledger v2.6.2, which I believe is now in
Debian stable, does allow the =-f -= argument despite the documentation
not stating this.  So, for me at least, I am indeed using a version of
ledger that supports this.

-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1
: using Org-mode version 7.4 (release_7.4.646.g57806)