call_*() is not working inside #+DATE

call_*() is not working inside #+DATE

[Rafael Laboissiere]

The following used to work for me in the past:

    #+NAME: date
    #+BEGIN_SRC sh :results silent :exports results :tangle no
    date
    #+END_SRC
    #+TITLE: Sample
    #+AUTHOR: Me
    #+DATE: call_date()

and I saw the output of the shell command "date" when exporting the file 
to LaTeX.  However does not work for the current HEAD of the Git 
repository.

Using git-bisect, I discovered that the culprit is commit 85ff663, i.e. 
my code above works for commit 85ff663^ but fails for commit 85ff663.

Commit 85ff663 was a pretty large commit and, besides that, the new code 
introduced by it was posteriorly changed, so it is hard to find where the 
bug comes from.  Could someone more acquainted with the code try to look 
at this bug, please?

Thanks,

Rafael Laboissière

Re: [PATCH] call_*() is not working inside #+DATE

[Rafael Laboissiere]

[-- Attachment #1: Type: text/plain, Size: 1877 bytes --]

* Rafael Laboissiere <rafael@laboissiere.net> [2016-03-07 02:58]:

> The following used to work for me in the past:
>
>   #+NAME: date
>   #+BEGIN_SRC sh :results silent :exports results :tangle no
>   date
>   #+END_SRC
>   #+TITLE: Sample
>   #+AUTHOR: Me
>   #+DATE: call_date()
>
> and I saw the output of the shell command "date" when exporting the 
> file to LaTeX.  However does not work for the current HEAD of the Git 
> repository.
>
> Using git-bisect, I discovered that the culprit is commit 85ff663, 
> i.e. my code above works for commit 85ff663^ but fails for commit 
> 85ff663.
>
> Commit 85ff663 was a pretty large commit and, besides that, the new 
> code introduced by it was posteriorly changed, so it is hard to find 
> where the bug comes from.  Could someone more acquainted with the code 
> try to look at this bug, please?

I investigated this issue further and discovered that the constructs 
call_<name>(args) and src_<lang>{code} are not evaluated at all when they 
appear in a keyword line (starting with "#+<keyword>:").  Org-babel 
behaves in this way probably on purpose.  At any rate, it would be better 
if the current behavior is documented somewhere.  This may be 
accomplished with the patch attached below.

Best,

Rafael Laboissière

P.S.: For those who are reading this message and are interested in a 
solution for my original problem, here is the way I am getting around it 
right now.  When exporting to LaTeX, I wanted to have, as the contents of 
the \date{} macro, the date and the hash of the last Git commit of the 
current checkout HEAD, to which my Org document belongs.  Here is how I 
am doing it now:

    #+TITLE: Sample
    #+AUTHOR: Me
    #+BEGIN_SRC sh :results silent :exports results :tangle no
    git show -s --date=short --format="%cd [%h]" HEAD > GitDateCommit.tex
    #+END_SRC
    #+DATE: \input{GitDateCommit}

[-- Attachment #2: 0001-Document-lack-of-evaluation-of-inline-code-blocks-in.patch --]
[-- Type: text/x-diff, Size: 2030 bytes --]

From fa47439f9897c4bab436ecf4c9f08fa686ff0231 Mon Sep 17 00:00:00 2001
From: Rafael Laboissiere <rafael@laboissiere.net>
Date: Fri, 11 Mar 2016 23:04:00 +0100
Subject: [PATCH] Document lack of evaluation of inline code blocks in keyword
 lines

doc/org.texi (Evaluating code blocks): Add footnote explaining that
inline code blocks are not evaluated inside keyword lines.
---
 doc/org.texi | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/doc/org.texi b/doc/org.texi
index e423df7..ce478f5 100644
--- a/doc/org.texi
+++ b/doc/org.texi
@@ -15030,13 +15030,16 @@ evaluation from the @kbd{C-c C-c} key binding.}.  This will call the
 its results into the Org mode buffer.
 
 @cindex #+CALL
-It is also possible to evaluate named code blocks from anywhere in an Org
-mode buffer or an Org mode table.  These named code blocks can be located in
-the current Org mode buffer or in the ``Library of Babel'' (@pxref{Library of
-Babel}).  Named code blocks can be evaluated with a separate @code{#+CALL:}
-line or inline within a block of text.  In both cases the result is wrapped
-according to the value of @code{org-babel-inline-result-wrap}, which by
-default is @code{"=%s="} for markup that produces verbatim text.
+It is also possible to evaluate named code blocks from
+anywhere@footnote{Actually, the constructs call_<name>() and src_<lang>@{@}
+are not evaluated when they appear in a keyword line (i.e. lines starting
+with @code{#+KEYWORD:}, @pxref{In-buffer settings}).}  in an Org mode buffer
+or an Org mode table.  These named code blocks can be located in the current
+Org mode buffer or in the ``Library of Babel'' (@pxref{Library of Babel}).
+Named code blocks can be evaluated with a separate @code{#+CALL:} line or
+inline within a block of text.  In both cases the result is wrapped according
+to the value of @code{org-babel-inline-result-wrap}, which by default is
+@code{"=%s="} for markup that produces verbatim text.
 
 The syntax of the @code{#+CALL:} line is
 
-- 
2.7.0

Re: [PATCH] call_*() is not working inside #+DATE

[Eric S Fraga]

On Saturday, 12 Mar 2016 at 08:57, Rafael Laboissiere wrote:
> P.S.: For those who are reading this message and are interested in a 
> solution for my original problem, here is the way I am getting around it 
> right now.

Thanks for this alternate solution.
-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.0.91.1, Org release_8.3.4-626-gb62d55

Re: [PATCH] call_*() is not working inside #+DATE

[Rafael Laboissiere]

* Eric S Fraga <e.fraga@ucl.ac.uk> [2016-03-12 08:51]:

> On Saturday, 12 Mar 2016 at 08:57, Rafael Laboissiere wrote:
>> P.S.: For those who are reading this message and are interested in a
>> solution for my original problem, here is the way I am getting around it
>> right now.
>
> Thanks for this alternate solution.

You are welcome.

It would be much better if the following construct worked:

    #+DATE: src_sh{git show -s --date=short --format="%cd [%h]" HEAD}

Unfortunately, it does not.  This behavior (or misbehavior, I do not 
know) can be traced down to the org-element-context function.  Suppose 
that you have the following content in a org-mode buffer:

    #+DATE: src_sh{date}
    src_sh{date}

With the cursor just after the underscore in the #+DATE line, 
org-element-context returns:

    (keyword
     (:key "DATE" :value "src_sh{date}" :begin 1 :end 22 :post-blank 0 :post-affiliated 1 :parent nil))

On the other hand, with the cursor just after the underscore in the next 
line, org-element-context returns (as it should be):

    (inline-src-block
     (:language "sh" :value "date" :parameters nil :begin 22 :end  34 :post-blank 0
       :parent (paragraph (:begin 22 :end 35 :contents-begin 22 :contents-end 35 :post-blank 0
                           :post-affiliated 22 :parent nil))))

This is the reason why Org-babel does not evaluate the inline source 
block in the #+DATE line.

Best,

Rafael Laboissière

Re: [PATCH] call_*() is not working inside #+DATE

[Nicolas Goaziou]

Hello,

Rafael Laboissiere <rafael@laboissiere.net> writes:

> It would be much better if the following construct worked:
>
>     #+DATE: src_sh{git show -s --date=short --format="%cd [%h]" HEAD}
>
> Unfortunately, it does not.  This behavior (or misbehavior, I do not 
> know) can be traced down to the org-element-context function.  Suppose 
> that you have the following content in a org-mode buffer:
>
>     #+DATE: src_sh{date}
>     src_sh{date}
>
> With the cursor just after the underscore in the #+DATE line, 
> org-element-context returns:
>
>     (keyword
>      (:key "DATE" :value "src_sh{date}" :begin 1 :end 22 :post-blank 0 :post-affiliated 1 :parent nil))
>
> On the other hand, with the cursor just after the underscore in the next 
> line, org-element-context returns (as it should be):
>
>     (inline-src-block
>      (:language "sh" :value "date" :parameters nil :begin 22 :end  34 :post-blank 0
>        :parent (paragraph (:begin 22 :end 35 :contents-begin 22 :contents-end 35 :post-blank 0
>                            :post-affiliated 22 :parent nil))))
>
> This is the reason why Org-babel does not evaluate the inline source 
> block in the #+DATE line.

Values from keyword are not parsed. Org used to make an exception for an
arbitrary bunch of them (e.g., DATE, TITLE, etc.). Now it is up to the
export back-ends to decide what keyword is going to be parsed.

I can think of two ways to solve this:

  1. Only evaluate the Babel code during export. Upon exporting the
     document, parsed keywords are known, so
     `org-babel-exp-process-buffer' may try to find "hidden" Babel code
     and execute it.

     This would however introduce a discrepancy between
     org-babel-execute-buffer and the behaviour upon exporting.

  2. Sort parsed keywords from regular ones at the syntax level, much
     like we did for export blocks recently. I.e., every keyword is
     parsed expect those marked as verbatim. I don't have an idea bout
     the involved syntax.

     This would probably induce some backward incompatibility.

WDYT?


Regards,

-- 
Nicolas Goaziou

Re: [PATCH] call_*() is not working inside #+DATE

[Rafael Laboissiere]

* Nicolas Goaziou <mail@nicolasgoaziou.fr> [2016-03-13 18:24]:

> Rafael Laboissiere <rafael@laboissiere.net> writes:
>
>> It would be much better if the following construct worked:
>>
>>     #+DATE: src_sh{git show -s --date=short --format="%cd [%h]" HEAD}
>>
>> Unfortunately, it does not.  This behavior (or misbehavior, I do not 
>> know) can be traced down to the org-element-context function.
>>
>> [snip]
>
> Values from keyword are not parsed. Org used to make an exception for an 
> arbitrary bunch of them (e.g., DATE, TITLE, etc.). Now it is up to the 
> export back-ends to decide what keyword is going to be parsed.
>
> I can think of two ways to solve this:
>
>  1. Only evaluate the Babel code during export. Upon exporting the
>     document, parsed keywords are known, so
>     `org-babel-exp-process-buffer' may try to find "hidden" Babel code
>     and execute it.
>
>     This would however introduce a discrepancy between
>     org-babel-execute-buffer and the behaviour upon exporting.
>
>  2. Sort parsed keywords from regular ones at the syntax level, much
>     like we did for export blocks recently. I.e., every keyword is
>     parsed expect those marked as verbatim. I don't have an idea bout
>     the involved syntax.
>
>     This would probably induce some backward incompatibility.
>
> WDYT?

Thanks for your proposal.

I would vote for solution #1.  The discrepancy between 
org-babel-execute-buffer and the behaviour upon exporting that you 
mention would not bother me.

Whatever solution is adopted (or if the current behavior is kept), the 
info documentation must be updated.  Do you agree with the patch that I 
proposed earlier in this thread?

Best,

Rafael

Re: [PATCH] call_*() is not working inside #+DATE

[Rafael Laboissiere]

* Rafael Laboissiere <rafael@laboissiere.net> [2016-03-12 08:57]:
>
> [snip]
>
> I investigated this issue further and discovered that the constructs 
> call_<name>(args) and src_<lang>{code} are not evaluated at all when 
> they appear in a keyword line (starting with "#+<keyword>:"). 
> Org-babel behaves in this way probably on purpose.  At any rate, it 
> would be better if the current behavior is documented somewhere.  This 
> may be accomplished with the patch attached below.

I went ahead and committed the patch.

Rafael Laboissière

Re: [PATCH] call_*() is not working inside #+DATE

[Nicolas Goaziou]

Hello,

Rafael Laboissiere <rafael@laboissiere.net> writes:

> * Rafael Laboissiere <rafael@laboissiere.net> [2016-03-12 08:57]:
>>
>> [snip]
>>
>> I investigated this issue further and discovered that the constructs
>> call_<name>(args) and src_<lang>{code} are not evaluated at all when
>> they appear in a keyword line (starting with "#+<keyword>:").
>> Org-babel behaves in this way probably on purpose.  At any rate, it
>> would be better if the current behavior is documented somewhere.
>> This may be accomplished with the patch attached below.
>
> I went ahead and committed the patch.

I think this is a bit premature, as we're still discussing how to fix
this issue, so this documentation patch is likely to be removed soon.

Moreover, I don't think the problem doesn't appear on maint branch,
where you applied your patch. So, at the very least, it should be
removed from there.


Regards,

-- 
Nicolas Goaziou

Re: [PATCH] call_*() is not working inside #+DATE

[Rafael Laboissiere]

* Nicolas Goaziou <mail@nicolasgoaziou.fr> [2016-03-14 20:41]:

> Rafael Laboissiere <rafael@laboissiere.net> writes:
>
>> * Rafael Laboissiere <rafael@laboissiere.net> [2016-03-12 08:57]:
>>>
>>> [snip]
>>>
>> I went ahead and committed the patch.
>
> I think this is a bit premature, as we're still discussing how to fix 
> this issue, so this documentation patch is likely to be removed soon.

Hopefully, a solution will be found soon and the documentation will be 
adjusted accordingly.  Notice that, in section "Evaluating code blocks", 
it is written "It is also possible to evaluate named code blocks from 
anywhere […]".

> Moreover, I don't think the problem doesn't appear on maint branch, 
> where you applied your patch. So, at the very least, it should be 
> removed from there.

The problem affects both master and maint, currently.

Rafael Laboissière

Re: [PATCH] call_*() is not working inside #+DATE

[Nicolas Goaziou]

Hello,

Rafael Laboissiere <rafael@laboissiere.net> writes:

> Hopefully, a solution will be found soon and the documentation will be
> adjusted accordingly.  Notice that, in section "Evaluating code
> blocks", it is written "It is also possible to evaluate named code
> blocks from anywhere […]".

The problem is that the documentation patch is (partly) wrong. From
maint, you can try calling `org-babel-execute-buffer' in the following
document

  #+DATE: src_emacs-lisp{(+ 1 1)}

It is possible to evaluate code snippets in keywords. It is not
possible, AFAIR, to evaluate them during export.

> The problem affects both master and maint, currently.

See above. I don't see the point of documenting a bug. It doesn't help
fixing it. Anyway, it's a minor issue. Let's not waste too much time on
this.


Regards,

-- 
Nicolas Goaziou

Re: [PATCH] call_*() is not working inside #+DATE

[Rafael Laboissiere]

* Nicolas Goaziou <mail@nicolasgoaziou.fr> [2016-03-15 19:50]:

> The problem is that the documentation patch is (partly) wrong. From 
> maint, you can try calling `org-babel-execute-buffer' in the following 
> document
>
>  #+DATE: src_emacs-lisp{(+ 1 1)}
>
> It is possible to evaluate code snippets in keywords.

It is strange, the inline source code block above is not evaluated for me 
from maint [commit 3e79c60] when calling `org-babel-execute-buffer'.

Rafael