orgtbl export to latex :fmt() fails

orgtbl export to latex :fmt() fails

[Thorsten Grothe]

Hi List,

I have a problem with a orgtbl which I would like to export with certain
options to latex.

The table looks like this:

#+TBLNAME: test
#+ORGTBL: SEND test orgtbl-to-latex :skip 2 :splice t :fmt (4"\\num{%s}")

| Soll |   |  | Haben |
|------+---+--+-------|
|      |   |  | 39000 |

I would like to export the 4 column of the table with the latex \num{} tag,
see :fmt (4"\\num{%s}"), I got this from the manual.

this works fine but *only* if the first column is *not* empty like so:

| Soll |   |  | Haben |
|------+---+--+-------|
|  0   |   |  | 39000 |

Result:
% BEGIN RECEIVE ORGTBL test
0 &  &  & \num{39000} \\
% END RECEIVE ORGTBL test


if the first column is *empty* the result looks like this:

| Soll |   |  | Haben |
|------+---+--+-------|
|      |   |  | 39000 |

Result:
% BEGIN RECEIVE ORGTBL test
&  & 39000 \\
% END RECEIVE ORGTBL test

the \num{} tag is not exported and one "&" in the table is missing.

I don't know what I'm doing wrong???

Thanks in advance for your help!!

Regards
Th. Grothe

Re: orgtbl export to latex :fmt() fails

[Nick Dokos]

Thorsten Grothe <info@th-grothe.de> writes:

> Hi List,
>
> I have a problem with a orgtbl which I would like to export with certain
> options to latex.
>
> The table looks like this:
>
> #+TBLNAME: test
> #+ORGTBL: SEND test orgtbl-to-latex :skip 2 :splice t :fmt (4"\\num{%s}")
>
> | Soll |   |  | Haben |
> |------+---+--+-------|
> |      |   |  | 39000 |
>
> I would like to export the 4 column of the table with the latex \num{} tag,
> see :fmt (4"\\num{%s}"), I got this from the manual.
>
> this works fine but *only* if the first column is *not* empty like so:
>
> | Soll |   |  | Haben |
> |------+---+--+-------|
> |  0   |   |  | 39000 |
>
> Result:
> % BEGIN RECEIVE ORGTBL test
> 0 &  &  & \num{39000} \\
> % END RECEIVE ORGTBL test
>
>
> if the first column is *empty* the result looks like this:
>
> | Soll |   |  | Haben |
> |------+---+--+-------|
> |      |   |  | 39000 |
>
> Result:
> % BEGIN RECEIVE ORGTBL test
> &  & 39000 \\
> % END RECEIVE ORGTBL test
>
> the \num{} tag is not exported and one "&" in the table is missing.
>
> I don't know what I'm doing wrong???
>

Nothing - there is a bug in org-table.el:org-table-clean-before-export
where the regexp that matches special chars in the first column (see

  (info "(org)Advanced features")

for the details) inadvertently matches "| | | | 3900|" and deletes the
first column. The regexp is set like this:

--8<---------------cut here---------------start------------->8---
  (let ((special (if maybe-quoted
		     "^[ \t]*| *\\\\?[\#!$*_^/ ] *|"
		   "^[ \t]*| *[\#!$*_^/ ] *|"))
--8<---------------cut here---------------end--------------->8---

and in each case I think that the space inside the second [...] is
spurious.

So try this patch:

--8<---------------cut here---------------start------------->8---
diff --git a/lisp/org-table.el b/lisp/org-table.el
index d7ef615..864493e 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -447,8 +447,8 @@ available parameters."
   "Check if the table has a marking column.
 If yes remove the column and the special lines."
   (let ((special (if maybe-quoted
-		     "^[ \t]*| *\\\\?[\#!$*_^/ ] *|"
-		   "^[ \t]*| *[\#!$*_^/ ] *|"))
+		     "^[ \t]*| *\\\\?[\#!$*_^/] *|"
+		   "^[ \t]*| *[\#!$*_^/] *|"))
 	(ignore  (if maybe-quoted
 		     "^[ \t]*| *\\\\?[!$_^/] *|"
 		   "^[ \t]*| *[!$_^/] *|")))
--8<---------------cut here---------------end--------------->8---

I think it's OK for the non-quoted case, but I'm not sure
about the quoted case (maybe-quotes is t). If there are no
objections, I'll push it later on tonight.

Just to be sure: this is a bug, so it should be committed
to the maint branch and then a merge should be done onto master -
correct?

Nick

Re: orgtbl export to latex :fmt() fails

[Thorsten Grothe]

Nick,

> Nothing - there is a bug in org-table.el:org-table-clean-before-export
> where the regexp that matches special chars in the first column (see
> 
>   (info "(org)Advanced features")
> 
> for the details) inadvertently matches "| | | | 3900|" and deletes the
> first column. The regexp is set like this:
> 
> --8<---------------cut here---------------start------------->8---
>   (let ((special (if maybe-quoted
> 		     "^[ \t]*| *\\\\?[\#!$*_^/ ] *|"
> 		   "^[ \t]*| *[\#!$*_^/ ] *|"))
> --8<---------------cut here---------------end--------------->8---
> 
> and in each case I think that the space inside the second [...] is
> spurious.
> 
> So try this patch:
> 
> --8<---------------cut here---------------start------------->8---
> diff --git a/lisp/org-table.el b/lisp/org-table.el
> index d7ef615..864493e 100644
> --- a/lisp/org-table.el
> +++ b/lisp/org-table.el
> @@ -447,8 +447,8 @@ available parameters."
>    "Check if the table has a marking column.
>  If yes remove the column and the special lines."
>    (let ((special (if maybe-quoted
> -		     "^[ \t]*| *\\\\?[\#!$*_^/ ] *|"
> -		   "^[ \t]*| *[\#!$*_^/ ] *|"))
> +		     "^[ \t]*| *\\\\?[\#!$*_^/] *|"
> +		   "^[ \t]*| *[\#!$*_^/] *|"))
>  	(ignore  (if maybe-quoted
>  		     "^[ \t]*| *\\\\?[!$_^/] *|"
>  		   "^[ \t]*| *[!$_^/] *|")))
> --8<---------------cut here---------------end--------------->8---
> 
> I think it's OK for the non-quoted case, but I'm not sure
> about the quoted case (maybe-quotes is t). If there are no
> objections, I'll push it later on tonight.
> 
> Just to be sure: this is a bug, so it should be committed
> to the maint branch and then a merge should be done onto master -
> correct?

thank you very much for your response, well I'm not an emacs guru, so my simple
question is, how to apply this patch? I'm working with archlinux here and I
installed orgmode systemwide not locally, I guess I should first install it in
my local homedir and than patch it, but how?

Sorry about this simple question :-)


Regards
Th. Grothe

Re: orgtbl export to latex :fmt() fails

[Nick Dokos]

Thorsten Grothe <info@th-grothe.de> writes:

> thank you very much for your response, well I'm not an emacs guru, so
> my simple question is, how to apply this patch? I'm working with
> archlinux here and I installed orgmode systemwide not locally, I guess
> I should first install it in my local homedir and than patch it, but
> how?
>

It depends on how exactly you installed: from git, from elpa, from a
tarball, using the org mode that came with your emacs, some other way?

If you don't mind living somewhat dangerously, you can find out where
org-table is located (M-x locate-library RET org-table RET) which is
going to be either a byte-compiled file (org-table.elc) or a .el file.
If a .el file, you can just edit it and delete the spaces (carefully: it
should like look the + lines in the patch after you are done). If it's a
.elc file, then you'll have to locate the .el file, edit it as before,
byte-compile it and then replace the .elc file with the new file.
In either case, you'll need to reload the file or restart your emacs.

I can't in all honesty recommend this procedure (it's too error-prone),
but desperate situations call for desperate measures. OTOH, if you
are not desperate, then waiting for the change to happen upstream and
then percolate down to you is a much safer (but slower) alternative.

HTH,
Nick

Re: orgtbl export to latex :fmt() fails

[Nick Dokos]

Nick Dokos <ndokos@gmail.com> writes:

> Nothing - there is a bug in org-table.el:org-table-clean-before-export
> where the regexp that matches special chars in the first column (see
>
>   (info "(org)Advanced features")
>
> for the details) inadvertently matches "| | | | 3900|" and deletes the
> first column. The regexp is set like this:
>
>   (let ((special (if maybe-quoted
> 		     "^[ \t]*| *\\\\?[\#!$*_^/ ] *|"
> 		   "^[ \t]*| *[\#!$*_^/ ] *|"))
>
>

Checking the list of special chars in the Advanced Features section
above, I see #!$*_^/ only - I'm not sure what the \ is doing in the
[...] character class. I propose getting rid of it as well as the space.

Also, org-table-clean-before-export is called from two places in
org-table.el, neither of which uses the maybe-quoted argument[fn:1], so
I propose getting rid of it and the if clauses as well. Like this:

--8<---------------cut here---------------start------------->8---
diff --git a/lisp/org-table.el b/lisp/org-table.el
index 6d649ab..bfe396e 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -436,15 +436,11 @@ available parameters."
 					   "[ \t]*|[ \t]*")))))))
 
 (defvar org-table-clean-did-remove-column nil) ; dynamically scoped
-(defun org-table-clean-before-export (lines &optional maybe-quoted)
+(defun org-table-clean-before-export (lines)
   "Check if the table has a marking column.
 If yes remove the column and the special lines."
-  (let ((special (if maybe-quoted
-		     "^[ \t]*| *\\\\?[\#!$*_^/ ] *|"
-		   "^[ \t]*| *[\#!$*_^/ ] *|"))
-	(ignore  (if maybe-quoted
-		     "^[ \t]*| *\\\\?[!$_^/] *|"
-		   "^[ \t]*| *[!$_^/] *|")))
+  (let ((special "^[ \t]*| *[#!$*_^/] *|")
+	(ignore "^[ \t]*| *[!$_^/] *|"))
     (setq org-table-clean-did-remove-column
 	  (not (memq nil
 		     (mapcar
--8<---------------cut here---------------end--------------->8---

Any objections?

Footnotes:

[fn:1] I checked contrib as well and nobody uses it there either. Although
       it's possible that somebody uses it out in the wild, I think it's
       very unlikely. But if you do use it, let me know.

Thanks,
-- 
Nick

Re: orgtbl export to latex :fmt() fails

[Thorsten Grothe]

Nick,

thanks for  your patience :-)

> It depends on how exactly you installed: from git, from elpa, from a
> tarball, using the org mode that came with your emacs, some other way?
> (...)

I installed orgmode now from elpa (org-plus-contrib) in my local homedir. So is
there another way to apply your patch or do I  have to copy and paste the
codelines?

Thank you!


Regards
Th. Grothe

Re: orgtbl export to latex :fmt() fails

[Nick Dokos]

Thorsten Grothe <info@th-grothe.de> writes:

> Nick,
>
> thanks for  your patience :-)
>
>> It depends on how exactly you installed: from git, from elpa, from a
>> tarball, using the org mode that came with your emacs, some other way?
>> (...)
>
> I installed orgmode now from elpa (org-plus-contrib) in my local
> homedir. So is there another way to apply your patch or do I have to
> copy and paste the codelines?
>

Save the message that contains the patch into a file, say
/tmp/org-table-clean-before-export.patch, change directory
to wherever elpa installed org (the top-level directory: you should
be able to see lisp/, contrib/,  etc. when you do an `ls')
and then from the shell say

$ patch -p1 < /tmp/org-table-clean-before-export.patch

I got the following output when I did that against maint - you
might get slightly different output:

,----
| patching file lisp/org-table.el
| Hunk #1 succeeded at 440 (offset -7 lines).
`----

Note btw, that `patch' saves the original file under org-table.el.orig,
so if things go wrong at any point, you can go back by copying it over
the modified file.

If that succeeds, you'll want to do a `make install' or whatever other
make target you used to install it in the first place: `make help'
describes all the options.

-- 
Nick

Re: orgtbl export to latex :fmt() fails

[Thorsten Grothe]

Nick,

> ...
> $ patch -p1 < /tmp/org-table-clean-before-export.patch
> 
> I got the following output when I did that against maint - you
> might get slightly different output:
> ...

yipieehhh it works, it was a lot of fiddling for a emacs newbe like me but now
I got it. I will test and report!

Thank you very much for your help!!

Regards
Th. Grothe

Re: orgtbl export to latex :fmt() fails

[Nick Dokos]

Nick Dokos <ndokos@gmail.com> writes:

> Nick Dokos <ndokos@gmail.com> writes:
>
>> Nothing - there is a bug in org-table.el:org-table-clean-before-export
>> where the regexp that matches special chars in the first column (see
>>
>>   (info "(org)Advanced features")
>>
>> for the details) inadvertently matches "| | | | 3900|" and deletes the
>> first column. The regexp is set like this:
>>
>>   (let ((special (if maybe-quoted
>> 		     "^[ \t]*| *\\\\?[\#!$*_^/ ] *|"
>> 		   "^[ \t]*| *[\#!$*_^/ ] *|"))
>>
>>
>
> Checking the list of special chars in the Advanced Features section
> above, I see #!$*_^/ only - I'm not sure what the \ is doing in the
> [...] character class. I propose getting rid of it as well as the space.
>
> Also, org-table-clean-before-export is called from two places in
> org-table.el, neither of which uses the maybe-quoted argument[fn:1], so
> I propose getting rid of it and the if clauses as well. Like this:
>
> diff --git a/lisp/org-table.el b/lisp/org-table.el
> index 6d649ab..bfe396e 100644
> --- a/lisp/org-table.el
> +++ b/lisp/org-table.el
> @@ -436,15 +436,11 @@ available parameters."
>  					   "[ \t]*|[ \t]*")))))))
>  
>  (defvar org-table-clean-did-remove-column nil) ; dynamically scoped
> -(defun org-table-clean-before-export (lines &optional maybe-quoted)
> +(defun org-table-clean-before-export (lines)
>    "Check if the table has a marking column.
>  If yes remove the column and the special lines."
> -  (let ((special (if maybe-quoted
> -		     "^[ \t]*| *\\\\?[\#!$*_^/ ] *|"
> -		   "^[ \t]*| *[\#!$*_^/ ] *|"))
> -	(ignore  (if maybe-quoted
> -		     "^[ \t]*| *\\\\?[!$_^/] *|"
> -		   "^[ \t]*| *[!$_^/] *|")))
> +  (let ((special "^[ \t]*| *[#!$*_^/] *|")
> +	(ignore "^[ \t]*| *[!$_^/] *|"))
>      (setq org-table-clean-did-remove-column
>  	  (not (memq nil
>  		     (mapcar
>
> Any objections?
>
> Footnotes:
>
> [fn:1] I checked contrib as well and nobody uses it there either. Although
>        it's possible that somebody uses it out in the wild, I think it's
>        very unlikely. But if you do use it, let me know.
>
> Thanks,


I pushed this to maint and merged it into master.
-- 
Nick

Re: orgtbl export to latex :fmt() fails

[Bastien]

Hi Nick,

Nick Dokos <ndokos@gmail.com> writes:

> I pushed this to maint and merged it into master.

Thanks for tracking this bug and fixing!

-- 
 Bastien