Bug formatting source code in new latex exporter

Bug formatting source code in new latex exporter

[Rick Frankel]

Sorry, previous mail seems to have gotten munged, lets' try again.

There is a bug with ox-latex and long listings. If the listing has a
label (name) or caption, it is wrapped in a '\begin{listing}[H]'
block. This causes listings longer than one page to be truncated if
they have labels, which means you can't have callable code longer than
one page (~40 lines for US Letter paper) and print it with minted.

The problem is on line 2178 of ox-latex:
 
  (when (or label caption)

should probably be:
       
  (when caption


An example document is below.

rick

---
#+TITLE:    Test
#+OPTIONS:  H:3 num:nil toc:nil \n:nil @:t ::t |:t ^:{} -:t f:t *:t <:t

* Setup and short examples
:PROPERTIES:
:EXPORTS:  code
:END:
*NOTE:* Run this section to generate the long examples before
generating the latex/pdf output.
** latex listing options
#+BEGIN_SRC elisp :results silent
  (setq
  org-latex-listings 'minted
  org-latex-minted-options
  '(("linenos" "true") ("stepnumber" "5") ("numbersep" "0.25em")
    ("frame" "leftline") ("framerule" "1pt")
    ("rulecolor" "\\color{framecolor}")))
#+END_SRC

** source w/ no label
#+BEGIN_SRC perl :eval never
  foreach my  $i qw(with without) {
      print join(
          "\n",
          sprintf("* Long listing %s label", $i),
          ($i eq 'with' ? '#+name: long-listing' : ''),
          "#+BEGIN_SRC perl :exports code",
          (map { "print '$_'" } 1..60),
          "#+END_SRC\n",
      );
  }
#+END_SRC
** src w/ with label
#+name: generate-listing
#+BEGIN_SRC perl :results raw
  foreach my  $i qw(with without) {
      print join(
          "\n",
          sprintf("* Long listing %s label", $i),
          ($i eq 'with' ? '#+name: long-listing' : ''),
          "#+BEGIN_SRC perl :exports code",
          (map { "print '$_'" } 1..60),
          "#+END_SRC\n",
      );
  }
#+END_SRC

Re: Bug formatting source code in new latex exporter

[Nicolas Goaziou]

Hello,

Rick Frankel <rick@rickster.com> writes:

> There is a bug with ox-latex and long listings. If the listing has a
> label (name) or caption, it is wrapped in a '\begin{listing}[H]'
> block. This causes listings longer than one page to be truncated if
> they have labels, which means you can't have callable code longer than
> one page (~40 lines for US Letter paper) and print it with minted.
>
> The problem is on line 2178 of ox-latex:
>  
>   (when (or label caption)
>
> should probably be:
>        
>   (when caption
>
>
> An example document is below.

This is a limitation from floats. But wrapping code within a listings
environment is, IMO, the right thing to do, otherwise, cross-references
will not work.

A hack around this would be to drop the environment when source code
exceeds 30 lines, but that's cheesy for sure.

Or, maybe, drop the environment when there's only the label, but only
when there is no cross-reference pointing to the src-block within the
whole parse-tree.


Regards,

-- 
Nicolas Goaziou

Re: Bug formatting source code in new latex exporter

[Rick Frankel]

At Thu, 21 Mar 2013 20:26:34 +0100,
Nicolas Goaziou wrote:
> 
> Hello,
> 
> Rick Frankel <rick@rickster.com> writes:
> 
> > The problem is on line 2178 of ox-latex:
> >  
> >   (when (or label caption)
> >
> > should probably be:
> >        
> >   (when caption
> >
> This is a limitation from floats. But wrapping code within a listings
> environment is, IMO, the right thing to do, otherwise, cross-references
> will not work.

Personally, I generate a lot of long listings (e.g., complex sql
statements, where i use babel and org to build up a large query) and
not a lot of cross references.

> A hack around this would be to drop the environment when source code
> exceeds 30 lines, but that's cheesy for sure.

> Or, maybe, drop the environment when there's only the label, but only
> when there is no cross-reference pointing to the src-block within the
> whole parse-tree.

That sounds like the most sophisticated approach. Still, as cheesy as
it seems, I think dropping lines of source code is a bigger
problem. The cross reference approach seems clever, but maybe a
simpler approach would simply be to add an ATTR_LaTeX(:longlisting)
and leave it up to the user. Currently, I've hacked my copy of
ox-latex as show above (ignore `label' as a float wrapping specifier).


rick

Re: Bug formatting source code in new latex exporter

[Nicolas Goaziou]

[-- Attachment #1: Type: text/plain, Size: 376 bytes --]

Hello,

Rick Frankel <rick@rickster.com> writes:

> The cross reference approach seems clever, but maybe a simpler
> approach would simply be to add an ATTR_LaTeX(:longlisting) and leave
> it up to the user.

That's the most reasonable option, indeed.

The following patch implements :long-listing attribute for src-blocks.

What do you think?


Regards,

-- 
Nicolas Goaziou

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: implement long-listing attribute --]
[-- Type: text/x-patch, Size: 3702 bytes --]

From 561ac2144f6cfd21f6160a641d999e38f6f47381 Mon Sep 17 00:00:00 2001
From: Nicolas Goaziou <n.goaziou@gmail.com>
Date: Sat, 23 Mar 2013 22:10:35 +0100
Subject: [PATCH] ox-latex: :long-listing avoids wrapping src-blocks within
 floats

* lisp/ox-latex.el (org-latex-long-listings): New variable.
(org-latex-src-block): Use new variable.
---
 lisp/ox-latex.el | 41 +++++++++++++++++++++++++++++++++--------
 1 file changed, 33 insertions(+), 8 deletions(-)

diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index 715212b..c743c89 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -95,6 +95,9 @@
 ;; Special blocks accept `:options' as attribute.  Its value will be
 ;; appended as-is to the opening string of the environment created.
 ;;
+;; Source blocks accept `:long-listing' attribute, which prevents the
+;; block to be wrapped within a float when non-nil.
+;;
 ;; This back-end also offers enhanced support for footnotes.  Thus, it
 ;; handles nested footnotes, footnotes in tables and footnotes in item
 ;; descriptions.
@@ -831,6 +834,20 @@ options will be applied to blocks of all languages."
 	   (string :tag "Minted option name ")
 	   (string :tag "Minted option value"))))
 
+(defcustom org-latex-long-listings nil
+  "When non-nil no listing will be wrapped within a float.
+
+Removing floats may break some functionalities.  For example, it
+will be impossible to use cross-references to listings when using
+`minted' set-up when this variable is non-nil.
+
+This value can be locally ignored with \":long-listing t\" and
+\":long-listing nil\" LaTeX attributes."
+  :group 'org-export-latex
+  :version "24.4"
+  :package-version '(Org . "8.0")
+  :type 'boolean)
+
 (defvar org-latex-custom-lang-environments nil
   "Alist mapping languages to language-specific LaTeX environments.
 
@@ -2156,16 +2173,23 @@ contextual information."
 	   (num-start (case (org-element-property :number-lines src-block)
 			(continued (org-export-get-loc src-block info))
 			(new 0)))
-	   (retain-labels (org-element-property :retain-labels src-block)))
+	   (retain-labels (org-element-property :retain-labels src-block))
+	   (long-listing
+	    (let ((attr (org-export-read-attribute :attr_latex src-block)))
+	      (if (plist-member attr :long-listing)
+		  (plist-get attr :long-listing)
+		org-latex-long-listings))))
       (cond
        ;; Case 1.  No source fontification.
        ((not org-latex-listings)
-	(let ((caption-str (org-latex--caption/label-string src-block info))
-	      (float-env (and caption "\\begin{figure}[H]\n%s\n\\end{figure}")))
+	(let* ((caption-str (org-latex--caption/label-string src-block info))
+	       (float-env (and (not long-listing)
+			       (or label caption)
+			       (format "\\begin{figure}[H]\n%s%%s\n\\end{figure}"
+				       caption-str))))
 	  (format
 	   (or float-env "%s")
-	   (concat caption-str
-		   (format "\\begin{verbatim}\n%s\\end{verbatim}"
+	   (concat (format "\\begin{verbatim}\n%s\\end{verbatim}"
 			   (org-export-format-code-default src-block info))))))
        ;; Case 2.  Custom environment.
        (custom-env (format "\\begin{%s}\n%s\\end{%s}\n"
@@ -2175,9 +2199,10 @@ contextual information."
        ;; Case 3.  Use minted package.
        ((eq org-latex-listings 'minted)
 	(let ((float-env
-	       (when (or label caption)
-		 (format "\\begin{listing}[H]\n%%s\n%s\\end{listing}"
-			 (org-latex--caption/label-string src-block info))))
+	       (and (not long-listing)
+		    (or label caption)
+		    (format "\\begin{listing}[H]\n%%s\n%s\\end{listing}"
+			    (org-latex--caption/label-string src-block info))))
 	      (body
 	       (format
 		"\\begin{minted}[%s]{%s}\n%s\\end{minted}"
-- 
1.8.2

Re: Bug formatting source code in new latex exporter

[Rick Frankel]

> > The cross reference approach seems clever, but maybe a simpler
> > approach would simply be to add an ATTR_LaTeX(:longlisting) and leave
> > it up to the user.
> 
> That's the most reasonable option, indeed.
> 
> The following patch implements :long-listing attribute for src-blocks.
> 
> What do you think?

Works for me. BTW, a couple of other small things:

      1. I think `elisp' should be added to the default 
      `org-latex-minted-langs'.
      2. Unrelated, but I spent some time trying to get relative file
      links working. At least in Acrobat Reader on windows, the only
      way file links work is with no protocol at all
      (\url{path/to/file}).

rick

Re: Bug formatting source code in new latex exporter

[Nicolas Goaziou]

Hello,

Rick Frankel <rick@rickster.com> writes:

>> > The cross reference approach seems clever, but maybe a simpler
>> > approach would simply be to add an ATTR_LaTeX(:longlisting) and leave
>> > it up to the user.
>> 
>> That's the most reasonable option, indeed.
>> 
>> The following patch implements :long-listing attribute for src-blocks.
>> 
>> What do you think?
>
> Works for me.

Good. I wonder if :long wouldn't be better. Since the attribute only
applies to src-blocks, the "listing" is redundant.

> BTW, a couple of other small things:
>
>       1. I think `elisp' should be added to the default 
>       `org-latex-minted-langs'.

There is no "elisp" language in Babel, is it? I think it's "emacs-lisp".

>       2. Unrelated, but I spent some time trying to get relative file
>       links working. At least in Acrobat Reader on windows, the only
>       way file links work is with no protocol at all
>       (\url{path/to/file}).

Do you mean the "file:" part should be dropped for files with a relative
path?


Regards,

-- 
Nicolas Goaziou

Re: Bug formatting source code in new latex exporter

[Rick Frankel]

On Mon, Mar 25, 2013 at 10:09:37PM +0100, Nicolas Goaziou wrote:
> Hello,
> 
> Rick Frankel <rick@rickster.com> writes:
> 
> >> > The cross reference approach seems clever, but maybe a simpler
> >> > approach would simply be to add an ATTR_LaTeX(:longlisting) and leave
> >> > it up to the user.
> >> 
> >> That's the most reasonable option, indeed.
> >> 
> >> The following patch implements :long-listing attribute for src-blocks.
> >> 
> >> What do you think?
> >
> > Works for me.
> 
> Good. I wonder if :long wouldn't be better. Since the attribute only
> applies to src-blocks, the "listing" is redundant.

Either way. `:long t' seems a bit less self-documenting than
`:long-listing', but shorter is always better w/ attributes.

> > BTW, a couple of other small things:
> >
> >       1. I think `elisp' should be added to the default 
> >       `org-latex-minted-langs'.
> 
> There is no "elisp" language in Babel, is it? I think it's "emacs-lisp".

Maybe it's an in-built alias. This works w/ emacs -Q [...]:

#+BEGIN_SRC elisp
  "hello"
#+END_SRC

> >       2. Unrelated, but I spent some time trying to get relative file
> >       links working. At least in Acrobat Reader on windows, the only
> >       way file links work is with no protocol at all
> >       (\url{path/to/file}).
> 
> Do you mean the "file:" part should be dropped for files with a relative
> path?

Yes. Actually, I think it should be dropped from ALL "file:" urls,
relative or absolute.

rick

Re: Bug formatting source code in new latex exporter

[Nicolas Goaziou]

Hello,

Rick Frankel <rick@rickster.com> writes:

> Either way. `:long t' seems a bit less self-documenting than
> `:long-listing', but shorter is always better w/ attributes.

Applied, with :long-listing.

:long is too short...

>> > BTW, a couple of other small things:
>> >
>> >       1. I think `elisp' should be added to the default 
>> >       `org-latex-minted-langs'.
>> 
>> There is no "elisp" language in Babel, is it? I think it's "emacs-lisp".
>
> Maybe it's an in-built alias. This works w/ emacs -Q [...]:
>
> #+BEGIN_SRC elisp
>   "hello"
> #+END_SRC

I don't see any reference to it in the "Languages" section in the
manual. Therefore, I don't think it deserves an entry in
`org-latex-minted-langs' until it becomes more official.

>> >       2. Unrelated, but I spent some time trying to get relative file
>> >       links working. At least in Acrobat Reader on windows, the only
>> >       way file links work is with no protocol at all
>> >       (\url{path/to/file}).
>> 
>> Do you mean the "file:" part should be dropped for files with a relative
>> path?
>
> Yes. Actually, I think it should be dropped from ALL "file:" urls,
> relative or absolute.

I think it could break things. Did you use correct path to relative
files (i.e. did you include "./" before the file name)?


Regards,

-- 
Nicolas Goaziou

Re: Bug formatting source code in new latex exporter

[Rick Frankel]

On Wed, Mar 27, 2013 at 03:17:26PM +0100, Nicolas Goaziou wrote:
> Hello,
> 
> > Maybe it's an in-built alias. This works w/ emacs -Q [...]:

Probably.

> I don't see any reference to it in the "Languages" section in the
> manual. Therefore, I don't think it deserves an entry in
> `org-latex-minted-langs' until it becomes more official.

> > Yes. Actually, I think it should be dropped from ALL "file:" urls,
> > relative or absolute.
> 
> I think it could break things. Did you use correct path to relative
> files (i.e. did you include "./" before the file name)?

Yes. I've tried all possible combinations, and (with acrobat reader),
the only way to get relative urls to work is w/o a protocol
prefix. The absolute urls work just fine w/o the file: prefix as well.

Here's an org snippet you can process to test the various possiblities
(you need a file "t.html" in the same directory as the pdf):

- [[file:/Users/rick/tmp/t.html]]
  - file ::
  #+LaTeX: \url{file:/Users/rick/tmp/t.html}
  - raw ::
  #+LaTeX: \url{/Users/rick/tmp/t.html}
- [[file:./t.html]]
  - file ::
  #+LaTeX: \url{file:./t.html}
  - raw ::
  #+LaTeX: \url{./t.html}
- [[file:t.html]]
  - file ::
  #+LaTeX: \url{file:t.html}
  - raw ::
  #+LaTeX: \url{t.html}

Re: Bug formatting source code in new latex exporter

[Nicolas Goaziou]

Hello,

Rick Frankel <rick@rickster.com> writes:

> Yes. I've tried all possible combinations, and (with acrobat reader),
> the only way to get relative urls to work is w/o a protocol
> prefix. The absolute urls work just fine w/o the file: prefix as well.

It looks like "file://" is still compulsory for absolute paths on my
side. Anyway, I pushed a commit to remove the prefix for relative paths.
Does it work as expected for you, now?

Thanks for reporting this.


Regards,

-- 
Nicolas Goaziou

Bug formatting source code in new latex exporter

[Rick Frankel]

[-- Attachment #1: Type: application/pgp, Size: 1841 bytes --]