Making an index in latex export --- surprisingly difficult

Making an index in latex export --- surprisingly difficult

[Robert Goldman]

I was trying to make an index in latex export and found that it was very
difficult to make it work.  I wonder if this could be simplified.
Here's what I had to do:

1.  put \makeindex and \usepackage{makeidx} in latex export header [no
big deal]
2.  put in \index commands (I don't /believe/ #+INDEX works, but I could
be wrong)
3.  Put a \printindex in the end. [again, no big deal]
4.  Modify the org-latex-to-pdf-process to

("pdflatex -interaction nonstopmode -output-directory %o %f" "makeindex
-o %b.ind %b.idx" "pdflatex -interaction nonstopmode -output-directory
%o %f" "pdflatex -interaction nonstopmode -output-directory %o %f")

[This was /somewhat/ of a big deal.  Suggest we add support for indexing
as a built-in option, like bibtex...]

5.  The REAL big deal:  edit my texmf.cf file to break the security
protection of openout_any=p.  By default, makeindex will refuse to open
an absolute pathname.  But org-mode will only pass absolute pathnames to
makeindex (AFAICT).  Question:  it seems like bibtex should suffer from
this same restriction.  Has anyone had that problem with it?

Cheers,
r

Re: Making an index in latex export --- surprisingly difficult

[Suvayu Ali]

On Tue, 26 Apr 2011 16:09:22 -0500
Robert Goldman <rpgoldman@sift.info> wrote:

> The REAL big deal:  edit my texmf.cf file to break the security
> protection of openout_any=p.  By default, makeindex will refuse to
> open an absolute pathname.  But org-mode will only pass absolute
> pathnames to makeindex (AFAICT).  Question:  it seems like bibtex
> should suffer from this same restriction.  Has anyone had that
> problem with it?

I did and no clean solution has been found yet.
http://thread.gmane.org/gmane.emacs.orgmode/38247/focus=38562

HTH

-- 
Suvayu

Open source is the future. It sets us free.

Re: Making an index in latex export --- surprisingly difficult

[Robert Goldman]

On 4/26/11 Apr 26 -4:26 PM, Suvayu Ali wrote:
> On Tue, 26 Apr 2011 16:09:22 -0500
> Robert Goldman <rpgoldman@sift.info> wrote:
> 
>> The REAL big deal:  edit my texmf.cf file to break the security
>> protection of openout_any=p.  By default, makeindex will refuse to
>> open an absolute pathname.  But org-mode will only pass absolute
>> pathnames to makeindex (AFAICT).  Question:  it seems like bibtex
>> should suffer from this same restriction.  Has anyone had that
>> problem with it?
> 
> I did and no clean solution has been found yet.
> http://thread.gmane.org/gmane.emacs.orgmode/38247/focus=38562
> 
> HTH
> 

I looked at that thread and unfortunately it petered out (partly because
it went off into a different direction to solve an easier problem with
conflicting style files).

The last message from Eric Fraga states:

> Oh well, there goes that theory.  The web link you gave yesterday would seem to indicate that the problem is present if you invoke the bibtex command from another directory and this does not appear to be the case here.  Very strange.

It's not actually invoking the bibtex command from another directory,
AFAICT, but invoking the bibtex command on an argument that is an
/absolute/ pathname.  This is now forbidden for makeindex and bibtex (I
don't know if it's permitted for pdflatex or not, but I suspect it is,
since the pdflatex part of the export process is working --- pdflatex
may not honor openout_any=p).

I don't see why, per the discussion in the texlive distribution notes,
it should matter that you change the TEXMF pathname environment
variable, since makeindex isn't doing any metafonting!

I also don't see any obvious way for programs like org-mode to adapt to
this except possibly by explicitly changing their working directory to
be the working directory of the org files and then using relative
pathnames instead of absolute ones.  And I would like to say right now
that I don't have the FOGGIEST idea what that would mean on Windows,
which seems to think about working directories somewhat differently!

Best,
r

Re: Making an index in latex export --- surprisingly difficult

[Nick Dokos]

Robert Goldman <rpgoldman@sift.info> wrote:

> I was trying to make an index in latex export and found that it was very
> difficult to make it work.  I wonder if this could be simplified.
> Here's what I had to do:
> 
> 1.  put \makeindex and \usepackage{makeidx} in latex export header [no
> big deal]
> 2.  put in \index commands (I don't /believe/ #+INDEX works, but I could
> be wrong)
> 3.  Put a \printindex in the end. [again, no big deal]
> 4.  Modify the org-latex-to-pdf-process to
> 
> ("pdflatex -interaction nonstopmode -output-directory %o %f" "makeindex
> -o %b.ind %b.idx" "pdflatex -interaction nonstopmode -output-directory
> %o %f" "pdflatex -interaction nonstopmode -output-directory %o %f")
> 
> [This was /somewhat/ of a big deal.  Suggest we add support for indexing
> as a built-in option, like bibtex...]
> 

Maybe you can try the texi2dvi option (I think it runs makeindex), but
texi2dvi did have a bug that has caused problems here in the past:
that's the reason it's not the default setting for
org-latex-to-pdf-process. If you run into the bug but still want to try
texi2dvi anyway, see

  http://lists.gnu.org/archive/html/bug-texinfo/2010-03/msg00032.html

for the fix Karl Berry applied to the development sources.

> 5.  The REAL big deal:  edit my texmf.cf file to break the security
> protection of openout_any=p.  By default, makeindex will refuse to open
> an absolute pathname.  But org-mode will only pass absolute pathnames to
> makeindex (AFAICT).  Question:  it seems like bibtex should suffer from
> this same restriction.  Has anyone had that problem with it?
> 

I haven't had the problem, primarily because I haven't used bibtex
through org yet :-) But bibtex should exhibit the same problem: the
bibtex change predated the makeindex one. Maybe bibtex is only called
with a relative path (if that's the case, then the same method should
cure makeindex as well). BTW, ".." is not allowed in the relative path:
you can only use subdirectories of the current directory.

Nick

Re: Making an index in latex export --- surprisingly difficult

[Robert Goldman]

On 4/26/11 Apr 26 -4:52 PM, Nick Dokos wrote:
> Robert Goldman <rpgoldman@sift.info> wrote:
> 
>> I was trying to make an index in latex export and found that it was very
>> difficult to make it work.  I wonder if this could be simplified.
>> Here's what I had to do:
>>
....
>> 2.  put in \index commands (I don't /believe/ #+INDEX works, but I could
>> be wrong)

Am I right about #+INDEX not being translated in the latex back end (it
seems not to work, but it's hard to verify a negative)?

If so, would this be hard to fix?  If it wouldn't, seems like that would
be A Good Thing.

....
>> 4.  Modify the org-latex-to-pdf-process to
>>
>> ("pdflatex -interaction nonstopmode -output-directory %o %f" "makeindex
>> -o %b.ind %b.idx" "pdflatex -interaction nonstopmode -output-directory
>> %o %f" "pdflatex -interaction nonstopmode -output-directory %o %f")
>>
>> [This was /somewhat/ of a big deal.  Suggest we add support for indexing
>> as a built-in option, like bibtex...]
>>
> 
> Maybe you can try the texi2dvi option (I think it runs makeindex), but
> texi2dvi did have a bug that has caused problems here in the past:
> that's the reason it's not the default setting for
> org-latex-to-pdf-process. If you run into the bug but still want to try
> texi2dvi anyway, see
> 
>   http://lists.gnu.org/archive/html/bug-texinfo/2010-03/msg00032.html
> 
> for the fix Karl Berry applied to the development sources.

Once one has makeindex hacked in there (oh, and you've broken the
security!), it works fine, thanks.
> 
>> 5.  The REAL big deal:  edit my texmf.cf file to break the security
>> protection of openout_any=p.  By default, makeindex will refuse to open
>> an absolute pathname.  But org-mode will only pass absolute pathnames to
>> makeindex (AFAICT).  Question:  it seems like bibtex should suffer from
>> this same restriction.  Has anyone had that problem with it?
>>
> 
> I haven't had the problem, primarily because I haven't used bibtex
> through org yet :-) But bibtex should exhibit the same problem: the
> bibtex change predated the makeindex one. Maybe bibtex is only called
> with a relative path (if that's the case, then the same method should
> cure makeindex as well). BTW, ".." is not allowed in the relative path:
> you can only use subdirectories of the current directory.
> 
> Nick
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 

Re: Making an index in latex export --- surprisingly difficult

[Nick Dokos]

Robert Goldman <rpgoldman@sift.info> wrote:

> >> 2.  put in \index commands (I don't /believe/ #+INDEX works, but I could
> >> be wrong)
> 
> Am I right about #+INDEX not being translated in the latex back end (it
> seems not to work, but it's hard to verify a negative)?
> 

Seems to be a publishing thingie only. Adding a publishing project

--8<---------------cut here---------------start------------->8---
        ...
        ("foo"
         :base-directory "~/src/org/latex/index/"
         :publishing-directory "~/src/org/latex/index"
         :publishing-function org-publish-org-to-latex
         :makeindex t
        )
        ...
--8<---------------cut here---------------end--------------->8---

to org-publish-project-alist and publishing the following org file

--8<---------------cut here---------------start------------->8---

#+INDEX: foo


* this

is a test of foo
--8<---------------cut here---------------end--------------->8---


generates \index{foo} in the TeX file.

Nick

> If so, would this be hard to fix?  If it wouldn't, seems like that would
> be A Good Thing.
> 

Re: Making an index in latex export --- surprisingly difficult

[Robert Goldman]

On 4/26/11 Apr 26 -6:14 PM, Nick Dokos wrote:
> Robert Goldman <rpgoldman@sift.info> wrote:
> 
>>>> 2.  put in \index commands (I don't /believe/ #+INDEX works, but I could
>>>> be wrong)
>>
>> Am I right about #+INDEX not being translated in the latex back end (it
>> seems not to work, but it's hard to verify a negative)?
>>
> 
> Seems to be a publishing thingie only. 

...snip...

Working on a patch to org-latex that will translate #+INDEX:  So far
it's a little more difficult than I'd hoped.  If we have that, then
adding something to automagically invoke the right latex fu to
accumulate and print the index should be relatively trivial.  Even
adding a new export method should be ok (modulo fixing the general
bibtex + makeindex privs issue).

R

Re: Making an index in latex export --- surprisingly difficult

[Nick Dokos]

Robert Goldman <rpgoldman@sift.info> wrote:

> >> 4.  Modify the org-latex-to-pdf-process to
> >>
> >> ("pdflatex -interaction nonstopmode -output-directory %o %f" "makeindex
> >> -o %b.ind %b.idx" "pdflatex -interaction nonstopmode -output-directory
> >> %o %f" "pdflatex -interaction nonstopmode -output-directory %o %f")
> >>
> >> [This was /somewhat/ of a big deal.  Suggest we add support for indexing
> >> as a built-in option, like bibtex...]
> >>
> > 
> > Maybe you can try the texi2dvi option (I think it runs makeindex), but
> > texi2dvi did have a bug that has caused problems here in the past:
> > that's the reason it's not the default setting for
> > org-latex-to-pdf-process. If you run into the bug but still want to try
> > texi2dvi anyway, see
> > 
> >   http://lists.gnu.org/archive/html/bug-texinfo/2010-03/msg00032.html
> > 
> > for the fix Karl Berry applied to the development sources.
> 
> Once one has makeindex hacked in there (oh, and you've broken the
> security!), it works fine, thanks.
> > 
> >> an absolute pathname.  But org-mode will only pass absolute pathnames to
> >> makeindex (AFAICT).  Question:  it seems like bibtex should suffer from
> >> this same restriction.  Has anyone had that problem with it?
> >>
> > 
> > I haven't had the problem, primarily because I haven't used bibtex
> > through org yet :-) But bibtex should exhibit the same problem: the
> > bibtex change predated the makeindex one. Maybe bibtex is only called
> > with a relative path (if that's the case, then the same method should
> > cure makeindex as well). BTW, ".." is not allowed in the relative path:
> > you can only use subdirectories of the current directory.
> > 

Here's a step-by-step that uses texi2dvi[fn:1]:

o Download a bibliography file:

  cd /home/nick/lib/bibtex
  wget -nd http://ftp.math.utah.edu/pub//tex/bib/linux.html
  mv linux.html linux.bib

o Set the BIBINPUTS environment variable in .profile (or similar):

  BIBINPUTS=/home/nick/lib/bibtex:$BIBINPUTS
  export BIBINPUTS

o Set org-latex-to-pdf-process in .emacs (or similar):

  (setq org-latex-to-pdf-process '("texi2dvi --pdf --tidy --batch %f"))

  The --tidy keeps all the intermediate files tucked away in a subdirectory.
  You can use --clean instead if you want automatic cleanup of intermediate
  files.

That's all for one-time setup. There is no problem with bibtex/makeindex
security because all the output files end up in a subdirectory and (I guess)
texi2dvi uses relative pathnames.

Here's an org file with an index and a bibliography that is properly exported
to PDF with the above setup:

--8<---------------cut here---------------start------------->8---
#+LATEX_HEADER: \usepackage{makeidx}
#+LATEX_HEADER: \makeindex

#+LaTeX: \bibliographystyle{plain}

* this

#+LaTeX: \index{foo}
is a test of foo \cite{Torvalds:1999:LE}

#+LaTeX: \bibliography{linux}
#+LaTeX: \printindex
--8<---------------cut here---------------end--------------->8---

Nick

Footnotes:

[fn:1] My egrep does not choke on the suspect [A-z] pattern that
texi2dvi uses, so I don't need Karl Berry's patch - if your egrep does
choke, the patch just changes the pattern to [A-Za-z] which although not
exactly the same, since it omits a few punctuation chars that come
between Z and a in ASCII[fn:2], is good enough.

[fn:2] The pattern tries to recognize legal drive names on DOS/Windoze,
and these characters are *legal drive names*!! We have Novell to thank
for that though, not M$. OTOH, if you are on Windoze and have a drive
named `[', watch out ;-)

Re: Making an index in latex export --- surprisingly difficult

[Thomas S. Dye]

Aloha Nick,

Brilliant.  Thanks!

Tom

On Apr 26, 2011, at 8:35 PM, Nick Dokos wrote:

> Robert Goldman <rpgoldman@sift.info> wrote:
>
>>>> 4.  Modify the org-latex-to-pdf-process to
>>>>
>>>> ("pdflatex -interaction nonstopmode -output-directory %o %f"  
>>>> "makeindex
>>>> -o %b.ind %b.idx" "pdflatex -interaction nonstopmode -output- 
>>>> directory
>>>> %o %f" "pdflatex -interaction nonstopmode -output-directory %o %f")
>>>>
>>>> [This was /somewhat/ of a big deal.  Suggest we add support for  
>>>> indexing
>>>> as a built-in option, like bibtex...]
>>>>
>>>
>>> Maybe you can try the texi2dvi option (I think it runs makeindex),  
>>> but
>>> texi2dvi did have a bug that has caused problems here in the past:
>>> that's the reason it's not the default setting for
>>> org-latex-to-pdf-process. If you run into the bug but still want  
>>> to try
>>> texi2dvi anyway, see
>>>
>>>  http://lists.gnu.org/archive/html/bug-texinfo/2010-03/msg00032.html
>>>
>>> for the fix Karl Berry applied to the development sources.
>>
>> Once one has makeindex hacked in there (oh, and you've broken the
>> security!), it works fine, thanks.
>>>
>>>> an absolute pathname.  But org-mode will only pass absolute  
>>>> pathnames to
>>>> makeindex (AFAICT).  Question:  it seems like bibtex should  
>>>> suffer from
>>>> this same restriction.  Has anyone had that problem with it?
>>>>
>>>
>>> I haven't had the problem, primarily because I haven't used bibtex
>>> through org yet :-) But bibtex should exhibit the same problem: the
>>> bibtex change predated the makeindex one. Maybe bibtex is only  
>>> called
>>> with a relative path (if that's the case, then the same method  
>>> should
>>> cure makeindex as well). BTW, ".." is not allowed in the relative  
>>> path:
>>> you can only use subdirectories of the current directory.
>>>
>
> Here's a step-by-step that uses texi2dvi[fn:1]:
>
> o Download a bibliography file:
>
>  cd /home/nick/lib/bibtex
>  wget -nd http://ftp.math.utah.edu/pub//tex/bib/linux.html
>  mv linux.html linux.bib
>
> o Set the BIBINPUTS environment variable in .profile (or similar):
>
>  BIBINPUTS=/home/nick/lib/bibtex:$BIBINPUTS
>  export BIBINPUTS
>
> o Set org-latex-to-pdf-process in .emacs (or similar):
>
>  (setq org-latex-to-pdf-process '("texi2dvi --pdf --tidy --batch %f"))
>
>  The --tidy keeps all the intermediate files tucked away in a  
> subdirectory.
>  You can use --clean instead if you want automatic cleanup of  
> intermediate
>  files.
>
> That's all for one-time setup. There is no problem with bibtex/ 
> makeindex
> security because all the output files end up in a subdirectory and  
> (I guess)
> texi2dvi uses relative pathnames.
>
> Here's an org file with an index and a bibliography that is properly  
> exported
> to PDF with the above setup:
>
> --8<---------------cut here---------------start------------->8---
> #+LATEX_HEADER: \usepackage{makeidx}
> #+LATEX_HEADER: \makeindex
>
> #+LaTeX: \bibliographystyle{plain}
>
> * this
>
> #+LaTeX: \index{foo}
> is a test of foo \cite{Torvalds:1999:LE}
>
> #+LaTeX: \bibliography{linux}
> #+LaTeX: \printindex
> --8<---------------cut here---------------end--------------->8---
>
> Nick
>
> Footnotes:
>
> [fn:1] My egrep does not choke on the suspect [A-z] pattern that
> texi2dvi uses, so I don't need Karl Berry's patch - if your egrep does
> choke, the patch just changes the pattern to [A-Za-z] which although  
> not
> exactly the same, since it omits a few punctuation chars that come
> between Z and a in ASCII[fn:2], is good enough.
>
> [fn:2] The pattern tries to recognize legal drive names on DOS/ 
> Windoze,
> and these characters are *legal drive names*!! We have Novell to thank
> for that though, not M$. OTOH, if you are on Windoze and have a drive
> named `[', watch out ;-)
>

Re: Making an index in latex export --- surprisingly difficult

[Eric S Fraga]

Robert Goldman <rpgoldman@sift.info> writes:

[...]

> I looked at that thread and unfortunately it petered out (partly because
> it went off into a different direction to solve an easier problem with
> conflicting style files).
>
> The last message from Eric Fraga states:
>
>> Oh well, there goes that theory.  The web link you gave yesterday
>> would seem to indicate that the problem is present if you invoke the
>> bibtex command from another directory and this does not appear to be
>> the case here.  Very strange.
>
> It's not actually invoking the bibtex command from another directory,
> AFAICT, but invoking the bibtex command on an argument that is an
> /absolute/ pathname.  This is now forbidden for makeindex and bibtex (I
> don't know if it's permitted for pdflatex or not, but I suspect it is,
> since the pdflatex part of the export process is working --- pdflatex
> may not honor openout_any=p).

If the file is indeed in the /current/ directory, then a simple fix may
be to use %b for the file argument to makeindex?  I have no problems
with bibtex using

,----
| org-latex-to-pdf-process is a variable defined in `org-latex.el'.
| Its value is ("pdflatex %f" "bibtex %b" "pdflatex %f" "pdflatex %f")
`----

Note the use of %b instead of %o or %f.  I've never used makeindex so
cannot be sure this would work.

-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1
: using Org-mode version 7.5 (release_7.5.209.g1a687)

Re: Making an index in latex export --- surprisingly difficult

[Robert Goldman]

On 4/28/11 Apr 28 -4:28 AM, Eric S Fraga wrote:
> Robert Goldman <rpgoldman@sift.info> writes:
> 
> [...]
> 
>> I looked at that thread and unfortunately it petered out (partly because
>> it went off into a different direction to solve an easier problem with
>> conflicting style files).
>>
>> The last message from Eric Fraga states:
>>
>>> Oh well, there goes that theory.  The web link you gave yesterday
>>> would seem to indicate that the problem is present if you invoke the
>>> bibtex command from another directory and this does not appear to be
>>> the case here.  Very strange.
>>
>> It's not actually invoking the bibtex command from another directory,
>> AFAICT, but invoking the bibtex command on an argument that is an
>> /absolute/ pathname.  This is now forbidden for makeindex and bibtex (I
>> don't know if it's permitted for pdflatex or not, but I suspect it is,
>> since the pdflatex part of the export process is working --- pdflatex
>> may not honor openout_any=p).
> 
> If the file is indeed in the /current/ directory, then a simple fix may
> be to use %b for the file argument to makeindex?  I have no problems
> with bibtex using
> 
> ,----
> | org-latex-to-pdf-process is a variable defined in `org-latex.el'.
> | Its value is ("pdflatex %f" "bibtex %b" "pdflatex %f" "pdflatex %f")
> `----
> 
> Note the use of %b instead of %o or %f.  I've never used makeindex so
> cannot be sure this would work.
> 

No, actually this does not work, since the expansion of %b is still an
absolute pathname, rather than a relative pathname.  The only difference
is that the file extension is removed.  Here's a snippet from my
*trace-output* buffer when I trace the shell-command function:

command="makeindex -o
/Users/rpg/obtw/obtw-trunk/memos/plan-representation/manual.ind
/Users/rpg/obtw/obtw-trunk/memos/plan-representation/manual.idx"

The entry from my org-latex-to-pdf-process is:

"makeindex -o %b.ind %b.idx"

So using the basename is orthogonal to the underlying problem, which is
that absolute pathnames are always used.

When I restore the openout_any = p setting, I see:

rpg% makeindex -o
/Users/rpg/obtw/obtw-trunk/memos/plan-representation/manual.ind
/Users/rpg/obtw/obtw-trunk/memos/plan-representation/manual.idx


makeindex: Not writing to
/Users/rpg/obtw/obtw-trunk/memos/plan-representation/manual.ind
(openout_any = p).
Can't create output index file
/Users/rpg/obtw/obtw-trunk/memos/plan-representation/manual.ind.

Note that this isn't something one needs any org-mode testing to verify
--- you can just figure out what %b will expand to, and test
interactively with the shell using makeindex or bibtex, if you have a
recent texlive, with the paranoid settings.

The only "solution" I know (aside from disabling the texmf.cnf security
setting) would be to have org ensure that its CWD is the document
directory when running these programs and use relative pathnames.  I
don't know that this solution is compatible with correct use of
EXPORT_FILE_NAME, however....

This seems like a very crippling security setting -- it would break many
plausible uses of the tex suite when they are run under program control
(I could easily imagine scenarios with make that would result in the use
of absolute pathnames, too); I'm surprised it was made.

Best,
r

Re: Making an index in latex export --- surprisingly difficult

[Eric S Fraga]

Robert Goldman <rpgoldman@sift.info> writes:

[...]

>> If the file is indeed in the /current/ directory, then a simple fix may
>> be to use %b for the file argument to makeindex?  I have no problems
>> with bibtex using
>> 
>> ,----
>> | org-latex-to-pdf-process is a variable defined in `org-latex.el'.
>> | Its value is ("pdflatex %f" "bibtex %b" "pdflatex %f" "pdflatex %f")
>> `----
>> 
>> Note the use of %b instead of %o or %f.  I've never used makeindex so
>> cannot be sure this would work.
>> 
>
> No, actually this does not work, since the expansion of %b is still an
> absolute pathname, rather than a relative pathname.  The only difference
> is that the file extension is removed.  Here's a snippet from my
> *trace-output* buffer when I trace the shell-command function:

Okay, you are correct!  I have sometimes used the following shell script
in lieu of the sequence of commands:

--8<---------------cut here---------------start------------->8---
#!/bin/sh -f
# first argument should be base latex file name but may be absolute
F=$1
B=$(basename $F)
echo 'Original file: ' $F
echo 'Base file: ' $B
echo 'Invoked in dir: ' $PWD
echo '--------------------------------- running pdflatex first ---------------------------------------------------'
pdflatex $F
echo '--------------------------------- running bibtex next -------------------------------------------------------'
bibtex  $B
echo '--------------------------------- running pdflatex again (and again) ----------------------------------------'
pdflatex $F
pdflatex $F
--8<---------------cut here---------------end--------------->8---

which is =/home/ucecesf/s/bin/org2pdf.sh= (for me). I then define

#+begin_src emacs-lisp
(setq org-latex-to-pdf-process '("/home/ucecesf/s/bin/org2pdf.sh %b"))
#+end_src

You could add a line of the form:

makeindex -o ${B}.ind ${B}.idx

to this script, probably before the bibtex line?

Of course, this doesn't some any of the other problems, as you note
below:

[...]

> The only "solution" I know (aside from disabling the texmf.cnf security
> setting) would be to have org ensure that its CWD is the document
> directory when running these programs and use relative pathnames.  I
> don't know that this solution is compatible with correct use of
> EXPORT_FILE_NAME, however....

Yes, the same applies to my simple script above: it assumes that the
current directory *is* the one in which the files are.

> This seems like a very crippling security setting -- it would break many
> plausible uses of the tex suite when they are run under program control
> (I could easily imagine scenarios with make that would result in the use
> of absolute pathnames, too); I'm surprised it was made.

It does seem like a silly restriction.
-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1
: using Org-mode version 7.5 (release_7.5.209.g1a687)

Re: Making an index in latex export --- surprisingly difficult

[Tom Dye]

On 2011-04-26 20:35:06 -1000, Nick Dokos <nicholas.dokos@hp.com> said:

> Robert Goldman <rpgoldman@sift.info> wrote:
> 
>>>> 4.  Modify the org-latex-to-pdf-process to
>>>> 
>>>> ("pdflatex -interaction nonstopmode -output-directory %o %f" "makeindex
>>>> -o %b.ind %b.idx" "pdflatex -interaction nonstopmode -output-directory
>>>> %o %f" "pdflatex -interaction nonstopmode -output-directory %o %f")
>>>> 
>>>> [This was /somewhat/ of a big deal.  Suggest we add support for indexing
>>>> as a built-in option, like bibtex...]
>>>> 
>>> 
>>> Maybe you can try the texi2dvi option (I think it runs makeindex), but
>>> texi2dvi did have a bug that has caused problems here in the past:
>>> that's the reason it's not the default setting for
>>> org-latex-to-pdf-process. If you run into the bug but still want to try
>>> texi2dvi anyway, see
>>> 
>>> http://lists.gnu.org/archive/html/bug-texinfo/2010-03/msg00032.html
>>> 
>>> for the fix Karl Berry applied to the development sources.
>> 
>> Once one has makeindex hacked in there (oh, and you've broken the
>> security!), it works fine, thanks.
>>> 
>>>> an absolute pathname.  But org-mode will only pass absolute pathnames to
>>>> makeindex (AFAICT).  Question:  it seems like bibtex should suffer from
>>>> this same restriction.  Has anyone had that problem with it?
>>>> 
>>> 
>>> I haven't had the problem, primarily because I haven't used bibtex
>>> through org yet :-) But bibtex should exhibit the same problem: the
>>> bibtex change predated the makeindex one. Maybe bibtex is only called
>>> with a relative path (if that's the case, then the same method should
>>> cure makeindex as well). BTW, ".." is not allowed in the relative path:
>>> you can only use subdirectories of the current directory.
>>> 
> 
> Here's a step-by-step that uses texi2dvi[fn:1]:
> 
> o Download a bibliography file:
> 
>   cd /home/nick/lib/bibtex
>   wget -nd http://ftp.math.utah.edu/pub//tex/bib/linux.html
>   mv linux.html linux.bib
> 
> o Set the BIBINPUTS environment variable in .profile (or similar):
> 
>   BIBINPUTS=/home/nick/lib/bibtex:$BIBINPUTS
>   export BIBINPUTS
> 
> o Set org-latex-to-pdf-process in .emacs (or similar):
> 
>   (setq org-latex-to-pdf-process '("texi2dvi --pdf --tidy --batch %f"))
> 
>   The --tidy keeps all the intermediate files tucked away in a subdirectory.
>   You can use --clean instead if you want automatic cleanup of intermediate
>   files.
> 
> That's all for one-time setup. There is no problem with bibtex/makeindex
> security because all the output files end up in a subdirectory and (I guess)
> texi2dvi uses relative pathnames.
> 
> Here's an org file with an index and a bibliography that is properly exported
> to PDF with the above setup:
> 
> --8<---------------cut here---------------start------------->8---
> #+LATEX_HEADER: \usepackage{makeidx}
> #+LATEX_HEADER: \makeindex
> 
> #+LaTeX: \bibliographystyle{plain}
> 
> * this
> 
> #+LaTeX: \index{foo}
> is a test of foo \cite{Torvalds:1999:LE}
> 
> #+LaTeX: \bibliography{linux}
> #+LaTeX: \printindex
> --8<---------------cut here---------------end--------------->8---
> 
> Nick
> 
> Footnotes:
> 
> [fn:1] My egrep does not choke on the suspect [A-z] pattern that
> texi2dvi uses, so I don't need Karl Berry's patch - if your egrep does
> choke, the patch just changes the pattern to [A-Za-z] which although not
> exactly the same, since it omits a few punctuation chars that come
> between Z and a in ASCII[fn:2], is good enough.
> 
> [fn:2] The pattern tries to recognize legal drive names on DOS/Windoze,
> and these characters are *legal drive names*!! We have Novell to thank
> for that though, not M$. OTOH, if you are on Windoze and have a drive
> named `[', watch out ;-)

Aloha all,

I've found it convenient to add paths to system-wide bibliographic 
databases to BIBINPUTS in .profile, as Nick suggests, and then use a sh 
source code block for project-specific databases, like this for a 
project in /org/tsdye/:

#+source: bibinputs
#+begin_src sh
  BIBINPUTS=/org/tsdye/:$BIBINPUTS
  export BIBINPUTS
#+end_src

If I remember to execute this before I start editing, then I can add 
references to my work from either the system-wide or project-specific 
databases and LaTeX export produces an error-free pdf file full of 
references.

Tom

-- 
Tom Dye
T. S. Dye & Colleagues, Archaeologists, Inc.
Honolulu, Hawai`i