Multi-file and master files

Multi-file and master files

[Phillip Lord]

I've been writing a long document in org-mode recently. I decided to use
org-mode for two reasons: firstly, I wrote the outline plan in org
anyway and switching tools was irritating; second, I want both PDF and
HTML output. My tool of choice would have been latex and auctex where it
not for the HTML requirement.

Setting it up was a little difficult, but in general it's going okay
now. However, I am really missing the notion of "master" documents from
auctex. I have a "main" file which just includes other files each of
which contain a specific section.

Now, the export functionality works, but many org commands appear not
to work propertly. So, for example, footnotes adds new footnotes
starting from 1 in each file, but in the HTML export the earlier
footnotes get overwritten. Likewise, there is no good tool for putting
in crosslinks, compared to reftex in latex.

Am I missing things here? Or would it make more sense to move to latex,
with all the hacking that this entails for the HTML output.

Phil

Re: Multi-file and master files

[Rasmus]

Hi Phillip,

phillip.lord@newcastle.ac.uk (Phillip Lord) writes:

> I've been writing a long document in org-mode recently. I decided to use
> org-mode for two reasons: firstly, I wrote the outline plan in org
> anyway and switching tools was irritating; second, I want both PDF and
> HTML output. My tool of choice would have been latex and auctex where it
> not for the HTML requirement.
>
> Setting it up was a little difficult, but in general it's going okay
> now. However, I am really missing the notion of "master" documents from
> auctex. I have a "main" file which just includes other files each of
> which contain a specific section.
>
> Now, the export functionality works, but many org commands appear not
> to work propertly. So, for example, footnotes adds new footnotes
> starting from 1 in each file, but in the HTML export the earlier
> footnotes get overwritten. Likewise, there is no good tool for putting
> in crosslinks, compared to reftex in latex.
>
> Am I missing things here? Or would it make more sense to move to latex,
> with all the hacking that this entails for the HTML output.

I might be misunderstanding your question here, so bear(?) with me.

I think #+INCLUDE: "$FILE" should take care of what you want.
Basically, at export time, INCLUDE is expanded to the content of $FILE
in the buffer.  Footnotes should work.

Your structure should be something like:

> cat master.org
#+TITLE: my title
* intro
  Txt with footnotes
#+INCLUDE: "chapter1.org"

> cat chapter1.org
* chapter 1
  Txt with footnotes

To include something put "<I" at the beginning of the line and click
TAB.  This will make sure syntax is correct, (even if the above is
not).

Hope it helps,
Rasmus

-- 
I almost cut my hair, it happened just the other day

Re: Multi-file and master files

[Phillip Lord]

Rasmus <rasmus@gmx.us> writes:

> Hi Phillip,
>
> phillip.lord@newcastle.ac.uk (Phillip Lord) writes:
>
> I might be misunderstanding your question here, so bear(?) with me.
>
> I think #+INCLUDE: "$FILE" should take care of what you want.
> Basically, at export time, INCLUDE is expanded to the content of $FILE
> in the buffer.  Footnotes should work.

So

master.org
  include  file1.org
  include  file2.org

Now, in file1.org add a footnote gets --

 [fn:1]


* Footnote

[fn:1] Footnote one


In file2.org add a footnote gets

 [fn:1]

* Foonote
 [fn:1] Footnote two

And footnote two gets lost.



I have a similar problem with hyperlinks. The normal store link
remembers the input.org file that the link is too. So, if in the
example, above I like between file2.org and file1.org using the Store
Link menu item, the HTML is wrong (since there is no file2.html
generated).

With latex and reftex, to put in a crossref, reftex knows about the
document structure and puts the hyperlink in as an internal reference,
since it will be internal when generated.

Phil

Re: Multi-file and master files

[Rasmus]

phillip.lord@newcastle.ac.uk (Phillip Lord) writes:

> Rasmus <rasmus@gmx.us> writes:
>
>> Hi Phillip,
>>
>> phillip.lord@newcastle.ac.uk (Phillip Lord) writes:
>>
>> I might be misunderstanding your question here, so bear(?) with me.
>>
>> I think #+INCLUDE: "$FILE" should take care of what you want.
>> Basically, at export time, INCLUDE is expanded to the content of $FILE
>> in the buffer.  Footnotes should work.
>
> So
>
> master.org
>   include  file1.org
>   include  file2.org
>
> Now, in file1.org add a footnote gets --
>
>  [fn:1]
>
>
> * Footnote
>
> [fn:1] Footnote one
>
>
> In file2.org add a footnote gets
>
>  [fn:1]
>
> * Foonote
>  [fn:1] Footnote two
>
> And footnote two gets lost.


Here's a complete example that works as expected (footnotes and
hyperlinks).  Perhaps one needs to specify :minlevel to get the
desired output.  I don't know if this is desired behavior.  If not you
can make another bug report.


>$ for f in $(ls -A /tmp/include); do echo ""; echo "→ $f ←"; echo "~~~~~~~~~~~~~~~"; echo ""; cat /tmp/include/$f; done


→ c1.org ←
~~~~~~~~~~~~~~~

* chapter 1
txt1[fn:1]

* Footnotes

[fn:1] fn1


→ c2.org ←
~~~~~~~~~~~~~~~

* chapter 2
txt2[fn:1]

* Footnotes

[fn:1] fn2 



→ m.org ←
~~~~~~~~~~~~~~~

#+OPTIONS: toc:nil
#+TITLE: Overall title
#+INCLUDE: "/tmp/include/c1.org" :minlevel 1 
#+INCLUDE: "/tmp/include/c2.org" :minlevel 1
* chapter 3
  Everything was said in [[*chapter 1]] and [[*chapter 2]]...

→ m.txt ←
~~~~~~~~~~~~~~~

			    ━━━━━━━━━━━━━━━
			     OVERALL TITLE
			    ━━━━━━━━━━━━━━━


1 chapter 1
═══════════

  txt1[1]


2 chapter 2
═══════════

  txt2[2]


3 chapter 3
═══════════

  Everything was said in 1 and 2…



Footnotes
─────────

[1] fn1

[2] fn2



> I have a similar problem with hyperlinks. The normal store link
> remembers the input.org file that the link is too. So, if in the
> example, above I like between file2.org and file1.org using the Store
> Link menu item, the HTML is wrong (since there is no file2.html
> generated).

I was not able to reproduce.  But please make a receipt to get this
behavior and it can be looked into.

—Rasmus

-- 
The Kids call him Billy the Saint

Re: Multi-file and master files

[Phillip Lord]

Hmmm. I seem to have trashed by org install by loading from ELPA;
currently, interactively, I can't export at all.


debugger entered--Lisp error: (invalid-function org-with-silent-modifications)
  org-with-silent-modifications(nil)
  org-refresh-category-properties()
  org-get-category()

It's still working through by batch process which is using cask and
probably has a different version of org.

Is there a stable org-mode package repo? At the moment, both ELPA and
the org-mode repo appear to be bleeding egde...

Phil


Rasmus <rasmus@gmx.us> writes:

> phillip.lord@newcastle.ac.uk (Phillip Lord) writes:
>
>> Rasmus <rasmus@gmx.us> writes:
>>
>>> Hi Phillip,
>>>
>>> phillip.lord@newcastle.ac.uk (Phillip Lord) writes:
>>>
>>> I might be misunderstanding your question here, so bear(?) with me.
>>>
>>> I think #+INCLUDE: "$FILE" should take care of what you want.
>>> Basically, at export time, INCLUDE is expanded to the content of $FILE
>>> in the buffer.  Footnotes should work.
>>
>> So
>>
>> master.org
>>   include  file1.org
>>   include  file2.org
>>
>> Now, in file1.org add a footnote gets --
>>
>>  [fn:1]
>>
>>
>> * Footnote
>>
>> [fn:1] Footnote one
>>
>>
>> In file2.org add a footnote gets
>>
>>  [fn:1]
>>
>> * Foonote
>>  [fn:1] Footnote two
>>
>> And footnote two gets lost.
>
>
> Here's a complete example that works as expected (footnotes and
> hyperlinks).  Perhaps one needs to specify :minlevel to get the
> desired output.  I don't know if this is desired behavior.  If not you
> can make another bug report.
>
>
>>$ for f in $(ls -A /tmp/include); do echo ""; echo "→ $f ←"; echo
>> "~~~~~~~~~~~~~~~"; echo ""; cat /tmp/include/$f; done
>
>
> → c1.org ←
> ~~~~~~~~~~~~~~~
>
> * chapter 1
> txt1[fn:1]
>
> * Footnotes
>
> [fn:1] fn1
>
>
> → c2.org ←
> ~~~~~~~~~~~~~~~
>
> * chapter 2
> txt2[fn:1]
>
> * Footnotes
>
> [fn:1] fn2 
>
>
>
> → m.org ←
> ~~~~~~~~~~~~~~~
>
>
> #+OPTIONS: toc:nil
> #+TITLE: Overall title
>
> #+INCLUDE: "/tmp/include/c1.org" :minlevel 1 
> #+INCLUDE: "/tmp/include/c2.org" :minlevel 1
>
> * chapter 3
>   Everything was said in [[*chapter 1]] and [[*chapter 2]]...
>
> → m.txt ←
> ~~~~~~~~~~~~~~~
>
> 			    ━━━━━━━━━━━━━━━
> 			     OVERALL TITLE
> 			    ━━━━━━━━━━━━━━━
>
>
> 1 chapter 1
> ═══════════
>
>   txt1[1]
>
>
> 2 chapter 2
> ═══════════
>
>   txt2[2]
>
>
> 3 chapter 3
> ═══════════
>
>   Everything was said in 1 and 2…
>
>
>
> Footnotes
> ─────────
>
> [1] fn1
>
> [2] fn2
>
>
>
>> I have a similar problem with hyperlinks. The normal store link
>> remembers the input.org file that the link is too. So, if in the
>> example, above I like between file2.org and file1.org using the Store
>> Link menu item, the HTML is wrong (since there is no file2.html
>> generated).
>
> I was not able to reproduce.  But please make a receipt to get this
> behavior and it can be looked into.
>
> —Rasmus

-- 
Phillip Lord,                           Phone: +44 (0) 191 222 7827
Lecturer in Bioinformatics,             Email: phillip.lord@newcastle.ac.uk
School of Computing Science,            http://homepages.cs.ncl.ac.uk/phillip.lord
Room 914 Claremont Tower,               skype: russet_apples
Newcastle University,                   twitter: phillord
NE1 7RU                                 

Re: Multi-file and master files

[Rasmus]

Hi Phillip,

phillip.lord@newcastle.ac.uk (Phillip Lord) writes:

> Hmmm. I seem to have trashed by org install by loading from ELPA;
> currently, interactively, I can't export at all.

Do C-h f on some org functions and check that they are from the elpa
installation.

See also:

     http://orgmode.org/org.html#Installation

Namely:
> Important: you need to do this in a session where no .org file has
> been visited, i.e. where no Org built-in function have been
> loaded. Otherwise autoload Org functions will mess up the
> installation.

If you didn't adhere to this, maybe try to delete your
~/.emacs.d/elpa/org... and try installing again from emacs -q.

> It's still working through by batch process which is using cask and
> probably has a different version of org.

I don't understand what you're trying to say here.

> Is there a stable org-mode package repo? At the moment, both ELPA and
> the org-mode repo appear to be bleeding egde...

Yeah, it's the maint branch in the git repo.  Are you sure that
org-elpa isn't using maint?

  http://orgmode.org/elpa.html

BTW: I now see the bug you are talking about (missing footnotes for
second file) when I start with emacs -Q, i.e. using the org version
shipped with Emacs.

—Rasmus

-- 
May contains speling mistake

Re: Multi-file and master files

[Phillip Lord]

Rasmus <rasmus@gmx.us> writes:
> phillip.lord@newcastle.ac.uk (Phillip Lord) writes:
>
>> Hmmm. I seem to have trashed by org install by loading from ELPA;
>> currently, interactively, I can't export at all.
>
> Do C-h f on some org functions and check that they are from the elpa
> installation.
>
> See also:
>
>      http://orgmode.org/org.html#Installation
>
> Namely:
>> Important: you need to do this in a session where no .org file has
>> been visited, i.e. where no Org built-in function have been
>> loaded. Otherwise autoload Org functions will mess up the
>> installation.
>
> If you didn't adhere to this, maybe try to delete your
> ~/.emacs.d/elpa/org... and try installing again from emacs -q.

Oh dear, yes, forgot to restart after org updated.



>> It's still working through by batch process which is using cask and
>> probably has a different version of org.
>
> I don't understand what you're trying to say here.

Cask is a command line tool that I use for doing installs from ELPA
while doing continuous integration, and some batch processes. So the
org-mode that I am using in my interactive emacs and the one that is
being used in my unit tests can be different.

>
>> Is there a stable org-mode package repo? At the moment, both ELPA and
>> the org-mode repo appear to be bleeding egde...
>
> Yeah, it's the maint branch in the git repo.  Are you sure that
> org-elpa isn't using maint?
>
>   http://orgmode.org/elpa.html

from org/archive-contents

(1 (org              . [(20140929) ( ) "Outline-based notes management and organizer" tar])
   (org-plus-contrib . [(20140929) ( ) "Outline-based notes management and organizer" tar]))

from elpa/archive-contents

 (org .
      [(20140929)
       nil "Outline-based notes management and organizer" tar nil])


Far as I can tell both ELPA and the Org elpa repo have the same
version, and it's recent and fast updated. The Download and Install info
on orgmode.org says...

Stable version 8.2.7c (Jui. 2014) — tar.gz or zip (release notes).

Development version (cgit):

~$ git clone git://orgmode.org/org-mode.git

M-x list-packages RET (see Org ELPA) 

i.e. ELPA is on dailies.

Slightly confusing. I would have expected ELPA to do a stable install,
and then git for those wanting the bleeding edge. Is the only purpose
for the org-mode ELPA to hold plus-contrib which probably includes no
FSF assigned code?

I can do a git based install if I have to, but would rather not if I
don't!

> BTW: I now see the bug you are talking about (missing footnotes for
> second file) when I start with emacs -Q, i.e. using the org version
> shipped with Emacs.


Okay, I will check this more carefully!

Thanks for the help.

Phil

Re: Multi-file and master files

[Nicolas Goaziou]

Hello,

phillip.lord@newcastle.ac.uk (Phillip Lord) writes:

> Slightly confusing. I would have expected ELPA to do a stable install,
> and then git for those wanting the bleeding edge. Is the only purpose
> for the org-mode ELPA to hold plus-contrib which probably includes no
> FSF assigned code?

ELPA installs the stable (i.e. bugfix-only) branch of Org. Bleeding-edge
is 8.3_beta.


Regards,

-- 
Nicolas Goaziou

Re: Multi-file and master files

[Phillip Lord]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
> phillip.lord@newcastle.ac.uk (Phillip Lord) writes:
>
>> Slightly confusing. I would have expected ELPA to do a stable install,
>> and then git for those wanting the bleeding edge. Is the only purpose
>> for the org-mode ELPA to hold plus-contrib which probably includes no
>> FSF assigned code?
>
> ELPA installs the stable (i.e. bugfix-only) branch of Org. Bleeding-edge
> is 8.3_beta.


And the orgmode.org ELPA? As far as I can see, they have the same files
in their org package (org-mode.org has org-plus-contrib as well).

Phil