Exporting to multiple files

Exporting to multiple files

[Marcin Borkowski]

Hi all,

I'd like to export an Org-mode file to /multiple/ HTML files.  For
instance, I might want to convert all first and second level headings
to files, and third-level headings to <h1>, fourth-level ones to <h2>
inside these files etc.  Is that possible?  I looked into the docs, but
didn't find anything like this.

Of course, there's an issue of /naming/ these files.  Basing the names
on the text of the heading would be the best thing to have, but it's
not strictly necessary for me - numbers or (as a /very last/ resort)
some kind of SHA hashes might do as well.

(In fact, I'd like to write a custom HTML exporter based on the HTML
one, so that I'd be fine with some Elisp to do this as opposed to just
setting some variables.)

TIA,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Adam Mickiewicz University

Re: Exporting to multiple files

[Marcin Borkowski]

Dnia 2014-03-09, o godz. 02:34:09
Marcin Borkowski <mbork@wmi.amu.edu.pl> napisał(a):

> Hi all,
> 
> I'd like to export an Org-mode file to /multiple/ HTML files.  For
> instance, I might want to convert all first and second level headings
> to files, and third-level headings to <h1>, fourth-level ones to <h2>
> inside these files etc.  Is that possible?  I looked into the docs,
> but didn't find anything like this.
> 
> Of course, there's an issue of /naming/ these files.  Basing the names
> on the text of the heading would be the best thing to have, but it's
> not strictly necessary for me - numbers or (as a /very last/ resort)
> some kind of SHA hashes might do as well.
> 
> (In fact, I'd like to write a custom HTML exporter based on the HTML
> one, so that I'd be fine with some Elisp to do this as opposed to just
> setting some variables.)

It occurred to me that I could just walk through the file, creating
temporary buffers with my subtrees, demote them and export them one by
one to HTML.  This seems doable (and even not too complicated), but is
there a better/simpler way, or has someone already invented that wheel?

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Adam Mickiewicz University

Re: Exporting to multiple files

[Bastien]

Hi Marcin,

Marcin Borkowski <mbork@wmi.amu.edu.pl> writes:

> It occurred to me that I could just walk through the file, creating
> temporary buffers with my subtrees, demote them and export them one by
> one to HTML.  This seems doable (and even not too complicated), but is
> there a better/simpler way, or has someone already invented that wheel?

Well, no.  I thought about it a while ago and ended up willing to
competely rewrite ox-publish.el... so I stopped there :)

-- 
 Bastien

Re: Exporting to multiple files

[Fabrice Popineau]

[-- Attachment #1: Type: text/plain, Size: 1830 bytes --]

You could have a look at o-blog for example. It does something which is not
too far from what you want (IIUC).
It does it in a way similar to what you describe.

Regards,

Fabrice


2014-03-09 10:34 GMT+01:00 Marcin Borkowski <mbork@wmi.amu.edu.pl>:

> Dnia 2014-03-09, o godz. 02:34:09
> Marcin Borkowski <mbork@wmi.amu.edu.pl> napisał(a):
>
> > Hi all,
> >
> > I'd like to export an Org-mode file to /multiple/ HTML files.  For
> > instance, I might want to convert all first and second level headings
> > to files, and third-level headings to <h1>, fourth-level ones to <h2>
> > inside these files etc.  Is that possible?  I looked into the docs,
> > but didn't find anything like this.
> >
> > Of course, there's an issue of /naming/ these files.  Basing the names
> > on the text of the heading would be the best thing to have, but it's
> > not strictly necessary for me - numbers or (as a /very last/ resort)
> > some kind of SHA hashes might do as well.
> >
> > (In fact, I'd like to write a custom HTML exporter based on the HTML
> > one, so that I'd be fine with some Elisp to do this as opposed to just
> > setting some variables.)
>
> It occurred to me that I could just walk through the file, creating
> temporary buffers with my subtrees, demote them and export them one by
> one to HTML.  This seems doable (and even not too complicated), but is
> there a better/simpler way, or has someone already invented that wheel?
>
> Best,
>
> --
> Marcin Borkowski
> http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
> Adam Mickiewicz University
>
>


-- 
Fabrice Popineau
-----------------------------
SUPELEC
Département Informatique
3, rue Joliot Curie
91192 Gif/Yvette Cedex
Tel direct : +33 (0) 169851950
Standard : +33 (0) 169851212
------------------------------

[-- Attachment #2: Type: text/html, Size: 2644 bytes --]

Re: Exporting to multiple files

[Richard Lawrence]

Hi Marcin,

Marcin Borkowski <mbork@wmi.amu.edu.pl> writes:
>
> I'd like to export an Org-mode file to /multiple/ HTML files.  For
> instance, I might want to convert all first and second level headings
> to files, and third-level headings to <h1>, fourth-level ones to <h2>
> inside these files etc.  Is that possible?  I looked into the docs, but
> didn't find anything like this.

I once wrote a function that does something like this.  (I haven't used
it since the pre-8.0 days, though, so it probably needs updating.)  It
creates individual PDFs from the subtrees under a headline, then
concatenates them into one PDF using an external program (pdftk).
Naming is done based on the EXPORT_FILE_NAME property as usual.  Maybe
you can use it as a skeleton:

#+BEGIN_SRC elisp
;; utilities for exporting the subtrees of a tree as individual PDFS
;; and as a single, concatenated PDF
(defun org-export-individual-pdfs-and-concat ()
  (interactive)
  (setq export-files nil
	pdf-files nil
	; point must be in main tree to be exported (not a subtree)
	concat-pdf-name (get-property-or-fail (point) "CONCATENATED_PDF_NAME"))
  (progn
    (org-map-entries
     (lambda ()
       (setq org-map-continue-from (outline-next-heading))
       (org-mark-subtree)
       ; org-map-entries positions point at the beginning of each subtree
       (if org-map-continue-from ; non-nil if outline-next-heading found a heading
	   (let ((org-trust-scanner-tags t))
	     (push (get-property-or-fail (point) "EXPORT_FILE_NAME") export-files)))
       (mapcar 'message (org-get-tags))
       (org-export-as-pdf nil)) ; TODO: why doesn't this respect noexport tag?
     nil 'tree)
    (concat-pdfs (nreverse (mapcar 'tex-name-to-pdf-name export-files))
		 concat-pdf-name)))

(defun get-property-or-fail (pom property)
  (or
   ; probably some opportunity for optimization here...see function
   ; documentation for org-map-entries
   (org-entry-get pom property)
   (error (format "Entry at %s does not define property %s" (org-heading-components) property))))

(defun tex-name-to-pdf-name (filename)
  (concat (file-name-sans-extension filename) ".pdf"))

(defun concat-pdfs (in-files out-file)
  (shell-command
   (format "pdftk %s cat output %s"
	   (mapconcat (lambda (s) s) in-files " ") ; join pdf names with spaces
	   out-file)))

#+END_SRC

Another option that occurs to me -- though it may not serve your needs
-- is to export your Org file to texinfo format.  I believe the texinfo
compiler can then generate separate separate HTML files for the
different sections in your .texi file.  Might be worth a try.
 
-- 
Best,
Richard

Re: Exporting to multiple files

[Marcin Borkowski]

Dnia 2014-03-13, o godz. 12:59:42
Bastien <bzg@gnu.org> napisał(a):

> Hi Marcin,
> 
> Marcin Borkowski <mbork@wmi.amu.edu.pl> writes:
> 
> > It occurred to me that I could just walk through the file, creating
> > temporary buffers with my subtrees, demote them and export them one
> > by one to HTML.  This seems doable (and even not too complicated),
> > but is there a better/simpler way, or has someone already invented
> > that wheel?
> 
> Well, no.  I thought about it a while ago and ended up willing to
> competely rewrite ox-publish.el... so I stopped there :)
> 

I see...  Thanks anyway!

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Adam Mickiewicz University

Re: Exporting to multiple files

[Marcin Borkowski]

Dnia 2014-03-13, o godz. 10:38:37
Richard Lawrence <richard.lawrence@berkeley.edu> napisał(a):

> Hi Marcin,
> 
> Marcin Borkowski <mbork@wmi.amu.edu.pl> writes:
> >
> > I'd like to export an Org-mode file to /multiple/ HTML files.  For
> > instance, I might want to convert all first and second level
> > headings to files, and third-level headings to <h1>, fourth-level
> > ones to <h2> inside these files etc.  Is that possible?  I looked
> > into the docs, but didn't find anything like this.
> 
> I once wrote a function that does something like this.  (I haven't
> used it since the pre-8.0 days, though, so it probably needs
> updating.)  It creates individual PDFs from the subtrees under a
> headline, then concatenates them into one PDF using an external
> program (pdftk). Naming is done based on the EXPORT_FILE_NAME
> property as usual.  Maybe you can use it as a skeleton:
> 
> #+BEGIN_SRC elisp
> ;; utilities for exporting the subtrees of a tree as individual PDFS
> ;; and as a single, concatenated PDF
> (defun org-export-individual-pdfs-and-concat ()
>   (interactive)
>   (setq export-files nil
> 	pdf-files nil
> 	; point must be in main tree to be exported (not a subtree)
> 	concat-pdf-name (get-property-or-fail (point)
> "CONCATENATED_PDF_NAME")) (progn
>     (org-map-entries
>      (lambda ()
>        (setq org-map-continue-from (outline-next-heading))
>        (org-mark-subtree)
>        ; org-map-entries positions point at the beginning of each
> subtree (if org-map-continue-from ; non-nil if outline-next-heading
> found a heading (let ((org-trust-scanner-tags t))
> 	     (push (get-property-or-fail (point) "EXPORT_FILE_NAME")
> export-files))) (mapcar 'message (org-get-tags))
>        (org-export-as-pdf nil)) ; TODO: why doesn't this respect
> noexport tag? nil 'tree)
>     (concat-pdfs (nreverse (mapcar 'tex-name-to-pdf-name
> export-files)) concat-pdf-name)))
> 
> (defun get-property-or-fail (pom property)
>   (or
>    ; probably some opportunity for optimization here...see function
>    ; documentation for org-map-entries
>    (org-entry-get pom property)
>    (error (format "Entry at %s does not define property
> %s" (org-heading-components) property))))
> 
> (defun tex-name-to-pdf-name (filename)
>   (concat (file-name-sans-extension filename) ".pdf"))
> 
> (defun concat-pdfs (in-files out-file)
>   (shell-command
>    (format "pdftk %s cat output %s"
> 	   (mapconcat (lambda (s) s) in-files " ") ; join pdf names
> with spaces out-file)))
> 
> #+END_SRC
> 
> Another option that occurs to me -- though it may not serve your needs
> -- is to export your Org file to texinfo format.  I believe the
> texinfo compiler can then generate separate separate HTML files for
> the different sections in your .texi file.  Might be worth a try.
>  

Thanks a lot!  Your code seems inspiring - I've only skimmed through it
now, but e.g. the org-map-entries is something very useful I had no
idea existed...

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Adam Mickiewicz University

Re: Exporting to multiple files

[Sacha Chua]

Marcin Borkowski <mbork@wmi.amu.edu.pl> writes:

Hello, Marcin, all!

>> > I'd like to export an Org-mode file to /multiple/ HTML files.  For
>> > instance, I might want to convert all first and second level
...
> Thanks a lot!  Your code seems inspiring - I've only skimmed through it
> now, but e.g. the org-map-entries is something very useful I had no
> idea existed...

This might also be useful: https://github.com/iani/org-publish-meta
It seems to allow you to export subtrees to different projects. Hope
that helps!

Sacha

Re: Exporting to multiple files

[Iannis Zannos]

[-- Attachment #1: Type: text/plain, Size: 813 bytes --]

Hello Sacha and other Emacs people,

thanks for pointing this out, org-publish-meta is my "child".
It is now in early beta.  I hope to start test-driving it soon.

Iannis



On Sun, Apr 6, 2014 at 11:45 PM, Sacha Chua <sacha@sachachua.com> wrote:

> Marcin Borkowski <mbork@wmi.amu.edu.pl> writes:
>
> Hello, Marcin, all!
>
> >> > I'd like to export an Org-mode file to /multiple/ HTML files.  For
> >> > instance, I might want to convert all first and second level
> ...
> > Thanks a lot!  Your code seems inspiring - I've only skimmed through it
> > now, but e.g. the org-map-entries is something very useful I had no
> > idea existed...
>
> This might also be useful: https://github.com/iani/org-publish-meta
> It seems to allow you to export subtrees to different projects. Hope
> that helps!
>
> Sacha
>
>
>

[-- Attachment #2: Type: text/html, Size: 1509 bytes --]

Re: Exporting to multiple files

[Marcin Borkowski]

On 2014-03-09, at 02:34, Marcin Borkowski wrote:

> Hi all,
>
> I'd like to export an Org-mode file to /multiple/ HTML files.  For
> instance, I might want to convert all first and second level headings
> to files, and third-level headings to <h1>, fourth-level ones to <h2>
> inside these files etc.  Is that possible?  I looked into the docs, but
> didn't find anything like this.

Hi all,

I'm now coming back to this old question of mine, now a bit more
determined to actually code something (as opposed to just thinking about
it).

One question: is it a good idea to implement my custom options, set in
the

#+:OPTIONS

line?  If yes (that's what I assume), how to do it?

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Adam Mickiewicz University

Re: Exporting to multiple files

[Eric Abrahamsen]

Marcin Borkowski <mbork@wmi.amu.edu.pl> writes:

> On 2014-03-09, at 02:34, Marcin Borkowski wrote:
>
>> Hi all,
>>
>> I'd like to export an Org-mode file to /multiple/ HTML files.  For
>> instance, I might want to convert all first and second level headings
>> to files, and third-level headings to <h1>, fourth-level ones to <h2>
>> inside these files etc.  Is that possible?  I looked into the docs, but
>> didn't find anything like this.
>
> Hi all,
>
> I'm now coming back to this old question of mine, now a bit more
> determined to actually code something (as opposed to just thinking about
> it).
>
> One question: is it a good idea to implement my custom options, set in
> the
>
> #+:OPTIONS
>
> line?  If yes (that's what I assume), how to do it?

I'm pretty sure there's nothing out of the box that does this.

I've spent time thinking about this off and on in the past, as I've been
(very slowly) chewing through a direct-to-epub export backend, and it
can be nice to have multiple HTML files in the epub package. Splitting
the files is a bit of a problem, but the real pain in the neck is
re-writing internal links so they're still valid. I thought about using
Org publishing mechanism instead of exporting, but the two systems seem
pretty locked in their paradigms: publishing is many-to-many, exporting
is one-to-one (or many-to-one, with includes). There's no built-in
option for one-to-many.

If you don't care about internal links, you'll still have to largely
short-circuit the export process. You could use org-element-map to zip
over the headlines, breaking off subtrees where appropriate and putting
them in a temporary Org-mode buffer, using org-promote-subtree to change
the depth, and then exporting the buffer to an appropriate file.

Or you could go the parsed route: split the file into buffers, parse
each buffer, remove top-level headings, increment the :level property of
all remaining headings, and then use `org-export-data' to get the HTML.

I'm sort of just thinking out loud here, I'm not sure what the best
approach would be.

Eric

Re: Exporting to multiple files

[Rasmus]

Marcin Borkowski <mbork@wmi.amu.edu.pl> writes:

> On 2014-03-09, at 02:34, Marcin Borkowski wrote:
>
>> Hi all,
>>
>> I'd like to export an Org-mode file to /multiple/ HTML files.  For
>> instance, I might want to convert all first and second level headings
>> to files, and third-level headings to <h1>, fourth-level ones to <h2>
>> inside these files etc.  Is that possible?  I looked into the docs, but
>> didn't find anything like this.
>
> Hi all,
>
> I'm now coming back to this old question of mine, now a bit more
> determined to actually code something (as opposed to just thinking about
> it).


> One question: is it a good idea to implement my custom options, set in
> the
>
> #+:OPTIONS
>
> line?

Sure; if you think so.

> If yes (that's what I assume), how to do it?

See `org-export-define-derived-backend'.  There's examples in the
contrib folder, e.g.

ox-confluence.el, ox-deck.el, ox-freemind.el, ox-gfm.el,
ox-koma-letter.el, ox-rss.el, ox-s5.el

—Rasmus

-- 
And I faced endless streams of vendor-approved Ikea furniture. . .

Re: Exporting to multiple files

[Nicolas Goaziou]

Hello,

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> I thought about using Org publishing mechanism instead of exporting,
> but the two systems seem pretty locked in their paradigms: publishing
> is many-to-many, exporting is one-to-one (or many-to-one, with
> includes). There's no built-in option for one-to-many.

Publishing can give you one-to-many since :publishing-function accepts
a lists of functions.


Regards,

-- 
Nicolas Goaziou

Re: Exporting to multiple files

[Marcin Borkowski]

On 2014-10-01, at 14:44, Nicolas Goaziou wrote:

> Hello,
>
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> I thought about using Org publishing mechanism instead of exporting,
>> but the two systems seem pretty locked in their paradigms: publishing
>> is many-to-many, exporting is one-to-one (or many-to-one, with
>> includes). There's no built-in option for one-to-many.
>
> Publishing can give you one-to-many since :publishing-function accepts
> a lists of functions.

Is it a bad idea to write an /export/ backend with one-to-many
functionality?  (Epub is probably the most obvious use case, but there
are others.)

> Regards,

Best,

-- 

Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Adam Mickiewicz University

Re: Exporting to multiple files

[Nicolas Goaziou]

Hello,

Marcin Borkowski <mbork@wmi.amu.edu.pl> writes:

> Is it a bad idea to write an /export/ backend with one-to-many
> functionality?  (Epub is probably the most obvious use case, but there
> are others.)

I don't know if that is a bad idea, but it sounds odd. In particular,
I don't get why it is "obvious" that ePub should export to many formats.


Regards,

-- 
Nicolas Goaziou

Re: Exporting to multiple files

[Eric Abrahamsen]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Hello,
>
> Marcin Borkowski <mbork@wmi.amu.edu.pl> writes:
>
>> Is it a bad idea to write an /export/ backend with one-to-many
>> functionality?  (Epub is probably the most obvious use case, but there
>> are others.)
>
> I don't know if that is a bad idea, but it sounds odd. In particular,
> I don't get why it is "obvious" that ePub should export to many formats.

Not many formats: many files. To be honest, it's not necessary for epub
to export to many separate files, but it's often done. It is easier to
deal with, if you're editing the epub afterwards. Of course, the usual
Org-mode approach is: get the file right in Org, and the exported
results should just be used as-is. But I still don't know how epubs
should be approached.

E 

Re: Exporting to multiple files

[Nicolas Goaziou]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Not many formats: many files. To be honest, it's not necessary for epub
> to export to many separate files, but it's often done. It is easier to
> deal with, if you're editing the epub afterwards. 

In this case, the back-end can generate multiple files. AFAIK, ox-odt
does something like that, except that the files are eventually zipped.


Regards,

-- 
Nicolas Goaziou

Re: Exporting to multiple files

[Eric Abrahamsen]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Not many formats: many files. To be honest, it's not necessary for epub
>> to export to many separate files, but it's often done. It is easier to
>> deal with, if you're editing the epub afterwards. 
>
> In this case, the back-end can generate multiple files. AFAIK, ox-odt
> does something like that, except that the files are eventually zipped.

I was already borrowing quite heavily from ox-odt (epub is a zipped
directory, too, and has a bunch of the same manifest/metadata stuff),
but hadn't yet tackled the problem of preserving inter-file links (I
don't think ox-odt splits the actual text into multiple files). If you
can think of any more clever approach than just
find-and-replace-plus-bookkeeping, please let me know!

Eric