PDF docs from Org-mode: not valid PDF?

PDF docs from Org-mode: not valid PDF?

[Michael Hannon]

Greetings.  The woman that runs our research group complains that she is
unable to print PDF documents that a couple of us in the group send her, and
that the two of us are the only source of such troublesome documents.

It occurred to me that the two of us are the only ones in the group that use
Emacs Org-mode to produce PDF documents.

I know there are web sites that can validate (i.e., measure conformance to the
standard of) HTML documents.  I wondered if there might be something similar
for PDF documents.

I found the following site:

    http://www.validatepdfa.com/

I don't much about the PDF standard, and I don't know anything about the
reliability of this site, but I've found that PDF documents that I generate
using Org-mode and send to that site come back with a boatload of validation
errors.

I've appended a sample Org-mode file that generates errors when I try to
validate it.  I've appended details of my configuration below the sample file.

I should mention that I have no trouble viewing the PDF file (using "evince"
on my Fedora system) or printing this file (to a Brother b/w printer).  Ditto for
my colleague on a MacOS X system.  Hence, I personally have no reason to
complain.  But there's that nagging issue with the boss.

Can anybody comment as to what I should expect in the way of validation?
And/or are there any settings that would affect this behavior?  I.e.,
something like:

    (setq org-export-pdf-use-strict t)

Thanks.

-- Mike

###### Beginning of sample Org-mode file
#+PROPERTY: session *R*
#+PROPERTY: cache yes
#+PROPERTY: results output verbatim
#+PROPERTY: tangle yes


* This is a test

#+BEGIN_SRC R :exports both
  
  x <- 7.2
  y <- 3
  
  print(x * y)
  
  
#+END_SRC

#+RESULTS:
: 21.6
###### End of sample Org-mode file

Emacs  : GNU Emacs 24.1.1 (x86_64-redhat-linux-gnu, GTK+ Version 2.24.13)
 of 2012-11-17 on buildvm-30.phx2.fedoraproject.org
Package: Org-mode version 7.9.2 (release_7.9.2-656-g2f9dda @
/usr/local/emacs.d/org-mode/org-devel/org-mode/lisp/)

current state:
==============
(setq
 org-export-preprocess-before-selecting-backend-code-hook
'(org-beamer-select-beamer-code)
 org-tab-first-hook '(org-hide-block-toggle-maybe
              org-src-native-tab-command-maybe
              org-babel-hide-result-toggle-maybe
              org-babel-header-arg-expand)
 org-speed-command-hook '(org-speed-command-default-hook
              org-babel-speed-command-hook)
 org-export-html-after-blockquotes-hook
'(org-special-blocks-convert-html-special-cookies)
 org-occur-hook '(org-first-headline-recenter)
 org-metaup-hook '(org-babel-load-in-session-maybe)
 org-export-preprocess-before-normalizing-links-hook
'(org-remove-file-link-modifiers)
 org-confirm-shell-link-function 'yes-or-no-p
 org-export-latex-after-blockquotes-hook
'(org-special-blocks-convert-latex-special-cookies)
 org-export-latex-final-hook '(org-beamer-amend-header org-beamer-fix-toc
                   org-beamer-auto-fragile-frames
                   org-beamer-place-default-actions-for-lists)
 org-export-latex-after-initial-vars-hook '(org-beamer-after-initial-vars)
 org-after-todo-state-change-hook '(org-clock-out-if-current)
 org-from-is-user-regexp "\\<Michael Hannon\\>"
 org-src-mode-hook '(org-src-babel-configure-edit-buffer
             org-src-mode-configure-edit-buffer)
 org-agenda-before-write-hook '(org-agenda-add-entry-text)
 org-babel-pre-tangle-hook '(save-buffer)
 org-mode-hook '(#[nil "\300\301\302\303\304$\207"
           [org-add-hook change-major-mode-hook org-show-block-all
            append local]
           5]
         #[nil "\300\301\302\303\304$\207"
           [org-add-hook change-major-mode-hook
            org-babel-show-result-all append local]
           5]
         org-babel-result-hide-spec org-babel-hide-all-hashes)
 org-ctrl-c-ctrl-c-hook '(org-babel-hash-at-point
              org-babel-execute-safely-maybe)
 org-cycle-hook '(org-cycle-hide-archived-subtrees org-cycle-hide-drawers
          org-cycle-hide-inline-tasks org-cycle-show-empty-lines
          org-optimize-window-after-visibility-change)
 org-export-latex-format-toc-function 'org-export-latex-format-toc-default
 org-export-first-hook '(org-beamer-initialize-open-trackers)
 org-babel-tangle-lang-exts '(("C++" . "cpp") ("latex" . "tex")
                  ("python" . "py") ("lisp" . "lisp")
                  ("emacs-lisp" . "el"))
 org-confirm-elisp-link-function 'yes-or-no-p
 org-metadown-hook '(org-babel-pop-to-session-maybe)
 org-structure-template-alist '(("r" "#+begin_src R\n\n#+end_src"
                 "<src lang=\"R\">\n\n</src>")
                ("S" "#+BEGIN_SRC ?\n\n#+END_SRC"
                 "<src lang=\"?\">\n\n</src>")
                ("s" "#+BEGIN_SRC ?\n\n#+END_SRC"
                 "<src lang=\"?\">\n\n</src>")
                ("e" "#+BEGIN_EXAMPLE\n?\n#+END_EXAMPLE"
                 "<example>\n?\n</example>")
                ("q" "#+BEGIN_QUOTE\n?\n#+END_QUOTE"
                 "<quote>\n?\n</quote>")
                ("v" "#+BEGIN_VERSE\n?\n#+END_VERSE"
                 "<verse>\n?\n</verse>")
                ("c" "#+BEGIN_CENTER\n?\n#+END_CENTER"
                 "<center>\n?\n</center>")
                ("l" "#+BEGIN_LaTeX\n?\n#+END_LaTeX"
                 "<literal style=\"latex\">\n?\n</literal>")
                ("L" "#+LaTeX: "
                 "<literal style=\"latex\">?</literal>")
                ("h" "#+BEGIN_HTML\n?\n#+END_HTML"
                 "<literal style=\"html\">\n?\n</literal>")
                ("H" "#+HTML: "
                 "<literal style=\"html\">?</literal>")
                ("a" "#+BEGIN_ASCII\n?\n#+END_ASCII")
                ("A" "#+ASCII: ")
                ("i" "#+INDEX: ?" "#+INDEX: ?")
                ("I" "#+INCLUDE: %file ?"
                 "<include file=%file markup=\"?\">")
                )
 org-babel-load-languages '((emacs-lisp . t) (lisp . t) (gnuplot . t) (R . t)
                (dot . t) (python . t) (sh . t) (latex . t)
                (C . t) (octave \.t))
 org-agenda-files '("~/Documents/scratch.org")
 org-clock-out-hook '(org-clock-remove-empty-clock-drawer)
 org-confirm-babel-evaluate 'my-org-confirm-babel-evaluate
 org-export-preprocess-after-blockquote-hook
'(org-special-blocks-make-special-cookies)
 org-src-fontify-natively t
 )

Re: PDF docs from Org-mode: not valid PDF?

[Karl Voit]

* Michael Hannon <jm_hannon@yahoo.com> wrote:
>
> I found the following site:
>     http://www.validatepdfa.com/
>
> I don't much about the PDF standard, and I don't know anything about the
> reliability of this site, but I've found that PDF documents that I generate
> using Org-mode and send to that site come back with a boatload of validation
> errors.

Please notice that PDF/A is a special version of the PDF standard as
described on [1]. «Normal» PDF files usually do not fulfill the
strict PDF/A standard.

Therefore I do think that this failed PDF/A validation does not
necessarily mean anything at all.

Could you post the PDF software and its version which causes
printing issues? Upload an example PDF file that causes those issues
on some web space as well.

Thanks!

  1. https://en.wikipedia.org/wiki/PDF/A
-- 
Karl Voit

Re: PDF docs from Org-mode: not valid PDF?

[Bastien]

Hi Michael,

Michael Hannon <jm_hannon@yahoo.com> writes:

> Greetings.  The woman that runs our research group complains that she is
> unable to print PDF documents that a couple of us in the group send her, and
> that the two of us are the only source of such troublesome documents.
>
> It occurred to me that the two of us are the only ones in the group that use
> Emacs Org-mode to produce PDF documents.

pdftex and other usual tools to convert from .tex to .pdf 
do not validate PDF/A, as Karl said.

Here are some information on how to make a .tex that will
validate:

http://support.river-valley.com/wiki/index.php?title=Generating_PDF/A_compliant_PDFs_from_pdftex

I'm not in favor of importing this machinery into our LaTeX
exporter.

Hope this helps anyway,

-- 
 Bastien

Re: PDF docs from Org-mode: not valid PDF?

[Michael Hannon]

Bastien <bzg@altern.org> wrote:

> Michael Hannon <jm_hannon@yahoo.com> writes:

>> Greetings.  The woman that runs our research group complains that she is
>> unable to print PDF documents that a couple of us in the group send her, and
>> that the two of us are the only source of such troublesome documents.
>>
>> It occurred to me that the two of us are the only ones in the group that use
>> Emacs Org-mode to produce PDF documents.

> pdftex and other usual tools to convert from .tex to .pdf
> do not validate PDF/A, as Karl said.
> 
> Here are some information on how to make a .tex that will
> validate:
> 
> http://support.river-valley.com/wiki/index.php?title=Generating_PDF/A_compliant_PDFs_from_pdftex

> I'm not in favor of importing this machinery into our LaTeX
> exporter.

Thanks, Bastien.  Interesting link.  I agree: I don't see that it would be
desirable to incorporate this into the LaTeX exporter.

My original motivation for this was to respond to the boss's complaints about
printing PDF documents by saying something like: see, the Org-mode PDF files
pass the validation test, so the problem must be elsewhere (maybe your
printer).  Knowing almost nothing about PDF internals, I glommed onto the
first validation test that I came across, and, oops, my document failed
miserably.

Karl was kind enough to point out that MOST PDF documents would fail the 'A'
test, so the failure is a red herring.

To proceed with this, I need to generate a series of PDF documents and have
the boss's admin assistant try to print them.  Unfortunately, I've been
swamped with some other stuff this week and haven't had a chance to work on
this.

Thanks again,

-- Mike

Re: PDF docs from Org-mode: not valid PDF?

[Jambunathan K]

LibreOffice's export to PDF has this option

    File->Export as PDF->PDF/A-1a

If you change the preferences in the UI, I am not sure whether it will
kicked in with command line export.

I will be interested in knowing what you find with the above option.

ps: You can export to PDF via OpenDocument/ODT export.  See or search
for `org-odt-preferred-document-format' or
`org-e-odt-preferred-document-format' in the manual/mailing list.


> Greetings.  The woman that runs our research group complains that she is
> unable to print PDF documents that a couple of us in the group send her, and
> that the two of us are the only source of such troublesome documents.
>
> It occurred to me that the two of us are the only ones in the group that use
> Emacs Org-mode to produce PDF documents.
>
> I know there are web sites that can validate (i.e., measure conformance to the
> standard of) HTML documents.  I wondered if there might be something similar
> for PDF documents.
>
> I found the following site:
>
>     http://www.validatepdfa.com/
>
> I don't much about the PDF standard, and I don't know anything about the
> reliability of this site, but I've found that PDF documents that I generate
> using Org-mode and send to that site come back with a boatload of validation
> errors.
>
> I've appended a sample Org-mode file that generates errors when I try to
> validate it.  I've appended details of my configuration below the sample file.
>
> I should mention that I have no trouble viewing the PDF file (using "evince"
> on my Fedora system) or printing this file (to a Brother b/w printer).  Ditto for
> my colleague on a MacOS X system.  Hence, I personally have no reason to
> complain.  But there's that nagging issue with the boss.
>
> Can anybody comment as to what I should expect in the way of validation?
> And/or are there any settings that would affect this behavior?  I.e.,
> something like:
>
>     (setq org-export-pdf-use-strict t)
>
> Thanks.
>
> -- Mike
>
> ###### Beginning of sample Org-mode file
>
> #+PROPERTY: session *R*
> #+PROPERTY: cache yes
>
> #+PROPERTY: results output verbatim
> #+PROPERTY: tangle yes
>
>
>
> * This is a test
>
>
> #+BEGIN_SRC R :exports both
>  
>  x <- 7.2
>  y <- 3
>  
>  print(x * y)
>  
>  
> #+END_SRC
>
>
> #+RESULTS:
> : 21.6
> ###### End of sample Org-mode file
>
> Emacs  : GNU Emacs 24.1.1 (x86_64-redhat-linux-gnu, GTK+ Version 2.24.13)
>  of 2012-11-17 on buildvm-30.phx2.fedoraproject.org
> Package: Org-mode version 7.9.2 (release_7.9.2-656-g2f9dda @
> /usr/local/emacs.d/org-mode/org-devel/org-mode/lisp/)
>
> current state:
> ==============
> (setq
>  org-export-preprocess-before-selecting-backend-code-hook
> '(org-beamer-select-beamer-code)
>  org-tab-first-hook '(org-hide-block-toggle-maybe
>               org-src-native-tab-command-maybe
>               org-babel-hide-result-toggle-maybe
>               org-babel-header-arg-expand)
>  org-speed-command-hook '(org-speed-command-default-hook
>               org-babel-speed-command-hook)
>  org-export-html-after-blockquotes-hook
> '(org-special-blocks-convert-html-special-cookies)
>  org-occur-hook '(org-first-headline-recenter)
>  org-metaup-hook '(org-babel-load-in-session-maybe)
>  org-export-preprocess-before-normalizing-links-hook
> '(org-remove-file-link-modifiers)
>  org-confirm-shell-link-function 'yes-or-no-p
>  org-export-latex-after-blockquotes-hook
> '(org-special-blocks-convert-latex-special-cookies)
>  org-export-latex-final-hook '(org-beamer-amend-header org-beamer-fix-toc
>                    org-beamer-auto-fragile-frames
>                    org-beamer-place-default-actions-for-lists)
>  org-export-latex-after-initial-vars-hook '(org-beamer-after-initial-vars)
>  org-after-todo-state-change-hook '(org-clock-out-if-current)
>  org-from-is-user-regexp "\\<Michael Hannon\\>"
>  org-src-mode-hook '(org-src-babel-configure-edit-buffer
>              org-src-mode-configure-edit-buffer)
>  org-agenda-before-write-hook '(org-agenda-add-entry-text)
>  org-babel-pre-tangle-hook '(save-buffer)
>  org-mode-hook '(#[nil "\300\301\302\303\304$\207"
>            [org-add-hook change-major-mode-hook org-show-block-all
>             append local]
>            5]
>          #[nil "\300\301\302\303\304$\207"
>            [org-add-hook change-major-mode-hook
>             org-babel-show-result-all append local]
>            5]
>          org-babel-result-hide-spec org-babel-hide-all-hashes)
>  org-ctrl-c-ctrl-c-hook '(org-babel-hash-at-point
>               org-babel-execute-safely-maybe)
>  org-cycle-hook '(org-cycle-hide-archived-subtrees org-cycle-hide-drawers
>           org-cycle-hide-inline-tasks org-cycle-show-empty-lines
>           org-optimize-window-after-visibility-change)
>  org-export-latex-format-toc-function 'org-export-latex-format-toc-default
>  org-export-first-hook '(org-beamer-initialize-open-trackers)
>  org-babel-tangle-lang-exts '(("C++" . "cpp") ("latex" . "tex")
>                   ("python" . "py") ("lisp" . "lisp")
>                   ("emacs-lisp" . "el"))
>  org-confirm-elisp-link-function 'yes-or-no-p
>  org-metadown-hook '(org-babel-pop-to-session-maybe)
>  org-structure-template-alist '(("r" "#+begin_src R\n\n#+end_src"
>                  "<src lang=\"R\">\n\n</src>")
>                 ("S" "#+BEGIN_SRC ?\n\n#+END_SRC"
>                  "<src lang=\"?\">\n\n</src>")
>                 ("s" "#+BEGIN_SRC ?\n\n#+END_SRC"
>                  "<src lang=\"?\">\n\n</src>")
>                 ("e" "#+BEGIN_EXAMPLE\n?\n#+END_EXAMPLE"
>                  "<example>\n?\n</example>")
>                 ("q" "#+BEGIN_QUOTE\n?\n#+END_QUOTE"
>                  "<quote>\n?\n</quote>")
>                 ("v" "#+BEGIN_VERSE\n?\n#+END_VERSE"
>                  "<verse>\n?\n</verse>")
>                 ("c" "#+BEGIN_CENTER\n?\n#+END_CENTER"
>                  "<center>\n?\n</center>")
>                 ("l" "#+BEGIN_LaTeX\n?\n#+END_LaTeX"
>                  "<literal style=\"latex\">\n?\n</literal>")
>                 ("L" "#+LaTeX: "
>                  "<literal style=\"latex\">?</literal>")
>                 ("h" "#+BEGIN_HTML\n?\n#+END_HTML"
>                  "<literal style=\"html\">\n?\n</literal>")
>                 ("H" "#+HTML: "
>                  "<literal style=\"html\">?</literal>")
>                 ("a" "#+BEGIN_ASCII\n?\n#+END_ASCII")
>                 ("A" "#+ASCII: ")
>                 ("i" "#+INDEX: ?" "#+INDEX: ?")
>                 ("I" "#+INCLUDE: %file ?"
>                  "<include file=%file markup=\"?\">")
>                 )
>  org-babel-load-languages '((emacs-lisp . t) (lisp . t) (gnuplot . t) (R . t)
>                 (dot . t) (python . t) (sh . t) (latex . t)
>                 (C . t) (octave \.t))
>  org-agenda-files '("~/Documents/scratch.org")
>  org-clock-out-hook '(org-clock-remove-empty-clock-drawer)
>  org-confirm-babel-evaluate 'my-org-confirm-babel-evaluate
>  org-export-preprocess-after-blockquote-hook
> '(org-special-blocks-make-special-cookies)
>  org-src-fontify-natively t
>  )
>
>

-- 

Re: PDF docs from Org-mode: not valid PDF?

[Achim Gratz]

Michael Hannon writes:
> To proceed with this, I need to generate a series of PDF documents and have
> the boss's admin assistant try to print them.  Unfortunately, I've been
> swamped with some other stuff this week and haven't had a chance to work on
> this.

Would it be possible that you first check what that error really is?
The one thing that comes to mind is that your PDF maybe references fonts
that are not available and the easiest solution would be to either
change to standard fonts or embed them in the PDF (if the license is
permitting that).


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra

Re: PDF docs from Org-mode: not valid PDF?

[Radhakrishnan CV]

[-- Attachment #1: Type: text/plain, Size: 1939 bytes --]

On Sat, Dec 15, 2012 at 12:37 AM, Achim Gratz <Stromeko@nexgo.de> wrote:

> Michael Hannon writes:
> > To proceed with this, I need to generate a series of PDF documents and
> have
> > the boss's admin assistant try to print them.  Unfortunately, I've been
> > swamped with some other stuff this week and haven't had a chance to work
> on
> > this.
>
> Would it be possible that you first check what that error really is?
> The one thing that comes to mind is that your PDF maybe references fonts
> that are not available and the easiest solution would be to either
> change to standard fonts or embed them in the PDF (if the license is
> permitting that).
>

As far as I know, PDF/A-1a is not meant for universal printing, instead it
is meant for accessibility. The current stock versions of pdfTeX
distributed are not capable of producing accessible PDF as per PDF/A-1a
specs. Main problem is that TeX does not use a space character and as such
finding word boundary becomes  difficult in pdf's created by pdfTeX; read
aloud applications will fail to voice render PDF's by pdfTeX.

However, if the OP is happy with a PDF as per PDF/A-1b, of course, pdfx.sty
distributed with TeXLive (and available at CTAN also) is capable of
creating.  This is for web reading and archival purposes.

If universal printing is what is wanted, then the spec to follow is
PDF/X-1a. Again, pdfx.sty does help to generate this too. I think, it is
not a big deal to extend pdfx package to generate pdf's compliant to later
specs like PDF/X-2, 3, ... (all for printing purpose). Later specs have
some advantages too like inclusion of icc color profile is not mandatory,
instead url will do.

I think, orgmode can even help to create a small XMP meta file (for Dublin
Core metadata compliance) needed by pdfx package while creating standards
compliant PDF's which at the moment is created by pdfTeX in an
unsatisfactory fashion.

Best regards
-- 
Radhakrishnan

[-- Attachment #2: Type: text/html, Size: 2301 bytes --]

Re: PDF docs from Org-mode: not valid PDF?

[Bastien]

Hi Radhakrishnan,

thanks for the information.  Am I correctly assuming you're one of 
the authors of the pdfx.sty package?

Radhakrishnan CV <cvr@river-valley.org> writes:

> I think, orgmode can even help to create a small XMP meta file (for
> Dublin Core metadata compliance) needed by pdfx package while
> creating standards compliant PDF's which at the moment is created by
> pdfTeX in an unsatisfactory fashion.

I don't know enough about this to be of real help, but if Org and 
Orgers can help, don't hesitate let us know precisely how!

Thanks,

-- 
 Bastien

Re: PDF docs from Org-mode: not valid PDF?

[Radhakrishnan CV]

[-- Attachment #1: Type: text/plain, Size: 838 bytes --]

Dear Bastien,

Sorry for the late reply.

On Sat, Dec 15, 2012 at 1:55 PM, Bastien <bzg@altern.org> wrote:

> Hi Radhakrishnan,
>
> thanks for the information.  Am I correctly assuming you're one of
> the authors of the pdfx.sty package?
>

You are right, Bastien.


> Radhakrishnan CV <cvr@river-valley.org> writes:
>
> > I think, orgmode can even help to create a small XMP meta file (for
> > Dublin Core metadata compliance) needed by pdfx package while
> > creating standards compliant PDF's which at the moment is created by
> > pdfTeX in an unsatisfactory fashion.
>
> I don't know enough about this to be of real help, but if Org and
> Orgers can help, don't hesitate let us know precisely how!
>

I can help with LaTeX part of the stuff when needed.

Best regards

Radhakrishnan
http://maps.google.com/maps?q=8.494174%2c76.963313

[-- Attachment #2: Type: text/html, Size: 1475 bytes --]

Re: PDF docs from Org-mode: not valid PDF?

[Bastien]

Hi,

Radhakrishnan CV <cvr@river-valley.org> writes:

>     I don't know enough about this to be of real help, but if Org and
>     Orgers can help, don't hesitate let us know precisely how!
>
>
> I can help with LaTeX part of the stuff when needed.

Thanks,

-- 
 Bastien