new exporter corrupts ODT

new exporter corrupts ODT

[Michael Brand]

[-- Attachment #1: Type: text/plain, Size: 327 bytes --]

Hi all

If OpenOffice.org 3.2.0 is too old to count please tell me and forget.

With an Org file containing only "* a\n" the old exporter is ok but the
new exporter release_7.8.10-633 on 23.3.1 makes an ODT reported as
"corrupt" by OOo. Attached are the Org and ODT exported as old, new
and new repaired by OOo 3.2.0.

Michael

[-- Attachment #2: release_7.8.10-633.org --]
[-- Type: application/octet-stream, Size: 4 bytes --]

* a

[-- Attachment #3: release_7.8.10-633.org.new.odt --]
[-- Type: application/vnd.oasis.opendocument.text, Size: 9636 bytes --]

[-- Attachment #4: release_7.8.10-633.org.new.odt.repaired_3_2_0.odt --]
[-- Type: application/vnd.oasis.opendocument.text, Size: 12355 bytes --]

[-- Attachment #5: release_7.8.10-633.org.old.odt --]
[-- Type: application/vnd.oasis.opendocument.text, Size: 9774 bytes --]

Re: new exporter corrupts ODT

[Jambunathan K]

Michael Brand <michael.ch.brand@gmail.com> writes:

> Hi all
>
> If OpenOffice.org 3.2.0 is too old to count please tell me and forget.
>
> With an Org file containing only "* a\n" the old exporter is ok but the
> new exporter release_7.8.10-633 on 23.3.1 makes an ODT reported as
> "corrupt" by OOo. Attached are the Org and ODT exported as old, new
> and new repaired by OOo 3.2.0.

I am not able to reproduce this issue locally.  My LibreOffice is fairly
recent. (LibreOffice 3.5.2.2 Build ID: 281b639-6baa1d3-ef66a77-d866f25-f36d45f)

If I pass the corrupted file through an online validator [1], I get the
below error.  The culprit is the mimetype file.  Note the leading
whitespace and the trailing newline.  If I fix this extraneous
whitespace by hand, then I see that the reported corruption goes away.

,----
|         application/vnd.oasis.opendocument.text
`----

I can confirm that ODT exporter, never creates the above leading and
trailing whitespaces.

Unless you hand-edited the ODT document, I think the error could be
because of one of the customizations you have.  (Do you have
`org-e-odt-prettify-xml' set to t.  If yes, what mode does mimetype file
open for you.  For me, mimetype opens in Fundamental mode.)

,----
| ODF Validator Result Page
| Result for release_7.8.10-633.org.new.odt
| 
| The document is NOT conformant ODF1.2!
| Details:
| release_7.8.10-633.org.new.odt: Info: ODF version of root document: 1.2
| release_7.8.10-633.org.new.odt/mimetype: Error: The ODF package 'release_7.8.10-633.org.new.odt' contains a 'mimetype' file containing ' application/vnd.oasis.opendocument.text', which differs from the mediatype of the root document 'application/vnd.oasis.opendocument.text'!
| release_7.8.10-633.org.new.odt: Error: The ODF mimetype ' application/vnd.oasis.opendocument.text' is invalid for the ODF XML Schema document!
| release_7.8.10-633.org.new.odt/mimetype: Info: mimetype is not an ODFMediaTypes mimetype.
| internal:/schema/odf1.2/OpenDocument-v1.2-cos01-manifest-schema.rng: Info: parsed.
| release_7.8.10-633.org.new.odt/META-INF/manifest.xml: Info: no errors, no warnings
| release_7.8.10-633.org.new.odt/mimetype: Info: 1 errors, no warnings
| release_7.8.10-633.org.new.odt: Info: Media Type: application/vnd.oasis.opendocument.text
| internal:/schema/odf1.2/OpenDocument-v1.2-cos01-schema.rng: Info: parsed.
| release_7.8.10-633.org.new.odt/meta.xml: Info: Generator: Generated by Org mode 7.8.10 in Emacs 23.3.1.
| release_7.8.10-633.org.new.odt/meta.xml: Info: no errors, no warnings
| release_7.8.10-633.org.new.odt/styles.xml: Info: no errors, no warnings
| release_7.8.10-633.org.new.odt/content.xml: Info: no errors, no warnings
| internal:/schema/odf1.2/OpenDocument-v1.2-cos01-dsig-schema.rng: Info: parsed.
| release_7.8.10-633.org.new.odt: Info: 2 errors, no warnings
`----


Footnotes: 
[1]  http://odf-validator.rhcloud.com/

-- 

Re: new exporter corrupts ODT

[Jambunathan K]

Jambunathan K <kjambunathan@gmail.com> writes:

> I think the error could be because of one of the customizations you
> have.

Good way to check this is with:

"C:\Program Files\emacs-24.0.97\bin\runemacs.exe" -Q -L
~/src/org-mode/lisp/ -L ~/src/org-mode/contrib/lisp/

(Note the -Q and adjusted load-paths)

M-x load-library RET org-e-odt RET
M-x org-export-dispatch RET etc etc

-- 

Re: new exporter corrupts ODT

[Jambunathan K]

Jambunathan K <kjambunathan@gmail.com> writes:

> Unless you hand-edited the ODT document, I think the error could be
> because of one of the customizations you have.  (Do you have
> `org-e-odt-prettify-xml' set to t.  If yes, what mode does mimetype
> file open for you.  For me, mimetype opens in Fundamental mode.)

After sending out the email, I realized that `org-e-odt-prettify-xml' is
ON by default in the org-e-odt.el.  Now I have turned it off.

Hopefully things are OK at your end with this "fix".  I am still curious
what mode does mimetype file open for you. (Do M-: mode-name in mimetype
buffer)

If the problem persists (even after this), try emacs -Q and isolate your
local config that messes up things.

-- 

Re: new exporter corrupts ODT

[Michael Brand]

[-- Attachment #1: Type: text/plain, Size: 919 bytes --]

Hi Jambunathan

On Sat, Jun 2, 2012 at 5:18 PM, Jambunathan K <kjambunathan@gmail.com> wrote:
> After sending out the email, I realized that `org-e-odt-prettify-xml' is
> ON by default in the org-e-odt.el.  Now I have turned it off.
>
> Hopefully things are OK at your end with this "fix".

Thank you for looking into this.

I just pulled and now get a backtrace when trying this:

cd /f/xmirror/git/org-mode
git pull
git describe
==> release_7.8.10-642-gca6f523
make cleanall info autoloads
(/Applications/Emacs.app/Contents/MacOS/Emacs -Q \
    -L "/f/xmirror/git/org-mode/lisp/" \
    -L "/f/xmirror/git/org-mode/contrib/lisp/" &)
M-x load-library RET org-e-odt RET
C-x C-f release_7.8.10-633.org RET
M-x org-export-dispatch RET o
==> see attached backtrace

> I am still curious
> what mode does mimetype file open for you. (Do M-: mode-name in mimetype
> buffer)

"M-: mode-name RET" shows "Fundamental".

Michael

[-- Attachment #2: backtrace.txt --]
[-- Type: text/plain, Size: 12775 bytes --]

Debugger entered--Lisp error: (void-function org-e-html-encode-plain-text)
  (org-e-html-encode-plain-text line)
  (setq line (org-e-html-encode-plain-text line))
  org-e-odt-encode-plain-text("a" t)
  (setq text (org-e-odt-encode-plain-text text t))
  org-e-odt-plain-text("a" (:author "br" :creator "Generated by Org mode 7.8.10 in Emacs 23.3.1." :date nil :description nil :email "b@aHostName" :exclude-tags ("noexport") :headline-levels 3 :keywords nil :language "en" :preserve-breaks nil :section-numbers t :select-tags ("export") :time-stamp-file t :title "release_7.8.10-633" :with-archived-trees headline :with-author t :with-clocks nil :with-creator comment :with-drawers t :with-email nil :with-emphasize t :with-entities t :with-fixed-width t :with-footnotes t :with-plannings nil ...))
  funcall(org-e-odt-plain-text "a" (:author "br" :creator "Generated by Org mode 7.8.10 in Emacs 23.3.1." :date nil :description nil :email "b@aHostName" :exclude-tags ("noexport") :headline-levels 3 :keywords nil :language "en" :preserve-breaks nil :section-numbers t :select-tags ("export") :time-stamp-file t :title "release_7.8.10-633" :with-archived-trees headline :with-author t :with-clocks nil :with-creator comment :with-drawers t :with-email nil :with-emphasize t :with-entities t :with-fixed-width t :with-footnotes t :with-plannings nil ...))
  (if transcoder (funcall transcoder data info) data)
  (let ((transcoder ...)) (if transcoder (funcall transcoder data info) data))
  (org-export-filter-apply-functions (plist-get info :filter-plain-text) (let (...) (if transcoder ... data)) info)
  (cond ((member data ...) nil) ((eq type ...) (org-export-filter-apply-functions ... ... info)) ((not ...) (org-export-data ... info)) ((not type) (mapconcat ... data "")) ((or ... ...) (let ... ...)) (t (let ... ...)))
  (let* ((type ...) (results ...)) (cond (... nil) (... results) (t ...)))
  org-export-data("a" (:author "br" :creator "Generated by Org mode 7.8.10 in Emacs 23.3.1." :date nil :description nil :email "b@aHostName" :exclude-tags ("noexport") :headline-levels 3 :keywords nil :language "en" :preserve-breaks nil :section-numbers t :select-tags ("export") :time-stamp-file t :title "release_7.8.10-633" :with-archived-trees headline :with-author t :with-clocks nil :with-creator comment :with-drawers t :with-email nil :with-emphasize t :with-entities t :with-fixed-width t :with-footnotes t :with-plannings nil ...))
  (lambda (obj) (org-export-data obj info))("a")
  mapconcat((lambda (obj) (org-export-data obj info)) ("a") "")
  (cond ((member data ...) nil) ((eq type ...) (org-export-filter-apply-functions ... ... info)) ((not ...) (org-export-data ... info)) ((not type) (mapconcat ... data "")) ((or ... ...) (let ... ...)) (t (let ... ...)))
  (let* ((type ...) (results ...)) (cond (... nil) (... results) (t ...)))
  org-export-data(("a") (:author "br" :creator "Generated by Org mode 7.8.10 in Emacs 23.3.1." :date nil :description nil :email "b@aHostName" :exclude-tags ("noexport") :headline-levels 3 :keywords nil :language "en" :preserve-breaks nil :section-numbers t :select-tags ("export") :time-stamp-file t :title "release_7.8.10-633" :with-archived-trees headline :with-author t :with-clocks nil :with-creator comment :with-drawers t :with-email nil :with-emphasize t :with-entities t :with-fixed-width t :with-footnotes t :with-plannings nil ...))
  (let* ((numberedp ...) (level ...) (text ...) (full-text ...)) (cond (... nil) (... ...) (t ...)))
  org-e-odt-headline((headline (:raw-value "a" :title ("a") :begin 1 :end 5 :pre-blank 0 :hiddenp nil :contents-begin 5 :contents-end 5 :level 1 :priority nil :tags nil :todo-keyword nil :todo-type nil :scheduled nil :deadline nil :timestamp nil :clock nil :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :quotedp nil :category nil)) nil (:author "br" :creator "Generated by Org mode 7.8.10 in Emacs 23.3.1." :date nil :description nil :email "b@aHostName" :exclude-tags ("noexport") :headline-levels 3 :keywords nil :language "en" :preserve-breaks nil :section-numbers t :select-tags ("export") :time-stamp-file t :title "release_7.8.10-633" :with-archived-trees headline :with-author t :with-clocks nil :with-creator comment :with-drawers t :with-email nil :with-emphasize t :with-entities t :with-fixed-width t :with-footnotes t :with-plannings nil ...))
  funcall(org-e-odt-headline (headline (:raw-value "a" :title ("a") :begin 1 :end 5 :pre-blank 0 :hiddenp nil :contents-begin 5 :contents-end 5 :level 1 :priority nil :tags nil :todo-keyword nil :todo-type nil :scheduled nil :deadline nil :timestamp nil :clock nil :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :quotedp nil :category nil)) nil (:author "br" :creator "Generated by Org mode 7.8.10 in Emacs 23.3.1." :date nil :description nil :email "b@aHostName" :exclude-tags ("noexport") :headline-levels 3 :keywords nil :language "en" :preserve-breaks nil :section-numbers t :select-tags ("export") :time-stamp-file t :title "release_7.8.10-633" :with-archived-trees headline :with-author t :with-clocks nil :with-creator comment :with-drawers t :with-email nil :with-emphasize t :with-entities t :with-fixed-width t :with-footnotes t :with-plannings nil ...))
  (and (fboundp transcoder) (funcall transcoder data nil info))
  (let ((transcoder ...)) (and (fboundp transcoder) (funcall transcoder data nil info)))
  (cond ((member data ...) nil) ((eq type ...) (org-export-filter-apply-functions ... ... info)) ((not ...) (org-export-data ... info)) ((not type) (mapconcat ... data "")) ((or ... ...) (let ... ...)) (t (let ... ...)))
  (let* ((type ...) (results ...)) (cond (... nil) (... results) (t ...)))
  org-export-data((headline (:raw-value "a" :title ("a") :begin 1 :end 5 :pre-blank 0 :hiddenp nil :contents-begin 5 :contents-end 5 :level 1 :priority nil :tags nil :todo-keyword nil :todo-type nil :scheduled nil :deadline nil :timestamp nil :clock nil :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :quotedp nil :category nil)) (:author "br" :creator "Generated by Org mode 7.8.10 in Emacs 23.3.1." :date nil :description nil :email "b@aHostName" :exclude-tags ("noexport") :headline-levels 3 :keywords nil :language "en" :preserve-breaks nil :section-numbers t :select-tags ("export") :time-stamp-file t :title "release_7.8.10-633" :with-archived-trees headline :with-author t :with-clocks nil :with-creator comment :with-drawers t :with-email nil :with-emphasize t :with-entities t :with-fixed-width t :with-footnotes t :with-plannings nil ...))
  (lambda (element) (org-export-data element info))((headline (:raw-value "a" :title ("a") :begin 1 :end 5 :pre-blank 0 :hiddenp nil :contents-begin 5 :contents-end 5 :level 1 :priority nil :tags nil :todo-keyword nil :todo-type nil :scheduled nil :deadline nil :timestamp nil :clock nil :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :quotedp nil :category nil)))
  mapconcat((lambda (element) (org-export-data element info)) ((headline (:raw-value "a" :title ... :begin 1 :end 5 :pre-blank 0 :hiddenp nil :contents-begin 5 :contents-end 5 :level 1 :priority nil :tags nil :todo-keyword nil :todo-type nil :scheduled nil :deadline nil :timestamp nil :clock nil :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :quotedp nil :category nil))) "")
  (let* ((greaterp ...) (objectp ...) (contents ...)) (funcall transcoder data (if greaterp ... contents) info))
  (progn (let* (... ... ...) (funcall transcoder data ... info)))
  (if transcoder (progn (let* ... ...)))
  (when transcoder (let* (... ... ...) (funcall transcoder data ... info)))
  (let ((transcoder ...)) (when transcoder (let* ... ...)))
  (cond ((member data ...) nil) ((eq type ...) (org-export-filter-apply-functions ... ... info)) ((not ...) (org-export-data ... info)) ((not type) (mapconcat ... data "")) ((or ... ...) (let ... ...)) (t (let ... ...)))
  (let* ((type ...) (results ...)) (cond (... nil) (... results) (t ...)))
  org-export-data((org-data nil (headline (:raw-value "a" :title ... :begin 1 :end 5 :pre-blank 0 :hiddenp nil :contents-begin 5 :contents-end 5 :level 1 :priority nil :tags nil :todo-keyword nil :todo-type nil :scheduled nil :deadline nil :timestamp nil :clock nil :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :quotedp nil :category nil))) (:author "br" :creator "Generated by Org mode 7.8.10 in Emacs 23.3.1." :date nil :description nil :email "b@aHostName" :exclude-tags ("noexport") :headline-levels 3 :keywords nil :language "en" :preserve-breaks nil :section-numbers t :select-tags ("export") :time-stamp-file t :title "release_7.8.10-633" :with-archived-trees headline :with-author t :with-clocks nil :with-creator comment :with-drawers t :with-email nil :with-emphasize t :with-entities t :with-fixed-width t :with-footnotes t :with-plannings nil ...))
  (org-element-normalize-string (org-export-data tree info))
  (let* ((body ...) (template ...) (output ...)) (when org-export-copy-to-kill-ring (org-kill-new output)) output)
  (let ((info ...) (tree ...)) (setq tree (org-export-filter-apply-functions ... tree info)) (setq info (org-combine-plists info ...)) (let* (... ... ...) (when org-export-copy-to-kill-ring ...) output))
  (save-restriction (cond (... ...) (subtreep ... ... ... ...)) (let (... ...) (setq tree ...) (setq info ...) (let* ... ... output)))
  (save-excursion (save-restriction (cond ... ...) (let ... ... ... ...)))
  org-export-as(e-odt nil nil nil nil nil)
  (let ((out ...) (buffer ...)) (with-current-buffer buffer (erase-buffer) (insert out) (goto-char ...)) buffer)
  org-export-to-buffer(e-odt #<buffer content.xml> nil nil nil)
  (let* ((outbuf ...) (target ...) (outdir ...) (default-directory outdir)) (setq org-current-export-file (file-name-directory ...)) (org-export-to-buffer (quote e-odt) outbuf (memq ... optns) (memq ... optns) (memq ... optns)) (setq org-lparse-opt-plist nil) (org-e-odt-save-as-outfile target nil) (if (not org-e-odt-preferred-output-format) target (or ... target)))
  org-e-odt-export-to-odt(nil nil nil)
  (cond ((eql --cl-var-- ...) nil) ((member* --cl-var-- ...) (require ...) (let ... ... ...)) ((member* --cl-var-- ...) (require ...) (org-e-ascii-export-to-ascii ... ... ... ...)) ((eql --cl-var-- ...) (require ...) (let ... ... ...)) ((eql --cl-var-- ...) (require ...) (org-e-latex-export-to-latex ... ... ... ...)) ((eql --cl-var-- ...) (require ...) (org-e-latex-export-to-pdf ... ... ...)) ((eql --cl-var-- ...) (require ...) (org-open-file ...)) ((eql --cl-var-- ...) (require ...) (let ... ... ...)) ((eql --cl-var-- ...) (require ...) (org-e-html-export-to-html ... ... ...)) ((eql --cl-var-- ...) (require ...) (org-open-file ...)) ((eql --cl-var-- ...) (require ...) (org-e-odt-export-to-odt ... ... ...)) ((eql --cl-var-- ...) (require ...) (org-open-file ...)) ((eql --cl-var-- ...) (require ...) (org-e-publish-current-file ...)) ((eql --cl-var-- ...) (require ...) (org-e-publish-current-project ...)) ((eql --cl-var-- ...) (require ...) (let ... ...)) ((eql --cl-var-- ...) (require ...) (org-e-publish-all ...)) (t (error "No command associated with key %s" ...)))
  (let ((--cl-var-- ...)) (cond (... nil) (... ... ...) (... ... ...) (... ... ...) (... ... ...) (... ... ...) (... ... ...) (... ... ...) (... ... ...) (... ... ...) (... ... ...) (... ... ...) (... ... ...) (... ... ...) (... ... ...) (... ... ...) (t ...)))
  (case (if (< raw-key 27) (+ raw-key 96) raw-key) (113 nil) ((65 78 85) (require ...) (let ... ... ...)) ((97 110 117) (require ...) (org-e-ascii-export-to-ascii ... ... ... ...)) (76 (require ...) (let ... ... ...)) (108 (require ...) (org-e-latex-export-to-latex ... ... ... ...)) (112 (require ...) (org-e-latex-export-to-pdf ... ... ...)) (100 (require ...) (org-open-file ...)) (72 (require ...) (let ... ... ...)) (104 (require ...) (org-e-html-export-to-html ... ... ...)) (98 (require ...) (org-open-file ...)) (111 (require ...) (org-e-odt-export-to-odt ... ... ...)) (79 (require ...) (org-open-file ...)) (70 (require ...) (org-e-publish-current-file ...)) (80 (require ...) (org-e-publish-current-project ...)) (88 (require ...) (let ... ...)) (69 (require ...) (org-e-publish-all ...)) (t (error "No command associated with key %s" ...)))
  (let* ((input ...) (raw-key ...) (optns ...)) (case (if ... ... raw-key) (113 nil) (... ... ...) (... ... ...) (76 ... ...) (108 ... ...) (112 ... ...) (100 ... ...) (72 ... ...) (104 ... ...) (98 ... ...) (111 ... ...) (79 ... ...) (70 ... ...) (80 ... ...) (88 ... ...) (69 ... ...) (t ...)))
  org-export-dispatch()
  call-interactively(org-export-dispatch t nil)
  execute-extended-command(nil)
  call-interactively(execute-extended-command nil nil)

Re: new exporter corrupts ODT

[Michael Brand]

Hi Jambunathan

On Sat, Jun 2, 2012 at 6:35 PM, Michael Brand
<michael.ch.brand@gmail.com> wrote:
[...]
> see attached backtrace
[...]

It seems to me you made the recent commit release_7.8.11-37-gec368e8
for my backtrace. So I tested and can confirm that it is resolved.
Also my original issue with the corrupt ODT is resolved. Now even for
org-e-odt-prettify-xml both on its new default nil or changed to t.
Originally on release_7.8.10-633-gb4d827c I tried only its old default
t that had the issue.

Thank you.

Michael

Re: new exporter corrupts ODT

[Achim Gratz]

[re-sent, sorry if you get this twice]

Michael Brand writes:
> With an Org file containing only "* a\n" the old exporter is ok but the
> new exporter release_7.8.10-633 on 23.3.1 makes an ODT reported as
> "corrupt" by OOo. Attached are the Org and ODT exported as old, new
> and new repaired by OOo 3.2.0.

Please do a 'git fetch --tags origin' — you seem to have pulled the
non-annotated release tag for 7.8.11 before Bastien corrected it.

Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: new exporter corrupts ODT

[Michael Brand]

Hi Achim

On Sun, Jun 3, 2012 at 3:13 PM, Achim Gratz <Stromeko@nexgo.de> wrote:
> Please do a 'git fetch --tags origin' — you seem to have pulled the
> non-annotated release tag for 7.8.11 before Bastien corrected it.

Oh yes, thanks for reminding me on that. I fetched tags after the
annonuncement on 2012-04-11 to do so for release_7.8.08 but seem
somehow to have lost track (on 7.8.10 again?) on one of my
Emacs/Org installations and missed to pay attention to it.

Michael