From mboxrd@z Thu Jan 1 00:00:00 1970 From: Robert Goldman Subject: Re: Re: [PATCH] org-export-generic, " text markup" -- and a request Date: Sun, 01 Aug 2010 22:02:49 -0500 Message-ID: <4C563559.6010304@sift.info> References: <20100723111931.GA29930@tomas> <201007240839.53358.DanielBausch@gmx.de> <20100724082329.GA21627@tomas> <87vd85rtev.wl%dmaus@ictsoc.de> <20100724140710.GB23108@tomas> <87hbjn7h4z.wl%dmaus@ictsoc.de> <20100725163540.GA19227@tomas> <87aap812cm.fsf@altern.org> Reply-To: rpgoldman@sift.info Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------010902090608070104030407" Return-path: Received: from [140.186.70.92] (port=51903 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OflIl-0004Lb-CW for emacs-orgmode@gnu.org; Sun, 01 Aug 2010 23:02:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OflIj-0000J0-2K for emacs-orgmode@gnu.org; Sun, 01 Aug 2010 23:02:55 -0400 Received: from mpls.sift.info ([75.146.46.193]:47970) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OflIi-0000Iq-Px for emacs-orgmode@gnu.org; Sun, 01 Aug 2010 23:02:53 -0400 In-Reply-To: <87aap812cm.fsf@altern.org> List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Bastien Cc: Wes Hardaker , emacs-orgmode@gnu.org This is a multi-part message in MIME format. --------------010902090608070104030407 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 7/31/10 Jul 31 -3:29 AM, Bastien wrote: > Robert Goldman writes: > >> FWIW, I have a number of substantial modifications to org-export-generic that >> fix its original inability to handle typefaces across line boundaries. >> >> However, my understanding is that all patches for org-export-generic wait on Wes >> Hardaker to approve them. >> >> If this would be a good time, I can ship the patches again. > > Let's wait for Wes' comeback for a while. > > Wes? > OK, attached is my sequence of patches for easier reference. --------------010902090608070104030407 Content-Type: text/plain; x-mac-type="0"; x-mac-creator="0"; name="0007-Added-a-call-to-insert-any-bodynewline-paragraph-val.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0007-Added-a-call-to-insert-any-bodynewline-paragraph-val.pa"; filename*1="tch" >From aefc56ca8b7ef9cf621a3833fa0100558f8823f0 Mon Sep 17 00:00:00 2001 From: Robert P. Goldman Date: Sun, 30 May 2010 15:30:11 -0500 Subject: [PATCH 7/7] Added a call to insert any bodynewline-paragraph value before the start of a line item. --- contrib/lisp/org-export-generic.el | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-) diff --git a/contrib/lisp/org-export-generic.el b/contrib/lisp/org-export-generic.el index 114769b..f8e8c4a 100644 --- a/contrib/lisp/org-export-generic.el +++ b/contrib/lisp/org-export-generic.el @@ -932,9 +932,13 @@ underlined headlines. The default is 3." (string-match "^\\([ \t]+\\)\\(\\*[ \t]*\\)" line)) ;; ;; plain list item - ;; ;; TODO: nested lists ;; + ;; first add a line break between any previous paragraph or line item and this + ;; one + (when bodynewline-paragraph + (insert bodynewline-paragraph)) + ;; I believe this gets rid of leading whitespace. (setq line (replace-match "" nil nil line)) -- 1.7.1 --------------010902090608070104030407 Content-Type: text/plain; x-mac-type="0"; x-mac-creator="0"; name="0006-Add-a-test-code-file.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0006-Add-a-test-code-file.patch" >From eca1acab3ad7522f6dca2314c634c4b86fbb0a04 Mon Sep 17 00:00:00 2001 From: Robert P. Goldman Date: Tue, 25 May 2010 09:08:04 -0500 Subject: [PATCH 6/7] Add a test code file. --- contrib/lisp/test-org-export-preproc.el | 39 +++++++++++++++++++++++++++++++ 1 files changed, 39 insertions(+), 0 deletions(-) create mode 100644 contrib/lisp/test-org-export-preproc.el diff --git a/contrib/lisp/test-org-export-preproc.el b/contrib/lisp/test-org-export-preproc.el new file mode 100644 index 0000000..3af8461 --- /dev/null +++ b/contrib/lisp/test-org-export-preproc.el @@ -0,0 +1,39 @@ +(require 'org-export-generic) + +(defun test-preproc () + (interactive) + (let ((string + (let ((region + (buffer-substring + (if (org-region-active-p) (region-beginning) (point-min)) + (if (org-region-active-p) (region-end) (point-max)))) + (opt-plist (org-combine-plists (org-default-export-plist) + (org-infile-export-plist))) + (export-plist '("tikiwiki" :file-suffix ".txt" :key-binding 85 :header-prefix "" :header-suffix "" :title-format "-= %s =-\n" :date-export nil :toc-export nil :body-header-section-numbers nil :body-section-prefix "\n" :body-section-header-prefix + ("! " "!! " "!!! " "!!!! " "!!!!! " "!!!!!! " "!!!!!!! ") + :body-section-header-suffix + (" \n" " \n" " \n" " \n" " \n" " \n") + :body-line-export-preformated t :body-line-format "%s " :body-line-wrap nil :body-line-fixed-format " %s\n" :body-list-format "* %s\n" :body-number-list-format "# %s\n" :blockquote-start "\n^\n" :blockquote-end "^\n\n" :body-newline-paragraph "\n" :bold-format "__%s__" :italic-format "''%s''" :underline-format "===%s===" :strikethrough-format "--%s--" :code-format "-+%s+-" :verbatim-format "~pp~%s~/pp~"))) + (org-export-preprocess-string + region + :for-ascii t + :skip-before-1st-heading + (plist-get opt-plist :skip-before-1st-heading) + :drawers (plist-get export-plist :drawers-export) + :tags (plist-get export-plist :tags-export) + :priority (plist-get export-plist :priority-export) + :footnotes (plist-get export-plist :footnotes-export) + :timestamps (plist-get export-plist :timestamps-export) + :todo-keywords (plist-get export-plist :todo-keywords-export) + :verbatim-multiline t + :select-tags (plist-get export-plist :select-tags-export) + :exclude-tags (plist-get export-plist :exclude-tags-export) + :emph-multiline t + :archived-trees + (plist-get export-plist :archived-trees-export) + :add-text (plist-get opt-plist :text))))) + (save-excursion + (switch-to-buffer "*preproc-temp*") + (point-max) + (insert string)))) + -- 1.7.1 --------------010902090608070104030407 Content-Type: text/plain; x-mac-type="0"; x-mac-creator="0"; name="0005-Expanded-docstring-for-org-emph-re.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="0005-Expanded-docstring-for-org-emph-re.patch" >From 1f51cb4b8ce7c62699e6905eaf0aa47ef508cca0 Mon Sep 17 00:00:00 2001 From: Robert P. Goldman Date: Mon, 24 May 2010 09:01:22 -0500 Subject: [PATCH 5/7] Expanded docstring for org-emph-re --- lisp/org.el | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/lisp/org.el b/lisp/org.el index 03399d7..63817d5 100644 --- a/lisp/org.el +++ b/lisp/org.el @@ -3313,6 +3313,8 @@ When nil, the \\name form remains in the buffer." (defvar org-emph-re nil "Regular expression for matching emphasis. After a match, the match groups contain these elements: +0 The match of the full regular expression, including the characters + before and after the proper match 1 The character before the proper match, or empty at beginning of line 2 The proper match, including the leading and trailing markers 3 The leading marker like * or /, indicating the type of highlighting -- 1.7.1 --------------010902090608070104030407 Content-Type: text/plain; x-mac-type="0"; x-mac-creator="0"; name="0004-Substantially-improved-org-export-generic-fontify-ba.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0004-Substantially-improved-org-export-generic-fontify-ba.pa"; filename*1="tch" >From d5f9d9793784f9e66af4d0c13736dc7e72eb7424 Mon Sep 17 00:00:00 2001 From: Robert P. Goldman Date: Fri, 21 May 2010 09:14:09 -0500 Subject: [PATCH 4/7] Substantially improved org-export-generic-fontify based on help from Carsten. --- contrib/lisp/org-export-generic.el | 28 +++++++++++----------------- 1 files changed, 11 insertions(+), 17 deletions(-) diff --git a/contrib/lisp/org-export-generic.el b/contrib/lisp/org-export-generic.el index 3a7af40..114769b 100644 --- a/contrib/lisp/org-export-generic.el +++ b/contrib/lisp/org-export-generic.el @@ -1338,35 +1338,29 @@ conversions.") "Convert fontification according to generic rules." (if (string-match org-emph-re string) ;; The match goes one char after the *string*, except at the end of a line - - ;; as far as I can tell from cargo-culting the code from - ;; the latex translation, we have the following: - ;; (match-string 1) is the material BEFORE the match - ;; -- should be unchanged - ;; (match-string 3) is the actual markup character - ;; (match-string 4) is the material that is to be - ;; marked up - ;; (match-string 5) is the remainder (let ((emph (assoc (match-string 3 string) org-export-generic-emphasis-alist)) - (beg (match-beginning 0))) + (beg (match-beginning 0)) + (end (match-end 0))) (unless emph (message "`org-export-generic-emphasis-alist' has no entry for formatting triggered by \"%s\"" (match-string 3 string))) ;; now we need to determine whether we have strikethrough or ;; a list, which is a bit nasty - (if (and (equal (match-string 3 str) "+") + (if (and (equal (match-string 3 string) "+") (save-match-data - (string-match "\\`-+\\'" (match-string 4 str)))) - ;; a list --- skip this match and recurse - (concat (substring str 0 (match-beginning 3)) - (org-export-generic-fontify (substring str (match-beginning 3)))) - (concat (substring str 0 beg) + (string-match "\\`-+\\'" (match-string 4 string)))) + ;; a list --- skip this match and recurse on the point after the + ;; first emph char... + (concat (substring string 0 (1+ (match-beginning 3))) + (org-export-generic-fontify (substring string (match-beginning 3)))) + (concat (substring string 0 beg) ;; part before the match (match-string 1 string) (org-export-generic-emph-format (second emph) (match-string 4 string) (third emph)) - (org-export-generic-fontify (match-string 5 string))))) + (or (match-string 5 string) "") + (org-export-generic-fontify (substring string end))))) string)) (defun org-export-generic-emph-format (format-varname string protect) -- 1.7.1 --------------010902090608070104030407 Content-Type: text/plain; x-mac-type="0"; x-mac-creator="0"; name="0003-Partial-solution-to-the-fontification-problem.-Havin.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0003-Partial-solution-to-the-fontification-problem.-Havin.pa"; filename*1="tch" >From 99f7bb368f73fcc5a6e2a4d05ce7dd4a4bca4919 Mon Sep 17 00:00:00 2001 From: Robert P. Goldman Date: Thu, 20 May 2010 21:49:32 -0500 Subject: [PATCH 3/7] Partial solution to the fontification problem. Having some trouble with the MATCH-STRING calls, but mostly ok. --- contrib/lisp/org-export-generic.el | 105 ++++++++++++++++++++++++++++++++++-- 1 files changed, 101 insertions(+), 4 deletions(-) diff --git a/contrib/lisp/org-export-generic.el b/contrib/lisp/org-export-generic.el index 0933f19..3a7af40 100644 --- a/contrib/lisp/org-export-generic.el +++ b/contrib/lisp/org-export-generic.el @@ -1,4 +1,4 @@ -;;; org-export-generic.el --- Export frameworg with custom backends +;; org-export-generic.el --- Export frameworg with custom backends ;; Copyright (C) 2009 Free Software Foundation, Inc. @@ -466,6 +466,15 @@ preformatted text\). A common non-nil value for this keyword is \"\\n\". Should typically be combined with a value for :body-line-format that does NOT end with a newline." :type string) + +;;; fontification keywords +(def-org-export-generic-keyword :bold-format) +(def-org-export-generic-keyword :italic-format) +(def-org-export-generic-keyword :underline-format) +(def-org-export-generic-keyword :strikethrough-format) +(def-org-export-generic-keyword :code-format) +(def-org-export-generic-keyword :verbatim-format) + @@ -623,6 +632,7 @@ underlined headlines. The default is 3." :verbatim-multiline t :select-tags (plist-get export-plist :select-tags-export) :exclude-tags (plist-get export-plist :exclude-tags-export) + :emph-multiline t :archived-trees (plist-get export-plist :archived-trees-export) :add-text (plist-get opt-plist :text)) @@ -671,6 +681,16 @@ underlined headlines. The default is 3." (bodylineform (or (plist-get export-plist :body-line-format) "%s")) (blockquotestart (or (plist-get export-plist :blockquote-start) "\n\n\t")) (blockquoteend (or (plist-get export-plist :blockquote-end) "\n\n")) + + ;; dynamic variables used heinously in fontification + ;; not referenced locally... + (format-boldify (plist-get export-plist :bold-format)) + (format-italicize (plist-get export-plist :italic-format)) + (format-underline (plist-get export-plist :underline-format)) + (format-strikethrough (plist-get export-plist :strikethrough-format)) + (format-code (plist-get export-plist :code-format)) + (format-verbatim (plist-get export-plist :verbatim-format)) + thetoc toctags have-headings first-heading-pos @@ -854,7 +874,7 @@ underlined headlines. The default is 3." (if org-export-generic-links-to-notes (push (cons desc0 link) link-buffer) (setq rpl (concat rpl " (" link ")") - wrap (+ (length line) (- (length (match-string 0) line)) + wrap (+ (length line) (- (length (match-string 0 line))) (length desc))))) (setq line (replace-match rpl t t line)))) (when custom-times @@ -936,7 +956,7 @@ underlined headlines. The default is 3." listcheckhalfend))) ) - (insert (format listformat line))) + (insert (format listformat (org-export-generic-fontify line)))) ((string-match "^\\([ \t]+\\)\\([0-9]+\\.[ \t]*\\)" line) ;; ;; numbered list item @@ -962,7 +982,7 @@ underlined headlines. The default is 3." listcheckhalfend))) ) - (insert (format numlistformat line))) + (insert (format numlistformat (org-export-generic-fontify line)))) ((equal line "ORG-BLOCKQUOTE-START") (setq line blockquotestart)) @@ -978,6 +998,9 @@ underlined headlines. The default is 3." ;; (org-export-generic-check-section "body" bodytextpre bodytextsuf) + (setq line + (org-export-generic-fontify line)) + ;; XXX: properties? list? (if (string-match "^\\([ \t]*\\)\\([-+*][ \t]+\\)\\(.*?\\)\\( ::\\)" line) (setq line (replace-match "\\1\\3:" t nil line))) @@ -1284,6 +1307,80 @@ REVERSE means to reverse the list if the plist match is a list (and vl (setcar vl nil)) vl)) + +;;; FIXME: this should probably turn into a defconstant later [2010/05/20:rpg] +(defvar org-export-generic-emphasis-alist + '(("*" format-boldify nil) + ("/" format-italicize nil) + ("_" format-underline nil) + ("+" format-strikethrough nil) + ("=" format-code t) + ("~" format-verbatim t)) + "Alist of org format -> formatting variables for fontification. +Each element of the list is a list of three elements. +The first element is the character used as a marker for fontification. +The second element is a variable name, set in org-export-generic. That +variable will be dereferenced to obtain a formatting string to wrap +fontified text with. +The third element decides whether to protect converted text from other +conversions.") + +;;; Cargo-culted from the latex translation. I couldn't figure out how +;;; to keep the structure since the generic export operates on lines, rather +;;; than on a buffer as in the latex export, meaning that none of the +;;; search forward code could be kept. This led me to rewrite the +;;; whole thing recursively. A huge lose for efficiency (potentially), +;;; but I couldn't figure out how to make the looping work. +;;; Worse, it's /doubly/ recursive, because this function calls +;;; org-export-generic-emph-format, which can call it recursively... +;;; [2010/05/20:rpg] +(defun org-export-generic-fontify (string) + "Convert fontification according to generic rules." + (if (string-match org-emph-re string) + ;; The match goes one char after the *string*, except at the end of a line + + ;; as far as I can tell from cargo-culting the code from + ;; the latex translation, we have the following: + ;; (match-string 1) is the material BEFORE the match + ;; -- should be unchanged + ;; (match-string 3) is the actual markup character + ;; (match-string 4) is the material that is to be + ;; marked up + ;; (match-string 5) is the remainder + (let ((emph (assoc (match-string 3 string) + org-export-generic-emphasis-alist)) + (beg (match-beginning 0))) + (unless emph + (message "`org-export-generic-emphasis-alist' has no entry for formatting triggered by \"%s\"" + (match-string 3 string))) + ;; now we need to determine whether we have strikethrough or + ;; a list, which is a bit nasty + (if (and (equal (match-string 3 str) "+") + (save-match-data + (string-match "\\`-+\\'" (match-string 4 str)))) + ;; a list --- skip this match and recurse + (concat (substring str 0 (match-beginning 3)) + (org-export-generic-fontify (substring str (match-beginning 3)))) + (concat (substring str 0 beg) + (match-string 1 string) + (org-export-generic-emph-format (second emph) + (match-string 4 string) + (third emph)) + (org-export-generic-fontify (match-string 5 string))))) + string)) + +(defun org-export-generic-emph-format (format-varname string protect) + "Return a string that results from applying the markup indicated by +FORMAT-VARNAME to STRING." + (let ((format (symbol-value format-varname))) + (let ((string-to-emphasize + (if protect + string + (org-export-generic-fontify string)))) + (if format + (format format string-to-emphasize) + string-to-emphasize)))) + (provide 'org-generic) (provide 'org-export-generic) -- 1.7.1 --------------010902090608070104030407 Content-Type: text/plain; x-mac-type="0"; x-mac-creator="0"; name="0002-Make-newline-handling-more-flexible-per-WH-declare-k.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0002-Make-newline-handling-more-flexible-per-WH-declare-k.pa"; filename*1="tch" >From 749ca33524ab0d0f96bb71e32bd0cc08070880d9 Mon Sep 17 00:00:00 2001 From: Robert P. Goldman Date: Sun, 9 May 2010 10:20:15 -0500 Subject: [PATCH 2/7] Make newline-handling more flexible (per WH), declare keywords. Followed Wes Hardaker's suggestion to make the translation of newlines more flexible --- instead of making a boolean for special translation of blank lines, I added the ability to specify the translation. Also added a macro for declaring generic translation keywords with type information and documentation. Hope this will make the generic translator easier to use. --- contrib/lisp/org-export-generic.el | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/contrib/lisp/org-export-generic.el b/contrib/lisp/org-export-generic.el index 23ecf9a..0933f19 100644 --- a/contrib/lisp/org-export-generic.el +++ b/contrib/lisp/org-export-generic.el @@ -465,7 +465,7 @@ newlines are interpreted as significant \(e.g., as indicating preformatted text\). A common non-nil value for this keyword is \"\\n\". Should typically be combined with a value for :body-line-format that does NOT end with a newline." - :type string)) + :type string) -- 1.7.1 --------------010902090608070104030407 Content-Type: text/plain; x-mac-type="0"; x-mac-creator="0"; name="0001-Revision-to-handling-of-blank-lines.-Start-of-declar.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0001-Revision-to-handling-of-blank-lines.-Start-of-declar.pa"; filename*1="tch" >From 3d4d938009cf04d987ae4522ddd90558c48dc962 Mon Sep 17 00:00:00 2001 From: Robert P. Goldman Date: Thu, 6 May 2010 14:16:37 -0500 Subject: [PATCH 1/7] Revision to handling of blank lines. Start of declaration protocol. Followed Wes Hardaker's idea of permitting alternative rewrites for blank lines, instead of making the blank line handler be a boolean and hard-wiring a newline character. Also added a declaration form, with type and documentation options, for the keywords used in defining a generic export method. --- contrib/lisp/org-export-generic.el | 27 ++++++++++++++++++++++++++- 1 files changed, 26 insertions(+), 1 deletions(-) diff --git a/contrib/lisp/org-export-generic.el b/contrib/lisp/org-export-generic.el index 1b099dd..23ecf9a 100644 --- a/contrib/lisp/org-export-generic.el +++ b/contrib/lisp/org-export-generic.el @@ -444,6 +444,31 @@ in this way, it will be wrapped." export definitions." (aput 'org-generic-alist type definition)) +;;; helper functions for org-set-generic-type +(defvar org-export-generic-keywords nil) +(defmacro* def-org-export-generic-keyword (keyword + &key documentation + type) + "Define KEYWORD as a legitimate element for inclusion in +the body of an org-set-generic-type definition." + `(progn + (pushnew ,keyword org-export-generic-keywords) + ;; TODO: push the documentation and type information + ;; somewhere where it will do us some good. + )) + +(def-org-export-generic-keyword :body-newline-paragraph + :documentation "Bound either to NIL or to a pattern to be +inserted in the output for every blank line in the input. + The intention is to handle formats where text is flowed, and +newlines are interpreted as significant \(e.g., as indicating +preformatted text\). A common non-nil value for this keyword +is \"\\n\". Should typically be combined with a value for +:body-line-format that does NOT end with a newline." + :type string)) + + + (defun org-export-generic-remember-section (type suffix &optional prefix) (setq org-export-generic-section-type type) (setq org-export-generic-section-suffix suffix) @@ -946,7 +971,7 @@ underlined headlines. The default is 3." ((string-match "^\\s-*$" line) ;; blank line (if bodynewline-paragraph - (insert "\n"))) + (insert bodynewline-paragraph))) (t ;; ;; body -- 1.7.1 --------------010902090608070104030407 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Emacs-orgmode mailing list Please use `Reply All' to send replies to the list. Emacs-orgmode@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-orgmode --------------010902090608070104030407--