Re: How to customize the org-mode's BEGIN_SRC HTML output

Re: How to customize the org-mode's BEGIN_SRC HTML output

[Benjamin Beckwith]

Rafael,

I had my shortcodes setup to accept the language directly. Your change
should work as you indicated.  I have an additional fix to my code that
should behave better for you.  I add a property, org-protected, that
prevents processing of the text.

I tried it on your code blocks by calling org-export directly (C-c C-e)
and then choosing 'H' to see the html in a buffer.  This really helps
with debug.

Fixed code follows
----------------------------------------------------
(defun bnb/org2blog-src-blocks-to-wp-syntaxhighlighter ()
  "Export #+BEGIN_SRC blocks as Wordpress Syntaxhighlighter
tags. There is a special header option, :syntaxhl that contains
the options to pass to syntaxhighlighter.

This is intended to be added to `org-export-preprocess-hooks'"
  (interactive)
  (save-window-excursion
    (let ((case-fold-search t)
	  (colon-re "^[ \t]*:\\([ \t]\\|$\\)")
	  lang body headers syntaxhl block
	  beg)
      (goto-char (point-min))
      (while (re-search-forward colon-re nil t)
	(replace-match (match-string 1))
	(beginning-of-line 1)
	(insert "[text light=\"true\"]\n")
	(setq beg (point))
	(while (looking-at colon-re)
	  (replace-match (match-string 1))
	  (end-of-line 1)
	  (or (eobp) (forward-char 1)))
	(end-of-line 1)
	(add-text-properties beg
			     (if (bolp)
				 (1- (point))
			       (point))
			     '(org-protected t))
	(insert "\n[/text]"))
      (unless (boundp 'org-babel-src-block-regexp)
	(require 'ob))
      (while (re-search-forward
	      (concat "\\(" org-babel-src-block-regexp
		      "\\|" org-babel-inline-src-block-regexp
		      "\\)")
	      nil t)
	(setq lang (match-string-no-properties 3))
	(if (string-match "-" lang)
	    (error "SyntaxHighlighter does not support languages with '-' in
the names"))
	(setq headers (match-string-no-properties 5))
	(setq body (match-string-no-properties 6))
	(save-match-data
	  (setq syntaxhl
		(if (string-match ":syntaxhl[ ]+\\([^ ]+\\)" headers)
		    (concat " " (replace-regexp-in-string "\;" " " (match-string 1
headers))))))
	(setq block (concat "\n\n[" lang syntaxhl "]\n" body "[/" lang "]\n"))
	(add-text-properties 0 (length block) '(org-protected t) block)
	(replace-match
	 block
	 nil t)))))
----------------------------------

I also remembered that I made the following setting as well, but I do
not know if I still need it.

(setq org-export-preprocess-hook
      (list
       'bnb/org2blog-src-blocks-to-wp-syntaxhighlighter
       'org-export-blocks-preprocess))

Re: Re: How to customize the org-mode's BEGIN_SRC HTML output

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 1107 bytes --]

On Tue, 24 Aug 2010 21:36:40 -0500, Rafael <rvf0068@gmail.com> wrote:
> 
> Benjamin Beckwith <bnbeckwith@gmail.com> writes:
> 
> > Hi, I also was interested in posting these blocks (through org2blog in
> > wordpress).  The code I posted below is added to
> > 'org-export-preprocess-hooks' where it looks for BEGIN_SRC blocks as
> > well as ':' blocks of code.
> >
> > In the case of BEGIN_SRC blocks, I add a header option, :syntaxhl where
> > I can pass in additional settings to the syntaxhighlighter code.
> >
> > The code below uses Wordpress shortcodes, but I am sure that you can
> > adapt for your own purposes.
> 
> Thanks! that seems to almost work for me, if I replace your penultimate
> line:
> 
> > (concat "\n\n[" lang syntaxhl "]\n" body "[/" lang "]\n")
> 
> by
> 
> (concat "\n\n[sourcecode language=\"" lang syntaxhl "\"]\n"
>                                           body "[/sourcecode]\n")
> 
> but then the line breaks are lost, and the latex exporter tries to be
> too smart and replace the code.. Please see

[...]

Actually, I lose the line breaks with the original version as well.

[-- Attachment #2: Type: text/plain, Size: 75 bytes --]

-- 
Eric S Fraga
GnuPG: 8F5C 279D 3907 E14A 5C29  570D C891 93D8 FFFC F67D

[-- Attachment #3: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: How to customize the org-mode's BEGIN_SRC HTML output

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 1329 bytes --]

On Tue, 24 Aug 2010 23:32:01 -0400, Benjamin Beckwith <bnbeckwith@gmail.com> wrote:
> 
> Rafael,
> 
> I had my shortcodes setup to accept the language directly. Your change
> should work as you indicated.  I have an additional fix to my code that
> should behave better for you.  I add a property, org-protected, that
> prevents processing of the text.
> 
> I tried it on your code blocks by calling org-export directly (C-c C-e)
> and then choosing 'H' to see the html in a buffer.  This really helps
> with debug.
> 
> Fixed code follows

[...]

Benjamin,

can you show us a simple example blog entry that works with this?  I
have tried and, with your version, I do not get the source code plugin
in Wordpress invoked.  If I use Rafael's change, the plugin does get
used.  Also, both your code and Rafael's cause all linebreaks in the
code to disappear.  

For instance, with some simple =MATLAB= code:

--8<---------------cut here---------------start------------->8---
#+srcname: matlabblock
#+begin_src matlab
function y = f(x)
  x = 1989;
  y = (2010-x)^2;
#+end_src
--8<---------------cut here---------------end--------------->8---

your most recent code generates

,----
| <p>    [matlab] function y = f(x)   x = 1989;   y = (2010-x)^2; [/matlab]
`----

which is not something wordpress seems to understand?

Thanks,
eric

[-- Attachment #2: Type: text/plain, Size: 75 bytes --]

-- 
Eric S Fraga
GnuPG: 8F5C 279D 3907 E14A 5C29  570D C891 93D8 FFFC F67D

[-- Attachment #3: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: How to customize the org-mode's BEGIN_SRC HTML output

[Rafael]

Eric S Fraga <ucecesf@ucl.ac.uk> writes:

> On Tue, 24 Aug 2010 23:32:01 -0400, Benjamin Beckwith <bnbeckwith@gmail.com> wrote:
>> 
>> Rafael,
>> 
>> I had my shortcodes setup to accept the language directly. Your change
>> should work as you indicated.  I have an additional fix to my code that
>> should behave better for you.  I add a property, org-protected, that
>> prevents processing of the text.
>> 
>> I tried it on your code blocks by calling org-export directly (C-c C-e)
>> and then choosing 'H' to see the html in a buffer.  This really helps
>> with debug.
>> 
>> Fixed code follows
>
> [...]
>
> Benjamin,
>
> can you show us a simple example blog entry that works with this?  I
> have tried and, with your version, I do not get the source code plugin
> in Wordpress invoked.  If I use Rafael's change, the plugin does get
> used.  Also, both your code and Rafael's cause all linebreaks in the
> code to disappear.  
>
> For instance, with some simple =MATLAB= code:
>
> #+srcname: matlabblock
> #+begin_src matlab
> function y = f(x)
>   x = 1989;
>   y = (2010-x)^2;
> #+end_src
>
> your most recent code generates
>
> ,----
> | <p>    [matlab] function y = f(x)   x = 1989;   y = (2010-x)^2; [/matlab]
> `----
>
> which is not something wordpress seems to understand?
>

I think Benjamin means that he has customized this

http://en.support.wordpress.com/shortcodes/

so that his code works directly. But it is not clear to me which changes
he had to do for that, nor if that is available to free wordpress
users...

Oh, and by the way, with Org-mode version 7.01h I concur with Eric in
that I unfortunately still lose the linebreaks and get LaTeX code
modified with Benjamin's new version.. 

Re: Re: How to customize the org-mode's BEGIN_SRC HTML output

[Puneeth]

Hi All,

I'm not sure if I should've jumped into the discussion much earlier,
but anyways,

On Wed, Aug 25, 2010 at 5:53 PM, Rafael <rvf0068@gmail.com> wrote:
> I think Benjamin means that he has customized this
>
> http://en.support.wordpress.com/shortcodes/
>
> so that his code works directly. But it is not clear to me which changes
> he had to do for that, nor if that is available to free wordpress
> users...

I do not understand the part about wordpress shortcodes and stuff. For
the theme I use on my wordpress.com blog, I do not get any syntax
highlighting from the direct html export of org(2blog).  I only
learnt, recently that wordpress has a special way to include code for
syntax highlighting. (Some themes seem to highlight code in documents
generated directly using org's export. for eg: [1])

> Oh, and by the way, with Org-mode version 7.01h I concur with Eric in
> that I unfortunately still lose the linebreaks and get LaTeX code
> modified with Benjamin's new version..

The line breaks being stripped off is due to code in org2blog. It has
nothing to do with org-mode's export. Wordpress does not ignore
linebreaks in the content, which looks very ugly for normal posts.
Code in org2blog strips off the line breaks from the html generated by
org-export-as-html. It checks for <pre> and <blockquote> tags and
leaves out the newlines within those tags. This is (most) probably
what is causing trouble.

I'll only be able to look into it in the weekend. Anybody is free to
beat me to that. :)

HTH,
Puneeth

[1] - http://rvftestblog.wordpress.com/2010/08/14/testing-code-blocks/

Re: Re: How to customize the org-mode's BEGIN_SRC HTML output

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 1717 bytes --]

On Wed, 25 Aug 2010 18:21:22 +0530, Puneeth <punchagan@gmail.com> wrote:
> 
> Hi All,
> 
> I'm not sure if I should've jumped into the discussion much earlier,
> but anyways,
> 
> On Wed, Aug 25, 2010 at 5:53 PM, Rafael <rvf0068@gmail.com> wrote:
> > I think Benjamin means that he has customized this
> >
> > http://en.support.wordpress.com/shortcodes/
> >
> > so that his code works directly. But it is not clear to me which changes
> > he had to do for that, nor if that is available to free wordpress
> > users...
> 
> I do not understand the part about wordpress shortcodes and stuff. For
> the theme I use on my wordpress.com blog, I do not get any syntax
> highlighting from the direct html export of org(2blog).  I only
> learnt, recently that wordpress has a special way to include code for
> syntax highlighting. (Some themes seem to highlight code in documents
> generated directly using org's export. for eg: [1])

It turns, having dug deeper, that there are two formats for source
code highlighting.  The two formats exist because users that have a
blog hosted by wordpress (e.g. someblog.wordpress.com) use one format
and those users that have their own hosted blogs based on
wordpress.org code use another.  The first is based on =[sourcecode
language...]= construct; the latter is based on a plugin and the
format is =[language]...[/language]=.  I don't understand why the
difference exists.

> 
> > Oh, and by the way, with Org-mode version 7.01h I concur with Eric in
> > that I unfortunately still lose the linebreaks and get LaTeX code
> > modified with Benjamin's new version..
> 
> The line breaks being stripped off is due to code in org2blog. It has

[...]

Ah ha!  This makes sense.  Thanks.

[-- Attachment #2: Type: text/plain, Size: 75 bytes --]

-- 
Eric S Fraga
GnuPG: 8F5C 279D 3907 E14A 5C29  570D C891 93D8 FFFC F67D

[-- Attachment #3: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: How to customize the org-mode's BEGIN_SRC HTML output

[Benjamin Beckwith]

Eric,

I should mention that I am using the "Syntaxhighlighter Evolved"
plugin for Wordpress.  It allows shortcodes with just the language
name.  I probably should use the [sourcecode lang="..."] variant
instead.  Maybe that is why my version did not work for you?

My latest page created with my code can be found at:
http://bnbeckwith.com/index.php/writegood-mode/

As for the linebreaks, in my haste to share I forgot that I was
advising the org2blog-strip-newlines function. Which effectively keeps
all line breaks.
------------------------------
(defadvice org2blog-strip-new-lines (around
bnb/org2blog-disable-newline-removal)
  "Disables the function the removes newlines from the generated HTML"
  (setq ad-return-value (ad-get-arg 0)))
------------------------------

Re: How to customize the org-mode's BEGIN_SRC HTML output

[Rafael]

Puneeth <punchagan@gmail.com> writes:

> The line breaks being stripped off is due to code in org2blog. It has
> nothing to do with org-mode's export. Wordpress does not ignore
> linebreaks in the content, which looks very ugly for normal posts.
> Code in org2blog strips off the line breaks from the html generated by
> org-export-as-html. It checks for <pre> and <blockquote> tags and
> leaves out the newlines within those tags. This is (most) probably
> what is causing trouble.
>
> I'll only be able to look into it in the weekend. Anybody is free to
> beat me to that. :)

This is a workaround, all based on work by Puneeth and Benjamin, that
seems to work for me. I did not know what to do with Benjamin's advice
stuff (Just evaluating it changed nothing in the output). 

;; Benjamin's stuff, one line changed
(defun bnb/org2blog-src-blocks-to-wp-syntaxhighlighter ()
  "Export #+BEGIN_SRC blocks as Wordpress Syntaxhighlighter
tags. There is a special header option, :syntaxhl that contains
the options to pass to syntaxhighlighter.
This is intended to be added to `org-export-preprocess-hooks'"
  (interactive)
  (save-window-excursion
    (let ((case-fold-search t)
	  (colon-re "^[ \t]*:\\([ \t]\\|$\\)")
	  lang body headers syntaxhl
	  beg)
      (goto-char (point-min))
      (while (re-search-forward colon-re nil t)
	(replace-match (match-string 1))
	(beginning-of-line 1)
	(insert "[text light=\"true\"]\n")
	(setq beg (point))
	(while (looking-at colon-re)
	  (replace-match (match-string 1))
	  (end-of-line 1)
	  (or (eobp) (forward-char 1)))
	(end-of-line 1)
	(add-text-properties beg
			     (if (bolp)
				 (1- (point))
			       (point))
			     '(org-protected t))
	(insert "\n[/text]"))
      (unless (boundp 'org-babel-src-block-regexp)
	(require 'ob))
      (while (re-search-forward
	      (concat "\\(" org-babel-src-block-regexp
		      "\\|" org-babel-inline-src-block-regexp
		      "\\)")
	      nil t)
	(setq lang (match-string-no-properties 3))
	(if (string-match "-" lang)
	    (error "SyntaxHighlighter does not support languages with '-' in the names"))
	(setq headers (match-string-no-properties 5))
	(setq body (match-string-no-properties 6))
	(save-match-data
	  (setq syntaxhl
		(if (string-match ":syntaxhl[ ]+\\([^ ]+\\)" headers)
		    (concat " " (replace-regexp-in-string "\;" " " (match-string 1 headers))))))
	(replace-match
         ;(concat "\n\n[" lang syntaxhl "]\n" body "[/" lang "]\n")
	 (concat "\n\n[sourcecode language=\"" lang syntaxhl "\"]\n" body "[/sourcecode]\n")
	 nil t)))))

(add-hook 'org-export-preprocess-hook 'bnb/org2blog-src-blocks-to-wp-syntaxhighlighter)

;; searching for [sourcecode ... ] ... [/sourcecode] so that newlines
;; are not removed
;; Puneeth's stuff, two lines changed
(defun org2blog-strip-new-lines (html)
  "Strip the new lines from the html, except in pre and blockquote tags."
  (save-excursion
    (with-temp-buffer
      (let* (start-pos end-pos)
        (insert html)
        (setq start-pos (point-min))
        (goto-char start-pos)
        (while (re-search-forward "\\[sourcecode.*" nil t 1)
          (setq end-pos (match-beginning 0))
          (replace-regexp "\\\n" " " nil start-pos end-pos)
          (re-search-forward "\\[/sourcecode.*" nil t 1)
          (setq start-pos (match-end 0))
          (goto-char start-pos))
        (setq end-pos (point-max))
        (replace-regexp "\\\n" " " nil start-pos end-pos)
        (buffer-substring-no-properties (point-min) (point-max))))))

Re: Re: How to customize the org-mode's BEGIN_SRC HTML output

[Puneeth]

On Wed, Aug 25, 2010 at 6:21 PM, Puneeth <punchagan@gmail.com> wrote:
>> Oh, and by the way, with Org-mode version 7.01h I concur with Eric in
>> that I unfortunately still lose the linebreaks and get LaTeX code
>> modified with Benjamin's new version..
>
> The line breaks being stripped off is due to code in org2blog. It has
> nothing to do with org-mode's export. Wordpress does not ignore
> linebreaks in the content, which looks very ugly for normal posts.
> Code in org2blog strips off the line breaks from the html generated by
> org-export-as-html. It checks for <pre> and <blockquote> tags and
> leaves out the newlines within those tags. This is (most) probably
> what is causing trouble.
>
> I'll only be able to look into it in the weekend. Anybody is free to
> beat me to that. :)

This has been fixed. Thanks to a patch from Benjamin Beckwith. You can
now use any WP shortcode blocks in your org2blog posts, without losing
line breaks.

Also, org2blog now directly posts babel src blocks as WP's sourcecode
blocks, without modifying the src blocks in your org file. Just like
with Benjamin's hook function, :syntaxhl header option can still be
used to pass additional arguments to WP's syntaxhighlighter.

Happy Blogging,
Puneeth

Re: How to customize the org-mode's BEGIN_SRC HTML output

[Rafael]

Puneeth <punchagan@gmail.com> writes:

> This has been fixed. Thanks to a patch from Benjamin Beckwith. You can
> now use any WP shortcode blocks in your org2blog posts, without losing
> line breaks.
>
> Also, org2blog now directly posts babel src blocks as WP's sourcecode
> blocks, without modifying the src blocks in your org file. Just like
> with Benjamin's hook function, :syntaxhl header option can still be
> used to pass additional arguments to WP's syntaxhighlighter.

That is just great, Puneeth, many thanks! I just checked and works as
intended. 

Now, in case you wanted to have some more feature requests, ;-), here
they are:

1. Making an option to disable the inclusion of ':light="true"'.

2. Note that Wordpress also includes support for inline math expressions
written in LaTeX, like this: $latex E=mc^2$, and displayed expressions,
like this: $latex \displaystyle E=mc^2$
(http://rvftestblog.wordpress.com/2010/08/24/code-blocks-again/ to see
what I mean). So it would be great if, the
exporter could translate $....$ into $latex ....$ and 

\begin{equation}
.......
\end{equation}

into

$latex \displaystyle ...... $.

Best regards,

Rafael

Re: Re: How to customize the org-mode's BEGIN_SRC HTML output

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 587 bytes --]

On Sun, 29 Aug 2010 15:07:11 +0530, Puneeth <punchagan@gmail.com> wrote:

[...]

> This has been fixed. Thanks to a patch from Benjamin Beckwith. You can
> now use any WP shortcode blocks in your org2blog posts, without losing
> line breaks.
> 
> Also, org2blog now directly posts babel src blocks as WP's sourcecode
> blocks, without modifying the src blocks in your org file. Just like
> with Benjamin's hook function, :syntaxhl header option can still be
> used to pass additional arguments to WP's syntaxhighlighter.
> 
> Happy Blogging,
> Puneeth

Thanks!  This works like a charm.

[-- Attachment #2: Type: text/plain, Size: 75 bytes --]

-- 
Eric S Fraga
GnuPG: 8F5C 279D 3907 E14A 5C29  570D C891 93D8 FFFC F67D

[-- Attachment #3: Type: text/plain, Size: 201 bytes --]

_______________________________________________
Emacs-orgmode mailing list
Please use `Reply All' to send replies to the list.
Emacs-orgmode@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Re: How to customize the org-mode's BEGIN_SRC HTML output

[Puneeth]

Rafael,

This has been Done.

Happy Blogging,
Puneeth

On Sun, Aug 29, 2010 at 11:48 PM, Rafael <rvf0068@gmail.com> wrote:
>
> Now, in case you wanted to have some more feature requests, ;-), here
> they are:
>
> 1. Making an option to disable the inclusion of ':light="true"'.
>
> 2. Note that Wordpress also includes support for inline math expressions
> written in LaTeX, like this: $latex E=mc^2$, and displayed expressions,
> like this: $latex \displaystyle E=mc^2$
> (http://rvftestblog.wordpress.com/2010/08/24/code-blocks-again/ to see
> what I mean). So it would be great if, the
> exporter could translate $....$ into $latex ....$ and
>
> \begin{equation}
> .......
> \end{equation}
>
> into
>
> $latex \displaystyle ...... $.

Re: How to customize the org-mode's BEGIN_SRC HTML output

[Rafael]

Puneeth <punchagan@gmail.com> writes:

>> On Sun, Aug 29, 2010 at 11:48 PM, Rafael <rvf0068@gmail.com> wrote:

>>> 2. Note that Wordpress also includes support for inline math expressions
>>> written in LaTeX, like this: $latex E=mc^2$, and displayed expressions,
>>> like this: $latex \displaystyle E=mc^2$
>>> (http://rvftestblog.wordpress.com/2010/08/24/code-blocks-again/ to see
>>> what I mean). 
>
> This has been Done.
>

Hi Puneeth and all,

Thanks a lot!
I could observe one bug and one "feature".

LaTeX expressions of one letter are not touched, for example $a$, $b$
etc.

Also, since by default stuff like \alpha and \cup is converted to &alpha
and &cup, you might want to include

(setq org-export-with-TeX-macros nil)

when exporting LaTeX.

Rafael