emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
* [ANN] Merge export-block type within special-block
@ 2014-07-27 12:37 Nicolas Goaziou
  2014-07-27 22:13 ` Bastien
  2014-08-29 14:02 ` KDr2
  0 siblings, 2 replies; 8+ messages in thread
From: Nicolas Goaziou @ 2014-07-27 12:37 UTC (permalink / raw)
  To: Org Mode List

Hello,

Export blocks are blocks dedicated to export back-ends, e.g.,
"#+BEGIN_LATEX". The way they are currently parsed is flawed.

Export blocks are back-end dependent. At the moment, back-ends register
their own export block in a variable, `org-element-block-name-alist', so
the parser can know if it needs to parse an export block or not. As
a consequence, the same block can be parsed differently if a given
export back-end is loaded or not. E.g.,

  #+BEGIN_HTML
  ...
  #+END_HTML

will be parsed as a `special-block' if "ox-html.el" is not loaded, or an
`export-block' otherwise. This is slightly... ugly. And it gets worse if
we include the cache, which will not update the block if it is not
modified.

I just committed a set of patches that solve the problem: `export-block'
elements do not exist anymore. Instead, such blocks are now parsed as
`special-block', always. This does not depend on the libraries loaded so
far.

Of course, special blocks are not treated exactly as export blocks. The
latter's contents are included as-is in the output whereas the former's
are interpreted. Therefore, special blocks now include another
property, :raw-value, which stores the pristine initial contents of the
block, and "ox.el" provides a new function,
`org-export-raw-special-block-p', which tells the difference between
a former export block and a special block. This makes sense since an
"export-block" is clearly, and only, an export concept. This is not
related to Org syntax.

This is more simple to handle than it sounds, and can be described with
two steps:

  1. `export-block' elements, translators and filters are now ignored.
     These can be removed from export back-ends (unless you want to
     preserve compatibility with Org 8.2, in which case leaving them
     will not hurt: they will be used in Org 8.2 and ignored in Org
     8.3).

  2. Translators for special blocks, e.g. `org-BACKEND-special-block'
     need to be updated and check first if current block is a raw
     special block or not. The following template is a suggestion.

     #+BEGIN_SRC emacs-lisp
     (defun org-latex-special-block (special-block contents info)
       (if (org-export-raw-special-block-p special-block info)
           (org-element-property :raw-value special-block)
         ;; Usual handling for special blocks goes here.
         ))
     #+END_SRC

     Note that if BACKEND is a derived back-end and doesn't implement
     its own special block translator already, there is nothing to
     change. The parent back-end will take care of such blocks.

     All back-ends in core and in contrib have been updated this way
     already.

I included `org-export-raw-special-block-p' in Org 8.2, as
a forward-compatibility measure, so back-end maintainers do not have to
do the `fboundp' dance.

BTW, for those in the back of the room: I didn't remove
"#+BEGIN_LATEX"-like constructs.


Regards,

-- 
Nicolas Goaziou                                                0x80A93738

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [ANN] Merge export-block type within special-block
  2014-07-27 12:37 [ANN] Merge export-block type within special-block Nicolas Goaziou
@ 2014-07-27 22:13 ` Bastien
  2014-08-29 14:02 ` KDr2
  1 sibling, 0 replies; 8+ messages in thread
From: Bastien @ 2014-07-27 22:13 UTC (permalink / raw)
  To: Nicolas Goaziou; +Cc: Org Mode List

Hi Nicolas,

thanks for this simplification and for the explanations.

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

>      All back-ends in core and in contrib have been updated this way
>      already.

And for this too!

-- 
 Bastien

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [ANN] Merge export-block type within special-block
  2014-07-27 12:37 [ANN] Merge export-block type within special-block Nicolas Goaziou
  2014-07-27 22:13 ` Bastien
@ 2014-08-29 14:02 ` KDr2
  2014-08-31 12:56   ` Nicolas Goaziou
  1 sibling, 1 reply; 8+ messages in thread
From: KDr2 @ 2014-08-29 14:02 UTC (permalink / raw)
  To: Org Mode List, mail

[-- Attachment #1: Type: text/plain, Size: 3498 bytes --]

Hi Nicolas,

This is nice, but it brought a bug, `[N]' in HTML block is recognized as
footnote, e.g.:

#+BEGIN_HTML
ONE[1]
<script>
console.log(v1[0]);
</script>
#+END_HTML

There are two footnotes in the generated HTML. Would you fix this please?

Thanks.




On Sun, Jul 27, 2014 at 8:37 PM, Nicolas Goaziou <mail@nicolasgoaziou.fr>
wrote:

> Hello,
>
> Export blocks are blocks dedicated to export back-ends, e.g.,
> "#+BEGIN_LATEX". The way they are currently parsed is flawed.
>
> Export blocks are back-end dependent. At the moment, back-ends register
> their own export block in a variable, `org-element-block-name-alist', so
> the parser can know if it needs to parse an export block or not. As
> a consequence, the same block can be parsed differently if a given
> export back-end is loaded or not. E.g.,
>
>   #+BEGIN_HTML
>   ...
>   #+END_HTML
>
> will be parsed as a `special-block' if "ox-html.el" is not loaded, or an
> `export-block' otherwise. This is slightly... ugly. And it gets worse if
> we include the cache, which will not update the block if it is not
> modified.
>
> I just committed a set of patches that solve the problem: `export-block'
> elements do not exist anymore. Instead, such blocks are now parsed as
> `special-block', always. This does not depend on the libraries loaded so
> far.
>
> Of course, special blocks are not treated exactly as export blocks. The
> latter's contents are included as-is in the output whereas the former's
> are interpreted. Therefore, special blocks now include another
> property, :raw-value, which stores the pristine initial contents of the
> block, and "ox.el" provides a new function,
> `org-export-raw-special-block-p', which tells the difference between
> a former export block and a special block. This makes sense since an
> "export-block" is clearly, and only, an export concept. This is not
> related to Org syntax.
>
> This is more simple to handle than it sounds, and can be described with
> two steps:
>
>   1. `export-block' elements, translators and filters are now ignored.
>      These can be removed from export back-ends (unless you want to
>      preserve compatibility with Org 8.2, in which case leaving them
>      will not hurt: they will be used in Org 8.2 and ignored in Org
>      8.3).
>
>   2. Translators for special blocks, e.g. `org-BACKEND-special-block'
>      need to be updated and check first if current block is a raw
>      special block or not. The following template is a suggestion.
>
>      #+BEGIN_SRC emacs-lisp
>      (defun org-latex-special-block (special-block contents info)
>        (if (org-export-raw-special-block-p special-block info)
>            (org-element-property :raw-value special-block)
>          ;; Usual handling for special blocks goes here.
>          ))
>      #+END_SRC
>
>      Note that if BACKEND is a derived back-end and doesn't implement
>      its own special block translator already, there is nothing to
>      change. The parent back-end will take care of such blocks.
>
>      All back-ends in core and in contrib have been updated this way
>      already.
>
> I included `org-export-raw-special-block-p' in Org 8.2, as
> a forward-compatibility measure, so back-end maintainers do not have to
> do the `fboundp' dance.
>
> BTW, for those in the back of the room: I didn't remove
> "#+BEGIN_LATEX"-like constructs.
>
>
> Regards,
>
> --
> Nicolas Goaziou                                                0x80A93738
>
>


-- 
-- 

KDr2, http://kdr2.com

[-- Attachment #2: Type: text/html, Size: 4627 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [ANN] Merge export-block type within special-block
  2014-08-29 14:02 ` KDr2
@ 2014-08-31 12:56   ` Nicolas Goaziou
  2014-09-25 15:12     ` KDr2
  2014-09-28  6:18     ` Aaron Ecay
  0 siblings, 2 replies; 8+ messages in thread
From: Nicolas Goaziou @ 2014-08-31 12:56 UTC (permalink / raw)
  To: KDr2; +Cc: Bastien Guerry, Org Mode List

Hello,

KDr2 <killy.draw@gmail.com> writes:

> This is nice, but it brought a bug, `[N]' in HTML block is recognized as
> footnote, e.g.:
>
> #+BEGIN_HTML
> ONE[1]
> <script>
> console.log(v1[0]);
> </script>
> #+END_HTML
>
> There are two footnotes in the generated HTML. Would you fix this
> please?

Unfortunately, no, I cannot fix it.

The problem is even deeper. Indeed, my approach is fundamentally wrong:
it is impossible to postpone choosing between parsed or raw data at
export time. This information must be obtained at parsing time.

Yet, I think syntax should not depend on the libraries loaded. So the
initial problem still needs a solution.

Special blocks and export blocks are just too similar.  We could make
them slightly different. One solution is to mark explicitly blocks meant
to insert raw code. E.g.,

  #+BEGIN_SOMETHING :special t
  ...
  #+END_SOMETHING

vs

  #+BEGIN_SOMETHING
  ...
  #+END_SOMETHING

In the first case contents would be parsed and the block treated as
a special block (i.e. depending on the back-end) whereas in the second
case, contents would be inserted as-is in the buffer, provided target
export back-ends accepts data from "SOMETHING" blocks (IOW "SOMETHING"
= "LATEX" if ox-latex is used).

This is clearly not backward-compatible. But it only modifies syntax for
special blocks, which, I guess, are much less used than their cousins,
export blocks. The ":special t" may be shorter, too.

Cc'ing Bastien for his opinion.


Regards,

-- 
Nicolas Goaziou

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [ANN] Merge export-block type within special-block
  2014-08-31 12:56   ` Nicolas Goaziou
@ 2014-09-25 15:12     ` KDr2
  2014-09-26  9:09       ` Nicolas Goaziou
  2014-09-28  6:18     ` Aaron Ecay
  1 sibling, 1 reply; 8+ messages in thread
From: KDr2 @ 2014-09-25 15:12 UTC (permalink / raw)
  To: Org Mode List

[-- Attachment #1: Type: text/plain, Size: 1933 bytes --]

I found this was fixed on both maint and master branch :)
Thanks for all your works, but would you tell us how did you do it? or give
the commit id? (Sorry I did not find it by myself...)

Thank you very much.

On Sun, Aug 31, 2014 at 8:56 PM, Nicolas Goaziou <mail@nicolasgoaziou.fr>
wrote:

> Hello,
>
> KDr2 <killy.draw@gmail.com> writes:
>
> > This is nice, but it brought a bug, `[N]' in HTML block is recognized as
> > footnote, e.g.:
> >
> > #+BEGIN_HTML
> > ONE[1]
> > <script>
> > console.log(v1[0]);
> > </script>
> > #+END_HTML
> >
> > There are two footnotes in the generated HTML. Would you fix this
> > please?
>
> Unfortunately, no, I cannot fix it.
>
> The problem is even deeper. Indeed, my approach is fundamentally wrong:
> it is impossible to postpone choosing between parsed or raw data at
> export time. This information must be obtained at parsing time.
>
> Yet, I think syntax should not depend on the libraries loaded. So the
> initial problem still needs a solution.
>
> Special blocks and export blocks are just too similar.  We could make
> them slightly different. One solution is to mark explicitly blocks meant
> to insert raw code. E.g.,
>
>   #+BEGIN_SOMETHING :special t
>   ...
>   #+END_SOMETHING
>
> vs
>
>   #+BEGIN_SOMETHING
>   ...
>   #+END_SOMETHING
>
> In the first case contents would be parsed and the block treated as
> a special block (i.e. depending on the back-end) whereas in the second
> case, contents would be inserted as-is in the buffer, provided target
> export back-ends accepts data from "SOMETHING" blocks (IOW "SOMETHING"
> = "LATEX" if ox-latex is used).
>
> This is clearly not backward-compatible. But it only modifies syntax for
> special blocks, which, I guess, are much less used than their cousins,
> export blocks. The ":special t" may be shorter, too.
>
> Cc'ing Bastien for his opinion.
>
>
> Regards,
>
> --
> Nicolas Goaziou
>



-- 
-- 

KDr2, http://kdr2.com

[-- Attachment #2: Type: text/html, Size: 2836 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [ANN] Merge export-block type within special-block
  2014-09-25 15:12     ` KDr2
@ 2014-09-26  9:09       ` Nicolas Goaziou
  0 siblings, 0 replies; 8+ messages in thread
From: Nicolas Goaziou @ 2014-09-26  9:09 UTC (permalink / raw)
  To: KDr2; +Cc: Org Mode List

Hello,

KDr2 <killy.draw@gmail.com> writes:

> I found this was fixed on both maint and master branch :)
> Thanks for all your works, but would you tell us how did you do it? or give
> the commit id? (Sorry I did not find it by myself...)

This is not really fixed. I just reverted the code base to its initial
state, which is bugged in a different, and less visible, way. The
problem needs further discussion to move on.


Regards,

-- 
Nicolas Goaziou

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [ANN] Merge export-block type within special-block
  2014-08-31 12:56   ` Nicolas Goaziou
  2014-09-25 15:12     ` KDr2
@ 2014-09-28  6:18     ` Aaron Ecay
  2014-09-28  7:26       ` Nicolas Goaziou
  1 sibling, 1 reply; 8+ messages in thread
From: Aaron Ecay @ 2014-09-28  6:18 UTC (permalink / raw)
  To: Nicolas Goaziou, KDr2; +Cc: Bastien Guerry, Org Mode List

Hi Nicolas,

I like the thrust of your initial patch very much; I would like to help
you make it happen.  Here’s some discussion as you requested in a
follow-up message.

2014ko abuztuak 31an, Nicolas Goaziou-ek idatzi zuen:
>
> Unfortunately, no, I cannot fix it.
> 
> The problem is even deeper. Indeed, my approach is fundamentally wrong:
> it is impossible to postpone choosing between parsed or raw data at
> export time. This information must be obtained at parsing time.
> 
> Yet, I think syntax should not depend on the libraries loaded. So the
> initial problem still needs a solution.
> 
> Special blocks and export blocks are just too similar.  We could make
> them slightly different. One solution is to mark explicitly blocks meant
> to insert raw code. E.g.,
> 
>   #+BEGIN_SOMETHING :special t
>   ...
>   #+END_SOMETHING
> 
> vs
> 
>   #+BEGIN_SOMETHING
>   ...
>   #+END_SOMETHING
>

I have a hunch that this is backwards.  It seems like the convention
has been to allow arbitrary special blocks in org files.  Sticking to
what I know best, these create arbitrarily-named environments in Latex.
Export blocks seem like the special case (e.g. the number of types is
constrained by the available export modules), and so they should have
to bear special marking.

What if you used the convention that all export blocks had the form
#+begin_export_latex, #+begin_export_html, etc.?  This should be
unambiguous to parse.  (It’s possible to bikeshed about the name, of
course: perhaps #+begin_literal_latex etc.)

HTH,

-- 
Aaron Ecay

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [ANN] Merge export-block type within special-block
  2014-09-28  6:18     ` Aaron Ecay
@ 2014-09-28  7:26       ` Nicolas Goaziou
  0 siblings, 0 replies; 8+ messages in thread
From: Nicolas Goaziou @ 2014-09-28  7:26 UTC (permalink / raw)
  To: KDr2; +Cc: Bastien Guerry, Org Mode List

Hello,

Aaron Ecay <aaronecay@gmail.com> writes:

> I have a hunch that this is backwards.  It seems like the convention
> has been to allow arbitrary special blocks in org files.  Sticking to
> what I know best, these create arbitrarily-named environments in Latex.
> Export blocks seem like the special case (e.g. the number of types is
> constrained by the available export modules), and so they should have
> to bear special marking.

Historically (i.e. pre-8.0), so-called export blocks were a core
feature, whereas special blocks were defined in an optional library
(IIRC "org-special-blocks.el"). That explains why my proposal was to
optionally activate special blocks and not the other way round.

We can instead use

  #+begin_latex :raw t
  ...
  #+end_latex

to optionally activate an export block instead of a special block.
I don't mind either way.

> What if you used the convention that all export blocks had the form
> #+begin_export_latex, #+begin_export_html, etc.?  This should be
> unambiguous to parse.  (It’s possible to bikeshed about the name, of
> course: perhaps #+begin_literal_latex etc.)

One drawback, however, is that is prevents any special block name from
starting with "export" (or any post-bikeshedding name that could be
issued).


Regards,

-- 
Nicolas Goaziou

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2014-09-28  7:26 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-07-27 12:37 [ANN] Merge export-block type within special-block Nicolas Goaziou
2014-07-27 22:13 ` Bastien
2014-08-29 14:02 ` KDr2
2014-08-31 12:56   ` Nicolas Goaziou
2014-09-25 15:12     ` KDr2
2014-09-26  9:09       ` Nicolas Goaziou
2014-09-28  6:18     ` Aaron Ecay
2014-09-28  7:26       ` Nicolas Goaziou

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).