Re: ox-md.el: Export TOC and Footnotes as Markdown rather than HTML

[Jake Romer <jkrmr.io@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 8407 bytes --]

Thanks for the pointers! I appreciate your patience -- I'm still a noob
with Org :-)

Here's the updated patch with entries in ORG-NEWS and the manual.

Cheers,

Jake



On Mon, Aug 15, 2016 at 8:34 AM, Nicolas Goaziou <mail@nicolasgoaziou.fr>
wrote:

> Hello,
>
> Jake Romer <jkrmr.io@gmail.com> writes:
>
> > I'm not as familiar with Org's TOC facilities as I should be, so I'll
> > update to handle these cases after some research. I want to avoid letting
> > these patches get too big, too.
>
> See `org-html-toc' and `org-html--toc-text' for an illustration.
>
> > Here's the patch for generating the footnotes section as Markdown, let me
> > know what you think.
>
> It looks good. However, I'm not able to apply them, they seem to be
> created from a subdirectory instead of the root dir.
>
> > From 63d985425d3995985bd340fe195d8e25bd7f2cc6 Mon Sep 17 00:00:00 2001
> > From: Jake Romer <jkrmr@github.com>
> > Date: Sat, 13 Aug 2016 11:45:34 -0400
> > Subject: [PATCH 1/4] Extract org-md-headline-title to generate title
> >
> > In order to generate a headline title in the preferred Markdown
> > style ('atx or 'setext), I extracted this bit of logic from
> > `org-md-headline'.
> >
> > Since `org-md-headline' doesn't process the headline for the footnotes
> > section, extracting `org-md-headline-title' will allow us to generate
> > the Footnote section header in the preferred Markdown style without
> > duplicating the logic for doing so.
> >
> > As an enhancement, `org-md-headline-title' places the anchor tag above
> > the section header. This makes it so that in a browser, clicking on the
> > TOC entry for that section scrolls to the section header without
> > obscuring it.
> >
> > Example generated section title:
> >
> >     <a id="orgheadline1"></a>
>
> Great. However, you need to supply modified variables and functions
> names in the commit message, e.g.,
>
> * lisp/ox-md.el (org-md-headline-title): New function.
> (org-md-headline): Use new function.
>
> >     Summary of Changes
> >     ==================
> > ---
> >  ox-md.el | 37 +++++++++++++++++++++++--------------
> >  1 file changed, 23 insertions(+), 14 deletions(-)
> >
> > diff --git a/ox-md.el b/ox-md.el
> > index 0aaade6..4bd445c 100644
> > --- a/ox-md.el
> > +++ b/ox-md.el
> > @@ -216,20 +216,29 @@ a communication channel."
> >                         (car (last (org-export-get-headline-number
> >                                     headline info))))
> >                        "."))))
> > -       (concat bullet (make-string (- 4 (length bullet)) ?\s) heading
> tags
> > -               "\n\n"
> > -               (and contents
> > -                    (replace-regexp-in-string "^" "    " contents)))))
> > -       ;; Use "Setext" style.
> > -       ((eq style 'setext)
> > -     (concat heading tags anchor "\n"
> > -             (make-string (length heading) (if (= level 1) ?= ?-))
> > -             "\n\n"
> > -             contents))
> > -       ;; Use "atx" style.
> > -       (t (concat (make-string level ?#) " " heading tags anchor "\n\n"
> > -               contents))))))
> > -
> > +       (concat bullet (make-string (- 4 (length bullet)) ?\s) heading
> tags "\n\n"
> > +               (and contents (replace-regexp-in-string "^" "    "
> contents)))))
> > +       (t (concat (org-md-headline-title style level title anchor tags)
> contents))))))
> > +
> > +
> > +;; Headline Title
> > +
> > +(defun org-md-headline-title (style level title &optional anchor tags)
>
> I suggest to name it `org-md--headline-title' to highlight the fact that
> it is an internal function.
>
> > +  "Generate a headline title in the preferred Markdown headline style.
> > +STYLE is the preferred style ('atx or 'setext)
> > +LEVEL is the header level.
> > +TITLE is the headline title.
> > +ANCHOR is the HTML anchor tag for the section as a string.
> > +TAGS are the tags set on the section."
> > +  (let ((anchor-lines (if anchor (concat anchor "\n\n") nil)))
>
>   (and anchor (concat anchor "\n\n"))
>
> > Subject: [PATCH 2/4] Add customizable footnotes-related variables
> >
> > Taking a cue from `ox-html.el', this patch adds
> > `org-md-footnotes-section' and `org-md-footnote-format' to allow user
> > customization of the formatting for the footnotes section and individual
> > footnotes, respectively.
>
> Ditto (variable names...)
>
> > +(defcustom org-md-footnotes-section "%s%s"
> > +  "Format for the footnotes section.
>
> Format string for...
>
> > +The first %s placeholder will be replaced with the localized Footnotes
> section
> > +heading, the second with the contents of the Footnotes section."
> > + :group 'org-export-md
> > +  :type 'string)
>
> You need to provide additional keywords, i.e.,
>
>   :version "25.1"
>   :package-version '(Org . "9.0")
>
>
> > +(defcustom org-md-footnote-format "<sup>%s</sup>"
> > +  "The format for the footnote reference.
> > +The %s will be replaced by the footnote reference itself."
> > +  :group 'org-export-md
> > +  :type 'string)
>
> Ditto.
>
> > Subject: [PATCH 3/4] Add footnote-generating functions
> >
> > Introduces the following functions:
> >
> > * `org-md-footnote-formatted'
> > * `org-md-footnote-section'
>
> See above.
>
> > +;;;; Footnote Section
> > +
> > +(defun org-md-footnote-formatted (footnote info)
>
> Suggestion: `org-md--footnote-formatted'
>
> > +  "Formats a single footnote entry FOOTNOTE.
>
> You should specify the data type for FOOTNOTE, i.e., a cons cell
>
>   (number . definition)
>
> > +INFO is a plist with contextual information."
> > +  (let* ((fn-num (car footnote))
> > +         (fn-text (cdr footnote))
> > +         (fn-format (plist-get info :md-footnote-format))
> > +         (fn-anchor (format "fn.%d" fn-num))
> > +         (fn-href (format " href=\"#fnr.%d\"" fn-num))
> > +         (fn-link-to-ref (org-html--anchor fn-anchor fn-num fn-href
> info)))
> > +    (concat (format fn-format fn-link-to-ref) " " fn-text "\n")))
> > +
> > +(defun org-md-footnote-section (info)
> > +  "Format the footnote section.
> > +INFO is a plist used as a communication channel."
> > +  (let* ((fn-alist (org-export-collect-footnote-definitions info))
> > +         (fn-alist (loop for (n type raw) in fn-alist collect
> > +                         (cons n (org-trim (org-export-data raw
> info)))))
>
> `cl-loop'
>
> > +         (headline-style (plist-get info :md-headline-style))
> > +         (section-title (org-html--translate "Footnotes" info)))
> > +    (when fn-alist
> > +      (format (plist-get info :md-footnotes-section)
> > +              (org-md-headline-title headline-style 1 section-title)
> > +              (mapconcat #'(lambda (fn) (org-md-footnote-formatted fn
> info))
>
>   (mapconcat (lambda (fn) ...) ...)
>
> > +                         fn-alist
> > +                         "\n")))))
> > +
> >  ;;;; Template
> >
> >  (defun org-md-inner-template (contents info)
> > --
> > 2.9.2
> >
> >
> > From a2e353da09d0db9bbde14641bd94e12e156a143c Mon Sep 17 00:00:00 2001
> > From: Jake Romer <jkrmr@github.com>
> > Date: Sat, 13 Aug 2016 13:11:26 -0400
> > Subject: [PATCH 4/4] Update org-md-inner-template to Markdown footnotes
> >
> > Updates `org-md-inner-template' to use `org-md-footnotes-section' to
> > generate a Markdown version of the footnotes section.
> >
> > Also appends `:md-footnote-format' and `md-footnotes-section' to the
> > exporter :options-alist.
> > ---
> >  ox-md.el | 15 +++++++++++++--
> >  1 file changed, 13 insertions(+), 2 deletions(-)
> >
> > diff --git a/ox-md.el b/ox-md.el
> > index 19b8912..7d78ff7 100644
> > --- a/ox-md.el
> > +++ b/ox-md.el
> > @@ -105,7 +105,10 @@ The %s will be replaced by the footnote reference
> itself."
> >                    (src-block . org-md-example-block)
> >                    (template . org-md-template)
> >                    (verbatim . org-md-verbatim))
> > -  :options-alist '((:md-headline-style nil nil org-md-headline-style)))
> > +  :options-alist
> > +  '((:md-headline-style nil nil org-md-headline-style)
> > +    (:md-footnote-format nil nil org-md-footnote-format)
> > +    (:md-footnotes-section nil nil org-md-footnotes-section)))
>
> These new parameters need to be documented in the manual: (info "(org)
> Publishing options").
>
> Would you mind providing an entry in ORG-NEWS and send again the patches
> from root directory so I can apply them?
>
> Thank you!
>
> Regards,
>
> --
> Nicolas Goaziou
>

[-- Attachment #1.2: Type: text/html, Size: 11166 bytes --]

[-- Attachment #2: render-fn-as-md.patch --]
[-- Type: application/octet-stream, Size: 8350 bytes --]

From c1d86d872800d35dc13e9523335154a6cc9a11ca Mon Sep 17 00:00:00 2001
From: Jake Romer <jkrmr@github.com>
Date: Sun, 21 Aug 2016 14:08:29 -0700
Subject: [PATCH] ox-md.el: Export footnotes section as Markdown

* lisp/ox-md.el (org-md--headline-title): New function.  Extract logic
used to generate section header in the preferred style of
Markdown (i.e., setext or atx).
(org-md-headline): Use `org-md-headline-title' to generate section
headers.
(org-md--footnote-formatted): New function.  Format a single footnote
entry as Markdown, using HTML only where necessary.
(org-md-footnote-section): New function.  Format the footnote section,
including the section title using `org-md--headline-title' and
individual entries using `org-md--footnote-formatted'.
(org-md-inner-template): Update to use `org-md-footnote-section'.
(org-md-footnotes-section): New customizable variable.  Format string for
footnotes section.
(org-md-footnote-format): New customizable variable.  Format string for
individual footnote.

Update ox-md.el to export the Footnotes section as Markdown, using HTML
only where necessary - namely, in footnote and footnote reference links.
---
 doc/org.texi  |   2 ++
 etc/ORG-NEWS  |  11 +++++++
 lisp/ox-md.el | 100 ++++++++++++++++++++++++++++++++++++++++++++++++----------
 3 files changed, 97 insertions(+), 16 deletions(-)

diff --git a/doc/org.texi b/doc/org.texi
index 9868113..26ef376 100644
--- a/doc/org.texi
+++ b/doc/org.texi
@@ -14444,6 +14444,8 @@ however, override everything.
 
 @multitable {@code{:md-headline-style}} {@code{org-md-headline-style}}
 @item @code{:md-headline-style} @tab @code{org-md-headline-style}
+@item @code{:md-footnote-format} @tab @code{org-md-footnote-format}
+@item @code{:md-footnotes-section} @tab @code{org-md-footnotes-section}
 @end multitable
 
 @subsubheading ODT specific properties
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 27b6ac9..91a458b 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -373,6 +373,12 @@ It replaces the deprecated ~next~ argument to ~org-previous-line-empty-p~.
 *** ~org-show-children~ 
 It is a faster implementation of ~outline-show-children~.
 
+*** ~org-md-headline~
+Generates a Markdown section headline in the preferred style (Setext or Atx).
+
+*** ~org-md-footnote-section~
+Exports a footnote section as Markdown.
+
 ** Removed functions
 *** ~org-agenda-filter-by-tag-refine~ has been removed.
 Use ~org-agenda-filter-by-tag~ instead.
@@ -503,6 +509,11 @@ modified anyway.
 They are still supported in Org 9.0 but will eventually be removed in
 a later release.  Use ~file~ link type along with universal arguments
 to force opening it in either Emacs or with system application.
+*** Markdown footnote export customization
+Variables ~org-md-footnotes-section~ and ~org-md-footnote-format~
+introduced for ox-md.el.  Both new variables define template strings
+which can be used to customize the format of the exported footnotes
+section and individual footnotes, respectively.
 * Version 8.3
 
 ** Incompatible changes
diff --git a/lisp/ox-md.el b/lisp/ox-md.el
index 7635fc7..467fe73 100644
--- a/lisp/ox-md.el
+++ b/lisp/ox-md.el
@@ -51,6 +51,25 @@ This variable can be set to either `atx' or `setext'."
 	  (const :tag "Use \"Setext\" style" setext)))
 
 
+;;;; Footnotes
+
+(defcustom org-md-footnotes-section "%s%s"
+  "Format string for the footnotes section.
+The first %s placeholder will be replaced with the localized Footnotes section
+heading, the second with the contents of the Footnotes section."
+ :group 'org-export-md
+ :type 'string
+ :version "25.1"
+ :package-version '(Org . "9.0"))
+
+(defcustom org-md-footnote-format "<sup>%s</sup>"
+  "Format string for the footnote reference.
+The %s will be replaced by the footnote reference itself."
+  :group 'org-export-md
+  :type 'string
+  :version "25.1"
+  :package-version '(Org . "9.0"))
+
 \f
 ;;; Define Back-End
 
@@ -89,7 +108,10 @@ This variable can be set to either `atx' or `setext'."
 		     (src-block . org-md-example-block)
 		     (template . org-md-template)
 		     (verbatim . org-md-verbatim))
-  :options-alist '((:md-headline-style nil nil org-md-headline-style)))
+  :options-alist
+  '((:md-headline-style nil nil org-md-headline-style)
+    (:md-footnote-format nil nil org-md-footnote-format)
+    (:md-footnotes-section nil nil org-md-footnotes-section)))
 
 \f
 ;;; Filters
@@ -215,20 +237,29 @@ a communication channel."
 			  (car (last (org-export-get-headline-number
 				      headline info))))
 			 "."))))
-	  (concat bullet (make-string (- 4 (length bullet)) ?\s) heading tags
-		  "\n\n"
-		  (and contents
-		       (replace-regexp-in-string "^" "    " contents)))))
-       ;; Use "Setext" style.
-       ((eq style 'setext)
-	(concat heading tags anchor "\n"
-		(make-string (length heading) (if (= level 1) ?= ?-))
-		"\n\n"
-		contents))
-       ;; Use "atx" style.
-       (t (concat (make-string level ?#) " " heading tags anchor "\n\n"
-		  contents))))))
-
+	  (concat bullet (make-string (- 4 (length bullet)) ?\s) heading tags "\n\n"
+		  (and contents (replace-regexp-in-string "^" "    " contents)))))
+       (t (concat (org-md--headline-title style level title anchor tags) contents))))))
+
+
+;; Headline Title
+
+(defun org-md--headline-title (style level title &optional anchor tags)
+  "Generate a headline title in the preferred Markdown headline style.
+STYLE is the preferred style ('atx or 'setext)
+LEVEL is the header level.
+TITLE is the headline title.
+ANCHOR is the HTML anchor tag for the section as a string.
+TAGS are the tags set on the section."
+  (let ((anchor-lines (and anchor (concat anchor "\n\n"))))
+    ;; Use "Setext" style
+    (if (and (eq style 'setext) (< level 3))
+        (let* ((underline-char (if (= level 1) ?= ?-))
+               (underline (concat (make-string (length title) underline-char) "\n")))
+          (concat "\n" anchor-lines title tags "\n" underline "\n"))
+        ;; Use "Atx" style
+        (let ((level-mark (make-string level ?#)))
+          (concat "\n" anchor-lines level-mark " " title tags "\n\n")))))
 
 ;;;; Horizontal Rule
 
@@ -465,6 +496,35 @@ a communication channel."
   contents)
 
 
+;;;; Footnote Section
+
+(defun org-md--footnote-formatted (footnote info)
+  "Formats a single footnote entry FOOTNOTE.
+FOOTNOTE is a cons cell of the form (number . definition).
+INFO is a plist with contextual information."
+  (let* ((fn-num (car footnote))
+         (fn-text (cdr footnote))
+         (fn-format (plist-get info :md-footnote-format))
+         (fn-anchor (format "fn.%d" fn-num))
+         (fn-href (format " href=\"#fnr.%d\"" fn-num))
+         (fn-link-to-ref (org-html--anchor fn-anchor fn-num fn-href info)))
+    (concat (format fn-format fn-link-to-ref) " " fn-text "\n")))
+
+(defun org-md-footnote-section (info)
+  "Format the footnote section.
+INFO is a plist used as a communication channel."
+  (let* ((fn-alist (org-export-collect-footnote-definitions info))
+         (fn-alist (cl-loop for (n type raw) in fn-alist collect
+                            (cons n (org-trim (org-export-data raw info)))))
+         (headline-style (plist-get info :md-headline-style))
+         (section-title (org-html--translate "Footnotes" info)))
+    (when fn-alist
+      (format (plist-get info :md-footnotes-section)
+              (org-md--headline-title headline-style 1 section-title)
+              (mapconcat (lambda (fn) (org-md--footnote-formatted fn info))
+                         fn-alist
+                         "\n")))))
+
 ;;;; Template
 
 (defun org-md-inner-template (contents info)
@@ -473,7 +533,15 @@ CONTENTS is the transcoded contents string.  INFO is a plist
 holding export options."
   ;; Make sure CONTENTS is separated from table of contents and
   ;; footnotes with at least a blank line.
-  (org-trim (org-html-inner-template (concat "\n" contents "\n") info)))
+  (org-trim (concat
+             ;; Table of contents.
+             (let ((depth (plist-get info :with-toc)))
+               (when depth (org-html-toc depth info)))
+             ;; Document contents.
+             contents
+             "\n"
+             ;; Footnotes section.
+             (org-md-footnote-section info))))
 
 (defun org-md-template (contents _info)
   "Return complete document string after Markdown conversion.
-- 
2.9.3