An org backend to Haddock

An org backend to Haddock

[Yuchen Pei]

Hello,

I decided to write an Org backend to Haddock[1], so that haskell library
documentation can be generated in org mode markup.  Compared to the
existing backends (html, latex and hoogle), the org format allows
utilising features like the infinite levels of heading hierarchy,
flexible folding / unfolding, cross-package linking, jumping to any
declaration with org-goto and the endless potentials of emacs
customisation.

It seems to me most information and haskell language features one can
find on displayed on hackage are supported by the org backend and
included in the output org files, though there are still some rough
edges and unsupported language features (like infix declarations and
linear types) which I aim to fix.  Some example output can be found at
[2] (I will need to rename the "assets" part of the url as it is not
accurate), including base[4], ghc-prim[5] and ghc-lib-parser[6] (I use
it for reference of the GHC API as it is easier to build than GHC).  For
more information, check out the README file[3].

Given this is not the official version of haddock and my changes are in
haddock-api, I'm calling it haddorg-api, for lack of a better name /
approach.  I'll be happy to contribute my changes upstream if a
different license covering my contribution (AGPLv3+) is accepted.

Let me know what you think.

[1] https://g.ypei.me/haddock.git/tree/haddock-api
[2] https://ypei.org/assets/haddorg-output/
[3] https://g.ypei.me/haddock.git/tree/haddock-api/README.org
[4] https://ypei.org/assets/haddorg-output/base-4.16.1.0.org.gz
[5] https://ypei.org/assets/haddorg-output/ghc-prim-0.8.0.org.gz
[6] https://ypei.org/assets/haddorg-output/ghc-lib-parser-9.2.2.20220307.org.gz

Best,
Yuchen

-- 
PGP Key: 47F9 D050 1E11 8879 9040  4941 2126 7E93 EF86 DFD0
          <https://ypei.org/assets/ypei-pubkey.txt>

Re: An org backend to Haddock

[Ihor Radchenko]

Yuchen Pei <id@ypei.org> writes:

> Given this is not the official version of haddock and my changes are in
> haddock-api, I'm calling it haddorg-api, for lack of a better name /
> approach.  I'll be happy to contribute my changes upstream if a
> different license covering my contribution (AGPLv3+) is accepted.
>
> Let me know what you think.

I am pretty sure that you are asking the Haskell people for upstreaming
;)

From Org side, it is always welcome when other Free software projects
are making use of Org.

Also, I looked into
https://g.ypei.me/haddock.git/tree/haddock-api/src/Haddock/Backends/Org/Types.hs

Beware of possible edge cases with "*/~|" in formatting. I am not
sure if text containing such symbols is possible as an input for your
library, but if it is, things may get formatted wrongly.

For example, text containing "|" anywhere inside a table field must have
"|" escaped using \vert entity. The markup can be escaped using
zero-width space.

Best,
Ihor

Re: An org backend to Haddock

[Yuchen Pei]

On Tue 2022-07-19 21:58:26 +0800, Ihor Radchenko wrote:

> Yuchen Pei <id@ypei.org> writes:
>
>> Given this is not the official version of haddock and my changes are in
>> haddock-api, I'm calling it haddorg-api, for lack of a better name /
>> approach.  I'll be happy to contribute my changes upstream if a
>> different license covering my contribution (AGPLv3+) is accepted.
>>
>> Let me know what you think.
>
> I am pretty sure that you are asking the Haskell people for upstreaming
> ;)

Yes - I sent the email to both haskell-cafe and emacs-orgmode, but
upstream here means haddock / haddock-api indeed, thanks for clarifying.

>
> From Org side, it is always welcome when other Free software projects
> are making use of Org.
>
> Also, I looked into
> https://g.ypei.me/haddock.git/tree/haddock-api/src/Haddock/Backends/Org/Types.hs
>
> Beware of possible edge cases with "*/~|" in formatting. I am not
> sure if text containing such symbols is possible as an input for your
> library, but if it is, things may get formatted wrongly.
>
> For example, text containing "|" anywhere inside a table field must have
> "|" escaped using \vert entity. The markup can be escaped using
> zero-width space.

True.  I have not addressed these edge cases in my code, except a quick
hack to prepend a space whenever any line a src block / result block has
a leading star.  Tables on the other hand are extremely rare in Haskell
package documentation strings, and having a | in them is even rarer.

I do wonder what is the best way to handle cross-package links.
Currently I'm using CUSTOM_ID in the format of
package-name-x.y.z.Module.Name.IdentifierName, as well as
Module.Name.IdentifierName.  But this is a "one big org file" approach,
and can make navigation slow when say the org file reaches 15+MB in
size.

Ideally one should be able to navigate the a hackage worth of
documentations easily, with working cross-package links and low latency
org-goto to jump to any identifier or module, but I don't see how to
achieve that.  To be further investigated.

>
> Best,
> Ihor


Best,
Yuchen

-- 
PGP Key: 47F9 D050 1E11 8879 9040  4941 2126 7E93 EF86 DFD0
          <https://ypei.org/assets/ypei-pubkey.txt>

Re: An org backend to Haddock

[Ihor Radchenko]

Yuchen Pei <id@ypei.org> writes:

> I do wonder what is the best way to handle cross-package links.
> Currently I'm using CUSTOM_ID in the format of
> package-name-x.y.z.Module.Name.IdentifierName, as well as
> Module.Name.IdentifierName.  But this is a "one big org file" approach,
> and can make navigation slow when say the org file reaches 15+MB in
> size.

> Ideally one should be able to navigate the a hackage worth of
> documentations easily, with working cross-package links and low latency
> org-goto to jump to any identifier or module, but I don't see how to
> achieve that.  To be further investigated.

If you know the file path, you can always explicitly link to a headline
in that file:

(4.8 Search Options in File Links)

     [[file:~/code/main.c::255]]
     [[file:~/xx.org::My Target]]
     [[file:~/xx.org::*My Target]]
     [[file:~/xx.org::#my-custom-id]]
     [[file:~/xx.org::/regexp/]]
     [[attachment:main.c::255]]

Also, you can make use of org-id and set org-id-extra-files as a
directory-local/file-local variable, possibly providing an Elisp sexp to
perform auto-calculation.

Finally, 15+Mb files should no longer be a problem in future releases
of Org. I am using a 20Mb notes files with no major problems daily (on
latest development version of Org).

Best,
Ihor