Re: Some projects

[Aaron Ecay <aaronecay@gmail.com>]

Hi Nicolas,

Thanks for writing this up.  It is important to think about, and
ultimately solve, all the issues you raise.

2015ko urriak 25an, Nicolas Goaziou-ek idatzi zuen:
> 
> Hello,
> 
> I'd like to see some features moving forward, and some important issues
> fixed, hopefully, in the next months. I'm sharing them here so that
> anyone interested can help.
> 
> 
> * Features
> 
> ** Citations
> 
> Development apparently stopped for some reason. We have a citation
> syntax for Org in wip-cite and some work done in wip-cite-awe and
> probably elsewhere.
> 
> I think we could at least provide features defined in Org Ref using the
> new syntax (minus hydra/helm related functions).
> 
> We don't need a silver bullet. Just something with a non-empty user
> base, and extensible. In any case, the work done so far shouldn't be
> wasted.

I was working on this rather intensively at one time, but I had to stop
because other aspects of life intruded.  I have just been coming back
towards a situation where I can imagine myself having some (still small,
but non-zero) chunks of time to devote to working on org.  So I hope I
will be able to pick this back up, but (regrettably) I’m not able to
make any promises.

Based on my recollection, here’s what the problems were when I stopped:

- The only “off the shelf”-capable citation processing library that we
  found last time is in Haskell, which introduced some difficulties for
  distributing the resulting tool.  I know some projects
  (e.g. git-annex) are written in Haskell and distributed as static
  binaries for windows/mac/linux/etc.  We’d need to figure out how to do
  this, or find another citation processing library in an
  easier-to-distribute language.  (I should say, all the work on the
  external tool was done by Richard Lawrence; I worked on the exporter
  for the citation syntax including the interface with an external
  tool.)

- There is a difference between citations as done by latex/bibtex/etc.,
  and those done in every other format (handled through CSL).  Assuming
  latex users want to keep their native processing rather than
  delegating to CSL, we need to solve the myriad small inconsistencies
  between these two tools.  I think this is an area where it’s important
  to get things right: users of citations generally have exacting
  requirements.  “Approximately Chicago-style” or “almost MLA” aren’t
  worth anything.

When I start working again I will take your words about silver bullets
to heart, and come up with something that covers at least basic use
cases for users who can compile a haskell program themselves.  We can
expand from there.

(I should also say, if someone else is interested in working on this
please don’t hesitate to jump right in.  I will help you however I can!)

> 
> ** Annotations
> 
> It would be nice to allow annotating text in Org. I already made
> a proposal for that feature a few months ago, without much success.
> Anyway, I'd like to implement it, in a simplified form. For the record,
> the idea is to use some markup for beginning and end of an annotation,
> e.g.,
> 
>   [@:id].....[@]
> 
> and store text within a dedicated section,
> 
>   * Annotations
> 
>   [@:id] ...
> 
> Like footnotes, you could easily browse remotely annotations at point.
> However, they would be, at least at the beginning, completely ignored
> during export.
> 
> ** Backslash escaping
> 
> Allowing to escape some symbols in plain text (e.g., emphasis markers,
> square brackets...) would remove a limitation in verbatim/code objects.
> As a small benefit, it would also permit to implement mid-word markup:
> b*o*ld.
> 
> There are some gotchas, however.
> 
> * Important fixes
> 
> ** Cache
> 
> The cache needs to be fixed. Its underlying implementation probably
> needs to be changed, too. At the moment, it uses an AVL tree, which
> isn't much tolerant and results in a freeze whenever the cache is
> corrupted. Shifting to text properties could improve the situation.
> 
> Also, it needs to do better analysis in order to prevent those
> corruptions. This is obviously the hardest part.
> 
> Again, an efficient cache can give us a better fontification mechanism,
> since Org syntax is not regular.
> 
> ** Link hexification
> 
> There are currently some subtle inconsistencies in link handling. See
> for example `org-make-link-string'.

At the risk of being a bit of an anarchist, I think this is somewhere we
could benefit from throwing backward compatibility out the window, at
least for brainstorming purposes.  The link-hexification bug threads
I’ve read give the impression of watching someone try to add one more
level onto a teetering house of cards.  It would be great if someone
could take inventory of the uses of links in org, and then try to design
a solid syntax that works for them from scratch.  We’d wait to worry
about implementation until having a well-engineered design.

(To give an example of what I have in mind, it may turn out that
hexification, borrowed from HTML/HTTP, is not even the best technical
solution to our actual problem, whatever it is.)

> 
> ** Use lexical binding everywhere
> 
> This is the easiest part: add appropriate cookie at the beginning of the
> file and fix resulting compiler warnings.

There are also a lot of places that org uses dynamic variables to
pass information around.  It would be good to eliminate these as much
as possible, since lexical binding allows generating more efficient
bytecode for lexically bound variable access (including function
arguments, AFAIK).  This is a further project to what you describe,
of course.

> 
> ** Find a solution for orgstruct minor mode
> 
> Org struct is really different from Org. It prevents Org from using
> better algorithms for outline navigation (see discussion about
> `org-show-children' on this list).
> 
> I think Org struct should be removed from "org.el" (and org-footnote
> ...) and added in its own library. It should also be built on top of
> Outline mode instead of Org mode. Org, OTOH, should remove dependencies
> on Outline mode and implement better navigation functions.

FWIW, this gets a +1 from me.  I’ve lost count of the number of times I’ve
started to make some “obvious” minor improvement to some org function,
only to have to discard the effort after 15 minutes when realizing that
supporting orgstruct mode makes it impossible to actually carry through
the change.

Thanks again,

-- 
Aaron Ecay