Re: adding a new org-element?

[John Kitchin <jkitchin@andrew.cmu.edu>]

Eric S Fraga writes:

> Interesting points raised in your last email.  And also reminiscent of
> the citation discussion... for better or for worse ;-)

True, I recall saying or thinking something to the effect of how
"someone" might consider how to use that expanded syntax ;)

> org currently has effective support for literate programming with babel;
> however, it has only rudimentary support for data: tables and properties
> (and maybe tags).  More and more we are finding the desire to work with
> data more generally outwith the constraints imposed by the current
> support.

I would add blocks (code and special) to this.

>
> Links provide another interface to data but also rather rudimentary.

I think I agree with this. The cite links, for example in org-ref point
to "Data" in a bibtex file, and it does so by an id/bibtex-key. If you
build in parsing of the path, and potentially the description it is
possible to use existing capability to expand this, but they are always
"links" and to operate on a particular type you have to filter them out.

> Maybe it is time to generalise some of these concepts while keeping
> parsing straightforward.

This is the gist of what I am thinking about for sure. Parsing and
authoring should always be as straightforward as possible. If parsing is
hard, nobody will write the code, and if authoring is difficult nobody
will use it.


> I would be strongly in favour of some type of structure that supported
> the equivalent of JSON in terms of data representation but with programming
> functionality for export, interaction and display as provided by links
> to some degree.

My current thinking is the s-expression is sufficiently general to
represent a lot of data. Nothing would keep you from doing something
like: $(json "\"employees\":[
    {\"firstName\":\"John\", \"lastName\":\"Doe\"},
    {\"firstName\":\"Anna\", \"lastName\":\"Smith\"},
    {\"firstName\":\"Peter\",\"lastName\":\"Jones\"}
]")$

or in s-exp:

$(employees
  '((:firstname "John" :lastname "Doe")
    (:firstname "Anna" :lastname "Smith")
    (:firstname "Peter" :lastname "Jones")))$

These both can be plain text, or there could be a representation
function that displays them as a list of names in an overlay, for
example, or as "List of employees" as a clickable link with actions, and
a tooltip of some kind.

> However, the easiest solution may be to extend the link syntax and
> implementation, or maybe just the implementation, to address some of the
> current limitations, especially in terms of display but also in terms of
> linking to other objects in the org file (or even to other org files)?
>
> At present, links have follow and export functionality.  The follow
> functionality is a start towards actions on the data and is complete, in
> the Turing sense, given that the full power of elisp is there.  Likewise
> for export.  Two things are missing: linking and display.

I am mostly satisfied with the follow capability; the ability to call a
function with a menu of options makes it pretty much possible to do
anything from a link!

Display is less missing, in the sense that what isn't handled at export
can be handled by font-lock. This is already done to some extent when
org-mode makes the [[]] disappear in a link, and collapses the links
with descriptions. org-ref also does it a bit by changing the color of
links, and uncollapsing links with descriptions. It isn't that easy to
do this, at the moment, and it takes some font-lock skill that is hard
to come by ;) That is a feature of font-lock though.

It would be great if it were possible to call a display function during
font-lock that did stuff on an org element, e.g. set faces, properties,
etc...

What is tricky about that for links, is they are all one
element(object?), but there are different types of links, so the
"org-font-lock-link" function might need to call out to another
"org-font-lock-link-<type>" function, or refer to some kind of alist of
(type . font-lock-func) to easily modify individual link types.

> Linking (confusing terminology: maybe cross-referencing) between "link
> objects" could be imposed on the description which can then be processed
> by the follow parameter.  Nevertheless, it probably would be desirable
> to have a naming capability for individual link instances, one of the
> aspects discussed in the citation thread IIRC.  What is missing entirely
> in links is display functionality; this could be added as a third
> argument to the link definition.

I don't think we need a third argument for this. I feel like this is
like a css sheet in HTML, you can choose the display representation and
change it, somewhat in between what you see in the buffer, and what you
see in export (which has a filter system to change the representation).
The fallback is plain text, which might determine if you put the data in
the text, or if you put it elsewhere and link to it. You would just load
different el files to determine the representation. These might even be
specified in the file, like they are for xml.

Being able to link to the $(employees)$ link above and even use it for
something is probably useful. It might not always be what you want to
do, compared to say, just querying a database in a code block, or
reading the data in from a file. But, suppose you wanted to mine papers
that contained $(molecule inchi-key LFQSCWFLJHTTHZ-UHFFFAOYSA-N formula
C2H5OH)$ and $(float id melting-point inchi-key
LFQSCWFLJHTTHZ-UHFFFAOYSA-N value -173.2 units "degF")$, then you might
link to the melting-point value somehow in a new paper without having to
copy it.

Anyway, the thoughts are still a little loose in my head. I want to try
it, and write a paper with it, and see if it was useful enough to write
another paper that way ;)


--
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu