Some projects

Some projects

[Nicolas Goaziou]

Hello,

I'd like to see some features moving forward, and some important issues
fixed, hopefully, in the next months. I'm sharing them here so that
anyone interested can help.


* Features

** Citations

Development apparently stopped for some reason. We have a citation
syntax for Org in wip-cite and some work done in wip-cite-awe and
probably elsewhere.

I think we could at least provide features defined in Org Ref using the
new syntax (minus hydra/helm related functions).

We don't need a silver bullet. Just something with a non-empty user
base, and extensible. In any case, the work done so far shouldn't be
wasted.

** Annotations

It would be nice to allow annotating text in Org. I already made
a proposal for that feature a few months ago, without much success.
Anyway, I'd like to implement it, in a simplified form. For the record,
the idea is to use some markup for beginning and end of an annotation,
e.g.,

  [@:id].....[@]

and store text within a dedicated section,

  * Annotations

  [@:id] ...

Like footnotes, you could easily browse remotely annotations at point.
However, they would be, at least at the beginning, completely ignored
during export.

** Backslash escaping

Allowing to escape some symbols in plain text (e.g., emphasis markers,
square brackets...) would remove a limitation in verbatim/code objects.
As a small benefit, it would also permit to implement mid-word markup:
b*o*ld.

There are some gotchas, however.

* Important fixes

** Cache

The cache needs to be fixed. Its underlying implementation probably
needs to be changed, too. At the moment, it uses an AVL tree, which
isn't much tolerant and results in a freeze whenever the cache is
corrupted. Shifting to text properties could improve the situation.

Also, it needs to do better analysis in order to prevent those
corruptions. This is obviously the hardest part.

Again, an efficient cache can give us a better fontification mechanism,
since Org syntax is not regular.

** Link hexification

There are currently some subtle inconsistencies in link handling. See
for example `org-make-link-string'.

** Use lexical binding everywhere

This is the easiest part: add appropriate cookie at the beginning of the
file and fix resulting compiler warnings.

** Find a solution for orgstruct minor mode

Org struct is really different from Org. It prevents Org from using
better algorithms for outline navigation (see discussion about
`org-show-children' on this list).

I think Org struct should be removed from "org.el" (and org-footnote
...) and added in its own library. It should also be built on top of
Outline mode instead of Org mode. Org, OTOH, should remove dependencies
on Outline mode and implement better navigation functions.

I have the feeling that Org struct will need to be rewritten almost from
scratch in the process.


Here we are. I hope finding enough spare time (it also depends on the
quantity of bugs to fix...) to investigate most, if not all of these
issues. Of course, feedback is welcome.

Regards,

-- 
Nicolas Goaziou                                                0x80A93738

Re: Some projects

[Marcin Borkowski]

On 2015-10-25, at 14:08, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:

> Hello,
>
> I'd like to see some features moving forward, and some important issues
> fixed, hopefully, in the next months. I'm sharing them here so that
> anyone interested can help.
>
>
> * Features
>
> ** Citations
>
> Development apparently stopped for some reason. We have a citation
> syntax for Org in wip-cite and some work done in wip-cite-awe and
> probably elsewhere.
>
> I think we could at least provide features defined in Org Ref using the
> new syntax (minus hydra/helm related functions).
>
> We don't need a silver bullet. Just something with a non-empty user
> base, and extensible. In any case, the work done so far shouldn't be
> wasted.

Interesting.  I don't use Org for writing scientific articles, but many
people do, and this might be really helpful.

> ** Annotations
>
> It would be nice to allow annotating text in Org. I already made
> a proposal for that feature a few months ago, without much success.
> Anyway, I'd like to implement it, in a simplified form. For the record,
> the idea is to use some markup for beginning and end of an annotation,
> e.g.,
>
>   [@:id].....[@]
>
> and store text within a dedicated section,
>
>   * Annotations
>
>   [@:id] ...
>
> Like footnotes, you could easily browse remotely annotations at point.
> However, they would be, at least at the beginning, completely ignored
> during export.

That sounds interesting, though I don't really see much difference
between this and plain simple comments.

> ** Backslash escaping
>
> Allowing to escape some symbols in plain text (e.g., emphasis markers,
> square brackets...) would remove a limitation in verbatim/code objects.
> As a small benefit, it would also permit to implement mid-word markup:
> b*o*ld.
>
> There are some gotchas, however.

And this one is probably the most interesting to me.  If I can help
(testing, suggestions, maybe coding - I'm in the process of transferring
copyright for my Emacs/Org-mode/AUCTeX contributions to the FSF), please
let me know.

> Here we are. I hope finding enough spare time (it also depends on the
> quantity of bugs to fix...) to investigate most, if not all of these
> issues. Of course, feedback is welcome.
>
> Regards,

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: Some projects

[Nicolas Goaziou]

Marcin Borkowski <mbork@mbork.pl> writes:

> That sounds interesting, though I don't really see much difference
> between this and plain simple comments.

  1. you cannot simply inline comments (you need to create a new link
     type or some such);
  2. you cannot mark exactly where the comment applies.

>> ** Backslash escaping
>>
>> Allowing to escape some symbols in plain text (e.g., emphasis markers,
>> square brackets...) would remove a limitation in verbatim/code objects.
>> As a small benefit, it would also permit to implement mid-word markup:
>> b*o*ld.
>>
>> There are some gotchas, however.
>
> And this one is probably the most interesting to me.  If I can help
> (testing, suggestions, maybe coding - I'm in the process of transferring
> copyright for my Emacs/Org-mode/AUCTeX contributions to the FSF), please
> let me know.

Here, I don't even have clear specifications. So the first step would be
to define them.

1. It should be unintrusive, i.e., you only need to escape ambiguous
   cases. E.g., \[1] makes sure that [1] will not be treated as
   a footnote reference, but \[*] is equivalent to [*].

2. It should allow to insert "=" within verbatim and "~" within code
   markup.

3. It should only be used in paragraphs, verse blocks or table cells.
   E.g., there is no escaping in #+NAME: ... or in node properties.

Assuming you can escape #, :, |, [, {, <, *, +, _, ^, /, ~, =, you only
need to escape \ if it is followed by any of the previous characters.

So, \\= means \= but \\! means \\!.

Unfortunately, there is a special entity, "\_ " which is incompatible
with the previous definition.

Implementation-wise, I think it is enough to resolve backslash escaping
when parsing a paragraph (or equivalent).


Regards,

Re: Some projects

[Eric S Fraga]

On Sunday, 25 Oct 2015 at 14:08, Nicolas Goaziou wrote:
> Hello,
>
> I'd like to see some features moving forward, and some important issues
> fixed, hopefully, in the next months. I'm sharing them here so that
> anyone interested can help.

Nicolas,

I look forward to all the advances you have proposed... except for this one:

> ** Backslash escaping
>
> Allowing to escape some symbols in plain text (e.g., emphasis markers,
> square brackets...) would remove a limitation in verbatim/code objects.
> As a small benefit, it would also permit to implement mid-word markup:
> b*o*ld.

I'm concerned that it will make typing normal text more onerous.  Right
now, org is quite non-intrusive in most of my writing yet is quite rich
in what it can encode.  Having to backslash symbols that I use in text
quite often (especially square brackets) would be annoying.  Maybe I
misunderstood; if so, I apologise for the noise!

Regarding citations, I have not played with the -wip branch at
all.  Maybe I should.  It just wasn't clear where that branch had got to
and whether it was at a point worth trying.  Can you (or somebody else)
update us with a summary of what is possible?  I'm always in the process
of writing papers so I may be happy to be a guinea pig.

Thanks,
eric

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.2, Org release_8.3.2-209-gba4d33

Re: Some projects

[Nicolas Goaziou]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> I'm concerned that it will make typing normal text more onerous.  Right
> now, org is quite non-intrusive in most of my writing yet is quite rich
> in what it can encode.  Having to backslash symbols that I use in text
> quite often (especially square brackets) would be annoying.  Maybe I
> misunderstood; if so, I apologise for the noise!

This is also my concern. I don't want to add complexity to typing in
Org, at all. Yet, if this feature is correctly defined, this should
disambiguate some annoying situations (e.g., [1] when you do not want
a footnote reference).

> Regarding citations, I have not played with the -wip branch at
> all.  Maybe I should.  It just wasn't clear where that branch had got to
> and whether it was at a point worth trying.  Can you (or somebody else)
> update us with a summary of what is possible?

I cannot. I just implemented the syntax. Others implemented tools on top
of it. I don't know the current state of this.

Regards,

Re: Some projects

[Marcin Borkowski]

On 2015-10-25, at 16:45, Eric S Fraga <e.fraga@ucl.ac.uk> wrote:

> On Sunday, 25 Oct 2015 at 14:08, Nicolas Goaziou wrote:
>> Hello,
>>
>> I'd like to see some features moving forward, and some important issues
>> fixed, hopefully, in the next months. I'm sharing them here so that
>> anyone interested can help.
>
> Nicolas,
>
> I look forward to all the advances you have proposed... except for this one:
>
>> ** Backslash escaping
>>
>> Allowing to escape some symbols in plain text (e.g., emphasis markers,
>> square brackets...) would remove a limitation in verbatim/code objects.
>> As a small benefit, it would also permit to implement mid-word markup:
>> b*o*ld.
>
> I'm concerned that it will make typing normal text more onerous.  Right
> now, org is quite non-intrusive in most of my writing yet is quite rich
> in what it can encode.  Having to backslash symbols that I use in text
> quite often (especially square brackets) would be annoying.  Maybe I
> misunderstood; if so, I apologise for the noise!

Eric, it's good that you write (IMO at least) - this is how the ideas
can get refined.

Since your point is quite valid - and OTOH, I would like to put anything
(or almost anything) in =code= markup, for instance (my use case: Emacs
keybindings, try =C-x ,= - Org won't recognize it as code!).  I could
mess up with org-emphasis-regexp-components in e.g. file local
variables, but this is far from clean.

Maybe a good solution would be to allow two syntaxes for markup:
"short", like *bold* or =code=, and "long", like \textbf{bold} and
\verb|code|.  If it is decided that such LaTeX-like syntax is fine, we
could only introduce escaping of backslash and curly braces, which seems
a decent compromise.

We could even use LaTeX-like constructs like \textbf, but with
\verb-like delimitation by two identical characters not present in the
text itself.  We might also use a convention that if the character right
after \textbf is an opening paren of some kind - the syntax table might
be reused for that - then we expect a closing one at the end, and if
not, we expect an identical one at the end.  This way, we could say
\textbf{Boldface} and \verb{verbatim}
but also
\textbf|{boldface in curly braces}| and \verb|C-x {|
etc.  This way only the backslash needs escaping.

There are a lot of possibilities, and plenty of time to settle on the
best one.

> Thanks,
> eric

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: Some projects

[Nicolas Goaziou]

Marcin Borkowski <mbork@mbork.pl> writes:

> Since your point is quite valid - and OTOH, I would like to put anything
> (or almost anything) in =code= markup, for instance (my use case: Emacs
> keybindings, try =C-x ,= - Org won't recognize it as code!).  I could
> mess up with org-emphasis-regexp-components in e.g. file local
> variables, but this is far from clean.

I cannot think of any bad consequence if we tailor "border" in
`org-emphasis-regexp-components' to allow everything but white spaces.

> Maybe a good solution would be to allow two syntaxes for markup:
> "short", like *bold* or =code=, and "long", like \textbf{bold} and
> \verb|code|.  If it is decided that such LaTeX-like syntax is fine, we
> could only introduce escaping of backslash and curly braces, which seems
> a decent compromise.

I don't think LaTeX-like syntax is good. It doesn't belong to
lightweight markup. Besides, Org already supports LaTeX macros so it
would probably be ambiguous.

I suggest to keep as close as possible to the existing markup.

Regards,

Re: Some projects

[Marcin Borkowski]

On 2015-10-25, at 17:37, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:

> Marcin Borkowski <mbork@mbork.pl> writes:
>
>> Since your point is quite valid - and OTOH, I would like to put anything
>> (or almost anything) in =code= markup, for instance (my use case: Emacs
>> keybindings, try =C-x ,= - Org won't recognize it as code!).  I could
>> mess up with org-emphasis-regexp-components in e.g. file local
>> variables, but this is far from clean.
>
> I cannot think of any bad consequence if we tailor "border" in
> `org-emphasis-regexp-components' to allow everything but white spaces.

I don't know, I guess I would welcome such change!

>> Maybe a good solution would be to allow two syntaxes for markup:
>> "short", like *bold* or =code=, and "long", like \textbf{bold} and
>> \verb|code|.  If it is decided that such LaTeX-like syntax is fine, we
>> could only introduce escaping of backslash and curly braces, which seems
>> a decent compromise.
>
> I don't think LaTeX-like syntax is good. It doesn't belong to
> lightweight markup. Besides, Org already supports LaTeX macros so it
> would probably be ambiguous.

I see.  It was only a proposal.  Another possibility could be XML's
<code> ... </code>.  Or maybe the whole idea of short and long syntax is
bad.  But I'm afraid that if we want "lightweight markup", we might get
"impossible to mark up certain strings" (like Emacs keystrokes, which
are sometimes rather weird, like =C-x ,= - which is unbound by default,
but possible to use).

> I suggest to keep as close as possible to the existing markup.
>
> Regards,

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: Some projects

[Nicolas Goaziou]

Marcin Borkowski <mbork@mbork.pl> writes:

> On 2015-10-25, at 17:37, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:
>
>> I cannot think of any bad consequence if we tailor "border" in
>> `org-emphasis-regexp-components' to allow everything but white spaces.
>
> I don't know, I guess I would welcome such change!

Done in b4af3f0. We'll see what horrible things happen.

> I see. It was only a proposal. Another possibility could be XML's
> <code> ... </code>. Or maybe the whole idea of short and long syntax
> is bad.

IMO, the latter.

> But I'm afraid that if we want "lightweight markup", we might get
> "impossible to mark up certain strings" (like Emacs keystrokes, which
> are sometimes rather weird, like =C-x ,= - which is unbound by
> default, but possible to use).

The point of escaping mechanism is to extend the set of possible
strings, at the price of a lesser readability. We could see what such
a mechanism could give us.

Regards,

Re: Some projects

[Thomas S. Dye]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Hello,
>
> I'd like to see some features moving forward, and some important issues
> fixed, hopefully, in the next months. I'm sharing them here so that
> anyone interested can help.
>
> ** Annotations
>
> It would be nice to allow annotating text in Org. I already made
> a proposal for that feature a few months ago, without much success.
> Anyway, I'd like to implement it, in a simplified form. For the record,
> the idea is to use some markup for beginning and end of an annotation,
> e.g.,
>
>   [@:id].....[@]
>
> and store text within a dedicated section,
>
>   * Annotations
>
>   [@:id] ...
>
> Like footnotes, you could easily browse remotely annotations at point.
> However, they would be, at least at the beginning, completely ignored
> during export.

+1

As an editor, I would use annotations a lot.  Sometimes an annotation
would be a comment on the text, meant to remind the author of some fact
or to suggest some other point of view.  Other times the annotation
would suggest an alternate wording.  In this case, it would be super
helpful to have a function that replaced annotated text with the
annotation, so the author could easily accept the editor's suggestion.

Would annotations also apply to source code blocks?

All the best,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Some projects

[Fabrice Popineau]

[-- Attachment #1: Type: text/plain, Size: 366 bytes --]

2015-10-25 18:57 GMT+01:00 Thomas S. Dye <tsd@tsdye.com>:


> would suggest an alternate wording.  In this case, it would be super
> helpful to have a function that replaced annotated text with the
> annotation, so the author could easily accept the editor's suggestion.
>
>
In this case, it would be nice if annotations could have an author too!

Regards,

Fabrice

[-- Attachment #2: Type: text/html, Size: 772 bytes --]

Re: Some projects

[Nicolas Goaziou]

Fabrice Popineau <fabrice.popineau@supelec.fr> writes:

> 2015-10-25 18:57 GMT+01:00 Thomas S. Dye <tsd@tsdye.com>:
>
>
>> would suggest an alternate wording.  In this case, it would be super
>> helpful to have a function that replaced annotated text with the
>> annotation, so the author could easily accept the editor's suggestion.
>>
>>
> In this case, it would be nice if annotations could have an author
> too!

The full proposal was at
<http://permalink.gmane.org/gmane.emacs.orgmode/97468> but it didn't get
much positive feedback. Therefore I suggested a simplified version, with
a single author.

Also, there is

  https://github.com/girzel/org-annotate

which is even simpler.

Regards,

Re: Some projects

[Eric S Fraga]

On Sunday, 25 Oct 2015 at 19:12, Nicolas Goaziou wrote:
> The full proposal was at
> <http://permalink.gmane.org/gmane.emacs.orgmode/97468> but it didn't get
> much positive feedback. Therefore I suggested a simplified version, with
> a single author.

It's a start and that's infinitely better than no provision for
annotations at all.  I know I would use this feature often.  I use
inlinetasks for annotations currently and this would be cleaner.

Multiple authors can be done manually for now, e.g. [@:1]blah[@] with
[1] later "ESF: I don't agree ;-)".

By the way, I personally would like inline annotations, along the lines
of inline footnotes.  Just saying...

thanks,
eric

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 25.0.50.2, Org release_8.3.2-209-gba4d33

Re: Some projects

[Marcin Borkowski]

On 2015-10-25, at 19:22, Eric S Fraga <e.fraga@ucl.ac.uk> wrote:

> By the way, I personally would like inline annotations, along the lines
> of inline footnotes.  Just saying...

+1

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: Some projects

[Marcin Borkowski]

On 2015-10-25, at 19:12, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:

> Fabrice Popineau <fabrice.popineau@supelec.fr> writes:
>
>> 2015-10-25 18:57 GMT+01:00 Thomas S. Dye <tsd@tsdye.com>:
>>
>>
>>> would suggest an alternate wording.  In this case, it would be super
>>> helpful to have a function that replaced annotated text with the
>>> annotation, so the author could easily accept the editor's suggestion.
>>>
>>>
>> In this case, it would be nice if annotations could have an author
>> too!
>
> The full proposal was at
> <http://permalink.gmane.org/gmane.emacs.orgmode/97468> but it didn't get
> much positive feedback. Therefore I suggested a simplified version, with
> a single author.

I like it!  I'm not sure I would use it a lot (and definitely not with
other people - there are not many people using Org-mode in my
institution, in fact I guess I'm the only one;-)), but it does seem
useful.

One thing: there /must/ be a way to export annotations.  For LaTeX, the
todonotes package (https://www.ctan.org/pkg/todonotes) seems a natural
choice.

> Also, there is
>
>   https://github.com/girzel/org-annotate
>
> which is even simpler.

I'll look into it, too, thanks!

> Regards,

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: Some projects

[Anders Johansson]

Marcin Borkowski <mbork <at> mbork.pl> writes:

> 
> 
> On 2015-10-25, at 19:12, Nicolas Goaziou <mail <at> nicolasgoaziou.fr> wrote:
> 
> > Fabrice Popineau <fabrice.popineau <at> supelec.fr> writes:
> >
> >> 2015-10-25 18:57 GMT+01:00 Thomas S. Dye <tsd <at> tsdye.com>:
> >>
> >>
> >>> would suggest an alternate wording.  In this case, it would be super
> >>> helpful to have a function that replaced annotated text with the
> >>> annotation, so the author could easily accept the editor's suggestion.
> >>>
> >>>
> >> In this case, it would be nice if annotations could have an author
> >> too!
> >
> > The full proposal was at
> > <http://permalink.gmane.org/gmane.emacs.orgmode/97468> but it didn't get
> > much positive feedback. Therefore I suggested a simplified version, with
> > a single author.
> 
> I like it!  I'm not sure I would use it a lot (and definitely not with
> other people - there are not many people using Org-mode in my
> institution, in fact I guess I'm the only one), but it does seem
> useful.
> 
> One thing: there /must/ be a way to export annotations.  For LaTeX, the
> todonotes package (https://www.ctan.org/pkg/todonotes) seems a natural
> choice.
> 

I hacked together something like this a while ago when I needed to add
inline-comments that would later be exported as odt-comments (but I also
made it work for latex).

I chose to implement it via a unicode-bracket ❰❙❱, like this:

❰Simple comment❱
❰Text that is commented on❙Comment❱
❰[Author] Simple comment❱
❰Commented text❙[Author] Comment❱

These are inserted with a custom command to make it easy (including an
author-history-list selectable with helm).

I've put it up in a gist if anyone finds it useful:
https://gist.github.com/andersjohansson/6baa1e42ad4d7353e125

Cheers,
Anders Johansson

Re: Some projects

[Anders Johansson]

Anders Johansson <mejlaandersj <at> gmail.com> writes:

> I hacked together something like this a while ago when I needed to add
> inline-comments that would later be exported as odt-comments (but I also
> made it work for latex).
> 
> I chose to implement it via a unicode-bracket ❰❙❱, like this:
> 
> ❰Simple comment❱
> ❰Text that is commented on❙Comment❱
> ❰[Author] Simple comment❱
> ❰Commented text❙[Author] Comment❱
> 
> These are inserted with a custom command to make it easy (including an
> author-history-list selectable with helm).
> 
> I've put it up in a gist if anyone finds it useful:
> https://gist.github.com/andersjohansson/6baa1e42ad4d7353e125
> 
> Cheers,
> Anders Johansson
> 

But taking a look at org-annotate (https://github.com/girzel/org-annotate) i
see that it is a much more complete first attempt at a solution. Stupid that
I didn't find that when I needed it this summer.

/Anders Johansson

Re: Some projects

[Thomas S. Dye]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> The full proposal was at
> <http://permalink.gmane.org/gmane.emacs.orgmode/97468> but it didn't get
> much positive feedback. Therefore I suggested a simplified version, with
> a single author.

In my case, the lack of positive feedback was "too much org, too little
time".

The original version, which includes the author Fabrice mentioned, seems
superior to me.  It potentially eases collaboration and also configuring
alternate versions of the same document.

I'm not clear about affiliated keyword for source code blocks.  Would
tangling honor  #+OPTIONS:  <at> :student1?

All the best,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Some projects

[Nicolas Goaziou]

Thomas S. Dye <tsd@tsdye.com> writes:

> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>
>> The full proposal was at
>> <http://permalink.gmane.org/gmane.emacs.orgmode/97468> but it didn't get
>> much positive feedback. Therefore I suggested a simplified version, with
>> a single author.
>
> In my case, the lack of positive feedback was "too much org, too little
> time".
>
> The original version, which includes the author Fabrice mentioned, seems
> superior to me.  It potentially eases collaboration and also configuring
> alternate versions of the same document.
>
> I'm not clear about affiliated keyword for source code blocks.  Would
> tangling honor  #+OPTIONS:  <at> :student1?

The limitation is that you can only replace the code block as a whole,
so with

  #+annotate: bugfix
  #+name: bikeshed
  #+begin_src emacs-lisp
  (paint 'bikeshed blue)
  #+end_src

  * Annotations

  [@:bugfig:me]
  # You're clearly wrong
  #+begin_src emacs-lisp
  (pain 'bikeshed green)
  #+end_src

when using

  #+OPTIONS: @:me

body becomes

  #+name: bikeshed
  #+begin_src emacs-lisp
  (paint 'bikeshed green)
  #+end_src


Regards,

Re: Some projects

[Thomas S. Dye]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Thomas S. Dye <tsd@tsdye.com> writes:
>
>> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>>
>>> The full proposal was at
>>> <http://permalink.gmane.org/gmane.emacs.orgmode/97468> but it didn't get
>>> much positive feedback. Therefore I suggested a simplified version, with
>>> a single author.
>>
>> In my case, the lack of positive feedback was "too much org, too little
>> time".
>>
>> The original version, which includes the author Fabrice mentioned, seems
>> superior to me.  It potentially eases collaboration and also configuring
>> alternate versions of the same document.
>>
>> I'm not clear about affiliated keyword for source code blocks.  Would
>> tangling honor  #+OPTIONS:  <at> :student1?
>
> The limitation is that you can only replace the code block as a whole,
> so with
>
>   #+annotate: bugfix
>   #+name: bikeshed
>   #+begin_src emacs-lisp
>   (paint 'bikeshed blue)
>   #+end_src
>
>   * Annotations
>
>   [@:bugfig:me]
>   # You're clearly wrong
>   #+begin_src emacs-lisp
>   (pain 'bikeshed green)
>   #+end_src
>
> when using
>
>   #+OPTIONS: @:me
>
> body becomes
>
>   #+name: bikeshed
>   #+begin_src emacs-lisp
>   (paint 'bikeshed green)
>   #+end_src

And also,

#+name: baz
#+begin_src emacs-lisp :noweb yes
(some code)
<<bikeshed>>
(other code)
#+end_src

so the various :noweb options, including evaluation of baz in the Org
mode buffer, honor #+OPTIONS: @:me?

All the best,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Some projects

[Nicolas Goaziou]

Thomas S. Dye <tsd@tsdye.com> writes:

> Would tangling honor #+OPTIONS: <at> :student1?

  #+OPTIONS: @:...

is for export, so I guess not. Though, it can be implemented later.

Regards,

Re: Some projects

[Rasmus]

Hi,

First, thanks for writing this up, Nicolas.  Org has been a bit slow
lately, at least for my part.

Perhaps, we should keep a file in org-mode.git which links back to
discussions (TODO.org).  There’s a couple of other things that we
discussed recently.

> ** Citations
>
> Development apparently stopped for some reason. We have a citation
> syntax for Org in wip-cite and some work done in wip-cite-awe and
> probably elsewhere.
>
> I think we could at least provide features defined in Org Ref using the
> new syntax (minus hydra/helm related functions).
>
> We don't need a silver bullet. Just something with a non-empty user
> base, and extensible. In any case, the work done so far shouldn't be
> wasted.

This is something I care deeply about, and I would like to work on it.
I’m a bit short on time these days, but still it’s the most important
missing feature IMO.

We sort of got stuck on syntax discussions the last time (besides
[cite]/[(cite)]), as well as tool choices (citeproc-java vs. some
org-specific Haskell implementation).

I would suggest we start with LaTeX, although it contains some danger of
making choices that are hard to make work with citeproc.  I’m not sure how
far Aaron got on this work.


> ** Annotations

I still don’t care for this, but no point in reiteration.

> ** Backslash escaping

Would be nice at times.

I strongly disagree with introducing XML or LaTeX like syntax as is
suggested a bit further down in the thread.

> ** Cache

Probably to complicated for me...

> ** Link hexification
>
> There are currently some subtle inconsistencies in link handling. See
> for example `org-make-link-string'.

Indeed, this is annoying.  Thanks for bringing it up again.

> ** Use lexical binding everywhere

Sounds good.  I am under the impression (from git log) that you have
already fixed a number of files in this regard, right?

> ** Find a solution for orgstruct minor mode

I would also try to dedicate time towards this.

> Org struct is really different from Org. It prevents Org from using
> better algorithms for outline navigation (see discussion about
> `org-show-children' on this list).
>
> I think Org struct should be removed from "org.el" (and org-footnote
> ...) and added in its own library.

I agree.  Orgstruct is powered by black voodoo.

> It should also be built on top of Outline mode instead of Org mode.

I don’t have an opinion on this.  Would you not think that it would be
better if orgstruct is a collections of functions that wrap around Org?  I
fear that if change X is made to Org, we would have to also implement the
change in orgstruct.


> Org, OTOH, should remove dependencies on Outline mode and implement
> better navigation functions.

Perhaps.  I don’t understand where the benefit would be.

> I have the feeling that Org struct will need to be rewritten almost from
> scratch in the process.

... Which is not all that bad.

—Rasmus

-- 
Evidence suggests Snowden used a powerful tool called monospaced fonts

Re: Some projects

[Marcin Borkowski]

On 2015-10-25, at 20:02, Rasmus <rasmus@gmx.us> wrote:

>> ** Backslash escaping
>
> Would be nice at times.
>
> I strongly disagree with introducing XML or LaTeX like syntax as is
> suggested a bit further down in the thread.

Since it seems I'm the only one that proposed (and liked) that idea, let
me reiterate that (1) I meant it only as a variant, that (2)
I personally would prefer LaTeX over XML, and that (3) if other ways of
making it possible to put /anything/ in =code= appear, that's completely
fine with me.

(Note also that I'm heavily biased, having been a (La)TeX user for about
two decades now.)

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: Some projects

[Matt Price]

[-- Attachment #1: Type: text/plain, Size: 1882 bytes --]

On Sun, Oct 25, 2015 at 3:02 PM, Rasmus <rasmus@gmx.us> wrote:

> Hi,
>
> First, thanks for writing this up, Nicolas.  Org has been a bit slow
> lately, at least for my part.
>

ditto.  and thanks to eveyrone in the thread for their input.

>
>
>
> > ** Citations
> >
> > Development apparently stopped for some reason. We have a citation
> > syntax for Org in wip-cite and some work done in wip-cite-awe and
> > probably elsewhere.
> >
> > I think we could at least provide features defined in Org Ref using the
> > new syntax (minus hydra/helm related functions).
> >
> > We don't need a silver bullet. Just something with a non-empty user
> > base, and extensible. In any case, the work done so far shouldn't be
> > wasted.
>
> This is something I care deeply about, and I would like to work on it.
> I’m a bit short on time these days, but still it’s the most important
> missing feature IMO.
>
> We sort of got stuck on syntax discussions the last time (besides
> [cite]/[(cite)]), as well as tool choices (citeproc-java vs. some
> org-specific Haskell implementation).
>
> I would suggest we start with LaTeX, although it contains some danger of
> making choices that are hard to make work with citeproc.  I’m not sure how
> far Aaron got on this work.
>
>
> I also regard this as the most important missing feature from Org. From my
perspective, latex-centred approaches are I guess fine but don't
immediately solve any issues for me. My use cases are (a) publication to
the web, either of papers or, especially,  of teaching resources; and (b)
circulation of scholarly work in  .docx or (in rare best-case scenarios)
markdown.  I would really be grateful if work on citations left the door
open for these formats, which are going to be with us for a long time.

Again, thanks to all you guys for your work on this.

m

[-- Attachment #2: Type: text/html, Size: 2481 bytes --]

Re: Some projects

[Samuel Wales]

i like some of these ideas, particularly lexical.

org already has a lot of syntax.  i am leery of introducing yet more
heterogenous syntax to org.  key word heterogenous.  i don't mind more
features if it is always using the same syntax framework and thus can
take advantage of everything else that uses it.  key word framework.

thus, i'd propose a single syntax framework that takes care of future
syntax.  a syntax framework like $[annotation ...] where ... is
specifiable as a lisp lambda list or similar could also be used for
other features, including long-form emphasis.

i don't think long-form emphasis is a bad idea at all.  it allows
export back end independence.  i only think it is bad if it means
introducing heterogenous, non-framework syntax.

a single framework takes care of future features too.  and as a bonus
it allows future subfeatures.  for example, there is no need to
implement authorship in annotations until we decide we want them
later.  when we do, just add a keyword option: $[annotation :author
"joe"].

and yet another bonus is that it could be used for user-defined
features.  all without adding heterogenous non-framework syntax.

===

i would need fontification to be able to fontify inline footnotes that
have more than one paragraph [i.e. have blank lines in them, which is
currently not allowed in 8.x org export.  fontification currently and
always has fontified them correctly from my perspective i.e. by
allowing multiple paragraphs].

Re: Some projects

[Samuel Wales]

naturally, long-form emphasis can look like $[emphasis ...].

Re: Some projects

[Aaron Ecay]

Hi Nicolas,

Thanks for writing this up.  It is important to think about, and
ultimately solve, all the issues you raise.

2015ko urriak 25an, Nicolas Goaziou-ek idatzi zuen:
> 
> Hello,
> 
> I'd like to see some features moving forward, and some important issues
> fixed, hopefully, in the next months. I'm sharing them here so that
> anyone interested can help.
> 
> 
> * Features
> 
> ** Citations
> 
> Development apparently stopped for some reason. We have a citation
> syntax for Org in wip-cite and some work done in wip-cite-awe and
> probably elsewhere.
> 
> I think we could at least provide features defined in Org Ref using the
> new syntax (minus hydra/helm related functions).
> 
> We don't need a silver bullet. Just something with a non-empty user
> base, and extensible. In any case, the work done so far shouldn't be
> wasted.

I was working on this rather intensively at one time, but I had to stop
because other aspects of life intruded.  I have just been coming back
towards a situation where I can imagine myself having some (still small,
but non-zero) chunks of time to devote to working on org.  So I hope I
will be able to pick this back up, but (regrettably) I’m not able to
make any promises.

Based on my recollection, here’s what the problems were when I stopped:

- The only “off the shelf”-capable citation processing library that we
  found last time is in Haskell, which introduced some difficulties for
  distributing the resulting tool.  I know some projects
  (e.g. git-annex) are written in Haskell and distributed as static
  binaries for windows/mac/linux/etc.  We’d need to figure out how to do
  this, or find another citation processing library in an
  easier-to-distribute language.  (I should say, all the work on the
  external tool was done by Richard Lawrence; I worked on the exporter
  for the citation syntax including the interface with an external
  tool.)

- There is a difference between citations as done by latex/bibtex/etc.,
  and those done in every other format (handled through CSL).  Assuming
  latex users want to keep their native processing rather than
  delegating to CSL, we need to solve the myriad small inconsistencies
  between these two tools.  I think this is an area where it’s important
  to get things right: users of citations generally have exacting
  requirements.  “Approximately Chicago-style” or “almost MLA” aren’t
  worth anything.

When I start working again I will take your words about silver bullets
to heart, and come up with something that covers at least basic use
cases for users who can compile a haskell program themselves.  We can
expand from there.

(I should also say, if someone else is interested in working on this
please don’t hesitate to jump right in.  I will help you however I can!)

> 
> ** Annotations
> 
> It would be nice to allow annotating text in Org. I already made
> a proposal for that feature a few months ago, without much success.
> Anyway, I'd like to implement it, in a simplified form. For the record,
> the idea is to use some markup for beginning and end of an annotation,
> e.g.,
> 
>   [@:id].....[@]
> 
> and store text within a dedicated section,
> 
>   * Annotations
> 
>   [@:id] ...
> 
> Like footnotes, you could easily browse remotely annotations at point.
> However, they would be, at least at the beginning, completely ignored
> during export.
> 
> ** Backslash escaping
> 
> Allowing to escape some symbols in plain text (e.g., emphasis markers,
> square brackets...) would remove a limitation in verbatim/code objects.
> As a small benefit, it would also permit to implement mid-word markup:
> b*o*ld.
> 
> There are some gotchas, however.
> 
> * Important fixes
> 
> ** Cache
> 
> The cache needs to be fixed. Its underlying implementation probably
> needs to be changed, too. At the moment, it uses an AVL tree, which
> isn't much tolerant and results in a freeze whenever the cache is
> corrupted. Shifting to text properties could improve the situation.
> 
> Also, it needs to do better analysis in order to prevent those
> corruptions. This is obviously the hardest part.
> 
> Again, an efficient cache can give us a better fontification mechanism,
> since Org syntax is not regular.
> 
> ** Link hexification
> 
> There are currently some subtle inconsistencies in link handling. See
> for example `org-make-link-string'.

At the risk of being a bit of an anarchist, I think this is somewhere we
could benefit from throwing backward compatibility out the window, at
least for brainstorming purposes.  The link-hexification bug threads
I’ve read give the impression of watching someone try to add one more
level onto a teetering house of cards.  It would be great if someone
could take inventory of the uses of links in org, and then try to design
a solid syntax that works for them from scratch.  We’d wait to worry
about implementation until having a well-engineered design.

(To give an example of what I have in mind, it may turn out that
hexification, borrowed from HTML/HTTP, is not even the best technical
solution to our actual problem, whatever it is.)

> 
> ** Use lexical binding everywhere
> 
> This is the easiest part: add appropriate cookie at the beginning of the
> file and fix resulting compiler warnings.

There are also a lot of places that org uses dynamic variables to
pass information around.  It would be good to eliminate these as much
as possible, since lexical binding allows generating more efficient
bytecode for lexically bound variable access (including function
arguments, AFAIK).  This is a further project to what you describe,
of course.

> 
> ** Find a solution for orgstruct minor mode
> 
> Org struct is really different from Org. It prevents Org from using
> better algorithms for outline navigation (see discussion about
> `org-show-children' on this list).
> 
> I think Org struct should be removed from "org.el" (and org-footnote
> ...) and added in its own library. It should also be built on top of
> Outline mode instead of Org mode. Org, OTOH, should remove dependencies
> on Outline mode and implement better navigation functions.

FWIW, this gets a +1 from me.  I’ve lost count of the number of times I’ve
started to make some “obvious” minor improvement to some org function,
only to have to discard the effort after 15 minutes when realizing that
supporting orgstruct mode makes it impossible to actually carry through
the change.

Thanks again,

-- 
Aaron Ecay

Re: Some projects

[Marcin Borkowski]

On 2015-10-26, at 00:03, Aaron Ecay <aaronecay@gmail.com> wrote:

> - The only “off the shelf”-capable citation processing library that we
>   found last time is in Haskell, which introduced some difficulties for
>   distributing the resulting tool.  I know some projects
>   (e.g. git-annex) are written in Haskell and distributed as static
>   binaries for windows/mac/linux/etc.  We’d need to figure out how to do
>   this, or find another citation processing library in an
>   easier-to-distribute language.  (I should say, all the work on the
>   external tool was done by Richard Lawrence; I worked on the exporter
>   for the citation syntax including the interface with an external
>   tool.)

This is probably a very naive question, but wouldn't it be possible to
have a pure Elisp tool?  I understand that lack of manpower may be the
main problem; are there any others?

> - There is a difference between citations as done by latex/bibtex/etc.,
>   and those done in every other format (handled through CSL).  Assuming
>   latex users want to keep their native processing rather than
>   delegating to CSL, we need to solve the myriad small inconsistencies
>   between these two tools.  I think this is an area where it’s important
>   to get things right: users of citations generally have exacting
>   requirements.  “Approximately Chicago-style” or “almost MLA” aren’t
>   worth anything.

Out of curiosity: what is CSL?

Best,
(and count me in the number of people who say a big Thank you),

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: Some projects

[Rasmus]

Marcin Borkowski <mbork@mbork.pl> writes:

> On 2015-10-26, at 00:03, Aaron Ecay <aaronecay@gmail.com> wrote:
>
>> - The only “off the shelf”-capable citation processing library that we
>>   found last time is in Haskell, which introduced some difficulties for
>>   distributing the resulting tool.  I know some projects
>>   (e.g. git-annex) are written in Haskell and distributed as static
>>   binaries for windows/mac/linux/etc.  We’d need to figure out how to do
>>   this, or find another citation processing library in an
>>   easier-to-distribute language.  (I should say, all the work on the
>>   external tool was done by Richard Lawrence; I worked on the exporter
>>   for the citation syntax including the interface with an external
>>   tool.)
>
> This is probably a very naive question, but wouldn't it be possible to
> have a pure Elisp tool?  I understand that lack of manpower may be the
> main problem; are there any others?

Sure, but it’s a lot of work. citeproc-java is 22K lines:

      https://github.com/michel-kraemer/citeproc-java/graphs/contributors

>> - There is a difference between citations as done by latex/bibtex/etc.,
>>   and those done in every other format (handled through CSL).  Assuming
>>   latex users want to keep their native processing rather than
>>   delegating to CSL, we need to solve the myriad small inconsistencies
>>   between these two tools.  I think this is an area where it’s important
>>   to get things right: users of citations generally have exacting
>>   requirements.  “Approximately Chicago-style” or “almost MLA” aren’t
>>   worth anything.
>
> Out of curiosity: what is CSL?

  https://en.wikipedia.org/wiki/Citation_Style_Language

What is used by e.g. Zotero and Pandoc and mostly everyone who's not using
LaTeX.  Unfortunately, it is not as flexible as biblatex.  E.g. to get
"author (year)" and "(author, year)" I /think/ you need two separate
styles, but I could easily be wrong on this.  This was discussed
extensively earlier.

Rasmus

-- 
Together we will make the possible totalllly impossible!

Re: Some projects

[Richard Lawrence]

Hi all,

Aaron Ecay <aaronecay@gmail.com> writes:

> Hi Nicolas,
>
> Thanks for writing this up.  It is important to think about, and
> ultimately solve, all the issues you raise.

Yes, thanks for this, Nicolas!  

> 2015ko urriak 25an, Nicolas Goaziou-ek idatzi zuen:
>> 
>> ** Citations
>> 
>> Development apparently stopped for some reason. We have a citation
>> syntax for Org in wip-cite and some work done in wip-cite-awe and
>> probably elsewhere.
>> 
>> I think we could at least provide features defined in Org Ref using the
>> new syntax (minus hydra/helm related functions).
>> 
>> We don't need a silver bullet. Just something with a non-empty user
>> base, and extensible. In any case, the work done so far shouldn't be
>> wasted.
>
> I was working on this rather intensively at one time, but I had to stop
> because other aspects of life intruded.  I have just been coming back
> towards a situation where I can imagine myself having some (still small,
> but non-zero) chunks of time to devote to working on org.  So I hope I
> will be able to pick this back up, but (regrettably) I’m not able to
> make any promises.
>
> Based on my recollection, here’s what the problems were when I stopped:
>
> - The only “off the shelf”-capable citation processing library that we
>   found last time is in Haskell, which introduced some difficulties for
>   distributing the resulting tool.  I know some projects
>   (e.g. git-annex) are written in Haskell and distributed as static
>   binaries for windows/mac/linux/etc.  We’d need to figure out how to do
>   this, or find another citation processing library in an
>   easier-to-distribute language.

Yes, this is my understanding, too.  In particular, there does not seem
to be an Elisp CSL library, and it would be a lot of work to write one.

The other CSL library that looks complete and usable is citeproc-js; but
like the Haskell library (pandoc-citeproc) it would need to be wrapped
somehow so that it can talk with Org.

It should be relatively straightforward for someone who knows Javascript
to write such a wrapper, if anyone wants to work on that.  But this does
not really solve the problem with distribution.  Either of the
off-the-shelf CSL libraries will require both a wrapper and a platform
for building/installing/running the wrapper and library as a complete
external tool.

> (I should say, all the work on the external tool was done by Richard
> Lawrence; I worked on the exporter for the citation syntax including
> the interface with an external tool.)

The tool I was working on is here:
https://github.com/wyleyr/org-citeproc

The branch of Org that it needs is here:
https://github.com/wyleyr/org-mode

At the moment, it supports single- and multiple-work citations in inline
styles (e.g. Chicago/Harvard type citations, and I think also styles
that use numbered references to the bibliography).  It doesn't presently
work with note-based styles, and making it work will require some
modifications on the Org side.  Specifically, the Org side will have to
get a bit smarter about how it inserts the formatted citations into the
document (Org needs to understand them as footnotes so that they get
correctly numbered, etc. amongst other non-citation footnotes).

> - There is a difference between citations as done by latex/bibtex/etc.,
>   and those done in every other format (handled through CSL).  Assuming
>   latex users want to keep their native processing rather than
>   delegating to CSL, we need to solve the myriad small inconsistencies
>   between these two tools.  I think this is an area where it’s important
>   to get things right: users of citations generally have exacting
>   requirements.  “Approximately Chicago-style” or “almost MLA” aren’t
>   worth anything.

I guess I would just add that it is not clear how much we need to solve
here, at least in the short term.  I can't remember whether we found any
concrete examples of needs people have that BibLaTeX can handle but
CSL cannot, or vice versa.  Anyway, there is a core set of citation
features that both types of backends handle readily, and I think it
would be a big win to have these accessible via a common syntax in Org.
No silver bullets is indeed the maxim to keep in mind.

> (I should also say, if someone else is interested in working on this
> please don’t hesitate to jump right in.  I will help you however I can!)

I also want to echo this.  I don't really have much time to work on this
myself right now (trying to get the ol' dissertation finished this year)
but I will help out however I can.

Best,
Richard

Re: Some projects

[Nicolas Goaziou]

Richard Lawrence <richard.lawrence@berkeley.edu> writes:

> Yes, this is my understanding, too.  In particular, there does not seem
> to be an Elisp CSL library, and it would be a lot of work to write
> one.

Here is a genuine question: what is the difficult part, or if there is
none, the longer parts of the work? AFAIU, CSL is XML and we can use
libxml for the low-level parsing.

Could you give some examples of the UI offered by the citeproc-hs
library?


Regards,

Re: Some projects

[Richard Lawrence]

Hi Nicolas and all,

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Richard Lawrence <richard.lawrence@berkeley.edu> writes:
>
>> Yes, this is my understanding, too.  In particular, there does not seem
>> to be an Elisp CSL library, and it would be a lot of work to write
>> one.
>
> Here is a genuine question: what is the difficult part, or if there is
> none, the longer parts of the work? AFAIU, CSL is XML and we can use
> libxml for the low-level parsing.

I can't speak to how difficult it would really be.  But I do think it
would be a lot of work, and tedious, unglamorous work at that.
Essentially it means implementing the CSL specification:

http://docs.citationstyles.org/en/stable/specification.html

The specification describes the XML format for CSL styles, which IIUC
are something like XSLT programs: they describe how to transform an
input document tree into an output document tree.  So, implementing CSL
is something like implementing a very special-purpose declarative
programming language, which has to handle declarations about every
detail of citation formatting (where to put periods, how to capitalize
names like "von Neumann", whether to use "&" or a translation of "and"
for the current locale, etc.).  Lisp is a good language to do this kind
of thing in, but parsing the XML is surely the easy part.

> Could you give some examples of the UI offered by the citeproc-hs
> library?

The library I ended up wrapping is actually pandoc-citeproc, which is a
fork of citeproc-hs (which seems to be unmaintained).

pandoc-citeproc essentially exports one main function, processCites.
This function transforms a Pandoc data structure containing Citation
nodes into another Pandoc where the citations have been rendered into
the text and the bibliography is appended at the end of the document.

processCites also requires a list of references (read from a
bibliography database, e.g. .bib files, via the readBiblioFile function)
and a CSL style (read from a file via the readCSLFile function).  So
basically, the pandoc-citeproc API consists of these three functions:
processCites, readBiblioFile, and readCSLFile.

Best,
Richard

Re: Some projects

[Matt Price]

[-- Attachment #1: Type: text/plain, Size: 2185 bytes --]

On Mon, Oct 26, 2015 at 6:23 PM, Richard Lawrence <
richard.lawrence@berkeley.edu> wrote:

> Hi Nicolas and all,
>
> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>
> > Richard Lawrence <richard.lawrence@berkeley.edu> writes:
> >
> >> Yes, this is my understanding, too.  In particular, there does not seem
> >> to be an Elisp CSL library, and it would be a lot of work to write
> >> one.
> >
> > Here is a genuine question: what is the difficult part, or if there is
> > none, the longer parts of the work? AFAIU, CSL is XML and we can use
> > libxml for the low-level parsing.
>
> I can't speak to how difficult it would really be.  But I do think it
> would be a lot of work, and tedious, unglamorous work at that.
> Essentially it means implementing the CSL specification:
>
> http://docs.citationstyles.org/en/stable/specification.html
>
> The specification describes the XML format for CSL styles, which IIUC
> are something like XSLT programs: they describe how to transform an
> input document tree into an output document tree.  So, implementing CSL
> is something like implementing a very special-purpose declarative
> programming language, which has to handle declarations about every
> detail of citation formatting (where to put periods, how to capitalize
> names like "von Neumann", whether to use "&" or a translation of "and"
> for the current locale, etc.).  Lisp is a good language to do this kind
> of thing in, but parsing the XML is surely the easy part.
>
> > Could you give some examples of the UI offered by the citeproc-hs
> > library?
>
> The library I ended up wrapping is actually pandoc-citeproc, which is a
> fork of citeproc-hs (which seems to be unmaintained).
>

The obvious advantage to pandoc-citeproc is that it keeps us close to
pandoc, and it would be good to encourage as much compatibility as possible
with those guys.

The disadvantage is that, from what I can tell, the javascript
implementation is the canonical version of citeproc, and the place where
improvements are pushed first.  So, for instance, if one wanted to
implement an org-syntax output format for citeproc, citeproc-js would be
the most likely project to support that work.

[-- Attachment #2: Type: text/html, Size: 3006 bytes --]

Re: Some projects

[Aaron Ecay]

Hi Richard, hi all,

2015ko urriak 26an, Richard Lawrence-ek idatzi zuen:
> 

[...]

>> I was working on this rather intensively at one time, but I had to stop
>> because other aspects of life intruded.  I have just been coming back
>> towards a situation where I can imagine myself having some (still small,
>> but non-zero) chunks of time to devote to working on org.  So I hope I
>> will be able to pick this back up, but (regrettably) I’m not able to
>> make any promises.
>> 
>> Based on my recollection, here’s what the problems were when I stopped:
>> 
>> - The only “off the shelf”-capable citation processing library that we
>> found last time is in Haskell, which introduced some difficulties for
>> distributing the resulting tool.  I know some projects
>> (e.g. git-annex) are written in Haskell and distributed as static
>> binaries for windows/mac/linux/etc.  We’d need to figure out how to do
>> this, or find another citation processing library in an
>> easier-to-distribute language.
> 
> Yes, this is my understanding, too.  In particular, there does not seem
> to be an Elisp CSL library, and it would be a lot of work to write one.
> 
> The other CSL library that looks complete and usable is citeproc-js; but
> like the Haskell library (pandoc-citeproc) it would need to be wrapped
> somehow so that it can talk with Org.
> 
> It should be relatively straightforward for someone who knows Javascript
> to write such a wrapper, if anyone wants to work on that.  But this does
> not really solve the problem with distribution.  

It solves many of the hard problems though.  Node.js is distributed
as a binary for many platforms.  We’d just have to direct users to
install this in the “normal way,” and use the installed binary to
interpret the JS source.  Whereas for haskell we’d be stuck building
the binary ourselves, worrying about static linking/dll hell/32-bit
dinosaurs/any of a half-dozen other problems that I don’t really
understand.

OTOH, pandoc-citeproc includes a bibtex parser; we’d need to write a JS
one and wire it up to citeproc-js.  When I looked (quite some time ago),
there did not seem to be any good bibtex parsing libraries in JS (and
several third-rate ones).

OT3rdH, responding to Matt’s message
<http://mid.gmane.org/CAN_Dec_52SP6ghR56PudHiH69KSPrQ0VDw2ZMnP5799-CtaVeQ@mail.gmail.com>,

> The disadvantage is that, from what I can tell, the javascript
> implementation is the canonical version of citeproc, and the place where
> improvements are pushed first.  So, for instance, if one wanted to
> implement an org-syntax output format for citeproc, citeproc-js would be
> the most likely project to support that work.

Pandoc can output org syntax, so it may be that we can just link with
the main pandoc haskell library as well as pandoc-citeproc and solve
this ourselves, without needing upstream support.

-- 
Aaron Ecay

Re: Some projects

[Rasmus]

Aaron Ecay <aaronecay@gmail.com> writes:

> Hi Richard, hi all,
>
> 2015ko urriak 26an, Richard Lawrence-ek idatzi zuen:
>> 
>
> [...]
>
>>> I was working on this rather intensively at one time, but I had to stop
>>> because other aspects of life intruded.  I have just been coming back
>>> towards a situation where I can imagine myself having some (still small,
>>> but non-zero) chunks of time to devote to working on org.  So I hope I
>>> will be able to pick this back up, but (regrettably) I’m not able to
>>> make any promises.
>>> 
>>> Based on my recollection, here’s what the problems were when I stopped:
>>> 
>>> - The only “off the shelf”-capable citation processing library that we
>>> found last time is in Haskell, which introduced some difficulties for
>>> distributing the resulting tool.  I know some projects
>>> (e.g. git-annex) are written in Haskell and distributed as static
>>> binaries for windows/mac/linux/etc.  We’d need to figure out how to do
>>> this, or find another citation processing library in an
>>> easier-to-distribute language.
>> 
>> Yes, this is my understanding, too.  In particular, there does not seem
>> to be an Elisp CSL library, and it would be a lot of work to write one.
>> 
>> The other CSL library that looks complete and usable is citeproc-js; but
>> like the Haskell library (pandoc-citeproc) it would need to be wrapped
>> somehow so that it can talk with Org.
>> 
>> It should be relatively straightforward for someone who knows Javascript
>> to write such a wrapper, if anyone wants to work on that.  But this does
>> not really solve the problem with distribution.  
>
> It solves many of the hard problems though.  Node.js is distributed
> as a binary for many platforms.  We’d just have to direct users to
> install this in the “normal way,” and use the installed binary to
> interpret the JS source.  Whereas for haskell we’d be stuck building
> the binary ourselves, worrying about static linking/dll hell/32-bit
> dinosaurs/any of a half-dozen other problems that I don’t really
> understand.

I would feel more comfortable relying on a JS library.  Perhaps it’s also
easier to find people who are willing to work on/knows JS over the long
haul...

> OTOH, pandoc-citeproc includes a bibtex parser; we’d need to write a JS
> one and wire it up to citeproc-js.  When I looked (quite some time ago),
> there did not seem to be any good bibtex parsing libraries in JS (and
> several third-rate ones).

Bibtex support is essential, of course.

Can someone remind me why citeproc-java isn’t good?  AFAIR, it has a
bibtex parser.  But probably it lacks in some other dimension...

> OT3rdH, responding to Matt’s message
> <http://mid.gmane.org/CAN_Dec_52SP6ghR56PudHiH69KSPrQ0VDw2ZMnP5799-CtaVeQ@mail.gmail.com>,
>
>> The disadvantage is that, from what I can tell, the javascript
>> implementation is the canonical version of citeproc, and the place where
>> improvements are pushed first.  So, for instance, if one wanted to
>> implement an org-syntax output format for citeproc, citeproc-js would be
>> the most likely project to support that work.
>
> Pandoc can output org syntax, so it may be that we can just link with
> the main pandoc haskell library as well as pandoc-citeproc and solve
> this ourselves, without needing upstream support.

Do we WANT to depend on Pandoc?  I would say "no".  In my OS, where we
finally got a binary distribution of pandoc, the size of pandoc is still
1600Mb!  I don’t know if this is representative of other systems, though.
E.g. what is the size of pandoc+deps in Debian?

Rasmus

-- 
Lasciate ogni speranza o voi che entrate: siete nella mani di'machellaio

Re: Some projects

[Aaron Ecay]

Hi Rasmus,

2015ko urriak 27an, Rasmus-ek idatzi zuen:

[...]

> 
> Can someone remind me why citeproc-java isn’t good?  AFAIR, it has a
> bibtex parser.  But probably it lacks in some other dimension...

You have to write java to use it, which is a big negative (at least for
me).  Last time I think we concluded that the CLI interface it exposes
is not adequate (though I actually started off trying to use it).

There’s also the issue that it is just a bundling of citeproc-js with
some java libraries (js interpreter, bibtex processor, ...), so someone
who tries to extend it actually has to know both java and js.

> Do we WANT to depend on Pandoc?  I would say "no".  In my OS, where we
> finally got a binary distribution of pandoc, the size of pandoc is still
> 1600Mb!  I don’t know if this is representative of other systems, though.
> E.g. what is the size of pandoc+deps in Debian?

It’s unfortunate, but pandoc has the most easily adaptable CSL
implementation for our use case, as far as I can see.  The other
contender is Zotero (in JS, possibly with XUL-specific bits).  It would
be good if somebody looked at what it would take for us to use Zotero’s
implementation, just as Richard did for Pandoc.

-- 
Aaron Ecay

Re: Some projects

[Rasmus]

Aaron Ecay <aaronecay@gmail.com> writes:

> You have to write java to use it, which is a big negative (at least for
> me).  Last time I think we concluded that the CLI interface it exposes
> is not adequate (though I actually started off trying to use it).

I see.  That's probably the reason.

> There’s also the issue that it is just a bundling of citeproc-js with
> some java libraries (js interpreter, bibtex processor, ...), so someone
> who tries to extend it actually has to know both java and js.

Another drawback.

Pure JS (or perhaps Haskell?) is probably more accessible then.  FWIW,
there's a couple of libraries that promise to read bibtex on npm, though I
haven’t tried any of those. 

        https://www.npmjs.com/search?q=bibtex

>> Do we WANT to depend on Pandoc?  I would say "no".  In my OS, where we
>> finally got a binary distribution of pandoc, the size of pandoc is still
>> 1600Mb!  I don’t know if this is representative of other systems, though.
>> E.g. what is the size of pandoc+deps in Debian?
>
> It’s unfortunate, but pandoc has the most easily adaptable CSL
> implementation for our use case, as far as I can see.  The other
> contender is Zotero (in JS, possibly with XUL-specific bits).  It would
> be good if somebody looked at what it would take for us to use Zotero’s
> implementation, just as Richard did for Pandoc.

Indeed.  I guess this is what they use:

         https://github.com/zotero/citeproc-node

It also looks rather complex...

Rasmus

-- 
9000!

Re: Some projects

[Matt Price]

[-- Attachment #1: Type: text/plain, Size: 1131 bytes --]

On Tue, Oct 27, 2015 at 9:51 AM, Rasmus <rasmus@gmx.us> wrote:

> Aaron Ecay <aaronecay@gmail.com> writes:
>
>
> >
> > It’s unfortunate, but pandoc has the most easily adaptable CSL
> > implementation for our use case, as far as I can see.  The other
> > contender is Zotero (in JS, possibly with XUL-specific bits).  It would
> > be good if somebody looked at what it would take for us to use Zotero’s
> > implementation, just as Richard did for Pandoc.
>
> Indeed.  I guess this is what they use:
>
>          https://github.com/zotero/citeproc-node
>
> It also looks rather complex...
>

FWIW, I just tried installing this on my Arch system, but it doesn't work
with node 0.12, and I am currently unable to switch to io.js due to
dependencies of several other projects. I guess tools like NVM can help
with this situation, but I worry that node is currently a moving target and
might lead to lots of platform-dependent buggy behaviour.

Though I have also personally had many problems with pandoc and haskell on
this same arch linux laptop, so it could be that I just suck at system
maintenance.

[-- Attachment #2: Type: text/html, Size: 1696 bytes --]

Re: Some projects

[Aldric Giacomoni]

[-- Attachment #1: Type: text/plain, Size: 522 bytes --]

On Tue, Oct 27, 2015 at 9:05 PM Matt Price <moptop99@gmail.com> wrote:

>
>
>>
> FWIW, I just tried installing this on my Arch system, but it doesn't work
> with node 0.12, and I am currently unable to switch to io.js due to
> dependencies of several other projects. I guess tools like NVM can help
> with this situation, but I worry that node is currently a moving target and
> might lead to lots of platform-dependent buggy behaviour.
>
> FYI - io.js has merged back into node.js, so we're back to just the one
project.

[-- Attachment #2: Type: text/html, Size: 1111 bytes --]

Re: Some projects

[Matt Lundin]

Matt Price <moptop99@gmail.com> writes:

> On Tue, Oct 27, 2015 at 9:51 AM, Rasmus <rasmus@gmx.us> wrote:
>
>
>     Aaron Ecay <aaronecay@gmail.com> writes:
>
>     Indeed. I guess this is what they use:
>
>     https://github.com/zotero/citeproc-node
>
>     It also looks rather complex...
>
>
> FWIW, I just tried installing this on my Arch system, but it doesn't
> work with node 0.12, and I am currently unable to switch to io.js due
> to dependencies of several other projects. I guess tools like NVM can
> help with this situation, but I worry that node is currently a moving
> target and might lead to lots of platform-dependent buggy behaviour. 

Testing it now... Works fine on my Arch system. Arch's current nodejs is
4.2. As I understand it, io.js has been merged back into node 4.+

The citeproc-node server itself is not very complex. It's just a node
wrapper around citeproc-js. The big limitation, it seems to me, is that
it only accepts a json format as input. Also it seems to use html markup
in all its output formats.

Though published by the Zotero programmers, citeproc-node, I believe, is
distinct from the citeproc-js implementation in Zotero, which is a XUL
application.

Matt

Re: Some projects

[Matt Price]

[-- Attachment #1: Type: text/plain, Size: 3015 bytes --]

On Tue, Oct 27, 2015 at 11:31 PM, Matt Lundin <mdl@imapmail.org> wrote:

> Matt Price <moptop99@gmail.com> writes:
>
> > On Tue, Oct 27, 2015 at 9:51 AM, Rasmus <rasmus@gmx.us> wrote:
> >
> >
> >     Aaron Ecay <aaronecay@gmail.com> writes:
> >
> >     Indeed. I guess this is what they use:
> >
> >     https://github.com/zotero/citeproc-node
> >
> >     It also looks rather complex...
> >
> >
> > FWIW, I just tried installing this on my Arch system, but it doesn't
> > work with node 0.12, and I am currently unable to switch to io.js due
> > to dependencies of several other projects. I guess tools like NVM can
> > help with this situation, but I worry that node is currently a moving
> > target and might lead to lots of platform-dependent buggy behaviour.
>
> Testing it now... Works fine on my Arch system. Arch's current nodejs is
> 4.2. As I understand it, io.js has been merged back into node 4.+
>

upgraded from 0.12, and it works fine.  My problem before was that some
important npm packages were still incompatible with node 4.2; it may be
that that's no longer an issue, and I spoke too soon.


>
> The citeproc-node server itself is not very complex. It's just a node
> wrapper around citeproc-js. The big limitation, it seems to me, is that
> it only accepts a json format as input. Also it seems to use html markup
> in all its output formats.
>

That does seem to be an issue, but I bet it wouldn't be too hard to fix.
Currently outputformat is hardcoded on line 94 of lib/citeServer.js; I'm a
little slow at reading JS but I think replacing line 259 with a switch
statement analogous to the one for responseformat at line 291 would allow
one to use the full capacities of citeproc.js.  I think the best route
would probably be to submit a patch to citeproc.js adding an org-mode
output format, propagating that up to citeproc-node, submitting a second
patch to citeproc-node, and then writing the org-internal functions
properly.  We might also want to add an additional responseformat option.

It might take a month or two to get all those changes accepted, but we
could probably do the org-mode development simultaneously, and it would be
worth the wait to have a stable solution.  My experience with the
citeproc-js maintainer is that he is very helpful and responsive to feature
requests.

I am a slow coder but would be happy to invest some time in this if it's
the direction the community wants to go in.  Adding a new output format to
citeproc-js is not that complex; see
https://bitbucket.org/fbennett/citeproc-js/src/tip/src/formats.js?fileviewer=file-view-default


> Though published by the Zotero programmers, citeproc-node, I believe, is
> distinct from the citeproc-js implementation in Zotero, which is a XUL
> application.
>
> I think that's right.  If we could avoid dependencies on Zotero that would
be good; it is a pretty substantial program, takes up a significant amount
of RAM at runtime, and relies on a GUI environment, which neither emacs nor
pandoc does.


> Matt
>

[-- Attachment #2: Type: text/html, Size: 4544 bytes --]

Re: Some projects

[Matt Price]

[-- Attachment #1.1: Type: text/plain, Size: 1106 bytes --]

On Wed, Oct 28, 2015 at 10:36 AM, Matt Price <moptop99@gmail.com> wrote:

>
> That does seem to be an issue, but I bet it wouldn't be too hard to fix.
> Currently outputformat is hardcoded on line 94 of lib/citeServer.js; I'm a
> little slow at reading JS but I think replacing line 259 with a switch
> statement analogous to the one for responseformat at line 291 would allow
> one to use the full capacities of citeproc.js.  I think the best route
> would probably be to submit a patch to citeproc.js adding an org-mode
> output format, propagating that up to citeproc-node, submitting a second
> patch to citeproc-node, and then writing the org-internal functions
> properly.  We might also want to add an additional responseformat option.
>
>> Matt
>>
>
>
here is a very half-assed aproach that adds an org-mode outputformat to
citeproc.  it's incomplete but does much of wha it ought to.  some of the
elements I left in html b/c I'm not sure how they oughtto be handled in
org.  e.g., how do we handle smallcaps?

(this is a patch to citeproc-node, so requires installing the repo from
github first).

[-- Attachment #1.2: Type: text/html, Size: 1753 bytes --]

[-- Attachment #2: add-org.patch --]
[-- Type: text/x-patch, Size: 4778 bytes --]

diff --git a/lib/citeServer.js b/lib/citeServer.js
index 691ca78..a5d82c7 100644
--- a/lib/citeServer.js
+++ b/lib/citeServer.js
@@ -91,7 +91,7 @@ var defaultCiteserverConfig = {
 var defaultRequestConfig = {
     bibliography: '1',
     citations: '0',
-    outputformat: 'html',
+    outputformat: 'org',
     responseformat: 'json',
     locale: 'en-US',
     style: 'chicago-author-date',
diff --git a/lib/citeproc.js b/lib/citeproc.js
index 08a8008..8622533 100644
--- a/lib/citeproc.js
+++ b/lib/citeproc.js
@@ -13140,6 +13140,106 @@ CSL.Output.Formats.prototype.html = {
         return "<a href=\"http://dx.doi.org/" + str + "\">" + str + "</a>";
     }
 };
+
+CSL.Output.Formats.prototype.org = {
+    "text_escape": function (text) {
+        if (!text) {
+            text = "";
+        }
+        return text.replace(/&/g, "&#38;")
+            .replace(/</g, "&#60;")
+            .replace(/>/g, "&#62;")
+            .replace("  ", "&#160; ", "g")
+            .replace(CSL.SUPERSCRIPTS_REGEXP,
+                     function(aChar) {
+                         return "<sup>" + CSL.SUPERSCRIPTS[aChar] + "</sup>";
+                     });
+    },
+    "bibstart": "* Bibliography\n",
+    "bibend": "\n",
+    "@font-style/italic": "/%%STRING%%/",
+    "@font-style/oblique": "/%%STRING%%/",
+    "@font-style/normal": "%%STRING%%",
+    "@font-variant/small-caps": "<span style=\"font-variant:small-caps;\">%%STRING%%</span>",
+    "@passthrough/true": CSL.Output.Formatters.passthrough,
+    "@font-variant/normal": false,
+    "@font-weight/bold": "*%%STRING%%*",
+    "@font-weight/normal": false,
+    "@font-weight/light": false,
+    "@text-decoration/none": false,
+    "@text-decoration/underline": "_%%STRING%%_",
+    "@vertical-align/sup": "^{%%STRING%%}",
+    "@vertical-align/sub": "_{%%STRING%%}",
+    "@vertical-align/baseline": false,
+    "@strip-periods/true": CSL.Output.Formatters.passthrough,
+    "@strip-periods/false": CSL.Output.Formatters.passthrough,
+    "@quotes/true": function (state, str) {
+        if ("undefined" === typeof str) {
+            return state.getTerm("open-quote");
+        }
+        return state.getTerm("open-quote") + str + state.getTerm("close-quote");
+    },
+    "@quotes/inner": function (state, str) {
+        if ("undefined" === typeof str) {
+            return "\u2019";
+        }
+        return state.getTerm("open-inner-quote") + str + state.getTerm("close-inner-quote");
+    },
+    "@quotes/false": false,
+    "@cite/entry": function (state, str) {
+        return state.sys.wrapCitationEntry(str, this.item_id, this.locator_txt, this.suffix_txt);
+	},
+    "@bibliography/entry": function (state, str) {
+        var insert = "";
+        if (state.sys.embedBibliographyEntry) {
+            insert = state.sys.embedBibliographyEntry(this.item_id) + "\n";
+        }
+        return "  \n" + str + "\n" + insert;
+    },
+    "@display/block": function (state, str) {
+        return "\n\n    <div class=\"csl-block\">" + str + "</div>\n";
+    },
+    "@display/left-margin": function (state, str) {
+        return "\n    <div class=\"csl-left-margin\">" + str + "</div>";
+    },
+    "@display/right-inline": function (state, str) {
+        return "<div class=\"csl-right-inline\">" + str + "</div>\n  ";
+    },
+    "@display/indent": function (state, str) {
+        return "<div class=\"csl-indent\">" + str + "</div>\n  ";
+    },
+    "@showid/true": function (state, str, cslid) {
+        if (!state.tmp.just_looking && ! state.tmp.suppress_decorations) {
+            if (cslid) {
+                return "<span class=\"" + state.opt.nodenames[cslid] + "\" cslid=\"" + cslid + "\">" + str + "</span>";
+            } else if ("string" === typeof str) {
+                var prePunct = "";
+                if (str) {
+                    var m = str.match(CSL.VARIABLE_WRAPPER_PREPUNCT_REX);
+                    prePunct = m[1];
+                    str = m[2];
+                }
+                var postPunct = "";
+                if (str && CSL.SWAPPING_PUNCTUATION.indexOf(str.slice(-1)) > -1) {
+                    postPunct = str.slice(-1);
+                    str = str.slice(0,-1);
+                }
+                return state.sys.variableWrapper(this.params, prePunct, str, postPunct);
+            } else {
+                return str;
+            }
+        } else {
+            return str;
+        }
+    },
+    "@URL/true": function (state, str) {
+        return "<a href=\"" + str + "\">" + str + "</a>";
+    },
+    "@DOI/true": function (state, str) {
+        return "<a href=\"http://dx.doi.org/" + str + "\">" + str + "</a>";
+    }
+};
+
 CSL.Output.Formats.prototype.text = {
     "text_escape": function (text) {
         if (!text) {

Re: Some projects

[Matt Price]

[-- Attachment #1: Type: text/plain, Size: 866 bytes --]

On Tue, Oct 27, 2015 at 9:03 AM, Aaron Ecay <aaronecay@gmail.com> wrote:

> Hi Rasmus,
>
> 2015ko urriak 27an, Rasmus-ek idatzi zuen:
>
> [...]
>
> >
>
> It’s unfortunate, but pandoc has the most easily adaptable CSL
> implementation for our use case, as far as I can see.  The other
> contender is Zotero (in JS, possibly with XUL-specific bits).  It would
> be good if somebody looked at what it would take for us to use Zotero’s
> implementation, just as Richard did for Pandoc.
>

You know about zotxt, right? https://github.com/egh/zotxt-emacs  It is
probably the best starting point, but is currently a little cumbersome to
set up, with several moving parts (zoterto, zotero zotxt extension,
zotxt-emacs library).  And in the end it also sort of makes use of pandoc,
too (https://github.com/egh/zotxt-emacs#for-pandoc-markdown-files)

[-- Attachment #2: Type: text/html, Size: 1427 bytes --]

Re: Some projects

[Rainer M Krug]

[-- Attachment #1: Type: text/plain, Size: 4783 bytes --]

Rasmus <rasmus@gmx.us> writes:

> Aaron Ecay <aaronecay@gmail.com> writes:
>
>> Hi Richard, hi all,
>>
>> 2015ko urriak 26an, Richard Lawrence-ek idatzi zuen:
>>> 
>>
>> [...]
>>
>>>> I was working on this rather intensively at one time, but I had to stop
>>>> because other aspects of life intruded.  I have just been coming back
>>>> towards a situation where I can imagine myself having some (still small,
>>>> but non-zero) chunks of time to devote to working on org.  So I hope I
>>>> will be able to pick this back up, but (regrettably) I’m not able to
>>>> make any promises.
>>>> 
>>>> Based on my recollection, here’s what the problems were when I stopped:
>>>> 
>>>> - The only “off the shelf”-capable citation processing library that we
>>>> found last time is in Haskell, which introduced some difficulties for
>>>> distributing the resulting tool.  I know some projects
>>>> (e.g. git-annex) are written in Haskell and distributed as static
>>>> binaries for windows/mac/linux/etc.  We’d need to figure out how to do
>>>> this, or find another citation processing library in an
>>>> easier-to-distribute language.
>>> 
>>> Yes, this is my understanding, too.  In particular, there does not seem
>>> to be an Elisp CSL library, and it would be a lot of work to write one.
>>> 
>>> The other CSL library that looks complete and usable is citeproc-js; but
>>> like the Haskell library (pandoc-citeproc) it would need to be wrapped
>>> somehow so that it can talk with Org.
>>> 
>>> It should be relatively straightforward for someone who knows Javascript
>>> to write such a wrapper, if anyone wants to work on that.  But this does
>>> not really solve the problem with distribution.  
>>
>> It solves many of the hard problems though.  Node.js is distributed
>> as a binary for many platforms.  We’d just have to direct users to
>> install this in the “normal way,” and use the installed binary to
>> interpret the JS source.  Whereas for haskell we’d be stuck building
>> the binary ourselves, worrying about static linking/dll hell/32-bit
>> dinosaurs/any of a half-dozen other problems that I don’t really
>> understand.
>
> I would feel more comfortable relying on a JS library.  Perhaps it’s also
> easier to find people who are willing to work on/knows JS over the long
> haul...
>
>> OTOH, pandoc-citeproc includes a bibtex parser; we’d need to write a JS
>> one and wire it up to citeproc-js.  When I looked (quite some time ago),
>> there did not seem to be any good bibtex parsing libraries in JS (and
>> several third-rate ones).
>
> Bibtex support is essential, of course.
>
> Can someone remind me why citeproc-java isn’t good?  AFAIR, it has a
> bibtex parser.  But probably it lacks in some other dimension...
>
>> OT3rdH, responding to Matt’s message
>> <http://mid.gmane.org/CAN_Dec_52SP6ghR56PudHiH69KSPrQ0VDw2ZMnP5799-CtaVeQ@mail.gmail.com>,
>>
>>> The disadvantage is that, from what I can tell, the javascript
>>> implementation is the canonical version of citeproc, and the place where
>>> improvements are pushed first.  So, for instance, if one wanted to
>>> implement an org-syntax output format for citeproc, citeproc-js would be
>>> the most likely project to support that work.
>>
>> Pandoc can output org syntax, so it may be that we can just link with
>> the main pandoc haskell library as well as pandoc-citeproc and solve
>> this ourselves, without needing upstream support.
>
> Do we WANT to depend on Pandoc?  I would say "no".  In my OS, where we
> finally got a binary distribution of pandoc, the size of pandoc is still
> 1600Mb!  I don’t know if this is representative of other systems, though.
> E.g. what is the size of pandoc+deps in Debian?

I don't know which OS you are using, but just checking on
[[https://github.com/jgm/pandoc/releases/1.15.1]]  and [[https://github.com/jgm/pandoc/releases/tag/1.15.1.1]]:

Windows: 19.7 MB
Mac    : 27.9 MB
Deb    : 20.2 MB

The 1600MB must be including LaTeX?

In General, I like the idea of using the Pandoc approach, as Pandoc
provides a very useful framework for all kinds of conversions (and I
don't like java...)

Nevertheless, I would be happy with whatever is used to implement
proper citation support.

Cheers,

Rainer

>
> Rasmus

-- 
Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys. (Germany)

Centre of Excellence for Invasion Biology
Stellenbosch University
South Africa

Tel :       +33 - (0)9 53 10 27 44
Cell:       +33 - (0)6 85 62 59 98
Fax :       +33 - (0)9 58 10 27 44

Fax (D):    +49 - (0)3 21 21 25 22 44

email:      Rainer@krugs.de

Skype:      RMkrug

PGP: 0x0F52F982

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]

Re: Some projects

[Rasmus]

Hi,

Rainer M Krug <Rainer@krugs.de> writes:

> I don't know which OS you are using, but just checking on
> [[https://github.com/jgm/pandoc/releases/1.15.1]] and
> [[https://github.com/jgm/pandoc/releases/tag/1.15.1.1]]:
>
> Windows: 19.7 MB
> Mac    : 27.9 MB
> Deb    : 20.2 MB
>
> The 1600MB must be including LaTeX?

Nope, but something like 100 haskell libraries and ghc.  I guess it’s not
build in a static way?

It’s much smaller on Debian, approx. 25Mb.  (I should switch to Fedora
full-time).

> In General, I like the idea of using the Pandoc approach, as Pandoc
> provides a very useful framework for all kinds of conversions (and I
> don't like java...)

Your first argument refers to a side effect (to us), and it is irrelevant
IMO.  Java run easily everywhere, easily, and if you use mathml for some
of your documents, you’re likely already using a jar file with Org.

That being said, it suggest that Pandoc /can/ be packed in a sensible way,
which is good.

Rasmus

-- 
A page of history is worth a volume of logic

Re: Some projects

[Ista Zahn]

On Tue, Oct 27, 2015 at 9:42 AM, Rasmus <rasmus@gmx.us> wrote:
> Hi,
>
> Rainer M Krug <Rainer@krugs.de> writes:
>
>> I don't know which OS you are using, but just checking on
>> [[https://github.com/jgm/pandoc/releases/1.15.1]] and
>> [[https://github.com/jgm/pandoc/releases/tag/1.15.1.1]]:
>>
>> Windows: 19.7 MB
>> Mac    : 27.9 MB
>> Deb    : 20.2 MB
>>
>> The 1600MB must be including LaTeX?
>
> Nope, but something like 100 haskell libraries and ghc.  I guess it’s not
> build in a static way?
>
> It’s much smaller on Debian, approx. 25Mb.  (I should switch to Fedora
> full-time).
>
>> In General, I like the idea of using the Pandoc approach, as Pandoc
>> provides a very useful framework for all kinds of conversions (and I
>> don't like java...)
>
> Your first argument refers to a side effect (to us), and it is irrelevant
> IMO.

I disagree. pandoc supports conversion to and from org-mode. Making
pandoc a requirement will enable other useful features (e.g., "Import
documents from...", alternative pandoc-based exporters etc. IMO pandoc
is easy on Windows and OSX. It is easy on some Linux distros but not
all. I use Arch LInux, where getting pandoc requires some work, but I
think that is an issue that the Linux distros need to work out. In
short, pandoc is extremely useful for org-mode users, and not only for
citations. I think pandoc is the way to go.

  Java run easily everywhere, easily, and if you use mathml for some
> of your documents, you’re likely already using a jar file with Org.
>
> That being said, it suggest that Pandoc /can/ be packed in a sensible way,
> which is good.
>
> Rasmus
>
> --
> A page of history is worth a volume of logic
>

Re: Some projects

[Rasmus Pank Roulund]

Hi,

Ista Zahn <istazahn@gmail.com> writes:

> I disagree. pandoc supports conversion to and from org-mode.

I fail to see how this is relevant for the discussion at hand.

> Making pandoc a requirement will enable other useful features (e.g.,
> "Import documents from...", alternative pandoc-based exporters etc.

AFAIK, nobody is working on this.

> IMO pandoc is easy on Windows and OSX. It is easy on some Linux distros
> but not all.

> I use Arch LInux, where getting pandoc requires some work, but I think
> that is an issue that the Linux distros need to work out.

No it doesn't: pacman -S pandoc.

The relevant question here is whether we can accept to rely on pandoc for
CSL support.  Any other feature is irrelevant.

At 25MB, a static, precompiled pandoc is probably fine, but for users of
some OSs, such as Archlinux or even worse some OS where pandoc is not
generally available as a precompiled package, it’s a really, really big
dependency.  I may still be the best option, though.

Rasmus

-- 
And when I’m finished thinking, I have to die a lot

Re: Some projects

[Ista Zahn]

[-- Attachment #1: Type: text/plain, Size: 1737 bytes --]

On Oct 27, 2015 11:09 AM, "Rasmus Pank Roulund" <rasmus@pank.eu> wrote:
>
> Hi,
>
> Ista Zahn <istazahn@gmail.com> writes:
>
> > I disagree. pandoc supports conversion to and from org-mode.
>
> I fail to see how this is relevant for the discussion at hand.
>
> > Making pandoc a requirement will enable other useful features (e.g.,
> > "Import documents from...", alternative pandoc-based exporters etc.
>
> AFAIK, nobody is working on this.

Not yet :-)
>
> > IMO pandoc is easy on Windows and OSX. It is easy on some Linux distros
> > but not all.
>
> > I use Arch LInux, where getting pandoc requires some work, but I think
> > that is an issue that the Linux distros need to work out.
>
> No it doesn't: pacman -S pandoc.

Good to know, thanks! That must be fairly recent.
>
> The relevant question here is whether we can accept to rely on pandoc for
> CSL support.  Any other feature is irrelevant.

That seems like a narrow way to look at it. Of course I agree that the
topic of discussion is citations, but I don't see why the bigger picture of
what the various options provide should be ignored.

I'll be happy to see improved citation support in org regardless of the
implementation details. But I do think pandoc deserves a serious look, and
I don't think the non-citation related possibilities it opens up are
irrelevant.

>
> At 25MB, a static, precompiled pandoc is probably fine, but for users of
> some OSs, such as Archlinux or even worse some OS where pandoc is not
> generally available as a precompiled package, it’s a really, really big
> dependency.  I may still be the best option, though.
>
> Rasmus
>
> --
> And when I’m finished thinking, I have to die a lot

[-- Attachment #2: Type: text/html, Size: 2218 bytes --]

Re: Some projects

[Rainer M Krug]

[-- Attachment #1: Type: text/plain, Size: 1823 bytes --]

Rasmus Pank Roulund <rasmus@pank.eu> writes:

> Hi,
>
> Ista Zahn <istazahn@gmail.com> writes:
>
>> I disagree. pandoc supports conversion to and from org-mode.
>
> I fail to see how this is relevant for the discussion at hand.

This is not really relevant for citations discussion itself, but having
pandoc will make it easier to use it for other interesting usages -
while the java option is as far as I understand citation only.
>
>> Making pandoc a requirement will enable other useful features (e.g.,
>> "Import documents from...", alternative pandoc-based exporters etc.
>
> AFAIK, nobody is working on this.

But it would be an interesting way for the future.

Rainer


>
>> IMO pandoc is easy on Windows and OSX. It is easy on some Linux distros
>> but not all.
>
>> I use Arch LInux, where getting pandoc requires some work, but I think
>> that is an issue that the Linux distros need to work out.
>
> No it doesn't: pacman -S pandoc.
>
> The relevant question here is whether we can accept to rely on pandoc for
> CSL support.  Any other feature is irrelevant.
>
> At 25MB, a static, precompiled pandoc is probably fine, but for users of
> some OSs, such as Archlinux or even worse some OS where pandoc is not
> generally available as a precompiled package, it’s a really, really big
> dependency.  I may still be the best option, though.
>
> Rasmus

-- 
Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys. (Germany)

Centre of Excellence for Invasion Biology
Stellenbosch University
South Africa

Tel :       +33 - (0)9 53 10 27 44
Cell:       +33 - (0)6 85 62 59 98
Fax :       +33 - (0)9 58 10 27 44

Fax (D):    +49 - (0)3 21 21 25 22 44

email:      Rainer@krugs.de

Skype:      RMkrug

PGP: 0x0F52F982

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]

Re: Some projects

[Matt Lundin]

Rasmus Pank Roulund <rasmus@pank.eu> writes:

>
> Ista Zahn <istazahn@gmail.com> writes:
>
>> IMO pandoc is easy on Windows and OSX. It is easy on some Linux distros
>> but not all.
>
>> I use Arch LInux, where getting pandoc requires some work, but I think
>> that is an issue that the Linux distros need to work out.
>
> No it doesn't: pacman -S pandoc.

> At 25MB, a static, precompiled pandoc is probably fine, but for users of
> some OSs, such as Archlinux or even worse some OS where pandoc is not
> generally available as a precompiled package, it’s a really, really big
> dependency.  I may still be the best option, though.

The best option on arch is probably pandoc-static in the AUR. It pulls
in the very large ghc as a build dependency. However, this can be
removed after the build. Still, pandoc-static is 118 MB on my machine.
(That's bigger than emacs itself).

In an ideal world, citeproc-js + node would be the most portable.
However, as others have pointed out, the lack of a good converter from
bibtex to citeproc's json format is a drawback. By contrast,
pandoc-citeproc offers both a converter and a CSL implementation.

Matt

Re: Some projects

[Richard Lawrence]

Hi Rasmus and all,

Rasmus <rasmus@gmx.us> writes:

> I would feel more comfortable relying on a JS library.  Perhaps it’s also
> easier to find people who are willing to work on/knows JS over the long
> haul...

Yes, I agree with both you and Aaron here.  JS is easier to distribute
and probably easier to find people to maintain.

>> OTOH, pandoc-citeproc includes a bibtex parser; we’d need to write a JS
>> one and wire it up to citeproc-js.  When I looked (quite some time ago),
>> there did not seem to be any good bibtex parsing libraries in JS (and
>> several third-rate ones).
>
> Bibtex support is essential, of course.
>
> Can someone remind me why citeproc-java isn’t good?  AFAIR, it has a
> bibtex parser.  But probably it lacks in some other dimension...

My recollection is that its command-line tool is not all that flexible,
and thus getting it to produce the kind of output we'd need requires
writing and maintaining Java, which no one really wanted to do.  Also, I
think the thought of firing up the JVM to process citations in an Org
document seemed a little...expensive.

Best,
Richard

Re: Some projects

[Matt Lundin]

Aaron Ecay <aaronecay@gmail.com> writes:
>
> - There is a difference between citations as done by latex/bibtex/etc.,
>   and those done in every other format (handled through CSL).  Assuming
>   latex users want to keep their native processing rather than
>   delegating to CSL, we need to solve the myriad small inconsistencies
>   between these two tools.  I think this is an area where it’s important
>   to get things right: users of citations generally have exacting
>   requirements.  “Approximately Chicago-style” or “almost MLA” aren’t
>   worth anything.

In my recollection, pandoc can juggle between CSL and biblatex. From the
pandoc manual:

,----
| --biblatex
| 
|    Use biblatex for citations in LaTeX output. This option is not for
|    use with the pandoc-citeproc filter or with PDF output. It is
|    intended for use in producing a LaTeX file that can be processed with
|    pdflatex and bibtex or biber.
`----

Perhaps someone with more experience of pandoc could assess how well it
toggles between CSL and biblatex, especially for more complicated styles
(e.g., Chicago). 

Matt

Re: Some projects

[Rasmus]

Matt Lundin <mdl@imapmail.org> writes:

> Perhaps someone with more experience of pandoc could assess how well it
> toggles between CSL and biblatex, especially for more complicated styles
> (e.g., Chicago). 

Can you point out an example of the expected format?  Bibtex.el does
pretty well and perhaps it’s trivial to transform an arbitrary bibtex key
into the json format on the Emacs side.

Rasmus

-- 
Got mashed potatoes. Ain't got no T-Bone. No T-Bone

Re: Some projects

[Rasmus]

Rasmus <rasmus@gmx.us> writes:

> Matt Lundin <mdl@imapmail.org> writes:
>
>> Perhaps someone with more experience of pandoc could assess how well it
>> toggles between CSL and biblatex, especially for more complicated styles
>> (e.g., Chicago). 
>
> Can you point out an example of the expected format?  Bibtex.el does
> pretty well and perhaps it’s trivial to transform an arbitrary bibtex key
> into the json format on the Emacs side.

Looking at,

    https://github.com/citation-style-language/schema/blob/master/csl-data.json

the main difficulty seems to be the author field, which is more "complex",
or perhaps just more sane, in the JSON equivalent.  It may be possible to
use bibtex-autokey-demangle-name to approximate a translation from bibtex
to JSON.

There are clearly other complex fields like "accessed", but I guess it
would rarely show up in an bib(la)tex entry.  But I have nothing to back
this claim up with.

Rasmus

-- 
When the facts change, I change my mind. What do you do, sir?

Re: Some projects

[Eric Abrahamsen]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Hello,
>
> I'd like to see some features moving forward, and some important issues
> fixed, hopefully, in the next months. I'm sharing them here so that
> anyone interested can help.
>

[...]

> * Important fixes
>
> ** Cache
>
> The cache needs to be fixed. Its underlying implementation probably
> needs to be changed, too. At the moment, it uses an AVL tree, which
> isn't much tolerant and results in a freeze whenever the cache is
> corrupted. Shifting to text properties could improve the situation.

I think we'll have to leave this one to you!

[...]

> ** Find a solution for orgstruct minor mode
>
> Org struct is really different from Org. It prevents Org from using
> better algorithms for outline navigation (see discussion about
> `org-show-children' on this list).
>
> I think Org struct should be removed from "org.el" (and org-footnote
> ...) and added in its own library. It should also be built on top of
> Outline mode instead of Org mode. Org, OTOH, should remove dependencies
> on Outline mode and implement better navigation functions.
>
> I have the feeling that Org struct will need to be rewritten almost from
> scratch in the process.

I've tried more than once over the past couple of years to attack
orgstruct mode, and been defeated every time. Largely because
indentation/filling is so damn complicated (too many layers of
indirection for my little brain), and orgstruct added yet another layer
of indirection on top of that. I think a rewrite would not be a bad
thing, and would be happy to participate.

[...]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

[...]

> The full proposal was at
> <http://permalink.gmane.org/gmane.emacs.orgmode/97468> but it didn't get
> much positive feedback. Therefore I suggested a simplified version, with
> a single author.

I may have been one of the lukewarm responders at the time, but now I
can't quite remember why. This looks good to me, and I'd like to pitch
in. I'm also +1 for inline annotations.

> Also, there is
>
>   https://github.com/girzel/org-annotate
>
> which is even simpler.

I'll be happy to rename/remove this repo when the time comes, to avoid
confusion!

The one thing I like about the package (indeed, it's what the bulk of
the code actually does) is the ability to "pop up" a single annotation's
text in a small, easily-dismissed special-mode buffer. Also, to display
all of a buffer's annotations in a tabulated-list pop-up buffer, where
you can easily jump to/edit/delete the various annotations.

Obviously Org footnotes (and future annotations) have jump there/jump
back capability, but for some reason I feel much more comfortable with
this style of browsing. If other people like this I could adapt the
existing code to work "on top of" footnotes and the future annotations
syntax, and that could be the whole point of the library. The base code
could remain entirely ignorant of the extra stuff.

------

These new features aside, is there any need to do any fundamental
refactoring? I mostly mean altering existing Org libraries to use the
element framework, stuff like that...

Eric

Re: Some projects

[Nicolas Goaziou]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> I've tried more than once over the past couple of years to attack
> orgstruct mode, and been defeated every time. Largely because
> indentation/filling is so damn complicated (too many layers of
> indirection for my little brain), and orgstruct added yet another layer
> of indirection on top of that. I think a rewrite would not be a bad
> thing, and would be happy to participate.

Basically, OrgStruct is Org with a very limited syntax, i.e., it only
recognizes headlines and lists. OrgStruct++ adds indentation and
filling.

I think OrgStruct may be redefined as an outline minor mode with Org
bindings. This is very different from Org. Even indentation and filling
should be new functions since Org's recognizes context that doesn't make
sense in OrgStruct.

> The one thing I like about the package (indeed, it's what the bulk of
> the code actually does) is the ability to "pop up" a single annotation's
> text in a small, easily-dismissed special-mode buffer. Also, to display
> all of a buffer's annotations in a tabulated-list pop-up buffer, where
> you can easily jump to/edit/delete the various annotations.
>
> Obviously Org footnotes (and future annotations) have jump there/jump
> back capability, but for some reason I feel much more comfortable with
> this style of browsing.

Have you tried C-c ' one a footnote reference? Unlike to C-c C-o, it
will pop-up a dedicated buffer for editing.

However, there is no global view of all footnotes. If it is useful, it
might be nice to add one and trigger it from, e.g., to
`org-footnote-action's menu.

> These new features aside, is there any need to do any fundamental
> refactoring? I mostly mean altering existing Org libraries to use the
> element framework, stuff like that...

Of course. Refactoring (e.g., replacing `setq' with `let'), adding code
comments, tests, using Element where applicable is always good. 

Regards,

Re: Some projects

[Eric Abrahamsen]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> I've tried more than once over the past couple of years to attack
>> orgstruct mode, and been defeated every time. Largely because
>> indentation/filling is so damn complicated (too many layers of
>> indirection for my little brain), and orgstruct added yet another layer
>> of indirection on top of that. I think a rewrite would not be a bad
>> thing, and would be happy to participate.
>
> Basically, OrgStruct is Org with a very limited syntax, i.e., it only
> recognizes headlines and lists. OrgStruct++ adds indentation and
> filling.

Right, I'd forgotten the distinction. I really only use orgstruct for
plain/numbered list editing (and I guess the occasional table), and
filling/indentation is pretty key there.

> I think OrgStruct may be redefined as an outline minor mode with Org
> bindings. This is very different from Org. Even indentation and filling
> should be new functions since Org's recognizes context that doesn't make
> sense in OrgStruct.

Would that mean that lists and tables wouldn't be supported? 

>> The one thing I like about the package (indeed, it's what the bulk of
>> the code actually does) is the ability to "pop up" a single annotation's
>> text in a small, easily-dismissed special-mode buffer. Also, to display
>> all of a buffer's annotations in a tabulated-list pop-up buffer, where
>> you can easily jump to/edit/delete the various annotations.
>>
>> Obviously Org footnotes (and future annotations) have jump there/jump
>> back capability, but for some reason I feel much more comfortable with
>> this style of browsing.
>
> Have you tried C-c ' one a footnote reference? Unlike to C-c C-o, it
> will pop-up a dedicated buffer for editing.

Hey, I didn't know about that. That's pretty nice! It's not *quite* as
no-brainer as the special-mode popups, but definitely close enough that
there's no need to write anything else.

> However, there is no global view of all footnotes. If it is useful, it
> might be nice to add one and trigger it from, e.g., to
> `org-footnote-action's menu.

I can adapt the code from org-annotate. The unfortunate thing about
tabulated list mode, which I otherwise really like, isn't able to show
multi-line list items. That really crimps its usefulness in showing
footnotes and annotations, since you can only see the first line.
Bummer.

>> These new features aside, is there any need to do any fundamental
>> refactoring? I mostly mean altering existing Org libraries to use the
>> element framework, stuff like that...
>
> Of course. Refactoring (e.g., replacing `setq' with `let'), adding code
> comments, tests, using Element where applicable is always good. 

Okay. I thought there might be some big chunk of Org that needed to be
shifted over on top of elements.

Eric

Re: Some projects

[Rasmus]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Right, I'd forgotten the distinction. I really only use orgstruct for
> plain/numbered list editing (and I guess the occasional table), and
> filling/indentation is pretty key there.

AFAIK, this doesn't work in commented lines (e.g., when using orgstruct
with init.el).

>> I think OrgStruct may be redefined as an outline minor mode with Org
>> bindings. This is very different from Org. Even indentation and filling
>> should be new functions since Org's recognizes context that doesn't make
>> sense in OrgStruct.
>
> Would that mean that lists and tables wouldn't be supported? 

ATM tables are supported via orgstruct++-mode, though at list basic list
support is provided by orgstruct-mode.

> The unfortunate thing about tabulated list mode, which I otherwise
> really like, isn't able to show multi-line list items. That really
> crimps its usefulness in showing footnotes and annotations, since you
> can only see the first line.  Bummer.

Did you open a bug for this?

>>> These new features aside, is there any need to do any fundamental
>>> refactoring? I mostly mean altering existing Org libraries to use the
>>> element framework, stuff like that...
>>
>> Of course. Refactoring (e.g., replacing `setq' with `let'), adding code
>> comments, tests, using Element where applicable is always good. 
>
> Okay. I thought there might be some big chunk of Org that needed to be
> shifted over on top of elements.

There’s plenty of stuff to refactor in org.el, org-agenda.el etc.

Rasmus

-- 
However beautiful the theory, you should occasionally look at the evidence

Re: Some projects

[Eric Abrahamsen]

Rasmus <rasmus@gmx.us> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Right, I'd forgotten the distinction. I really only use orgstruct for
>> plain/numbered list editing (and I guess the occasional table), and
>> filling/indentation is pretty key there.
>
> AFAIK, this doesn't work in commented lines (e.g., when using orgstruct
> with init.el).

That's something I've never tried. TBH, I only use orgstruct mode when
writing emails. But there I use it quite a bit.

>>> I think OrgStruct may be redefined as an outline minor mode with Org
>>> bindings. This is very different from Org. Even indentation and filling
>>> should be new functions since Org's recognizes context that doesn't make
>>> sense in OrgStruct.
>>
>> Would that mean that lists and tables wouldn't be supported? 
>
> ATM tables are supported via orgstruct++-mode, though at list basic list
> support is provided by orgstruct-mode.

Right, but my question is about a future reworking of the library.
Nicolas mentions refactoring it onto the outline mode, which AFAIK
doesn't do lists or tables.

>> The unfortunate thing about tabulated list mode, which I otherwise
>> really like, isn't able to show multi-line list items. That really
>> crimps its usefulness in showing footnotes and annotations, since you
>> can only see the first line.  Bummer.
>
> Did you open a bug for this?

I posted a query on emacs.help, prior to opening a bug :)

>>>> These new features aside, is there any need to do any fundamental
>>>> refactoring? I mostly mean altering existing Org libraries to use the
>>>> element framework, stuff like that...
>>>
>>> Of course. Refactoring (e.g., replacing `setq' with `let'), adding code
>>> comments, tests, using Element where applicable is always good. 
>>
>> Okay. I thought there might be some big chunk of Org that needed to be
>> shifted over on top of elements.
>
> There’s plenty of stuff to refactor in org.el, org-agenda.el etc.

I was fishing for something more specific, but if there aren't any
desperate necessities, I guess that's a good thing. I'll take a look.

Marcin Borkowski <mbork@mbork.pl> writes:

> On 2015-10-27, at 19:53, Eric Abrahamsen <eric@ericabrahamsen.net> wrote:
>
>> I can adapt the code from org-annotate. The unfortunate thing about
>> tabulated list mode, which I otherwise really like, isn't able to show
>> multi-line list items. That really crimps its usefulness in showing
>> footnotes and annotations, since you can only see the first line.
>> Bummer.
>
> BTW: have you looked at EWOC?
>
> http://mbork.pl/2015-07-18_TLM_vs_EWOC,_or_there_and_back_again

I had not, and that was really interesting! Thanks for the link. On
balance, I'd like TLM to support this, but we'll see what people think.

E

Re: Some projects

[Marcin Borkowski]

On 2015-10-27, at 19:53, Eric Abrahamsen <eric@ericabrahamsen.net> wrote:

> I can adapt the code from org-annotate. The unfortunate thing about
> tabulated list mode, which I otherwise really like, isn't able to show
> multi-line list items. That really crimps its usefulness in showing
> footnotes and annotations, since you can only see the first line.
> Bummer.

BTW: have you looked at EWOC?

http://mbork.pl/2015-07-18_TLM_vs_EWOC,_or_there_and_back_again

Hth,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: Some projects

[Nicolas Goaziou]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>
>> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>>
>>> I've tried more than once over the past couple of years to attack
>>> orgstruct mode, and been defeated every time. Largely because
>>> indentation/filling is so damn complicated (too many layers of
>>> indirection for my little brain), and orgstruct added yet another layer
>>> of indirection on top of that. I think a rewrite would not be a bad
>>> thing, and would be happy to participate.
>>
>> Basically, OrgStruct is Org with a very limited syntax, i.e., it only
>> recognizes headlines and lists. OrgStruct++ adds indentation and
>> filling.
>
> Right, I'd forgotten the distinction. I really only use orgstruct for
> plain/numbered list editing (and I guess the occasional table), and
> filling/indentation is pretty key there.
>
>> I think OrgStruct may be redefined as an outline minor mode with Org
>> bindings. This is very different from Org. Even indentation and filling
>> should be new functions since Org's recognizes context that doesn't make
>> sense in OrgStruct.
>
> Would that mean that lists and tables wouldn't be supported?

Tables are handled with OrgTbl minor mode, which is unrelated to
OrgStruct mode.

Lists would require slightly more work, but should be supported in
OrgStruct mode. It implies to advise/hook Some functions from
org-list.el (e.g., to avoid checking about context besides other items
and headlines).

Regards,

Re: Some projects

[Kaushal Modi]

Hi Nicolas,

Thanks for putting up this list.

> ** Backslash escaping

> Allowing to escape some symbols in plain text (e.g., emphasis markers,
> square brackets...) would remove a limitation in verbatim/code objects.
> As a small benefit, it would also permit to implement mid-word markup:
> b*o*ld.

> There are some gotchas, however.

I am definitely interested in a solution to this (based on a related
discussion we had here earlier):
http://www.mail-archive.com/emacs-orgmode@gnu.org/msg100527.html
I'd happy to contribute to it as I can, but I don't know where to
start. But as soon as I see some activity related to backslash
escaping, I'd happily be involved in the code development.

--
Kaushal Modi