gnowsys-mode a major mode extending org

gnowsys-mode a major mode extending org

[Nagarjuna G.]

Dear orgmode developers and users

This is a note to share with you our adventure with org.

We, at gnowledge.org, began a project that extends orgmode (without
spoiling or loosing any of the orgmode features) to support
collaborative knowledge organization and networking using the GNOWSYS
storage on the server side.  This is work in progress.  I am writing
this note to inform you about the on going work, as well as to thank
you all, specially Carsten for creating an app that took Emacs to yet
another level.

* What we are doing with orgmode: 

  1. We use Emacs as a client to collaboratively create, update
     knowledge networks in plain text.  The knowledge network is
     created by adding nodes (rendered in orgmode as a note item) and
     named relations between them (rendered in orgmode in the
     properties drawer of the note item). The storage of the network
     is compatible to RDF model of storing networks, though we do not
     store or process XML either on the server side or on the client
     side. Thus, every orgmode item is added in a form that can be
     serialized as a RDF triple.  Some examples given below.

  2. We are using xmlrpc (xmlrpc.el), a builtin feature in Emacs, to
     connect to GNOWSYS server. GNOWSYS publishes its API as
     webservice (which has an API to perform the authentication,
     search the knowledge base, return and update the nodes and its
     properties.)

  3. Each node when returned from the GNOWSYS server to the client
     opens in a org buffer (temporary) as a note item.  Since we
     render the linkeddata in a text format orgmode renders everything
     nicely.  We have defined more types of links, extending the ones
     available already, so that the linked nodes' URI is always
     available.

  4. Any text written below the drawers is stored as the value of an
     attribute 'content' of the current node on the server side.

  5. We are adding more and more functions to complete the entire
     application, thanks to orgmode and Emacs, where difficult tasks
     are tamed to such an extent, we are obviously excited!

* What missing feature of orgmode will be available in gnowsys-mode

  We can do networking between items making frame based orgmode items
  linked with any other items in any which way by explicitly naming
  the relations.  This feature makes orgmode ready for semantic
  web. We plan to use the properties drawer to hold all the metadata
  of each node.  In-built version control of all notes as well as the
  networks (a feature of GNOWSYS).  Orgmode with the extensions that
  we are applying can demonstrate how complex application data can be
  represented and manipulated in plain text including semantic web.

* What this extension may mean to the users of orgmode and others.

  These are the use cases for extended orgmode/gnowsys-mode:

  1. Each note item (*) can be published at a remote server as a node,
     with its content and properties stored as attributes.  Thus all
     data gets published on the server, though optionally the user can
     save all data as text files on his/her PC. Published nodes
     can be read and edited by others, allowing collaborative
     authoring.  Using this feature, we can use orgmode as a wiki
     source file.

  2. GNOWSYS supports version control of all changes made to the node,
     thus changes can be tracked. No additional version control like
     cvs, svn, git, bazaar etc. are required.  (GNOWSYS should soon
     support more functions to improve this.) Nothing is deleted from
     the server. Nodes can be delinked or marked deleted, nodes with
     deleted mark still exist on the server after making a new
     snapshot of the node.

  3. Ontologies or application schema can be made and published
     collaboratively.  This is useful as a semantic web
     application. All this can be done without using XML on either
     server side or client side, though we can import and export data
     serialized in RDF format.  This makes it a pure text based app
     for semantic web.

  4. Complex documents like courses, manuals, books can be written in
     orgmode collaboratively with bibliogrpahy support etc.  LaTeX and
     DocBook export feature could make use of this to make orgmode as
     a desktop publishing system supporting collaborative editing. (We
     want to discuss this project with guidance from org team.)

  5. Considering that a website is nothing but a set of complex
     documents with internal as well as external hyperlinks, complete
     website can be made, managed and published by orgmode with the
     source code of the site completely encoded in text.

  6. GNOWSYS is made for collaborative creation of semantic networks,
     so this use case needs no more elaboration.  Concept graphs,
     mindmaps included. 


We realize that much of the above use needs a good and advanced search
facility on the GNOWSYS side.  Though crude search is possible now,
and integrated within gnowsys-mode, a lot of improvement is possible
here.

The bleeding-edge gnowsys-mode can be obtained using "svn co
svn://svn.savannah.gnu.org/gnowsys/gnowsys-mode". 

Divya (a student hacker) did this project as her GSoC 2009 under Rajiv
and me as mentors. She keeps on updating with more and more functions
regularly.  We will feel encouraged if some of you could try this and
send us feedback. As first time emacs-lisp hackers our code may not be
written elegantly.  If you detect blunders do let us know.  We already
learned a lot from the existing examples.

gnowsys-mode will be demonstrated at ISWC 2009 to be held this October
near Washington for the semantic web community.  If any of you are
aruond Washington/Virginia I would be glad to meetup.

Thanks again for org!

Nagarjuna

Re: gnowsys-mode a major mode extending org

[Samuel Wales]

Hi Nagarjuna,

On Tue, Sep 29, 2009 at 17:19, Nagarjuna G. <nagarjun@gnu.org> wrote:
>  We can do networking between items making frame based orgmode items
>  linked with any other items in any which way by explicitly naming
>  the relations.  This feature makes orgmode ready for semantic
>  web. We plan to use the properties drawer to hold all the metadata
>  of each node.  In-built version control of all notes as well as the
>  networks (a feature of GNOWSYS).  Orgmode with the extensions that
>  we are applying can demonstrate how complex application data can be
>  represented and manipulated in plain text including semantic web.

I haven't posted most of my related ideas yet, but I proposed
something similar (a prerequisite for it) called ID markers, in
http://www.mail-archive.com/emacs-orgmode@gnu.org/msg11845.html .  I
still need to reply to the replies, and will do so at some point.  I
also need to motivate one of the goals,for ID markers, which is
graph-theoretic graphs (in particular robust bidirectional links) for
personal use.

You can have any place (which can be loosely defined) be a node
(including being a source and a target of arcs) merely by putting an
ID marker there.  This frees you from needing nodes to be entries with
properties.

Don't know if you can use them or not.

I do hope that whatever ideas you implement can be put into contrib or
the core as general-purpose facilities for org.

-- 
Myalgic encephalomyelitis causes death (Jason et al. 2006)
and severe suffering.  Conflicts of interest are destroying
research.  What people "know" is wrong.  Silence = death.
http://www.meactionuk.org.uk/What_Is_ME_What_Is_CFS.htm

Re: gnowsys-mode a major mode extending org

[Samuel Wales]

By "something similar" I just mean personal org stuff -- not the
semantic web.  If org entries are sufficient for your nodes, then you
probably don't need ID markers.

I am curious to see where you go with your project.

Re: gnowsys-mode a major mode extending org

[Nagarjuna G]

On Wed, Sep 30, 2009 at 5:49 AM, Nagarjuna G. <nagarjun@gnu.org> wrote:
>
>  1. We use Emacs as a client to collaboratively create, update
>     knowledge networks in plain text.  The knowledge network is
>     created by adding nodes (rendered in orgmode as a note item) and
>     named relations between them (rendered in orgmode in the
>     properties drawer of the note item). The storage of the network
>     is compatible to RDF model of storing networks, though we do not
>     store or process XML either on the server side or on the client
>     side. Thus, every orgmode item is added in a form that can be
>     serialized as a RDF triple.  Some examples given below.
>

I forgot to add examples in the post.  We created a library of
screencasts here:
http://lab.gnowledge.org/download/gnowsys-mode-screencasts

Nagarjuna

Re: gnowsys-mode a major mode extending org

[Nagarjuna G.]

On Wed, Sep 30, 2009 at 6:53 AM, Samuel Wales <samologist@gmail.com> wrote:
>
> I haven't posted most of my related ideas yet, but I proposed
> something similar (a prerequisite for it) called ID markers, in
> http://www.mail-archive.com/emacs-orgmode@gnu.org/msg11845.html .  I
> still need to reply to the replies, and will do so at some point.  I
> also need to motivate one of the goals,for ID markers, which is
> graph-theoretic graphs (in particular robust bidirectional links) for
> personal use.
>

Indeed, I read the posts on the archive.  If we have a mechanism by
which each note/node has a URI it is possible to draw graphs.

I may have forgotten to add, we do draw graphs of each node using
graphviz library in SVG and show it in browser.  The graph file is
generated and its link provided in the org buffer.

> You can have any place (which can be loosely defined) be a node
> (including being a source and a target of arcs) merely by putting an
> ID marker there.  This frees you from needing nodes to be entries with
> properties.
>

Isn't adding an ID marker same as creating node with properties.  It
will be nice to make it as readable as possible using text, and also
make the network navigable for human beings.

>
> I do hope that whatever ideas you implement can be put into contrib or
> the core as general-purpose facilities for org.
>

Which algo did you use for generating unique IDs?

Nagarjuna

Re: gnowsys-mode a major mode extending org

[Carsten Dominik]

Hi Nagarjuna,

This sound very interesting indeed.  Thanks for sharing!

- Carsten

On Sep 30, 2009, at 2:19 AM, Nagarjuna G. wrote:

> Dear orgmode developers and users
>
> This is a note to share with you our adventure with org.
>
> We, at gnowledge.org, began a project that extends orgmode (without
> spoiling or loosing any of the orgmode features) to support
> collaborative knowledge organization and networking using the GNOWSYS
> storage on the server side.  This is work in progress.  I am writing
> this note to inform you about the on going work, as well as to thank
> you all, specially Carsten for creating an app that took Emacs to yet
> another level.
>
> * What we are doing with orgmode:
>
>  1. We use Emacs as a client to collaboratively create, update
>     knowledge networks in plain text.  The knowledge network is
>     created by adding nodes (rendered in orgmode as a note item) and
>     named relations between them (rendered in orgmode in the
>     properties drawer of the note item). The storage of the network
>     is compatible to RDF model of storing networks, though we do not
>     store or process XML either on the server side or on the client
>     side. Thus, every orgmode item is added in a form that can be
>     serialized as a RDF triple.  Some examples given below.
>
>  2. We are using xmlrpc (xmlrpc.el), a builtin feature in Emacs, to
>     connect to GNOWSYS server. GNOWSYS publishes its API as
>     webservice (which has an API to perform the authentication,
>     search the knowledge base, return and update the nodes and its
>     properties.)
>
>  3. Each node when returned from the GNOWSYS server to the client
>     opens in a org buffer (temporary) as a note item.  Since we
>     render the linkeddata in a text format orgmode renders everything
>     nicely.  We have defined more types of links, extending the ones
>     available already, so that the linked nodes' URI is always
>     available.
>
>  4. Any text written below the drawers is stored as the value of an
>     attribute 'content' of the current node on the server side.
>
>  5. We are adding more and more functions to complete the entire
>     application, thanks to orgmode and Emacs, where difficult tasks
>     are tamed to such an extent, we are obviously excited!
>
> * What missing feature of orgmode will be available in gnowsys-mode
>
>  We can do networking between items making frame based orgmode items
>  linked with any other items in any which way by explicitly naming
>  the relations.  This feature makes orgmode ready for semantic
>  web. We plan to use the properties drawer to hold all the metadata
>  of each node.  In-built version control of all notes as well as the
>  networks (a feature of GNOWSYS).  Orgmode with the extensions that
>  we are applying can demonstrate how complex application data can be
>  represented and manipulated in plain text including semantic web.
>
> * What this extension may mean to the users of orgmode and others.
>
>  These are the use cases for extended orgmode/gnowsys-mode:
>
>  1. Each note item (*) can be published at a remote server as a node,
>     with its content and properties stored as attributes.  Thus all
>     data gets published on the server, though optionally the user can
>     save all data as text files on his/her PC. Published nodes
>     can be read and edited by others, allowing collaborative
>     authoring.  Using this feature, we can use orgmode as a wiki
>     source file.
>
>  2. GNOWSYS supports version control of all changes made to the node,
>     thus changes can be tracked. No additional version control like
>     cvs, svn, git, bazaar etc. are required.  (GNOWSYS should soon
>     support more functions to improve this.) Nothing is deleted from
>     the server. Nodes can be delinked or marked deleted, nodes with
>     deleted mark still exist on the server after making a new
>     snapshot of the node.
>
>  3. Ontologies or application schema can be made and published
>     collaboratively.  This is useful as a semantic web
>     application. All this can be done without using XML on either
>     server side or client side, though we can import and export data
>     serialized in RDF format.  This makes it a pure text based app
>     for semantic web.
>
>  4. Complex documents like courses, manuals, books can be written in
>     orgmode collaboratively with bibliogrpahy support etc.  LaTeX and
>     DocBook export feature could make use of this to make orgmode as
>     a desktop publishing system supporting collaborative editing. (We
>     want to discuss this project with guidance from org team.)
>
>  5. Considering that a website is nothing but a set of complex
>     documents with internal as well as external hyperlinks, complete
>     website can be made, managed and published by orgmode with the
>     source code of the site completely encoded in text.
>
>  6. GNOWSYS is made for collaborative creation of semantic networks,
>     so this use case needs no more elaboration.  Concept graphs,
>     mindmaps included.
>
>
> We realize that much of the above use needs a good and advanced search
> facility on the GNOWSYS side.  Though crude search is possible now,
> and integrated within gnowsys-mode, a lot of improvement is possible
> here.
>
> The bleeding-edge gnowsys-mode can be obtained using "svn co
> svn://svn.savannah.gnu.org/gnowsys/gnowsys-mode".
>
> Divya (a student hacker) did this project as her GSoC 2009 under Rajiv
> and me as mentors. She keeps on updating with more and more functions
> regularly.  We will feel encouraged if some of you could try this and
> send us feedback. As first time emacs-lisp hackers our code may not be
> written elegantly.  If you detect blunders do let us know.  We already
> learned a lot from the existing examples.
>
> gnowsys-mode will be demonstrated at ISWC 2009 to be held this October
> near Washington for the semantic web community.  If any of you are
> aruond Washington/Virginia I would be glad to meetup.
>
> Thanks again for org!
>
> Nagarjuna
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Remember: use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: gnowsys-mode a major mode extending org

[Scot Becker]

This sounds great!  I have been wanting the idea of 'hard linked'
nodes in org (whose existence would be equally in hierarchy A and
hierarchy B).  And it sounds like this may provide that (along with
much more).

Re: gnowsys-mode a major mode extending org

[Samuel Wales]

On Tue, Sep 29, 2009 at 21:49, Nagarjuna G. <nagarjun@gnowledge.org> wrote:
> Isn't adding an ID marker same as creating node with properties.  It

No.

In this case Node = graph theory and Entry = org headline.

ID markers turn non-entries into nodes.

Never mind; it was just something i thought you might want.