most robust linking practices?

most robust linking practices?

[John Kitchin]

[-- Attachment #1: Type: text/plain, Size: 1688 bytes --]

Hi all,

I am using org-mode in a multiuser environment, (i.e. many people have
access to the org-files). I create org-files with links in them to other
files, and I am trying to find the most robust way to do that.

For example, in one file in section I type C-c l to store a link, and then
later C-c-C-l to insert it in another file. That link looks like this:

[[file:~/dft-book/dft.org::*Introduction%20to%20DFT][Introduction to DFT]]

It works for me, but not for other users, because of the ~ in it.

I have tried using org-id, with mixed results.  I set this up in my init
file

;; automatically create ids for links
(require 'org-id)
(setq org-id-link-to-org-use-id 'create-if-interactive-and-no-custom-id)


Now, when C-c l is typed, it creates a unique id in the heading, and the
link looks like this:
[[id:065443d5-59d7-4119-b530-7b63af28349b][Background]]

I haven't figured out a detail though. If the original file is not open,
org-mode does not seem to find it when I click on it.

Am I missing some setup for org-id? I can see here
http://orgmode.org/worg/org-api/org-id-api.html that there is some concept
of a database of ids, but I didn't see anything about using it.

How would another user click on that id link and get to the file if they
didn't have the database?

Finally, the end goal here is to package a set of interlinked org-files
that someone else would use as a standalone package. What is the best link
strategy for that?

Thanks!

John

-----------------------------------
John Kitchin
Associate Professor
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
http://kitchingroup.cheme.cmu.edu

[-- Attachment #2: Type: text/html, Size: 2181 bytes --]

Re: most robust linking practices?

[Nick Dokos]

John Kitchin <jkitchin@andrew.cmu.edu> writes:

> Hi all,
>
> I am using org-mode in a multiuser environment, (i.e. many people have access to the org-files). I create org-files with links in
> them to other files, and I am trying to find the most robust way to do that.
>
> For example, in one file in section I type C-c l to store a link, and then later C-c-C-l to insert it in another file. That link
> looks like this:
>
> [[file:~/dft-book/dft.org::*Introduction%20to%20DFT][Introduction to DFT]]
>
> It works for me, but not for other users, because of the ~ in it.
>

How do the other users have access to this file? Is it in a shared
filesystem? Are *all* the files you want to share in a shared filesystem?
Does everybody have write access or are they read-only?

There is org-link-file-path-type which can be set to noabbrev to use
links with absolute paths (without ~). That would work in a single
namespace but not e.g. if everybody mounts some shared FS over NFS
and uses a different mount point. Relative paths would work better
in that case.

> I have tried using org-id, with mixed results.  I set this up in my init file
>
> ;; automatically create ids for links
> (require 'org-id)
> (setq org-id-link-to-org-use-id 'create-if-interactive-and-no-custom-id)
>
> Now, when C-c l is typed, it creates a unique id in the heading, and the link looks like this:
> [[id:065443d5-59d7-4119-b530-7b63af28349b][Background]]
>
> I haven't figured out a detail though. If the original file is not
> open, org-mode does not seem to find it when I click on it.
>

In the same emacs process or a different one?
I haven't seen this but the last time I used IDs was some years ago
(but see below).

> Am I missing some setup for org-id? I can see here
> http://orgmode.org/worg/org-api/org-id-api.html that there is some
> concept of a database of ids, but I didn't see anything about using
> it.
>
> How would another user click on that id link and get to the file if they didn't have the database?
>

The id database is kept in a file:

,----
| org-id-locations-file is a variable defined in `org-id.el'.
| Its value is "~/.emacs.d/.org-id-locations"
| 
| Documentation:
| The file for remembering in which file an ID was defined.
| This variable is only relevant when `org-id-track-globally' is set.
| 
`----

so it would have to be in a shared place for others to use. But it seems
that writing this file out is racy. It can be made read-only of course
but you would not be able to create new links.

The problem is that as you create links the id locations are kept in a
variable org-id-locations in memory. The value of the variable is saved
to the file when emacs exits and when org-id-find is called and cannot
find the id (I think), or you eval

     (org-id-locations-save)

explicitly.

In particular, if the database file is up-to-date, then starting another
emacs and following an id-link works whether the target file is already
visited or not. Maybe what you are seeing is this discrepancy.

> Finally, the end goal here is to package a set of interlinked
> org-files that someone else would use as a standalone package. What is
> the best link strategy for that?

My guess would be relative file links: all the files are in the
hieararchy under a single directory and all the file links are limited
to point strictly within the hierarchy, using relative pathnames.

Nick

Re: most robust linking practices?

[John Kitchin]

[-- Attachment #1: Type: text/plain, Size: 4345 bytes --]

Thanks for the clarifying questions.

The files are all on a unix file system served over nfs, so everyone has
the same / root. the users (students) have read access to my files.

I am working towards creating "packages" of notes in org-mode (they might
even be installed as emacs packages) for the courses that I teach. Having
relative paths within a package certainly makes sense. I would like to link
to notes in other packages too, as the courses are related, and build on
each other. but I won't know in advance where those get installed. It
sounds like those packages will have to have some variables configured to
make that work out.

Thanks for the tips on tweaking link formats and id behavior!

j

On Thu, Jan 16, 2014 at 5:32 PM, Nick Dokos <ndokos@gmail.com> wrote:

> John Kitchin <jkitchin@andrew.cmu.edu> writes:
>
> > Hi all,
> >
> > I am using org-mode in a multiuser environment, (i.e. many people have
> access to the org-files). I create org-files with links in
> > them to other files, and I am trying to find the most robust way to do
> that.
> >
> > For example, in one file in section I type C-c l to store a link, and
> then later C-c-C-l to insert it in another file. That link
> > looks like this:
> >
> > [[file:~/dft-book/dft.org::*Introduction%20to%20DFT][Introduction to
> DFT]]
> >
> > It works for me, but not for other users, because of the ~ in it.
> >
>
> How do the other users have access to this file? Is it in a shared
> filesystem? Are *all* the files you want to share in a shared filesystem?
> Does everybody have write access or are they read-only?
>
> There is org-link-file-path-type which can be set to noabbrev to use
> links with absolute paths (without ~). That would work in a single
> namespace but not e.g. if everybody mounts some shared FS over NFS
> and uses a different mount point. Relative paths would work better
> in that case.
>
> > I have tried using org-id, with mixed results.  I set this up in my init
> file
> >
> > ;; automatically create ids for links
> > (require 'org-id)
> > (setq org-id-link-to-org-use-id 'create-if-interactive-and-no-custom-id)
> >
> > Now, when C-c l is typed, it creates a unique id in the heading, and the
> link looks like this:
> > [[id:065443d5-59d7-4119-b530-7b63af28349b][Background]]
> >
> > I haven't figured out a detail though. If the original file is not
> > open, org-mode does not seem to find it when I click on it.
> >
>
> In the same emacs process or a different one?
> I haven't seen this but the last time I used IDs was some years ago
> (but see below).
>
> > Am I missing some setup for org-id? I can see here
> > http://orgmode.org/worg/org-api/org-id-api.html that there is some
> > concept of a database of ids, but I didn't see anything about using
> > it.
> >
> > How would another user click on that id link and get to the file if they
> didn't have the database?
> >
>
> The id database is kept in a file:
>
> ,----
> | org-id-locations-file is a variable defined in `org-id.el'.
> | Its value is "~/.emacs.d/.org-id-locations"
> |
> | Documentation:
> | The file for remembering in which file an ID was defined.
> | This variable is only relevant when `org-id-track-globally' is set.
> |
> `----
>
> so it would have to be in a shared place for others to use. But it seems
> that writing this file out is racy. It can be made read-only of course
> but you would not be able to create new links.
>
> The problem is that as you create links the id locations are kept in a
> variable org-id-locations in memory. The value of the variable is saved
> to the file when emacs exits and when org-id-find is called and cannot
> find the id (I think), or you eval
>
>      (org-id-locations-save)
>
> explicitly.
>
> In particular, if the database file is up-to-date, then starting another
> emacs and following an id-link works whether the target file is already
> visited or not. Maybe what you are seeing is this discrepancy.
>
> > Finally, the end goal here is to package a set of interlinked
> > org-files that someone else would use as a standalone package. What is
> > the best link strategy for that?
>
> My guess would be relative file links: all the files are in the
> hieararchy under a single directory and all the file links are limited
> to point strictly within the hierarchy, using relative pathnames.
>
> Nick
>
>
>

[-- Attachment #2: Type: text/html, Size: 5491 bytes --]

Re: most robust linking practices?

[Brett Viren]

[-- Attachment #1: Type: text/plain, Size: 1976 bytes --]

Hi John,

John Kitchin <jkitchin@andrew.cmu.edu> writes:

> The files are all on a unix file system served over nfs, so everyone
> has the same / root. the users (students) have read access to my
> files.
>
> I am working towards creating "packages" of notes in org-mode (they
> might even be installed as emacs packages) for the courses that I
> teach. Having relative paths within a package certainly makes sense. I
> would like to link to notes in other packages too, as the courses are
> related, and build on each other. but I won't know in advance where
> those get installed. It sounds like those packages will have to have
> some variables configured to make that work out.

How about defining a slew of links in org-link-abbrev-list.  Say, one
for each set of class notes.

Maybe you'd maintain two copies of such a list, one that assumes your
shared file system is being used and one that assumes some layout
convention in the user's home directory.  Your users could pick the best
one or use them as a starting point for their own customization.

The fact that the link definitions may contain inline lisp functions may
help to organize this.

I guess you would need some way to update your reader/user's copy of the
list as it evolves.  Immediately, I don't have any ideas about that.

I've started to use this approach a little.  So far, just to reference
some common external links.  Here's my setup:

#+BEGIN_SRC elisp
;; Custom external links
;; http://orgmode.org/manual/Adding-hyperlink-types.html#Adding-hyperlink-types
;; http://orgmode.org/manual/Link-abbreviations.html#Link-abbreviations
(defun bv-link-resolve-github (tag)
  (replace-regexp-in-string ":" "/blob/master/" tag))

(setq org-link-abbrev-alist
      '(
        ("ghsite" . "https://github.com/brettviren/%h")
        ("ghfile" . "https://github.com/brettviren/%(bv-link-resolve-github)")
        ("dbtrac" . "http://dayabay.ihep.ac.cn/tracs/dybsvn/ticket/%h")
        ))
#+END_SRC

-Brett.

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: most robust linking practices?

[Nick Dokos]

John Kitchin <jkitchin@andrew.cmu.edu> writes:

> The files are all on a unix file system served over nfs, so everyone
> has the same / root. the users (students) have read access to my
> files.
>
> I am working towards creating "packages" of notes in org-mode (they
> might even be installed as emacs packages) for the courses that I
> teach. Having relative paths within a package certainly makes sense. I
> would like to link to notes in other packages too, as the courses are
> related, and build on each other. but I won't know in advance where
> those get installed. It sounds like those packages will have to have
> some variables configured to make that work out.
>

IIUC, everybody sees the same namespace (students in read-only mode, you
in rw: but /a/b/c/foo.org is the same file for everybody). If that's the
case, then all methods (absolute or relative pathnames and/or ids)
should work, no?

I'd still do relative pathnames for individual "packages". For
interpackage links, you might want to do a sort of double
indirection[fn:1]: for each package, create a directory with a symlink
farm where the symlinks point off-package:

    package1: /p1/c/foo.org contains an org link to file:../farm/bar.org
              /p1/farm/bar.org -> /p2/c/bar.org (-> means symlink)

    package2: /p2/c/bar.org

It should be possible to construct the symlink farms mechanically (and if
not, see [fn:1] :-).) Assuming that the indirections are not too onerous, it
should be possible to arrange things so that installation consists of
setting one symlink in each package:


    package1: /p1/c/foo.org contains an org link to file:../farm/bar.org
              /p1/farm/bar.org -> ./bar/bar.org
              /p1/farm/bar -> /p2/c

    package2: /p2/c/bar.org
 
Only /p1/farm/bar needs to be adjusted.

Or just go whole-hog with ids (but take good care of the id file: double
and triple backups would not be excessive imo). Since only you can
modify the file, it should work OK.

Footnotes:

[fn:1] http://en.wikipedia.org/wiki/Indirection - the David Wheeler
       quote :-)

Nick

Re: most robust linking practices?

[John Kitchin]

[-- Attachment #1: Type: text/plain, Size: 2919 bytes --]

thanks for the ideas.

I put together a new kind of link that takes you to files inside of emacs
packages, or to paths relative to where a library is installed.

http://kitchingroup.cheme.cmu.edu/blog/2014/01/19/Making-org-mode-links-to-files-in-Emacs-packages/

basically I find where the library or package is installed, and then
construct an org-link relative to that.

from the tests at the post above, it seems to work pretty well!

John

-----------------------------------
John Kitchin
Associate Professor
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
http://kitchingroup.cheme.cmu.edu



On Fri, Jan 17, 2014 at 10:47 AM, Nick Dokos <ndokos@gmail.com> wrote:

> John Kitchin <jkitchin@andrew.cmu.edu> writes:
>
> > The files are all on a unix file system served over nfs, so everyone
> > has the same / root. the users (students) have read access to my
> > files.
> >
> > I am working towards creating "packages" of notes in org-mode (they
> > might even be installed as emacs packages) for the courses that I
> > teach. Having relative paths within a package certainly makes sense. I
> > would like to link to notes in other packages too, as the courses are
> > related, and build on each other. but I won't know in advance where
> > those get installed. It sounds like those packages will have to have
> > some variables configured to make that work out.
> >
>
> IIUC, everybody sees the same namespace (students in read-only mode, you
> in rw: but /a/b/c/foo.org is the same file for everybody). If that's the
> case, then all methods (absolute or relative pathnames and/or ids)
> should work, no?
>
> I'd still do relative pathnames for individual "packages". For
> interpackage links, you might want to do a sort of double
> indirection[fn:1]: for each package, create a directory with a symlink
> farm where the symlinks point off-package:
>
>     package1: /p1/c/foo.org contains an org link to file:../farm/bar.org
>               /p1/farm/bar.org -> /p2/c/bar.org (-> means symlink)
>
>     package2: /p2/c/bar.org
>
> It should be possible to construct the symlink farms mechanically (and if
> not, see [fn:1] :-).) Assuming that the indirections are not too onerous,
> it
> should be possible to arrange things so that installation consists of
> setting one symlink in each package:
>
>
>     package1: /p1/c/foo.org contains an org link to file:../farm/bar.org
>               /p1/farm/bar.org -> ./bar/bar.org
>               /p1/farm/bar -> /p2/c
>
>     package2: /p2/c/bar.org
>
> Only /p1/farm/bar needs to be adjusted.
>
> Or just go whole-hog with ids (but take good care of the id file: double
> and triple backups would not be excessive imo). Since only you can
> modify the file, it should work OK.
>
> Footnotes:
>
> [fn:1] http://en.wikipedia.org/wiki/Indirection - the David Wheeler
>        quote :-)
>
> Nick
>
>
>

[-- Attachment #2: Type: text/html, Size: 4483 bytes --]