[BUG] bug in org-publish and a (wrong) patch

[BUG] bug in org-publish and a (wrong) patch

[Nick Dokos]

org-publish-cache-ctime-of-src tries (but does not always succeed) to
deal with symlinks: file-symlink-p returns the target as a string, but
if the target is relative to the symlink, that's not going to fly.
e.g. if c is a symlink like this

    /a/b/c->../d/f

then (file-symlink-p "/a/b/c") -> "../d/f"
but if the current directory is any place other than /a/b, the target
will not be found, the file attributes are going to be nil and
the function will blow up.

Here is a patch born of about 5 mins of contemplation. It solved my
immediate problem but it is certainly wrong. It breaks absolute targets
(which I think are handled correctly by the original version). I'm not
even sure that it correctly handles *all* relative targets. It also
needs to treat the case of a non-existent symlink target (where
file-symlink-p returns t).

It might be safer also to check if the file attributes are
nil and deal with that, instead of blowing up.

--8<---------------cut here---------------start------------->8---
diff --git a/lisp/org-publish.el b/lisp/org-publish.el
index e944eea..dd192d6 100644
--- a/lisp/org-publish.el
+++ b/lisp/org-publish.el
@@ -1150,7 +1150,7 @@ Returns value on success, else nil."
 (defun org-publish-cache-ctime-of-src (filename)
   "Get the FILENAME ctime as an integer."
   (let ((src-attr (file-attributes (if (stringp (file-symlink-p filename))
-				       (file-symlink-p filename)
+				       (concat (file-name-directory filename) (file-symlink-p filename))
 				     filename))))
     (+
      (lsh (car (nth 5 src-attr)) 16)
--8<---------------cut here---------------end--------------->8---

Nick

Re: [BUG] bug in org-publish and a (wrong) patch

[Nick Dokos]

Nick Dokos <nicholas.dokos@hp.com> wrote:

> ...It also
> needs to treat the case of a non-existent symlink target (where
> file-symlink-p returns t).
> 

Scratch that: in that case, it gets the file attributes of the
symlink itself, so no problem.

Re: [BUG] bug in org-publish and a (wrong) patch

[Carsten Dominik]

Hi Nick,

I have not looked closely, but maybe you can use


(expand-file-name  .... (file-name-directory filename))

to fix this patch?  Not sure, I have not spent any time on it.

- Carsten

On Apr 7, 2011, at 7:11 AM, Nick Dokos wrote:

> org-publish-cache-ctime-of-src tries (but does not always succeed) to
> deal with symlinks: file-symlink-p returns the target as a string, but
> if the target is relative to the symlink, that's not going to fly.
> e.g. if c is a symlink like this
> 
>    /a/b/c->../d/f
> 
> then (file-symlink-p "/a/b/c") -> "../d/f"
> but if the current directory is any place other than /a/b, the target
> will not be found, the file attributes are going to be nil and
> the function will blow up.
> 
> Here is a patch born of about 5 mins of contemplation. It solved my
> immediate problem but it is certainly wrong. It breaks absolute targets
> (which I think are handled correctly by the original version). I'm not
> even sure that it correctly handles *all* relative targets. It also
> needs to treat the case of a non-existent symlink target (where
> file-symlink-p returns t).
> 
> It might be safer also to check if the file attributes are
> nil and deal with that, instead of blowing up.
> 
> --8<---------------cut here---------------start------------->8---
> diff --git a/lisp/org-publish.el b/lisp/org-publish.el
> index e944eea..dd192d6 100644
> --- a/lisp/org-publish.el
> +++ b/lisp/org-publish.el
> @@ -1150,7 +1150,7 @@ Returns value on success, else nil."
> (defun org-publish-cache-ctime-of-src (filename)
>   "Get the FILENAME ctime as an integer."
>   (let ((src-attr (file-attributes (if (stringp (file-symlink-p filename))
> -				       (file-symlink-p filename)
> +				       (concat (file-name-directory filename) (file-symlink-p filename))
> 				     filename))))
>     (+
>      (lsh (car (nth 5 src-attr)) 16)
> --8<---------------cut here---------------end--------------->8---
> 
> Nick
> 

- Carsten

Re: [BUG] bug in org-publish and a (wrong) patch

[Nick Dokos]

Carsten Dominik <carsten.dominik@gmail.com> wrote:

> Hi Nick,
> 
> I have not looked closely, but maybe you can use
> 
> 
> (expand-file-name  .... (file-name-directory filename))
> 
> to fix this patch?  Not sure, I have not spent any time on it.
> 

Almost but not quite: C-h v expand-file-name says

,----
| (expand-file-name NAME &optional DEFAULT-DIRECTORY)
| 
| Convert filename NAME to absolute, and canonicalize it.
| Second arg DEFAULT-DIRECTORY is directory to start with if NAME is relative
| (does not start with slash or tilde); if DEFAULT-DIRECTORY is nil or missing,
| the current buffer's value of `default-directory' is used.
`----

so you end up tacking it onto a completely unrelated directory (and my
experiments confirm this).

But there is a :base-directory for the project that could be obtained
from the project-plist and passed to expand-file-name.  I think that
would work but would require passing the project-plist down through a couple
of layers to org-publish-cache-ctime-of-src. Alternatively, it (or just
the base directory) could be bound dynamically in org-publish-file and
used in the ctime function.

What do you think would be preferable?

Thanks,
Nick

> - Carsten
> 
> On Apr 7, 2011, at 7:11 AM, Nick Dokos wrote:
> 
> > org-publish-cache-ctime-of-src tries (but does not always succeed) to
> > deal with symlinks: file-symlink-p returns the target as a string, but
> > if the target is relative to the symlink, that's not going to fly.
> > e.g. if c is a symlink like this
> > 
> >    /a/b/c->../d/f
> > 
> > then (file-symlink-p "/a/b/c") -> "../d/f"
> > but if the current directory is any place other than /a/b, the target
> > will not be found, the file attributes are going to be nil and
> > the function will blow up.
> > 
> > Here is a patch born of about 5 mins of contemplation. It solved my
> > immediate problem but it is certainly wrong. It breaks absolute targets
> > (which I think are handled correctly by the original version). I'm not
> > even sure that it correctly handles *all* relative targets. It also
> > needs to treat the case of a non-existent symlink target (where
> > file-symlink-p returns t).
> > 
> > It might be safer also to check if the file attributes are
> > nil and deal with that, instead of blowing up.
> > 
> > --8<---------------cut here---------------start------------->8---
> > diff --git a/lisp/org-publish.el b/lisp/org-publish.el
> > index e944eea..dd192d6 100644
> > --- a/lisp/org-publish.el
> > +++ b/lisp/org-publish.el
> > @@ -1150,7 +1150,7 @@ Returns value on success, else nil."
> > (defun org-publish-cache-ctime-of-src (filename)
> >   "Get the FILENAME ctime as an integer."
> >   (let ((src-attr (file-attributes (if (stringp (file-symlink-p filename))
> > -				       (file-symlink-p filename)
> > +				       (concat (file-name-directory filename) (file-symlink-p filename))
> > 				     filename))))
> >     (+
> >      (lsh (car (nth 5 src-attr)) 16)
> > --8<---------------cut here---------------end--------------->8---
> > 
> > Nick
> > 
> 
> - Carsten
> 
> 
> 
> 

Re: [BUG] bug in org-publish and a (wrong) patch

[David Maus]

[-- Attachment #1.1: Type: text/plain, Size: 1714 bytes --]

At Fri, 08 Apr 2011 12:58:06 -0400,
Nick Dokos wrote:
>
> Carsten Dominik <carsten.dominik@gmail.com> wrote:
>
> > Hi Nick,
> >
> > I have not looked closely, but maybe you can use
> >
> >
> > (expand-file-name  .... (file-name-directory filename))
> >
> > to fix this patch?  Not sure, I have not spent any time on it.
> >
>
> Almost but not quite: C-h v expand-file-name says
>
> ,----
> | (expand-file-name NAME &optional DEFAULT-DIRECTORY)
> |
> | Convert filename NAME to absolute, and canonicalize it.
> | Second arg DEFAULT-DIRECTORY is directory to start with if NAME is relative
> | (does not start with slash or tilde); if DEFAULT-DIRECTORY is nil or missing,
> | the current buffer's value of `default-directory' is used.
> `----
>
> so you end up tacking it onto a completely unrelated directory (and my
> experiments confirm this).
>
> But there is a :base-directory for the project that could be obtained
> from the project-plist and passed to expand-file-name.  I think that
> would work but would require passing the project-plist down through a couple
> of layers to org-publish-cache-ctime-of-src. Alternatively, it (or just
> the base directory) could be bound dynamically in org-publish-file and
> used in the ctime function.
>
> What do you think would be preferable?

Took some time, but attached patch fixes the problem w/o the need for
passing down :base-directory at all. Simply expand-filename only if
the symlink is relative; luckily the filename passed to this fun
already is absolute.

@Bastien: Didn't push because I assume you already started the release
process for Org 7.6.

Best,
  -- David
--
OpenPGP... 0x99ADB83B5A4478E6
Jabber.... dmjena@jabber.org
Email..... dmaus@ictsoc.de

[-- Attachment #1.2: 0001-Properly-handle-relative-symlinks-when-publishing.patch --]
[-- Type: text/plain, Size: 1660 bytes --]

From f6ed4d5707995f34a627886d0607dd7e6343144b Mon Sep 17 00:00:00 2001
From: David Maus <dmaus@ictsoc.de>
Date: Sun, 26 Jun 2011 20:02:42 +0200
Subject: [PATCH] Properly handle relative symlinks when publishing

* org-publish.el (org-publish-cache-ctime-of-src): Properly handle
relative symlinks.

At Thu, 07 Apr 2011 01:11:00 -0400,
Nick Dokos wrote:
>
> org-publish-cache-ctime-of-src tries (but does not always succeed) to
> deal with symlinks: file-symlink-p returns the target as a string, but
> if the target is relative to the symlink, that's not going to fly.
> e.g. if c is a symlink like this
>
>     /a/b/c->../d/f
>
> then (file-symlink-p "/a/b/c") -> "../d/f"
> but if the current directory is any place other than /a/b, the target
> will not be found, the file attributes are going to be nil and
> the function will blow up.
---
 lisp/org-publish.el |    7 ++++---
 1 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/lisp/org-publish.el b/lisp/org-publish.el
index 56cc80a..0d3d70a 100644
--- a/lisp/org-publish.el
+++ b/lisp/org-publish.el
@@ -1157,9 +1157,10 @@ Returns value on success, else nil."
 
 (defun org-publish-cache-ctime-of-src (filename)
   "Get the FILENAME ctime as an integer."
-  (let ((src-attr (file-attributes (if (stringp (file-symlink-p filename))
-				       (file-symlink-p filename)
-				     filename))))
+  (let* ((symlink-maybe (or (file-symlink-p filename) filename))
+	 (src-attr (file-attributes (if (file-name-absolute-p symlink-maybe)
+					symlink-maybe
+				      (expand-file-name symlink filename)))))
     (+
      (lsh (car (nth 5 src-attr)) 16)
      (cadr (nth 5 src-attr)))))
-- 
1.7.2.5


[-- Attachment #2: Type: application/pgp-signature, Size: 230 bytes --]

Re: [BUG] bug in org-publish and a (wrong) patch

[Nick Dokos]

David Maus <dmaus@ictsoc.de> wrote:

> At Fri, 08 Apr 2011 12:58:06 -0400,
> Nick Dokos wrote:
> >
> > Carsten Dominik <carsten.dominik@gmail.com> wrote:
> >
> > > Hi Nick,
> > >
> > > I have not looked closely, but maybe you can use
> > >
> > >
> > > (expand-file-name  .... (file-name-directory filename))
> > >
> > > to fix this patch?  Not sure, I have not spent any time on it.
> > >
> >
> > Almost but not quite: C-h v expand-file-name says
> >
> > ,----
> > | (expand-file-name NAME &optional DEFAULT-DIRECTORY)
> > |
> > | Convert filename NAME to absolute, and canonicalize it.
> > | Second arg DEFAULT-DIRECTORY is directory to start with if NAME is relative
> > | (does not start with slash or tilde); if DEFAULT-DIRECTORY is nil or missing,
> > | the current buffer's value of `default-directory' is used.
> > `----
> >
> > so you end up tacking it onto a completely unrelated directory (and my
> > experiments confirm this).
> >
> > But there is a :base-directory for the project that could be obtained
> > from the project-plist and passed to expand-file-name.  I think that
> > would work but would require passing the project-plist down through a couple
> > of layers to org-publish-cache-ctime-of-src. Alternatively, it (or just
> > the base directory) could be bound dynamically in org-publish-file and
> > used in the ctime function.
> >
> > What do you think would be preferable?
> 
> Took some time, but attached patch fixes the problem w/o the need for
> passing down :base-directory at all. Simply expand-filename only if
> the symlink is relative; luckily the filename passed to this fun
> already is absolute.
> 
> @Bastien: Didn't push because I assume you already started the release
> process for Org 7.6.
> 

> From f6ed4d5707995f34a627886d0607dd7e6343144b Mon Sep 17 00:00:00 2001
> From: David Maus <dmaus@ictsoc.de>
> Date: Sun, 26 Jun 2011 20:02:42 +0200
> Subject: [PATCH] Properly handle relative symlinks when publishing
> 
> * org-publish.el (org-publish-cache-ctime-of-src): Properly handle
> relative symlinks.
> 
> At Thu, 07 Apr 2011 01:11:00 -0400,
> Nick Dokos wrote:
> >
> > org-publish-cache-ctime-of-src tries (but does not always succeed) to
> > deal with symlinks: file-symlink-p returns the target as a string, but
> > if the target is relative to the symlink, that's not going to fly.
> > e.g. if c is a symlink like this
> >
> >     /a/b/c->../d/f
> >
> > then (file-symlink-p "/a/b/c") -> "../d/f"
> > but if the current directory is any place other than /a/b, the target
> > will not be found, the file attributes are going to be nil and
> > the function will blow up.
> ---
>  lisp/org-publish.el |    7 ++++---
>  1 files changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/lisp/org-publish.el b/lisp/org-publish.el
> index 56cc80a..0d3d70a 100644
> --- a/lisp/org-publish.el
> +++ b/lisp/org-publish.el
> @@ -1157,9 +1157,10 @@ Returns value on success, else nil."
>  
>  (defun org-publish-cache-ctime-of-src (filename)
>    "Get the FILENAME ctime as an integer."
> -  (let ((src-attr (file-attributes (if (stringp (file-symlink-p filename))
> -				       (file-symlink-p filename)
> -				     filename))))
> +  (let* ((symlink-maybe (or (file-symlink-p filename) filename))
> +	 (src-attr (file-attributes (if (file-name-absolute-p symlink-maybe)
> +					symlink-maybe
> +				      (expand-file-name symlink filename)))))
>      (+
>       (lsh (car (nth 5 src-attr)) 16)
>       (cadr (nth 5 src-attr)))))
> -- 
> 1.7.2.5
> 

I don't think it's correct as it stands: What is ``symlink'' on the last
line?  should it be be symlink-maybe perhaps? and expand-file-name
expands wrt to the default directory passed as its third argument. Maybe
the third argument can be (file-name-directory filename) as Carsten
suggested, but surely it cannot be just ``filename''.

Replacing the last line with

          (expand-file-name symlink-maybe (file-name-directory filename))...

my (very simple) test case gets published correctly, so it's
certainly better than what's there.

Nick

Re: [BUG] bug in org-publish and a (wrong) patch

[David Maus]

[-- Attachment #1.1: Type: text/plain, Size: 4484 bytes --]

At Mon, 27 Jun 2011 00:12:11 -0400,
Nick Dokos wrote:
>
> David Maus <dmaus@ictsoc.de> wrote:
>
> > At Fri, 08 Apr 2011 12:58:06 -0400,
> > Nick Dokos wrote:
> > >
> > > Carsten Dominik <carsten.dominik@gmail.com> wrote:
> > >
> > > > Hi Nick,
> > > >
> > > > I have not looked closely, but maybe you can use
> > > >
> > > >
> > > > (expand-file-name  .... (file-name-directory filename))
> > > >
> > > > to fix this patch?  Not sure, I have not spent any time on it.
> > > >
> > >
> > > Almost but not quite: C-h v expand-file-name says
> > >
> > > ,----
> > > | (expand-file-name NAME &optional DEFAULT-DIRECTORY)
> > > |
> > > | Convert filename NAME to absolute, and canonicalize it.
> > > | Second arg DEFAULT-DIRECTORY is directory to start with if NAME is relative
> > > | (does not start with slash or tilde); if DEFAULT-DIRECTORY is nil or missing,
> > > | the current buffer's value of `default-directory' is used.
> > > `----
> > >
> > > so you end up tacking it onto a completely unrelated directory (and my
> > > experiments confirm this).
> > >
> > > But there is a :base-directory for the project that could be obtained
> > > from the project-plist and passed to expand-file-name.  I think that
> > > would work but would require passing the project-plist down through a couple
> > > of layers to org-publish-cache-ctime-of-src. Alternatively, it (or just
> > > the base directory) could be bound dynamically in org-publish-file and
> > > used in the ctime function.
> > >
> > > What do you think would be preferable?
> >
> > Took some time, but attached patch fixes the problem w/o the need for
> > passing down :base-directory at all. Simply expand-filename only if
> > the symlink is relative; luckily the filename passed to this fun
> > already is absolute.
> >
> > @Bastien: Didn't push because I assume you already started the release
> > process for Org 7.6.
> >
>
> > From f6ed4d5707995f34a627886d0607dd7e6343144b Mon Sep 17 00:00:00 2001
> > From: David Maus <dmaus@ictsoc.de>
> > Date: Sun, 26 Jun 2011 20:02:42 +0200
> > Subject: [PATCH] Properly handle relative symlinks when publishing
> >
> > * org-publish.el (org-publish-cache-ctime-of-src): Properly handle
> > relative symlinks.
> >
> > At Thu, 07 Apr 2011 01:11:00 -0400,
> > Nick Dokos wrote:
> > >
> > > org-publish-cache-ctime-of-src tries (but does not always succeed) to
> > > deal with symlinks: file-symlink-p returns the target as a string, but
> > > if the target is relative to the symlink, that's not going to fly.
> > > e.g. if c is a symlink like this
> > >
> > >     /a/b/c->../d/f
> > >
> > > then (file-symlink-p "/a/b/c") -> "../d/f"
> > > but if the current directory is any place other than /a/b, the target
> > > will not be found, the file attributes are going to be nil and
> > > the function will blow up.
> > ---
> >  lisp/org-publish.el |    7 ++++---
> >  1 files changed, 4 insertions(+), 3 deletions(-)
> >
> > diff --git a/lisp/org-publish.el b/lisp/org-publish.el
> > index 56cc80a..0d3d70a 100644
> > --- a/lisp/org-publish.el
> > +++ b/lisp/org-publish.el
> > @@ -1157,9 +1157,10 @@ Returns value on success, else nil."
> >
> >  (defun org-publish-cache-ctime-of-src (filename)
> >    "Get the FILENAME ctime as an integer."
> > -  (let ((src-attr (file-attributes (if (stringp (file-symlink-p filename))
> > -				       (file-symlink-p filename)
> > -				     filename))))
> > +  (let* ((symlink-maybe (or (file-symlink-p filename) filename))
> > +	 (src-attr (file-attributes (if (file-name-absolute-p symlink-maybe)
> > +					symlink-maybe
> > +				      (expand-file-name symlink filename)))))
> >      (+
> >       (lsh (car (nth 5 src-attr)) 16)
> >       (cadr (nth 5 src-attr)))))
> > --
> > 1.7.2.5
> >
>
> I don't think it's correct as it stands: What is ``symlink'' on the last
> line?  should it be be symlink-maybe perhaps? and expand-file-name
> expands wrt to the default directory passed as its third argument. Maybe
> the third argument can be (file-name-directory filename) as Carsten
> suggested, but surely it cannot be just ``filename''.
>
> Replacing the last line with
>
>           (expand-file-name symlink-maybe (file-name-directory filename))...
>
> my (very simple) test case gets published correctly, so it's
> certainly better than what's there.

Damn, you are right. This patch was composed way to sloppy. Here's a
corrected one.

Best,
  -- David
--
OpenPGP... 0x99ADB83B5A4478E6
Jabber.... dmjena@jabber.org
Email..... dmaus@ictsoc.de

[-- Attachment #1.2: 0001-Properly-handle-relative-symlinks-when-publishing.patch --]
[-- Type: text/plain, Size: 1715 bytes --]

From 6ad9f33ccb2d8702dd1d4375dd82998f72078e60 Mon Sep 17 00:00:00 2001
From: David Maus <dmaus@ictsoc.de>
Date: Sun, 26 Jun 2011 20:02:42 +0200
Subject: [PATCH] Properly handle relative symlinks when publishing

* org-publish.el (org-publish-cache-ctime-of-src): Properly handle
relative symlinks.

At Thu, 07 Apr 2011 01:11:00 -0400,
Nick Dokos wrote:
>
> org-publish-cache-ctime-of-src tries (but does not always succeed) to
> deal with symlinks: file-symlink-p returns the target as a string, but
> if the target is relative to the symlink, that's not going to fly.
> e.g. if c is a symlink like this
>
>     /a/b/c->../d/f
>
> then (file-symlink-p "/a/b/c") -> "../d/f"
> but if the current directory is any place other than /a/b, the target
> will not be found, the file attributes are going to be nil and
> the function will blow up.
---
 lisp/org-publish.el |    9 ++++++---
 1 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/lisp/org-publish.el b/lisp/org-publish.el
index 56cc80a..5646430 100644
--- a/lisp/org-publish.el
+++ b/lisp/org-publish.el
@@ -1157,9 +1157,12 @@ Returns value on success, else nil."
 
 (defun org-publish-cache-ctime-of-src (filename)
   "Get the FILENAME ctime as an integer."
-  (let ((src-attr (file-attributes (if (stringp (file-symlink-p filename))
-				       (file-symlink-p filename)
-				     filename))))
+  (let* ((symlink-maybe (or (file-symlink-p filename) filename))
+	 (src-attr (file-attributes (if (file-name-absolute-p symlink-maybe)
+					symlink-maybe
+				      (expand-file-name 
+				       symlink-maybe
+				       (file-name-directory filename))))))
     (+
      (lsh (car (nth 5 src-attr)) 16)
      (cadr (nth 5 src-attr)))))
-- 
1.7.2.5


[-- Attachment #2: Type: application/pgp-signature, Size: 230 bytes --]