Re: Re: org-publish not publishing changed files

[Nick Dokos <nicholas.dokos@hp.com>]

Aidan Gauland <aidalgol@no8wireless.co.nz> wrote:

> 
> Yes it does.  Sorry, I should have worded that differently.  I meant
> that I have been repeatedly wiping ~/.org-timestamps/ in hopes that it
> will not get screwed up again and think none of my files have changed,
> and never re-publishes changed files (after an initial org-publish
> operation).  I think I have run into a bug in org-mode and I really
> want to track it down, so I thought the best place to start would be
> to look at what the timestamps are (in a human-readable format).
> 

OK - thanks. I started taking a look at the code, but I'm not getting
anywhere fast, so the more eyes we can have on it the better.

I believe what happens is that the in-memory cache is initialized from
the files in ~/.org-timestamps the first time that you try to publish
(I'm not sure yet when the cache is flushed back out to the files yet).
Here is one of the timestamp files:

,----
| (setq org-publish-cache (make-hash-table :test 'equal :weakness nil :size 100))
| (puthash ":project:" "worg-org-faq" org-publish-cache)
| (puthash ":cache-file:" "/home/nick/.org-timestamps/worg-org-faq.cache" org-publish-cache)
| (puthash "Xd2e1407553750278e966c6cda86f154ffa648149" 1299616514 org-publish-cache)
`----

It's a simple key-value store: (puthash key value table) does the
obvious thing, so the timestamp here is 1299616514 for the object with
key Xd2e1407553750278e966c6cda86f154ffa648149. The timestamp is the
ctime of the file, obtained by calling file-attributes, getting the
5th element, the last modification time. This is returned as a list
of two integers (high, low16) which are then combined into a single
integer: that's the time since 1970-01-01:00:00:00, the usual Unix
time. 

In the above case, I calculate as follows (lsh is a shift function -
(lsh val 16) shifts left by 16, (lsh val -16) shifts right by 16):

high bits: (lsh 1299616514 -16) -> 19830
low bits:  (- 1299616514 (lsh 19830 16)) -> 37634

So encoded time is (19830 37364)

Decoded time: (decode-time '(19830 37634)) -> (14 35 15 8 3 2011 2 nil -18000)

that is 14:35:15 08-03-2011 dow=2(=Tuesday) dst=nil(= no dst - the
switch happened later) zone=-18000 (in seconds - that is -5GMT in
hours).

So you might try the following simple helper function:

--8<---------------cut here---------------start------------->8---
(defun org-ts-to-hr (ts)
  (let* ((high (lsh ts -16))
         (low (- ts (lsh high 16))))
    (decode-time (list high low))))
--8<---------------cut here---------------end--------------->8---

(org-ts-to-hr 1299616514) -> (14 35 15 8 3 2011 2 nil -18000)

Note that it may not be the timestamp itself that's at fault: it may
be the key. If we calculate a key and go look in the cache and don't
find it, then the file is considered out of date. But I guess this
is the opposite behavior of what you see. Never mind...

HTH,
Nick