I've been having this same issue - the issue is quite reproducible for me, and it has been for years. I just finally grew tired of the issue and decided to investigate it, and yes, the issue is org-agenda-show-new-time.
I also have invisible entries in the org buffer, and the call to org-move-to-column apparently will move several lines forward, which causes us to process the same lines over and over.
Wrapping the call to org-move-to-column with a let setting the buffer-invisibility-spec to nil does fix the issue.