DST will recall that I encountered this bug before. He had given me a copy of his zelda game, which I worked on.
The player character moved in a similar fashion, and the view moved after he touched a side. The effect was the same. His view became progressively offset from where he wanted it.
What I learned was that it was a matter of the x (and y) accumulating a fractional portion until it exceeded 1.0. This caused the character to take an extra large step every now and then.
You can observe the effect by placing a grid behind or in front of the actors. You will see that they jump around but the grid does not. Therefore, the bug lies in the code pertaining to changing the value of x and y, or in the collision code.