The explanation for what you are seeing is most likely not what you think it is...
I would guess the alignments are correct, but the way they are displaying in the manual tool is not correct. Your are seeing the alignment from the previous/next frame on the frame you are viewing.
Sometimes closing out of the manual tool and re-opening fixes this.
Short version of why this sometimes happens: Normally a slight issue with the video file, or maybe a small bug somewhere in the manual tool.
Ultimately, it is not possible to seek to an exact frame in a video file in a timely manner. However, this is exactly what we have to be able to do (the only exact way to do this would be to scan through every single frame, counting frames, on each frame change.... this is clearly not feasible).
The manual tool will seek to the correct frame based on the Presentation Timestamp (PTS) of the frame. The PTS is calculated the first time you launch the manual tool. However it has to make assumptions about the accuracy of the video file. Sadly, not all video files are created equally. I have put many safeguards in place to handle problematic videos, most of which you will have never noticed, because they are working. However, it is not possible for me to cover every possible issue.