Same. I was really surprised when i found that Sidewise is the culprit. Right now this issue is only reason why I am using firefox instead of chrome.
There is another workaround - you can jump for already buffered parts of the video without any problems.