Nevermind, I did find the culprit.
Turns out it was a CSS conflict with Bootstrap3
Bootstrap uses the property border-sizing: border-box which makes all borders, margins, paddings, etc. part of the div's size.
If you add some CSS to your Template, it works fine:
div#colorbox {
box-sizing: content-box ;
}
That should do the trick.;-)