There should be no clearance between the inner races and the stem. Same for the outer races and the headstock. The bottom inner is usually a medium push fit and the upper inner is usually a light push fit, if the parts are at room temperature. Warming the lower inner race will help things along.