Set the vise jaws so they support the bearing but not the carrier. Drive the carrier off using wood or plastic driver and a hammer. Put the new bearing in position using a block of wood between the bearing the vise jaw and push the bearing in place by tightening the vise.
Basically you are using the vise as a primitive press.