Bench setting the valve clearances is best practice. If the seats and valves are cut correctly and concentrically, it won't change.
That said, BE CAREFUL. Because you are only moving one cam at a time, it possible for the valves to hit and bend. I would fit and adjust one cam at a time to avoid the issue.