Microstructural refinement in single phase materials terminates after sufficient strains. To enable such a dynamic equilibrium, restoration mechanisms are necessary that balance the refinement caused by further straining. We will present insights from split specimens, showing grain boundary migration to be the dominant restoration mechanism. Interestingly, when high pressure torsion deformed samples with already saturated microstructure were further deformed by cold rolling, with the long grain axis oriented parallel to the rolling direction, a substantial decrease in hardness with increasing thickness reduction was found to occur. It will be shown, that similar as in the steady state, grain boundary motion in normal direction occurs what might overcompensate the geometrical refinement at the beginning of the strain path change. The observed softening could be supressed when exactly the same samples are rolled along a different direction, indicating that boundary migration is not entirely driven by the applied stress level.