A Theoretical and Experimental Exploration in Permutation Randomization on Nonsmooth Nonconvex Optimization

While gradient-based optimizers that incorporate randomization often demonstrate superior performance on complex optimizations, the theoretical foundations of this advantage remain underexplored. A central question arises: What role does randomization play in dimension-free, nonsmooth, nonconvex optimization? To address this gap, we examine both the theoretical and empirical impact of permutation randomization within gradient-based optimization frameworks, using it as a representative case to in