What is an appropriate stop criteria for training on a non-stationary environment in reinforcement learning?

Question

I\'m currently studying reinforcement learning (RL) and would like to understand non-stationary environments better. So for stationary environments, the Q-values of all state-ac