Why is Openai's PPO2 implementation differentiable?

Question

I\'m trying to understand the concept behind the implementation of the OpenAI PPO2 algorithm. The loss function that is minimized is as follows: loss = pg_loss - entropy *