How to reproduce results in a neural network when using a custom activation function?

Question

I have built my autoencoder from scratch, but instead of using the already provided activation functions such as ReLU, I want to experiment with my custom ones. Example: