RNN Variational Autoencoder produces good reconstruction, but poor generation

Question

I am trying to reproduce the results of this paper by training the RNN based variational autoencoder. While reconstruction of the original text is working very well, the generat