What is the default activation function of cudnnlstm in tensorflow

问题

What's the default activation function of cudnnlstm in TensorFlow? How can I set an activation function such as relu? Maybe it's just linear model? I read the document, but I did not find it.

For example, the code is below:

lstmcell=tf.contrib.cudnn_rnn.CudnnLSTM(1,encoder_size,direction="bidirectional")

hq,_ =lstmcell(query)

And I read the document of TensorFlow From this link.

The function is below

__init__(

    num_layers,
    num_units,
    input_mode=CUDNN_INPUT_LINEAR_MODE,
    direction=CUDNN_RNN_UNIDIRECTION,
    dropout=0.0,
    seed=None,
    dtype=tf.float32,
    kernel_initializer=None,
    bias_initializer=None,
    name=None
)

And no keyword to set a parameter such as "activation = "tanh" just like tf.nn.rnn_cell.LSTMell.

So what's the default activation function of cudnnlstm in TensorFlow, and how to change it to leaky_relu.

回答1:

tf.contrib.cudnn_rnn.CudnnLSTM() : Tanh

This was given in the Keras github.

https://github.com/keras-team/keras/issues/8510#issuecomment-429255318

Nvidia documentation.

https://devblogs.nvidia.com/optimizing-recurrent-neural-networks-cudnn-5/

回答2:

To answer OP's 2nd question which was edited in later, there is currently no way to set a custom activation function for CudnnLSTM and CudnnGRU.

来源：https://stackoverflow.com/questions/52993397/what-is-the-default-activation-function-of-cudnnlstm-in-tensorflow