What is the difference between the TensorFlow batch normalization implementations?

╄→гoц情女王★ 提交于 2019-12-23 16:16:58

问题


TensorFlow seems to implement at least 3 versions of batch normalization:

  • tf.nn.batch_normalization
  • tf.layers.batch_normalization
  • tf.contrib.layers.batch_norm

These all have different arguments and documentation.

What is the difference between these, and which one should I use?


回答1:


They are actually very different.

  • nn.batch_normalization performs the basic operation (i.e. a simple normalization)
  • layers.batch_normalization is a batchnorm "layer", i.e. it takes care of setting up the trainable parameters etc. At the end of the day, it is a wrapper around nn.batch_normalization. Chances are this is the one you want to use, unless you want to take care of setting up variables etc. yourself.

It's similar to the difference between nn.conv2d and layers.conv2d, for example.

As for the contrib version, I can't say for sure, but it seems to me like an experimental version with some extra parameters not available in the "regular" layers one.




回答2:


The are all based on the same paper: http://arxiv.org/abs/1502.03167 So they should do the same thing.

Some functions move around the code but the old versions are kept to keep backwards compatibility and you end up with more than one version.

I would recommend using the simplest one that lets you do your project (that is tf.nn.batch_normalization). If you need features/parameters that are not offered pick the one that works for you.

Note: tf.contrib.* is not guarateed to remain backwards compatible (the api might change in a future version).



来源:https://stackoverflow.com/questions/48949318/what-is-the-difference-between-the-tensorflow-batch-normalization-implementation

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!