How to alternate train op's in tensorflow?

Question

I am implementing an alternating training scheme. The graph contains two training ops. The training should alternate between these.

This is relevant for research lik

Answer 1

I think the simplest way is just

for i in range(1000):
  batch_xs, batch_ys = mnist.train.next_batch(100)
  if i % 2 == 0:
    sess.run(train_step1, feed_dict={x: batch_xs, y_: batch_ys})
  else:
    sess.run(train_step2, feed_dict={x: batch_xs, y_: batch_ys})

But if it's necessary to make a switch via tensorflow conditional flow, do it this way:

optimizer = tf.train.GradientDescentOptimizer(0.5)
train_step = tf.cond(tf.equal(tf.mod(global_step, 2), 0),
                     true_fn=lambda: optimizer.apply_gradients(zip(tf.gradients(cross_entropy, tvars1), tvars1), global_step),
                     false_fn=lambda: optimizer.apply_gradients(zip(tf.gradients(cross_entropy, tvars2), tvars2), global_step))