问题
This is the piece of code that I am working with:
import tensorflow as tf
import numpy as np
from PIL import Image
from os import listdir
nodes_l1 = 500
nodes_l2 = 100
nodes_l3 = 500
num_batches = 20
num_epochs = 50
# Array of file dirs
human_file_array = listdir('human/')
human_file_array = [['human/'+human_file_array[i],[1,0]] for i in range(len(human_file_array))]
cucumber_file_array = listdir('cucumber/')
cucumber_file_array = [['cucumber/'+cucumber_file_array[i],[0,1]] for i in range(len(cucumber_file_array))]
file_array_shuffled = human_file_array + cucumber_file_array
np.random.shuffle(file_array_shuffled)
htest_file_array = listdir('human_test/')
htest_file_array = [['human_test/'+htest_file_array[i],[1,0]] for i in range(len(htest_file_array))]
ctest_file_array = listdir('cucumber_test/')
ctest_file_array = [['cucumber_test/'+ctest_file_array[i],[0,1]] for i in range(len(ctest_file_array))]
test_file_array = ctest_file_array + htest_file_array
np.random.shuffle(test_file_array)
input_data = tf.placeholder('float', [None, 250*250*3]
output_data = tf.placeholder('float')
hl1_vars = {
'weight': tf.Variable(tf.random_normal([250*250*3, nodes_l1])),
'bias': tf.Variable(tf.random_normal([nodes_l1]))
}
hl2_vars = {
'weight': tf.Variable(tf.random_normal([nodes_l1, nodes_l2])),
'bias': tf.Variable(tf.random_normal([nodes_l2]))
}
hl3_vars = {
'weight': tf.Variable(tf.random_normal([nodes_l2, nodes_l3])),
'bias': tf.Variable(tf.random_normal([nodes_l3]))
}
output_layer_vars = {
'weight': tf.Variable(tf.random_normal([nodes_l3, 2])),
'bias': tf.Variable(tf.random_normal([2]))
}
layer1 = tf.add(tf.matmul(input_data, hl1_vars['weight']),hl1_vars['bias'])
layer1 = tf.nn.softmax(layer1)
layer2 = tf.add(tf.matmul(layer1, hl2_vars['weight']), hl2_vars['bias'])
layer2 = tf.nn.softmax(layer2)
layer3 = tf.add(tf.matmul(layer2, hl3_vars['weight']), hl3_vars['bias'])
layer3 = tf.nn.softmax(layer3)
output = tf.add(tf.matmul(layer3, output_layer_vars['weight']), output_layer_vars['bias'])
output = tf.nn.softmax(output)
def convert_image(path):
with Image.open(path) as img:
img = img.resize((250,250))
img = img.convert('RGB')
return img
def train_network():
#prediction = output
cost = tf.reduce_sum(tf.nn.sigmoid_cross_entropy_with_logits(output, output_data)) # output is the prediction, output_data is key
optimizer = tf.train.AdamOptimizer().minimize(cost)
with tf.Session() as sess:
sess.run(tf.initialize_all_variables())
saver = tf.train.Saver()
for epoch in range(num_epochs):
epoch_error = 0
batch_size = int((len(file_array_shuffled)/num_batches))
for i in range(num_batches):
path_var = []
key_var = []
img_var = []
#Still Filename Batch!!
batch_file_array = file_array_shuffled[batch_size*i:(batch_size*i)+batch_size] #batch1['file&val array']['val']
for batch_val in batch_file_array:
path_var.append(batch_val[0])
key_var.append(batch_val[1])
#FROM HERE ON path_var AND key_var HAVE MATCHING INDEXES DO NOT RANDOMIZE!!!
#This section here is complicated!
for path in path_var:
img = convert_image(path)
img_var.append(np.reshape(np.array(img), 250*250*3))
#print np.shape(img_var),np.shape(key_var) #img_var is array of size (batch#, 64*64*3) key_var is the key [human, cucumber]
#End of complicationimage conversion
_,c = sess.run([optimizer, cost], feed_dict={input_data:img_var, output_data:key_var})
epoch_error += c
#print "Batch",i+1,"done out of",num_batches
print "Epoch",epoch+1,"completed out of",num_epochs,"\tError",epoch_error
save_path = saver.save(sess, "model.ckpt")
train_network()
def use_network():
#prediction = output
with tf.Session() as sess:
sess.run(tf.initialize_all_variables())
saver = tf.train.Saver()
saver.restore(sess, "model.ckpt")
for test_file in test_file_array:
#print test_file
img = np.reshape(np.array(convert_image(test_file[0])), 250*250*3)
result = output.eval(feed_dict={input_data:[img]})
print result,tf.argmax(result,1).eval(),test_file[1]
use_network()
http://pastebin.com/Gp6SVYJR
As I am still new to using tensorflow, I thought it would be a good idea to try and create a program that can identify the difference between Humans and Cucumbers. I pulled images from Image-Net, and put the human pictures into human/ and cucumber photos into cucumber/
I created a list of the steps that I believe the program is taking:
An array of filepaths and keys are made and then shuffled.
Batches are created of the file paths.
The filepath from the batches are turned into images, resized to 250x250, and added into an image batch array.(Keys and images still aligning at this point).
Image batch and key batch fed into the array.
At the end of all of the epochs, it tests the network against 10 of each images.
When I run use_network(), I get this output in the console:
[[ 0.53653401 0.46346596]] [0] [0, 1]
[[ 0.53653401 0.46346596]] [0] [0, 1]
[[ 0.53653401 0.46346596]] [0] [0, 1]
[[ 0.53653401 0.46346596]] [0] [1, 0]
[[ 0.53653401 0.46346596]] [0] [1, 0]
[[ 0.53653401 0.46346596]] [0] [0, 1]
[[ 0.53653401 0.46346596]] [0] [1, 0]
[[ 0.53653401 0.46346596]] [0] [1, 0]
[[ 0.53653401 0.46346596]] [0] [1, 0]
[[ 0.53653401 0.46346596]] [0] [0, 1]
[[ 0.53653401 0.46346596]] [0] [1, 0]
[[ 0.53653401 0.46346596]] [0] [0, 1]
[[ 0.61422414 0.38577583]] [0] [1, 0]
[[ 0.53653401 0.46346596]] [0] [0, 1]
[[ 0.53653401 0.46346596]] [0] [1, 0]
[[ 0.53653401 0.46346596]] [0] [1, 0]
[[ 0.53653401 0.46346596]] [0] [0, 1]
[[ 0.53653401 0.46346596]] [0] [0, 1]
[[ 0.53653401 0.46346596]] [0] [0, 1]
[[ 0.53653401 0.46346596]] [0] [1, 0]
The first array being the output nodes, the second array being the tf.argmax() of the output, and the third being what it was expected to be.
The actual learning seems to be rather small too, with this being the output from the learning:
Epoch 1 completed out of 50 Error 3762.83390808
Epoch 2 completed out of 50 Error 3758.51748657
Epoch 3 completed out of 50 Error 3753.70425415
Epoch 4 completed out of 50 Error 3748.32539368
Epoch 5 completed out of 50 Error 3742.45524597
Epoch 6 completed out of 50 Error 3736.21272278
Epoch 7 completed out of 50 Error 3729.56756592
...
Epoch 45 completed out of 50 Error 3677.34605408
Epoch 46 completed out of 50 Error 3677.34388733
Epoch 47 completed out of 50 Error 3677.34150696
Epoch 48 completed out of 50 Error 3677.3391571
Epoch 49 completed out of 50 Error 3677.33673096
Epoch 50 completed out of 50 Error 3677.33418274
I have attempted to do the following to try and change things:
Making images smaller, 32x32 for example, and/or in black and white. To see if smaller images would lead to changes in predictions.
Changing the cost equation between reduce_sum and reduce_mean, and the inner equation between sigmoid_cross_entropy to softmax_cross_entropy.
I have a few ideas as to why it isnt working, and they are as followed:
Just poor code
Input data too large and not enough nodes/layers to process.
Image and associated key getting scrambled somewhere.
回答1:
There are a few possible issues I see with this. The first is that you are using densely connected layers to process large image net images. You should be using convolutional networks for images. I think this is your biggest problem. Only after applying a pyramid of convolutional / pooling layers to reduce the spatial dimensions into "features" should you add a dense layer.
https://www.tensorflow.org/versions/r0.11/tutorials/deep_cnn/index.html
Secondly, even if you were going to use dense layers you should not apply the softmax function as an activation between hidden layers (with some exceptions such as in attention models but this is a more advanced concept.) Softmax forces the sum of every activation to in the layer to one which you probably don't want. I would change the activation between hidden layers to relu or at least tanh.
Finally, I have found that when networks approach a constant value it can help to lower the learning rate. I don't think this is your issue though. My first two comments are what you should focus on.
来源：https://stackoverflow.com/questions/40451874/tensorflow-always-converging-to-same-output-for-all-items-after-training