问题
Suppose I would like to build a tool that would tell me on a scale from 0 to 1 the similarity between images.
I could go ahead and download a pre-trained mage models such as vgg16, remove its last layers (classifier), convert the images into feature vectors using the pre-trained model and finally compute the similarity between the vectors using something like the cosine similarity.
The method is better explained here: https://mc.ai/find-more-like-this-product-using-transfer-learning/
... but what if I wanted to enhance the tool using my own dataset ? More specifically, I have a dataset where humans either said that two pictures where similar (1) or not similar (0):
I was wondering, if you see a way how I could in-cooperate this to enhance the similarity score tool ? Many thanks. :)
回答1:
You can use a siamese network with a VGG-16 base. The model will be trained with your labeled images, similar images will be trained with a similarity of 1, different images will be trained with the similarity of 0.
Triplet loss can be used to minimize the dissimilarity between the same objects.
Here is an implementation in keras: https://medium.com/@prabhnoor0212/siamese-network-keras-31a3a8f37d04
Training process with your dataset:
Let's say you have a simple classification dataset. So, you have a class for apples, one for cats, and so on.
- First, you'll want to make sure the images in a single class are very similar.
(If you put a car and a truck in the same class and expect them to have a similarity score of 1, it'll make your network perform poorly)
Once you have the dataset, you'll just need to generate pairs, if two images are from same class the label will be 1. and if not the label will be 0.
Now, just train the model with triplet loss and VGG siamese.
You can modify the code examples, and even find many resources which shares code.
来源：https://stackoverflow.com/questions/61666602/enhancing-pre-learned-model-with-human-input-to-better-determine-image-similarit