How is google search by image implemented?

Question

I can just drag and drop any image in google and get results. :)

How is it implimented ? What is the idea behind the algorithm ?

Is that image data converted to

Answer 1

Image search is an exciting field. Google Reverse Image Search uses a combination of Image processing techniques such as Scale Invariant Feature Transforms and Principle Component Analysis based Scale Invariant Feature Transform. Beyond the prowess of deep learning techniques and image processing algorithms, Google has an advantage in the deployment of large scale big data processing algorithms at scale. Google also creates really efficient indexes of images using advanced hashing techniques. It is interesting to see how Google is creating space-efficient data structures for images that are used subsequently in the image search algorithms.

Fundamentals of Reverse Image Search

In the entirety, Google Image Search is efficient not only because of image processing algorithms. It is also re-using and re-purposing the parametric search techniques of Google Text Search Engine.

Factors in Google Image Search Algorithm

Answer 2

Surprisingly, we can also use Google to answer this question!

What is the algorithm used by Google Search by Image

It is definitely not confirmed, but I'm sure Google uses many of these techniques/ a blend of them when identifying images

Answer 3

I think that google images uses a 3 combined algorithm

1. detect image's deformation (by rotation, scale, illumination,...)
2. detect by the colors
3. detect by the visual similarity

and a fourth algorithm that is a secret by google (to ranking for example) ;)

(see here -> http://www.quora.com/Algorithms/What-is-the-algorithm-used-by-Google-Search-by-Image-1 )

---

Update 2016

My original answer was on 2012 - in the meanwhile other studies and research have taken more and more importance and I learn some new stuff. ;-)

In my opinion the mains "philosophies" about image detections are three:

1. machine learning algorithms
2. deep learning (a machine learning algorithm)
3. (pattern recognition)

Today, I think that pattern recognition has lost its importance: machine learning is in my opinion the right way to work for searching by image.

With machine learning you can even search for similarly match (for example faces - that obviously are not equals between them). The difficult is how you will to teach correctly your machine. Different approach can be taken.

Deep learning is simply a machine learning algorithm. It goes deeper using differents layers to match a possible image, some example of layers could be:

• pixel-vector (color, intensity,..)
• shapes
• edges
• ...