What can Google Images See?

Like many SEOs I like the idea of advertorial content for link building purposes – it’s quality written content, which is rich in semantic keywords, and is genuinely interesting for customers. Furthermore, it drives engaged traffic, you have control over anchor texts, and the sites that carry advertorials are usually of high value.
However, Google doesn’t like them unless they use a ‘nofollow’ instruction (link attribute). While this is not necessarily a problem in itself, for a number of reasons, many people still want to be able to have followed links rather than be dictated to.
As part of the ever-changing landscape, advertorial content on news sites needs to be described using words such as ‘this is advertorial’ or ‘sponsored content’ and this isn’t even a decree from Google - it is a legal requirement. So, to avoid using crawlable, detectable words like ‘advertorial’ on a page, many publishers have been using an image with no alt text to ensure they are legally compliant without being ‘Google Compliant’ per se.
Unfortunately, Google appears to have been trying to stay ahead of this, learning to crawl text within images and as rumours have it, within flash. So, with all this in mind, how can we test what Google actually 'sees'?
From what we can see, aside from colour signals, Google is getting clues from the page the image appears on, such as; target page, proximal content, or content in general, alt tags etc.

Time for a Little Experiment...

A search Sponsored post image brings up these results:

On the right hand side of the results:

…exactly what I was after.
But if you check the page that the image is from, there are numerous references to ‘sponsored post’ on the page; alt, URL, article title, nearby text, so that’s easy for Google to discern:

A Google image search for the actual image URL brings up these results below… all pretty close matches and it seems to have got the ‘sponsor’ bit right. But; all the main results in the ‘pages that include matching images’ section are from pages that have at least one on-page clue that the image relates to ‘sponsor’, and the ‘visually similar images’ at the bottom are clearly colour based, rather than matching text. So, no actual evidence that Google is reading the images.

So I tried saving one of the sponsored images as something unrelated, ‘billybobharry’ in this case, and then uploading that file straight into to the search to see what I get. Lo and behold, I get exactly the same results as for the URL search above. I wasn’t expecting that:

The results are exactly the same; so you would have to deduce that Google can match images exactly.
I then tried a search with another image from my PC (the image can also be found online) and I only get the results of the exact same image from across the web, but nothing with the same subject (me) in a different context. So, Google can match images exactly, but it’s idea of visually similar is very rough at best - I look nothing like these people…

But … and here’s the good bit … if I create an image in paint with the words ‘Sponsored Post’ clear and bold across a white background, I get only visually similar images, no matches for the text itself (Google likes to put in the natural results for ‘time cost quality’ for this search. I couldn’t get rid of it at all). There are no clues as to the meaning of the image online for Google, and no online match for the file, so it just gives me visually similar images, and nothing even close to sponsored post images.

In short, if an image is going to be used to hide text, such as ‘sponsored post’, then I think this strategy will still work for the moment.