Google+ Photo Search With Image Recognition

Last year, Google Drive added an advanced image search feature powered by Goggles that recognizes objects and uses OCR technology to extract text. The same feature is now available in Google+: search for [sunflower], click "More", restrict the results to "Photos" and select "Most recent". You'll find sunflower images from Google+ posts that don't even include "sunflower", not even in the image filename.

This also works for the images you've uploaded to Picasa Web Albums/Google+ Photos or the images uploaded by your circles.

{ via Android Police }

Google Drive Goggles

Google Drive has the most powerful image search technology you can find in an online storage service. It uses Google Goggles technology to recognize objects, logos, landmarks, text and much more, so you can find an image even if the filename is a.jpg. The most impressive thing is that the processing happens as soon as you upload the images and there's no setting to enable.

For example, Google Drive found an image of the Android statue from the Googleplex when searching for [Android].

It also found a screenshot of Google's homepage and a Chrome logo when searching for [Google], this time using OCR technology.

"Let's say you upload a scanned image of an old newspaper clipping. You can search for a word from the text of the actual article. We also use image recognition so that if you drag and drop photos from your Grand Canyon trip into Drive, you can later search for [grand canyon] and photos of its gorges should pop up. This technology is still in its early stages, and we expect it to get better over time," explains Google.

Continuous Mode in Google Goggles

Google Goggles for Android has a new mode that lets you find information about objects without manually taking pictures. Just enable the continuous mode and Goggles will automatically analyze the scene, upload the photos to Google's servers and show the results at the bottom of the screen. "Continuous mode is a quick and easy new way to use Goggles. You can now get results instantly without having to take a picture - no shutter press required! Goggles will scan the scene continuously so you don't need to worry about taking multiple pictures. The new continuous mode works best with books, products, artwork, and landmarks," explains Google. For now, Goggles can't detect text in the continuous mode, so you need to switch to the snapshot mode to translate a text, find an address or call a phone number.

The "live" mode is useful if you want to scan many barcodes or if you want to find information about many objects without spending a lot of time taking the perfect photos. Goggles is still far from perfect, so it will sometimes detect a lot of incorrect objects. You'll need a fast Internet connection, good lighting and an Android phone with a decent camera. Unfortunately, the application doesn't save its findings. While it's obvious that cluttering the history is not a good idea, saving the list of detected objects to a temporary page would be really useful.

Google Goggles 1.7 also adds a feature that returns pages which include the text you've scanned, even if it's a long text. This is especially useful for news articles. Now you can quickly find the online version of the content you're reading.

You can improve Goggles by adding information about the objects you know. Tap "can you suggest a better result?", draw a box around the object and describe the object. "To date, hundreds of thousands of submissions have been made to improve Goggles. Starting today, if you choose to suggest a better result or submit a new object altogether, your suggestion could become a result for the next user who searches for a similar object with Goggles," informs Google.

If you have a device that runs Android Gingerbread, Honeycomb or Ice Cream Sandwich, install Goggles from the Android Market or update to the latest version. According to Google, about 50.6% of the Android devices run Gingerbread and 2.4% of the devices run Android Honeycomb.

Google Goggles Makes Your Phone's Camera Smarter

Google Goggles is an application that's sometimes useful, but it's not good enough to use it every time you want to find something about an object. The Android app has a new feature that integrates it with the Camera app, uploads all the photos you're taking to Google's servers and shows notifications in the status bar if Goggles found something useful. It may sound spooky, but it makes your phone's camera smarter.

"With this new opt-in feature in Goggles, you can simply photograph an image using your phone's camera, and Goggles will work in the background to analyze your image. If your photo contains items that Goggles can recognize, the app will notify you," explains Google. The feature is disabled by default, but you can enable it from the settings page by choosing "Search from Camera".



It's probably a good idea to only enable this option when you're on vacation or when you're planning to photograph barcodes for products you want to buy. It's also useful if you're in a bookstore and you want to "bookmark" some books.

Google Mobile's help center informs that "each Goggles query consumes approximately 100 KB of data" and you can limit the amount of data that's consumed by selecting "Search on WiFi networks only" under "Mobile Connection".

"Search from Camera" is one of the features that won't be available in the Google app for iPhone because iOS' background APIs aren't that powerful. If you have an Android device, install Google Goggles 1.6 from the Android Market.

Google Goggles History

Until Google releases a desktop version of Google Goggles, you can see your previous visual queries at http://www.google.com/goggles/history even when you don't have your phone with you. Click on one of the images and you'll see the results, similar images and links to related web pages.


It's surprising to see that this service is not yet integrated with Google Web History and it's not part of Google Image Search.

{ spotted by François Beaufort }

Google Goggles Solves Sudoku Puzzles

Google's visual search tool has a new trick up its sleeve: solving Sudoku puzzles. Google Goggles 1.3 for Android scans barcodes much faster, recognizes print ads from the major U.S. magazines and newspapers, but most people will probably ignore those useful features and will try to see if Goggles can actually solve Sudoku puzzles. The nice thing is that this feature also works in the latest version of Google Mobile App for iPhone, along with print ads recognition.

"Now, Goggles on Android and iPhone can recognize puzzles and provide answers to help make you faster than a Sudoku champ. So if you ever get stuck, take a clear picture of the entire puzzle with Goggles and we'll tell you the correct solution," explains Google.

Maybe providing a hint would also be useful, especially if you want to solve the puzzle.




The new features show that Google Goggles has a lot of potential, but it's still not useful enough in the real world. An interesting article from Xconomy quotes Hartmut Neven, Google's technical lead manager for image recognition, who says that Goggles is not yet an universal visual search tool. "What keeps me awake at night is, 'What are the honest-to-God use cases that we can deliver,' where it's not just an 'Oh, wow.' We call it the bar of daily engagement. Can we make it useful enough that every day you will take out Goggles and do something with it?"

Google Goggles-Enabled Ads

Google tests a way to monetize Goggles, the mobile application for visual search. "A year ago, Google introduced a smartphone application that lets users take photos of objects and get search results in return. The company will take that capability into the world of marketing with an experiment allowing five [US] brands to use the application in their promotional materials". Here are the five brands that are part of the experiment: Buick, Delta, Diageo, Disney and T-Mobile.

Android and iPhone users in the US will be able to scan the magazine ads for Buick Regal, T-Mobile G2 and the movie posters for Disney's "Tron: Legacy" to find more information from the official sites.

Michael Slinger, Google's head of mobile search advertising sales in North America, says that "it's a learning experiment for us more than an opportunity to make money," but it could be another reason to use Google Goggles.

Google Goggles for iPhone

Google Mobile App for iPhone started to become a bloated app, just like Google Toolbar. Instead of developing new applications, Google adds extra features to the Google Mobile App.

The latest new feature added to the application is Google Goggles, a visual search tool that helps you identify objects in a picture. "Computer vision is a hard problem and Google Goggles is still a Labs product. It works well for things such as landmarks, logos and the covers of books, DVDs and games. However, it doesn't yet work for some things you might want to try like animals, plants or food," explains Google.

Google Goggles is available in the Google Mobile App if you use an iPhone 3GS running iOS 4 or an iPhone 4. Goggles is also a standalone Android application.

Google Buys Plink to Improve Visual Mobile Search

Google acquired Plink, a small startup that developed Plink Art, an Android application that lets you identify paintings using your mobile phone's camera. Plink Art was one of the winners of the Android Developer Challenge 2. You can find the application in Android's market: just search for [Plink Art].

"The coolest feature of Plink Art is instant art identification. Just snap a photo and if the painting is in our database our visual search system will recognize it and tell you all about it. Currently Plink knows about tens of thousands of famous paintings," explains Plink's website.

I tried the application on my Nexus One by taking photos of Google Image Search results, but the results were terrible. Google Goggles found much better results for the same images.


The two Plink co-founders and only employees will work to improve Google Goggles, a visual search tool that already lets you find books, places and artwork. From Plink's blog:

"We started Plink to bring the power of visual search to everyone, and we're delighted to be taking a big step towards that goal today. Google has already shown that it's serious about investing in this space with Google Goggles, and for the Plink team the opportunity to take our algorithms to Google-scale was just too exciting to pass up. The visual search engines of today can do some pretty cool things, but they still have a long long way to go. We're looking forward to helping the Goggles team build a visual search engine that works not just for paintings or book covers, but for everything you see around you. There are beautiful things to be done with computer vision – it's going to be a lot of fun!"


{ via LA Times Blog}

Translating Text Using Google Goggles

Google tests a very useful feature: translating text using Google Goggles, the visual search mobile app available for Android phones. Instead of visiting Google Translate and typing the text you want to translate, you could use your phone's camera to photograph the text, send the photo to Google's servers and get the translation.

This feature is not yet available because it's quite difficult to perform OCR (optical character recognition) on a photo, especially if you don't even know the language of the text.

"Right now this technology only works for German-to-English translations and it's not yet ready for prime time. However, it shows a lot of promise for what the future might hold. Soon your phone will be able to translate signs, posters and other foreign text instantly into your language. Eventually, we're hoping to build a version of Google Goggles that can translate between all of the 52 languages currently supported by Google Translate — bringing even more information to you on the go," says Hartmut Neven.


Google tests an API for Google Docs that lets you perform OCR on a high-resolution image, but the results aren't impressive. Maybe the OCR service should be released in Google Labs.

Google Goggles, Mobile Visual Search

Google revealed some details about an upcoming mobile application that lets you upload photos and returns relevant information.

Google is working on Google Visual Search, a mobile application that lets users take a picture of a location from their Android-powered smartphone and trigger a Google search that pulls up information associated with the image.

"Imagine you're a tourist and you arrive at this place and you would like to know more about it, all you will have to do is take a shot of the [Santa Monica pier] sign and you see we recognized this as the Santa Monica pier," [Google Product Manager Hartmut] Neven said.

However, the technology, known internally as Google Goggles, didn't pass muster when Google tested it with a focus group in August. The company's engineers are working out the bugs and building out the immense database required to propel the technology.

Neven Vision, a company acquired by Google in 2006, had several patents on mobile visual search and object recognition, including a patent for an "image-based search engine for mobile phones with camera":

The present invention may be embodied in an image-based information retrieval system that includes a mobile telephone and a remote server. The mobile telephone has a built-in camera, a recognition engine for recognizing an object or feature in an image from the built-in camera, and a communication link for requesting information from the remote server related to a recognized object or feature.

In more detailed features of the invention, the object may be an advertising billboard and the related information may be a web page address. Alternatively, the object may be a car and the related information may be a car manual. Also, the object may be a product and the related information may be a payment confirmation. Further, the object may be a book and the related information may be an audio stream.

Last year, Google launched an iPhone app that allowed you to to do a Google search using your voice. Obtaining some search results just by uploading a picture brings Google even closer to the real world.

Image licensed as Creative Commons Attribution by Mac Funamizu.

Update: Google Goggles is now available in Google Labs. If you have an Android phone, go to the Android Market app and search for "Google Goggles".