An unofficial blog that watches Google's attempts to move your operating system online since 2005. Not affiliated with Google.

Send your tips to

February 11, 2008

Intuitive Mobile Search

Image licensed as Creative Commons Attribution by Mac Funamizu.

Mac Funamizu imagines a more intuitive way to search by just pointing at objects or selecting text. An Internet-enabled mobile device that incorporates a camera, scanner, GPS could make use of services like Google Maps or an improved image search to recognize objects and deliver useful information about them.

"You can use it when you want to know a car model, an insect name, what kind of food is served at a restaurant and how much, who built a bridge, etc. etc. But as a designer myself, I hope it's able to tell me a name of a font of the type I see, the size, color (in RGB), and so on," explains Mac.

There are already commercial applications like GeoVector 3D Search that let users access data about some points of interest, but this could be extended to non-local search. GeoVector "currently provides products and services which significantly simplify local searches, allowing users to point their mobile device toward objects of interest to access information about them. Users can point and click with their mobile phone the way a computer user navigates using a mouse," according to a press release.

Google tests introducing barcodes in print ads to increase their accountability. "2D barcodes (...) allow readers to click on interesting print ads with their cellphones and seamlessly connect to relevant online content." A barcode could encode URLs or any other short text. "When you capture a picture of it with your cellphone, special decoding software reads the encoded information, and prompts your phone's browser to visit a URL." Barcodes are very popular in Japan, where a lot of mobile phones include decoding software. "Japan, the first country with a highly developed 3G network and high usage of the mobile internet, was also the country where telecoms like NTTDoCoMo and KDDI achieved a breakthrough by bringing QR code readers to mobile phones. Today QR Codes are so pervasive in Japan that it's almost impossible without seeing one. You can find them in advertisements, mobile campaigns, on maps, in magazines, on billboards etc. and nobody want to miss them anymore."

Point-and-click is an intuitive way to interact with objects and obtain information from search engines, but it's necessary to be able to analyze images and recognize objects, the same way barcode decoders transform codes into text. Google's acquisition of Neven Vision is a step in this direction.


  1. As if we need more people fiddling with electronic crap out in public. Cell phones and iPhones are bad enough -- people using them are like zombies, only half aware of their surroundings.

    How about "interacting" with your surroundings using your own senses instead of relying on the damned internet to make sense of everything for you?

  2. It seems like these bar codes could also be used to provide text or audio versions of content to visually impaired individuals.

  3. I've read an article about Nokia working on a service that offers you information about an object (e.g. a building) send in as a photo. But I don't know, if this service is already available.

  4. Anybody remembers CueCat?

  5. Bill Gates showed off a future Windows Mobile prototype at CES that did something like that:

  6. Mr. Funamizu's vision is creative, solid and well executed! I'm appreciative of his use of cc here; it makes it easier for others to participate in this discussion!