Google's Visual Translation Supports 20 New Languages

Google bought Word Lens last year and brought a very useful feature to Google Translate: real-time visual translation. Use a phone or a tablet running Android or iOS, point the camera at a sign or text in a foreign language and Google will translate the text almost instantly, while preserving all the other details. It's like using a magic camera that translates text and lets you read street signs, restaurant menus, user manuals, newspaper articles even if they're written in foreign languages.

Visual translation now supports 20 additional languages. "You can now translate to and from English and Bulgarian, Catalan, Croatian, Czech, Danish, Dutch, Filipino, Finnish, Hungarian, Indonesian, Lithuanian, Norwegian, Polish, Romanian, Slovak, Swedish, Turkish and Ukrainian. You can also do one-way translations from English to Hindi and Thai." Back in January, the feature was launched with only 7 supported languages: English, French, German, Italian, Portuguese, Russian and Spanish.

This feature requires to pick the right languages before tapping the camera button and one of the languages must be English. You'll probably be prompted to download a small language pack, since you can use Word Lens offline.

Google Research Blog has more information about Word Lens. After finding the text regions in the picture, Google recognizes the letters using a convolutional neural network. "Letters out in the real world are marred by reflections, dirt, smudges, and all kinds of weirdness. So we built our letter generator to create all kinds of fake 'dirt' to convincingly mimic the noisiness of the real world—fake reflections, fake smudges, fake weirdness all around." After recognizing the letters, Google translates the text taking into account that text recognition might include mistakes, then it "renders the translation on top of the original words in the same style as the original". Google actually erases the original text using the colors surrounding the text and draws the translation using the initial foreground color. It's quite clever.

Here's a funny demo: