Niantic Labs: LinguaLens

"We built an AI translator for Niantic’s Innovation Cohort that uses your phone’s camera to convert any scene into six languages- including Klingon - by combining ChatGPT and Microsoft Azure object detection.”

Drawing on advanced translation from ChatGPT and real-time object detection via Microsoft Azure, this mobile tool seamlessly interprets whatever the camera sees. Whether it’s a street sign, a menu, or just everyday objects, the system quickly analyzes and translates text into multiple languages - even the fictional Klingon from Star Trek. Users simply hover their phone over the scene, and within a second, see translated text overlaid on their screen. By blending cutting-edge AI with user-friendly functionality, the translator exemplifies how technology can break language barriers, all while adding a touch of playful sci-fi."