This AI app helps visually impaired individuals discover and navigate objects of their atmosphere

What happened? For people who cannot rely on their eyesight, a new AI tool intervenes as an additional sensory group. Researchers at Penn State University have developed an app called NaviSense that can use voice commands to locate objects in real time and then guide users to them using spatial audio cues and vibration feedback from the phone.

• The app tracks the location of objects around you using a camera sensor input.
• It translates spatial information into auditory cues to guide users to the object.
• Because NaviSense works in real time, there is no lengthy setup or manual marking of objects required.
• Designed to help visually impaired users navigate indoor and outdoor spaces more safely.

Caleb Craig/Penn State

This is important because: Previous systems for the visually impaired required pre-installed object models, making them slow and limited. NaviSense overcomes this limitation by connecting to an external server powered by AI models to recognize objects in real-time based on voice commands without pre-training the environment.

It uses the smartphone camera, spatial audio and vibration to guide you to an object, tell users whether it’s left, right, up or down, and give a “bull’s eye” confirmation when their hand is in the right place. The app also tracks hand movements through phone movements, asks follow-up questions if your request is unclear, and instantly adapts to new spaces like kitchens, walkways, or stores.

Caleb Craig/Penn State

Why should I care? Even if you are not visually impaired, this shows how AI can close gaps and make everyday life more accessible. In a world where technology too often feels flashy, this feels deeply human.

• NaviSense could give visually impaired people far more independence and confidence when navigating unfamiliar spaces.
• The technology could be integrated into smartphones or wearables, making it more accessible in everyday life.

Okay, what’s next? Vijaykrishnan Narayanan, a professor at Evan Pugh University who works on NaviSense, said: “This technology is close to commercial release and we are working to make it even more accessible.” AI often sounds futuristic or abstract, but it’s the kind of real-world impact that immediately catches the eye.