As multimodal artificial intelligence (AI) continues to advance, researchers are developing various studies and services to assist people with disabilities. One recent innovation is a vision AI camera designed to recognize pedestrians and obstacles, guiding visually impaired individuals.
On Wednesday, a wearable vision AI system developed by Dr. Leilei Gu and others from Shanghai Jiao Tong University was featured in the international journal, Nature Machine Intelligence.
This wearable system consists of artificial skin patches that can be attached to glasses, wrists, and fingers. Each patch is equipped with an independently functioning vision AI camera.
The camera acts as an eye, alerting users to obstacles from various angles. A miniature computer analyzes video footage using machine learning algorithms to identify people, doors, walls, and furniture. Visually impaired pedestrians receive navigation instructions through left and right earbuds every 0.25 seconds.
The skin patch worn on the hand vibrates to warn users when it detects obstacles within 40 cm. It also provides feedback through vibration to indicate when users can grab an object as they extend their hands.
The research team conducted a test with 20 visually impaired participants, having them navigate a 25-meter indoor maze. Results showed that the camera system was over 25% more effective than using a cane in terms of distance walked and time taken.
Dr. Gu stated that he plans to improve device miniaturization, service reliability, and safety for practical application.
Some research projects have already transitioned into real-world services. In February of this year, Nvidia partnered with the American Society for Deaf Children to develop a platform called Signs, which helps deaf infants and toddlers learn American Sign Language (ASL).
The platform features a 3D avatar trained with sign language videos that demonstrate signs for users to imitate. A webcam AI tool analyzes the user’s sign language motions and provides real-time feedback.
Nvidia aims to enhance the service by acquiring 400,000 video clips containing 1,000 sign language words as training data.
Cheri Dowling, Executive Director of the American Society for Deaf Children, explained that the service allows deaf children as young as 6 to 8 months to begin communicating effectively with their families. She noted that the sign language data used has been verified by professional ASL teachers, ensuring its reliability.
In South Korea, SK Telecom collaborated with Tuat in 2022 to develop an AI-based visual assistance voice guidance application called Sullivan+. This project utilized SK Telecom’s voice AI, NUGU, along with vision AI technologies.
The AI reads text captured by the smartphone camera and informs users about their surroundings and people present.
A representative from SK Telecom explained that the development of these services is not driven by market potential but rather by the company’s commitment to environmental, social, and governance (ESG) initiatives aimed at expanding the benefits of technology.