Facial recognition is becoming the next big form of biometric authentication on mobile devices and applications. Google has come up with a new method to match faces and voices that includes new ability to recognize and match multiple faces and voices in a video. The search giant has filed patent for the same with the World Intellectual Property Organization.
The patent application describes that the method focuses on determining when somebody is talking and then matching their voice to their face. This approach gives the machine ability to hear and understand individual voices even in noisy environments by simply looking for a face that matches the sound pattern corresponding to a pre-defined voice profile.
The patent details Google using a voice diarization system that starts by finding faces then watches those faces in the video to determine when somebody’s talking. The goal is catch someone talking alone and then isolate their voice by confirming any audio matches with the movement of their mouth. Google then positively profiles that voice and files it together with the face to create a hard match.
In crowded situations or environments with multiple speakers, Google’s algorithm repeats the procedure for every person who speaks in the video. Once it creates a profile for everybody, the system becomes intelligent enough to tell who is speaking and when, as well as understand what they are saying by reading their lips. Android Headlines notes that the machine also transcribes the audio to check that mouth movement matches what the machine’s audio processing thinks the person is saying.
Watch: Asus Zenfone 5Z First Look Video
The new technology is developed using computer vision and machine learning and can be used in number of environments. In the patent application, Google describes using the technology for automatic captioning, which could be deployed as a feature on YouTube. YouTube already supports automatic captioning but its accuracy is not that good. The tech could also be used for improved facial recognition and security on hardware devices. There are many other use cases for the technology once WIPO authorizes the patent application.