For decades, Artificial Intelligence (AI) has been a part of the fantasy world and remained in the comic books and movies. And if you ve seen Marvel s Iron Man movies, you may have come across Jarvis, Tony Stark s AI-powered powered butler that runs the house. Of course, the concept sounds too futuristic, but with the kind of means and technology we have today, building a basic AI-based assistant isn t difficult. Earlier this year, Facebook CEO Mark Zuckerberg took a personal challenge to build a Jarvis-style AI to run his home. He not only managed to successfully complete the challenge, but also offered a sneak peek at its working. Also Read - Happy Friendship Day 2021: How to send Friendship Day wishes Stickers on WhatsAppAlso Read - Facebook is finally bringing 'smart glasses' in collaboration with Ray-Ban
Over the past three years or so, we have seen a mushrooming growth of AIs in different formats, right from voice assistants such as Google Assistant, Siri and Cortana, to chat bots that work within Facebook Messenger. However, these AIs are designed for simple tasks, such as searching for information online, sending texts or making phone calls, and for answering basic general knowledge queries. Google as also integrated Nest support which lets you control the thermostat or lights using voice commands. The command can be as simple as, OK Google, set the temperature to 20 degrees. Also Read - Google, Facebook make vaccination mandatory for employees returning to office
When Zuckerberg took the challenge, he mentioned that Jarvis-like AI could be designed to control almost everything in the house, from opening doors, to turning the lights on and off, to controlling the thermostat and appliances. The interviewer and noted comedian Jerry Seinfeld had mocked Zuckerberg AI saying, “Isn’t it funny how hard we work to just eliminate a little bit of effort. I mean to get the door is not much work.” While Zuckerberg did laugh it off then, he has now come out with whatever success he has achieved in these months. ALSO READ: Facebook CEO Mark Zuckerberg shares 360-degree video of baby Max s first steps
Zuckerberg recently offered Fast Company with a sneak peek into his house to demonstrate the working of his AI assistant. Written in Python, PHP and Objective C, and spending around 150 hours, Jarvis uses several artificial intelligence techniques such as natural language processing, speech and face recognition, and reinforcement learning. Zuckerberg also wants it to “learn his tastes and patterns, learn new words and concepts, and even entertain his daughter Max.”
“For assistants like Jarvis to be able to control everything in homes for more people, we need more devices to be connected and the industry needs to develop common APIs and standards for the devices to talk to each other. I had to reverse engineer APIs for some of these to even get to the point where I could issue a command from my computer to turn the lights on or get a song to play, Zuckerberg said. ALSO READ: Alphabet s Larry Page named the world s most powerful CEO, Mark Zuckerberg grabs second spot
Most home appliances aren t connected to the internet yet. While it is possible to use internet-connected power switches to remotely turn them on or off, it isn t enough. Giving an example, Zuckerberg explained that it is difficult to find a toaster that can push the bread down when its switched off. To make it happen, he had to look for a 1950s toaster and use a connected switch.
Zuckerberg built the first version of Jarvis app for iOS, and plans to build one for Android too. He created a Messenger chat bot to text commands to his assistant, rather than having a completely different app. To begin with, Zuckerberg first thought Jarvis to respond to simple keywords such as bedroom, lights, on, off, and more. He uses the custom iPhone app or Messenger bot to turn the lights on or off. ALSO READ: From GIF comments to offline Messenger; here are some interesting features coming to Facebook
This was followed by enabling voice, vision and facial recognition. It uses Facebook’s facial recognition technology to scan faces from cameras positioned at the front door. It s a two-step process that detects the face and runs face recognition to identify the person. Once identified, it will see if Zuckerberg is expecting that person, and then let them in, while also letting him know. It will also text him the photo of the person. ALSO READ: Inside Mark Zuckerberg s plan to make Facebook video-first
One challenge Zuckerberg found was to get Jarvis parse similar phrases. For instance, play Someone Like You means the specific song by Adele, instead it would recommend songs similar to that phrase. This is because phrases like someone like you, someone like Adele, and some Adele sound very similar. Also, to turn off the lights Zuckerberg and his wife uses different phrases like living room or family room and Jarvis cannot understand synonyms. ALSO READ: Mark Zuckerberg shares rare photos of Facebook s highly advanced data center
Zuckerberg s version of Jarvis is still learning and is set to improve over time. He also mentioned that he would open source the code, but as it’s currently too tightly tied with his own home, appliances and network configuration. If he manages to built a layer to abstract home automation functionality, he may make the code official.