comscore Mark Zuckerberg offers a sneak peek at Jarvis, the AI butler he uses to control his home
News

Mark Zuckerberg offers a sneak peek at Jarvis, the AI butler he uses to control his home

Zuckerberg spent around 150 hours to build the AI assistant. It uses natural language processing, face and speech recognition to carry out simple tasks.

mark-zuckerberg-facebook

For decades, Artificial Intelligence (AI) has been a part of the fantasy world and remained in the comic books and movies. And if you ve seen Marvel s Iron Man movies, you may have come across Jarvis, Tony Stark s AI-powered powered butler that runs the house. Of course, the concept sounds too futuristic, but with the kind of means and technology we have today, building a basic AI-based assistant isn t difficult. Earlier this year, Facebook CEO Mark Zuckerberg took a personal challenge to build a Jarvis-style AI to run his home. He not only managed to successfully complete the challenge, but also offered a sneak peek at its working. Also Read - Happy Friendship Day 2021: How to send Friendship Day wishes Stickers on WhatsApp

Also Read - Facebook is finally bringing 'smart glasses' in collaboration with Ray-Ban

Over the past three years or so, we have seen a mushrooming growth of AIs in different formats, right from voice assistants such as Google Assistant, Siri and Cortana, to chat bots that work within Facebook Messenger. However, these AIs are designed for simple tasks, such as searching for information online, sending texts or making phone calls, and for answering basic general knowledge queries. Google as also integrated Nest support which lets you control the thermostat or lights using voice commands. The command can be as simple as, OK Google, set the temperature to 20 degrees. Also Read - Google, Facebook make vaccination mandatory for employees returning to office

When Zuckerberg took the challenge, he mentioned that Jarvis-like AI could be designed to control almost everything in the house, from opening doors, to turning the lights on and off, to controlling the thermostat and appliances. The interviewer and noted comedian Jerry Seinfeld had mocked Zuckerberg AI saying, “Isn’t it funny how hard we work to just eliminate a little bit of effort. I mean to get the door is not much work.” While Zuckerberg did laugh it off then, he has now come out with whatever success he has achieved in these months. ALSO READ: Facebook CEO Mark Zuckerberg shares 360-degree video of baby Max s first steps

Zuckerberg recently offered Fast Company with a sneak peek into his house to demonstrate the working of his AI assistant. Written in Python, PHP and Objective C, and spending around 150 hours, Jarvis uses several artificial intelligence techniques such as natural language processing, speech and face recognition, and reinforcement learning. Zuckerberg also wants it to “learn his tastes and patterns, learn new words and concepts, and even entertain his daughter Max.”

“For assistants like Jarvis to be able to control everything in homes for more people, we need more devices to be connected and the industry needs to develop common APIs and standards for the devices to talk to each other. I had to reverse engineer APIs for some of these to even get to the point where I could issue a command from my computer to turn the lights on or get a song to play, Zuckerberg said. ALSO READ: Alphabet s Larry Page named the world s most powerful CEO, Mark Zuckerberg grabs second spot

Most home appliances aren t connected to the internet yet. While it is possible to use internet-connected power switches to remotely turn them on or off, it isn t enough. Giving an example, Zuckerberg explained that it is difficult to find a toaster that can push the bread down when its switched off. To make it happen, he had to look for a 1950s toaster and use a connected switch.

Zuckerberg built the first version of Jarvis app for iOS, and plans to build one for Android too. He created a Messenger chat bot to text commands to his assistant, rather than having a completely different app. To begin with, Zuckerberg first thought Jarvis to respond to simple keywords such as bedroom, lights, on, off, and more. He uses the custom iPhone app or Messenger bot to turn the lights on or off. ALSO READ: From GIF comments to offline Messenger; here are some interesting features coming to Facebook

This was followed by enabling voice, vision and facial recognition. It uses Facebook’s facial recognition technology to scan faces from cameras positioned at the front door. It s a two-step process that detects the face and runs face recognition to identify the person. Once identified, it will see if Zuckerberg is expecting that person, and then let them in, while also letting him know. It will also text him the photo of the person. ALSO READ: Inside Mark Zuckerberg s plan to make Facebook video-first

One challenge Zuckerberg found was to get Jarvis parse similar phrases. For instance, play Someone Like You means the specific song by Adele, instead it would recommend songs similar to that phrase. This is because phrases like someone like you, someone like Adele, and some Adele sound very similar. Also, to turn off the lights Zuckerberg and his wife uses different phrases like living room or family room and Jarvis cannot understand synonyms. ALSO READ: Mark Zuckerberg shares rare photos of Facebook s highly advanced data center

Zuckerberg s version of Jarvis is still learning and is set to improve over time. He also mentioned that he would open source the code, but as it’s currently too tightly tied with his own home, appliances and network configuration. If he manages to built a layer to abstract home automation functionality, he may make the code official.

For the latest tech news across the world, latest PC and Mobile games, tips & tricks, top-notch gadget reviews of most exciting releases follow BGR India’s Facebook, Twitter, subscribe our YouTube Channel. Also follow us on  Facebook Messenger for latest updates.
  • Published Date: December 20, 2016 6:10 PM IST



new arrivals in india

Micromax In 2b
Micromax In 2b

8,999

Vivo Y72 5G
Vivo Y72 5G

20,990

Tecno Camon 17
Tecno Camon 17

12,999

Tecno Camon 17 Pro
Tecno Camon 17 Pro

16,999

Realme C11 2021
Realme C11 2021

6,999

Oppo Reno6 Pro 5G
Oppo Reno6 Pro 5G

39,990

Oppo Reno6 5G
Oppo Reno6 5G

29,990

Samsung Galaxy M21 2021
Samsung Galaxy M21 2021

12,499

OnePlus Nord 2
OnePlus Nord 2

27,999

Poco F3 GT
Poco F3 GT

27,999

Samsung Galaxy A22 5G
Samsung Galaxy A22 5G

19,999

Xiaomi Redmi Note 10T 5G
Xiaomi Redmi Note 10T 5G

13,999

Samsung Galaxy F22
Samsung Galaxy F22

12,499

Xiaomi Mi 11 Lite
Xiaomi Mi 11 Lite

21,999

Infinix Note 10 Pro
Infinix Note 10 Pro

16,999

Infinix Note 10
Infinix Note 10

10,999

Vivo Y73
Vivo Y73

20,990

OnePlus Nord CE 5G
OnePlus Nord CE 5G

22,999

iQOO Z3
iQOO Z3

19,990

Realme C25s
Realme C25s

9,999

Poco M3 Pro 5G
Poco M3 Pro 5G

13,999

Realme X7 Max 5G
Realme X7 Max 5G

26,999

Oppo F19
Oppo F19

18,990

Motorola Moto G40 Fusion
Motorola Moto G40 Fusion

13,999

POCO M2 Reloaded
POCO M2 Reloaded

9,499

OPPO A74 5G
OPPO A74 5G

17,990

Oppo A53s 5G
Oppo A53s 5G

14,990

Vivo V21 5G
Vivo V21 5G

29,990

Realme C25
Realme C25

9,499

Realme C21
Realme C21

7,999

Realme C20
Realme C20

6,799

Motorola Moto G60
Motorola Moto G60

17,999

iQOO 7
iQOO 7

31,990

Samsung Galaxy M42 5G
Samsung Galaxy M42 5G

21,999

Xiaomi Mi 11 Ultra
Xiaomi Mi 11 Ultra

69,999

Xiaomi Mi 11X Pro 5G
Xiaomi Mi 11X Pro 5G

39,999

Xiaomi Mi 11X
Xiaomi Mi 11X

29,999

Realme 8 5G
Realme 8 5G

13,999

Samsung Galaxy F02s
Samsung Galaxy F02s

8,999

Samsung Galaxy F12
Samsung Galaxy F12

10,999

POCO X3 Pro
POCO X3 Pro

18,999

Realme 8 Pro
Realme 8 Pro

17,999

Realme 8
Realme 8

14,999

Vivo X60 Pro Plus
Vivo X60 Pro Plus

69,990

Vivo X60 Pro
Vivo X60 Pro

49,990

Vivo X60
Vivo X60

37,990

OnePlus 9 Pro 5G
OnePlus 9 Pro 5G

64,999

OnePlus 9R 5G
OnePlus 9R 5G

39,999

OnePlus 9 5G
OnePlus 9 5G

49,999

Samsung Galaxy A72
Samsung Galaxy A72

34,999

Samsung Galaxy A52
Samsung Galaxy A52

26,499

Micromax In 1
Micromax In 1

10,499

Asus ROG Phone 5
Asus ROG Phone 5

49,999

Samsung Galaxy M12
Samsung Galaxy M12

10,999

Motorola Moto G30
Motorola Moto G30

10,999

Motorola Moto G10 Power
Motorola Moto G10 Power

9,999

Oppo F19 Pro Plus 5G
Oppo F19 Pro Plus 5G

25,990

Oppo F19 Pro
Oppo F19 Pro

21,490

Xiaomi Redmi Note 10 Pro Max
Xiaomi Redmi Note 10 Pro Max

18,999

Xiaomi Redmi Note 10 Pro
Xiaomi Redmi Note 10 Pro

15,999

Xiaomi Redmi Note 10
Xiaomi Redmi Note 10

11,999

Realme Narzo 30A
Realme Narzo 30A

8,999

Realme Narzo 30 Pro
Realme Narzo 30 Pro

16,999

Infinix Smart 5
Infinix Smart 5

7,199

Samsung Galaxy F62
Samsung Galaxy F62

23,999

Samsung Galaxy A12
Samsung Galaxy A12

12,999

Nokia 5.4
Nokia 5.4

13,999

Nokia 3.4
Nokia 3.4

11,999

Realme X7 Pro 5G
Realme X7 Pro 5G

29,999

Realme X7
Realme X7

19,999

Vivo Y31
Vivo Y31

16,490

Oppo Reno5 Pro 5G
Oppo Reno5 Pro 5G

35,990

Samsung Galaxy S21 Ultra 5G
Samsung Galaxy S21 Ultra 5G

1,05,999

Samsung Galaxy S21 Plus 5G
Samsung Galaxy S21 Plus 5G

81,999

Samsung Galaxy S21 5G
Samsung Galaxy S21 5G

69,999

Vivo Y12s
Vivo Y12s

9,990

Vivo Y51A
Vivo Y51A

17,990

Samsung Galaxy M02s
Samsung Galaxy M02s

8,999

Xiaomi Mi 10i
Xiaomi Mi 10i

21,999

Oppo A15s
Oppo A15s

11,490

Tecno Spark 6 Go
Tecno Spark 6 Go

8,499

Vivo V20 2021
Vivo V20 2021

24,990

Vivo Y20A
Vivo Y20A

11,490

Xiaomi Redmi 9 Power
Xiaomi Redmi 9 Power

11,999

Motorola Moto G9 Power
Motorola Moto G9 Power

11,999

Motorola Moto G 5G
Motorola Moto G 5G

20,999

Vivo V20 Pro
Vivo V20 Pro

29,990

Xiaomi Mi 10T
Xiaomi Mi 10T

35,999

Xiaomi Redmi 9i
Xiaomi Redmi 9i

8,299

Xiaomi Mi 10T Pro
Xiaomi Mi 10T Pro

39,999

Infinix Hot 10
Infinix Hot 10

9,999

Vivo V20 SE
Vivo V20 SE

20,990

Vivo V20
Vivo V20

24,990

Micromax In 1b
Micromax In 1b

6,999

Micromax In Note 1
Micromax In Note 1

10,999

OnePlus 8T
OnePlus 8T

42,999

Samsung Galaxy F41
Samsung Galaxy F41

15,499

Best Sellers