This year Mark Zuckerberg set himself the goal of building an artificial intelligence system that could run his home. If you think¬†ol’ Zuck has watched Iron Man one to many times you’d be right because that’s what inspired the idea, and the name.

Jarvis, as the Facebook creator calls the system, can control many aspects of Zuckerberg’s home including appliances, lighting, temperature and security. This being an AI system it will constantly learn from the faces it sees, the voices it hears and the messages it receives.

Building Jarvis was not easy and making it so that the AI was able to recognise and act on verbal queues correctly was quite the challenge according to Zuckerberg, who detailed the build in a blog post.

The reason for this he says¬†comes down to context. “For example, when I tell it to turn the AC up in “my office”, that means something completely different from when Priscilla tells it the exact same thing,” Zuckerberg wrote.

Despite this and other problems such as having the system recognise conversational speech rather than commands, the Facebook creator says that Jarvis is a good base to build from and that he wants to make it seem more like a part of his home than a friendlier (for now) HAL 9000.

“Ever since I built voice into Jarvis, I’ve also wanted to build in more humour. Part of this is that now it can interact with Max and I want those interactions to be entertaining for her, but part of it is that it now feels like it’s present with us. I’ve taught it fun little games like Priscilla or I can ask it who we should tickle and it will randomly tell our family to all go tickle one of us, Max or Beast,” wrote Zuckerberg.

Texts from AI

Even though Zuckerberg wants to be able to talk to Jarvis he says that text is more important for communicating with AI than many think. “That’s because voice can be disruptive and text gives you more control of when you want to look at it. Even when I speak to Jarvis, if I’m using my phone, I often prefer it to text or display its response,” Zuckerberg says.

Text based commands allow Zuckerberg to use Facebook’s Messenger bot platform to command the system and allows him to easily manage permissions and look at responses from the AI when it’s convenient for him.

“Overall, this was a great challenge,” says Zuckerberg. “These challenges have a way of teaching me more than I expected at the beginning. This year I thought I’d learn about AI, and I also learned about home automation and Facebook’s internal technology too.”

Jarvis is not going to be coming to your home anytime soon, the system is deeply rooted in Zuckerberg’s home and he’d need to build a component that allows for more varied home automation.

With that having been said the creator says he’d considering open sourcing his code if he builds that component.

[Image – CC BY SA Andrew Feinberg]