Behind The Robots Wed, Sep 5, '18
Interview with Daniel Schofield, Solutions Engineer


At SoftBank Robotics America (SBRA) we build robots, but our biggest strength is our people. Through the Behind The Robots series we hope to showcase the passion and drive of SBRA employees from a wide variety of backgrounds, geographies, and skill sets, who have come together to make Pepper a uniquely human humanoid.

Today, we’ve interviewed Daniel Schofield, a Solutions Engineer at Softbank Robotics. Daniel plays a key role in helping customers create and actualize their vision for leveraging the power of Pepper to drive business results.

SBRA: What is your role at SoftBank? 


Daniel: My role is very multidisciplinary. I originally was hired as a technical trainer, and I still do a lot in the way of technical documentation, training course creation, and more. My primary role is to work alongside the professional services team, sales team, and customer success team to scope and build out a Pepper solution that can live up to the promise and personality of what we advertise. In that capacity, I’ve played a pivotal role in building solutions for some of our largest clients.

SBRA: Were you always interested in technology?

Daniel: Yes, but my relationship with it has changed over the years. I grew up coding and took computer science courses in high school, but as far as my career, I actually got my start in film. After film school, I spent five years in “Hollywood” working for companies like Dreamworks and AMC, producing big budget commercials, and even started writing my own sci-fi screenplays.

The turning point came after I got the advice to turn my screenplays into a novel series, and realized I didn’t have enough technical knowledge to finish it well. From there, I did a hard pivot back into technology, and the rest is kind of history.

SBRA: What kinds of solutions do you develop for Pepper?

Daniel: Ultimately, what customers want is for Pepper to feel that there is a real personality behind those big emotive eyes. And creating the “reality” of Pepper most often comes down to her language. Language is a big area of focus for my team, and where we spend a lot of time.

Customers want Pepper to talk about a lot of things out-of-the-box, so they only have to train her on understanding their business-specific issues. This means there must be a core Pepper personality layer. When customers come and ask Pepper, “Are robots going to take over the world,” “Who are you,” “Who made you,” and other questions of that nature, we have created a personality and an experience that is innate to Pepper.

SBRA: What goes into building a robot’s personality?

Daniel: When it comes to building out Pepper's core personality, it's a balancing act between keeping Pepper's persona brand-compliant and conservatively conscientious on the one hand and giving it spirit and flair on the other.

What's really interesting is looking at the data to see what people are actually saying to Pepper and then categorizing and subcategorizing that data in order to more clearly see the full range of topics about which people expect to be able to converse with Pepper.

For example, seeing user questions relating to Pepper's persona, such as inquiring about her favorites and least favorites, help us to realize that someone else will likely ask Pepper about her likes and dislikes. Only by categorizing the data in this way can we actually begin to see where the holes are in her current knowledge base and start to fill them in.

Another consideration we must take into account is the very different ways in which people speak and communicate. Through most of the 20th century, nearly all interaction with computers was done through tightly controlled, binary interfaces such as keyboards and mouses that registered clicks and key strokes as 1s and 0s.

With Pepper, the foundation of every interaction is conversation, which depends on an amazing, but imperfect, machine-learning-based technology called Natural Language Processing (NLP). In other words, it is a much more complex and error-prone interface that we have to account for. So what this all means for creating a Pepper experience is that we must really anticipate and build out a wide range of language diversity when designing the conversation's inputs and outputs.

For example, a simple prompt for bathroom directions can stretch anywhere between the extremes of "Pardon, can you direct me to the nearest ladies' room?" to "hey, you got a toilet?" To date, this dynamic range of user inputs still needs to be mapped to the desired output by manually training the machine learning model.

SBRA: What are the biggest challenges of building out Pepper’s dynamic responses?

Daniel: Brains, minds, and personalities are very complex, so this work can be complicated. A lot of work goes into building out the linguistic map, and we also have to work to avoid things like intent collision.

What is intent collision? Behind everything that you say there is an intent, and the form for how you express an intent is an utterance, what a person actually says. All of those utterances must be mapped to intents, which -- in chatbot language -- means a basic conversational unit about which the chatbot is capable of communicating. This is the basis for all chatbots, and also is the basis for how Pepper delivers a conversational experience. So going back to intent collisions: when the user’s inputs (utterances) for two different intents are too similar, then oftentimes a user input intending to trigger one of the intents will incorrectly end up triggering the other. That’s intent collision and can be a real problem with very large chatbots.

For example, when Pepper is in a supermarket she needs to be able to talk about avocados. There are a million things people can ask about avocados - how much they cost, where they are, or if they are organic. We have that all programmed in. But sometimes people throw a curve ball. Someone might ask what color avocados are, or whether avocados come from a tree or a bush. If that’s not in Pepper’s knowledge graph, she can’t reply with the same common sense as a human, and she might say how large an avocado is, when someone asked about the color.

This isn’t just a Pepper problem, it’s an unsolved problem in the field of computer science. Google has predictions that we will have it solved within the next five years, but in the meantime, we focus on the 80 / 20 rule, to make sure there is enough content for Pepper to be successful in the broadest set of experiences.

Additionally, because we are now living in the world of Siri, Alexa, Google Home, Cortana, etc., people are becoming increasingly comfortable speaking to an entity that is human-like. These chatbots have been grown through manual human efforts and machine-learning technology, and can process somewhere in the order of 60 – 100k intents. That’s not easy work to replicate, and it’s proprietary, so we can’t just go out and put Google on our robot. Essentially, we’ve had to build Pepper’s mind from scratch.

SBRA: As someone who has been very involved with the development of Pepper’s mind, is it possible to still be surprised by her?

Daniel: Believe it or not, I’m always surprised by Pepper. I’ve been at SoftBank for a long time, and have put a lot of hours into developing elements for Pepper’s conversational agent. But because you can access any part of Pepper’s intent engine at any point in the conversation, connections can be made that aren’t always obvious to the designer.

Recently, Pepper answered a question I hadn’t programmed into her chatbot. I had programmed something similar, and she used that knowledge to give me an answer that was even better than if we had programmed the direct question.

Moments like that are pure magic, and are what keep me inspired. You realize that, at some point, if you build a massive personality for Pepper, it will eventually be impossible to remember everything you created. And with the power of machine-learning, there is no limit to the freshness and realness with which Pepper can converse. In that sense, I’m constantly reminded that we’re truly bringing something to life here.

SBRA: What would you say to someone who wants to get involved with robotics but doesn’t have a computer science degree?

Daniel: Obviously having a computer science degree is great, but I was a film major, and now I’m here. So, even more important than the degree is the passion and drive.

Overall, my biggest recommendation for someone wanting to get into technology is to become a constant learner, and a self-learner. The industry -- and robotics in particular -- is moving at a furious pace, so if you have the willingness to understand it from an engineering perspective, and have the passion and drive, there is a lot of opportunity to grow.