Smartphone with opend Ask Mercedes application

Artificial intelligence meets personality

Augmented reality and the ‘Ask Mercedes’ chatbot project

No more browsing through instruction manuals needed

Take a seat in the car and let your eye wander over the still-unfamiliar cockpit. Is there a control that is not immediately obvious as to what it does or a function you cannot find straight away? Just Ask Mercedes. Daimler is enhancing communication with additional digital channels, making leafing through operating instructions, handbooks or quick start guides a thing of the past. The ‘Ask Mercedes’ application ensures that every user gets their bearings in the vehicle quickly. The virtual assistant is activated by voice, using a smartphone keyboard or camera. A system comprising artificial intelligence (AI) and augmented reality (AR) running in the background then ensures that even a modern car with its numerous functions, possible configurations and assistance systems can be understood at the drop of a hat.

But Ask Mercedes is just one iteration of a new concept that Daimler wants to use to reach and support its customers in an unprecedented way. Other chatbot applications and contents are coordinated via an ‘intelligent’ information hub.

More functions, less complexity

The idea for this chatbot application was born during Digital Life Day 2016. The aim of Daimler’s in-house format is to identify trends and digital topics. Employees are encouraged to present and submit their ideas in a mix of an Ideas Exchange and workshop. The participants identified the chatbot concept as a modern way of making it easier for customers to interact with their vehicles – vehicles which offer an ever-increasing number of functions, thus also making them more complicated.

Everything then happened very quickly following the Digital Life Day: A so-called minimum viable product (MVP) was developed to test the feasibility and the acceptance of the chatbot. This approach originated in the lean start-up movement and helps companies to better estimate a product or service’s chances of success. However, this is not done through surveys, but instead through feedback from users who try out early versions of a new product. This feedback is then used to enhance and improve the next version.

At the end of the MVP phase, the participants were certain that chatbots could ensure Mercedes-Benz customers get to know and experience their vehicle in a completely new way. The experts then added another future trend to the concept – augmented reality. The functionalities of Ask Mercedes are built on these two pillars.

Daimler is positioning Ask Mercedes precisely on a customer pain point. The range of functions in vehicles and the supplementary service offer is continuously increasing. That’s why it is becoming more and more important to engage in a close dialogue with customers. The company wants customers to be able to inform themselves and get answers to unresolved issues anytime and anywhere. The first steps of this process should be as simple and interesting as possible for the user, which is why additional quick start guides and tutorial videos are available on the website alongside the detailed operating instructions. Ask Mercedes is not replacing the previous sources of information, it is simply providing the driver with another way of getting to know their vehicle. The application is a new, interactive addition to the options for communicating with customers. For one thing, Ask Mercedes is capable of understanding and answering questions asked by the customer in natural language or submitted via text. As well as that, an integrated camera function powered by natural language understanding and augmented reality technologies allows the driver to scan controls and have the relevant explanations displayed on their smartphone.

This eliminates the need to search through pre-set structures such as alphabetised lists. Instead, Ask Mercedes always puts the needs of the customer and their situation first by giving them the exact information they need at that specific moment whilst remaining both charming and informative.

Short channels lead to the desired outcome

It was clear once the MVP phase had been successfully completed that this type of project could only be implemented with an experienced partner on board. The experts from adesso prevailed in an extensive selection process. The decision-makers at Daimler were impressed by the employees’ experience and expertise with chatbots, as well as adesso’s product and manufacturer-independent positioning.

This was important as it was clear to the participants from the start that having Ask Mercedes as the core component of the operating instructions is only one application case. Chatbot functions can also be used in a meaningful way in other company departments such as the Customer Contact Center, Daimler’s customer support. “We are developing an ‘intelligent’ information hub together with Daimler that acts as the core for all chatbot applications,” explains Stefan Hussmann, Director of the Automotive and Transportation Line of Business at adesso SE. “The hub is a content library and information hub. Different specialist departments add their content to the hub, then the chatbots take this content and present it to the user.”

Stefan Hussmann, Director of the Automotive and Transportation Line of Business at adesso SE

Caption: Stefan Hussmann, Director of the Automotive and Transportation Line of Business at adesso SE

This IT platform should not be dependent on one provider. This is because the project participants wanted to keep all options open to them in terms of being able to react to new developments or trends, given that they are operating in a technological environment in which much is still in a state of flux and in which more and more technologies with new possibilities are emerging. A suitable middleware was developed to implement this manufacturer-independent IT approach.

In April 2017, Daimler and adesso set about developing this infrastructure. They released the first iOS version of an app just seven months later. In this short period of time, they together established the technical foundations, developed the application and coordinated the content creation process.

“This project can’t be delivered that quickly without an agile approach,” says Stefan Hussman. “We are focussing on a model that we call ‘tamed agility’. This allows us to combine short development cycles and a flexible approach to the requirements of a company such as Daimler with reliability and predictability.” The participants spent the first few weeks establishing requirements, answering questions on the architecture and clarifying issues such as performance. Once they had the foundations, it was not long before they reached the development stage.

To kick off the Ask Mercedes project, the experts from Daimler and adesso chose IBM Watson as the AI basis and decided on components from Vuforia for the AR side. But this technology is just one aspect of Ask Mercedes. The team was faced with the challenge of giving an artificial intelligence a personality.

A chatbot with charm

Ask Mercedes is not just a pure information system. Interacting with the app should feel natural, be it via speech or text. This means the user should forget that they are communicating with a chatbot rather than a person.

The idea is that the chatbot does not simply display sections from the manual that might fit the search. It should be a knowledgeable chatbot having a conversion with a driver about their car.

The fact that Ask Mercedes comes across this way is down to the attention to detail displayed by the team during the development of the ‘Mercedes’ chatbot personality.

The team first designed a complete personality profile. The chatbot was then fed with facts about the car and taught to find the correct answers to the most important questions. To do this, information from every department involved was gathered, which the team then made compatible with the app. At the same time, the participants initially collected around 6,000 customer responses and analysed queries sent to the Daimler Customer Contact Center over the course of just under a year. This gave the team a detailed insight into the information needs of the customers.

While this took place, the participants set about breathing life into the personality profile. After all, it is important that Ask Mercedes’ unique personality also shines through in the way it answers, sometimes with a wink, sometimes with a bit of attitude. “A lot of users began to grill the app a little bit after they had asked the first few technical questions,” says Stefan Hussman. “We wanted to develop a system that doesn’t just provide information but is also just fun and encourages the customers to engage in a dialogue.” As part of the conversational user experience concept, adesso hired scriptwriters to come up with the most creative and surprising answers possible for Ask Mercedes to give in this situation. Just a hint: Ask about the lottery numbers.

Nobody knows where the AI journey will go next

Ask Mercedes has been available in the initial launch markets of India, Malaysia, South Africa and with a localised version in the USA since the end of 2017. Australia, New Zealand and Hong Kong followed in the middle of April. More countries, new models (the app is currently available only for Mercedes S, E, A and C class) and more languages and functions will be added gradually. The functions of the system will also be offered via other channels in the future. The next step will see Ask Mercedes integrated into the Facebook Messenger ecosystem, with content integration into voice assistants Google Home and Amazon Alexa following later. The aim is to enable users to remain in contact with their vehicle at any time and from anywhere. The advantages of the selected IT infrastructure with a flexible middleware are evident even when it comes to expanding Ask Mercedes. Experts can quickly and easily integrate new functions during operation.

There are already several chatbots that perform their services around the ‘intelligent’ information hub. Projects in areas such as CASE, smart or VAN are running alongside the previously mentioned Customer Contact Center. This allows the platform to be gradually expanded with other subject areas.

So that the team can expand the system as outlined, organisational and structural requirements must now be created for a viable long-term positioning. While the first few months of the project focussed on speed and agility, the focus has now shifted to developing stable processes and standardising workflows. The participants are now working together on an organisation that enables the chatbot and augmented reality to be developed further.

Ask Mercedes shows that these new technologies provide the opportunity to think ahead, for example, in order to interactively design communication with customers. What it needs is an IT structure that flexibly cooperates with any development. And, of course, a team that wants to find these paths, as well as see where they lead.

Do you have any questions?

There is no website or brochure which can replace a personal meeting to talk about your goals and topics. We are looking forward to an appointment on site.


Save this page. Remove this page.