Strategic planning and prototyping of chatbots Chatbots with great user experience

Like with other digital products that are part of a strategy and build on thorough testing, you should first be clear about technological possibilities, business objectives, and user needs. When aligning these three areas, you have good chances of creating a successful chatbot. After this strategic planning, it is time for prototyping. Step by step, the chatbot is being developed, always close to the target audience and based on their constant feedback. With this basic explanation of the process, it is time to have a closer look at the various aspects involved.

Creating a good chatbot strategy

The basis of every successful digital product or service is a good strategy. Chatbots are no exemption. The following chapters explain the basics of chatbot strategy, while the following paragraphs dig deeper into prototyping.

Understanding chatbot technology

One of the advantages of chatbots is their flexibility. Chatbots integrate into websites, apps, social networks, and messengers (Skype, Facebook, Messenger, Slack, Telegram). As customers are distributed across more and more channels, this means chatbots are an important part of the communication mix.

Like voice assistants (Alexa, Siri), chatbots are conversational interfaces, allowing users to interact via language, in this case using a chat interface. Many platforms also allow integrating graphical elements like buttons, image galleries, or maps. However, the primary communication between humans and bots is written language. Chatbots build on three principal building blocks to analyze language: utterances, intents, and entities.

• Everything that users write is an utterance. These utterances are the basis of communication between users and bots.
• Behind an utterance is the intent: whatever a user wants to achieve in a dialogue. This is what needs to be extracted. Developers often formulate intents as verbs with nouns, like “giveTrafficInfo” (in a bot about traffic information) or “showIBAN” (in a banking bot showing one's international bank account number).
• Finally, there are also entities. These are parts of the sentence which modify the intent, like “A1” or “of today.” This information acts as a filter allowing to adapt the answers of a bot to the situation of the user: It should look for traffic jams on the A1, not on other roads.

First, a chatbot registers the text that users enter, e.g., “tell me about the current traffic situation on the A1”. As the communication with a chatbot happens in written language, it does not need to convert spoken words in writing to process it. However, the bot needs to be tolerant regarding writing errors.

Second, the bot needs to identify intents and entities in the registered utterances. The solution to this problem is Natural Language Processing, a technology that allows computers to process natural language. Services like Microsoft's LUIS (Language Understanding Intelligent Service), Google's Dialogflow, or Facebook's wit.ai accept utterances in different languages as input. Then, they extract defined intents and entities. The result of this analysis, together with the context of a conversation, allows a chatbot to search for the right answer, e.g., by connecting to a company's data via an API.

There are different types of chatbots. Rule-based chatbots work with a range of predefined rules. However, this technology has its limits. In particular, the communication fails when users use phrases that the developer did not anticipate. For example, one might say “Is there a traffic jam on the A1 at the moment?” or “Should I leave now or continue working for an hour?” instead of the anticipated “Tell me about the current traffic situation on the A1”.

Alternatively, developers can use artificial intelligence (AI) technology to build chatbots. In particular, machine learning is a common solution, allowing a chatbot to learn from the communication with its users.

Finding use cases to build a chatbot strategy

Technological trends alone are no guarantee for a successful bot. It is also necessary to find the right use case. In principle, chatbots shine in smaller, well-defined use cases that allow for convincing dialogues. Bot directories like Botlist point to a large variety of bots that show the whole spectrum of use cases. However, case studies also help to understand the role that the chatbot technology can play. For example, Sharmadean Reid, who works as a social media influencer and runs the nail studio WAH in London, uses chatbots to organize appointments with her customers. Talking to Frank Pusch in iX magazine 6/2018, she describes that after just two months, 30% of the reservations are made using the automatic system. Thus, she is planning to expand the system and actively propose free slots to frequent customers. Reid's success story shows that even small and medium-sized companies can profit from chatbots.

Another case study shows the use of chatbots in marketing. The team behind the Jäm Bot of Jägermeister collaborated with the rappers Eko Fresh and Ali As. The artists created a large pool of fragments of a rap video. With the help of the bot, users could then create their own personal rap video.

Another advantage of chatbots is that users do not need to install them to interact. This means they are an alternative in situations where users might not want to install an app. An example of this strategy is the air company KLM (Netherlands) which allows customers to book flights via bots and receive information about their flights in Messenger.

Getting to know the users

If you want to create a successful chatbot, you should have a decent understanding of your audience. This understanding is likely the result of thorough user research. Surveys and statistics like the “2018 State of Chatbots Report” with 1,051 American consumers can provide a first orientation. This report contains evidence of the success of chatbots and gives ideas of using them. For example, around every seventh (15 %) consumer has interacted with chatbots. They particularly like that chatbots can give quick answers to questions, help to solve problems and provide details about issues.

Surveys like this can be a good reference point, but for an informed chatbot strategy, you should investigate the perspective of people from your right audience. The challenge is that the most straightforward methods are not always the best, which is particularly true for surveys like “What would you need a chatbot for?”. Potential users do often not know in detail, if and in which situations they would use a chatbot. As long as people do not know chatbots, they often cannot easily imagine use cases. In addition, what people do is not necessarily the same as what they say they will be doing. Thus, answers to surveys like this are always hypothetical and should be taken cautiously.

It is often better to ask people about what they do and how they experience it. How often do they take actions of interests that could potentially profit from a chatbot? What are their pain points, where do they reach limits? What questions, problems, and tasks do frequently come up?

• Obviously, a good source for answers to these questions are all the channels where customers reach out to a company, like people working in customer support, analytics data of frequently visited pages, and search queries.
• Another useful method is to provide potential users with something tangible, like a prototype or a story, which helps them to understand the vision of the creators. Then, you can invite these people to share their comments and begin understanding their needs.
• Personas are an excellent way to make the results of this research actionable. They make it considerably easier to assess whether a chatbot is the right choice for your audience than abstract statistics.

User-centered chatbot prototyping

Based on the knowledge about business goals, technological possibilities, and user needs, it is time for the next step: prototyping. I suggest an iterative approach, where you develop and test solutions for interactions with chatbots step by step, one after another. You could start with very simple prototypes (low fidelity) which you continuously improve based on user feedback. Errors are an inevitable and valuable part of this process. Instead of trying to create a “perfect” first attempt, it is preferable to get prototypes as quickly as possible into users' hands and learn from the feedback you get in this process.

Creating dialogues for chatbots

Every medium has its peculiarities, and with chatbots, this is the dialogue with users. Accordingly, the first round of prototyping is all about how a bot can provide the user with the required information: which intents do they have, which information (entities) do they need, and how do they ask for it (utterances)?

• Sticky notes are an excellent choice to register the flow of dialogue. You can move them around endlessly on a whiteboard to test alternatives. If you include a single part of the conversation on every sticky note, you have a lot of flexibility to rearrange the flow.
• Every utterance of the chatbot should move the conversation forward. It should also be rather short, but still, communicate the personality of the bot.
• In the first phases of the prototyping, the focus is mostly on intents and entities: what do users want to achieve, and in which order will they probably express their wishes? What information does the bot, in turn, need to respond to these needs? How should it ask for this, and what should it infer from the context?
• Typical elements of chat language are useful in the dialogues. This includes emojis and abbreviations like “LOL” (laughing out loud). The chatbot should know a reasonable answer to these utterances. Emojis and natural language allow a bot to communicate its own personality, like the Wienbot of the city of Vienna demonstrates. Besides its charming way of communicating, it also offers a ton of useful information about daily life in Vienna (like weather, parking, or directions).
• It is also worthwhile to think about alternative dialogues. How could a response sound if the bot reacts in a slightly more humorous way? Or if it pokes fun at itself?
• The onboarding deserves special attention. This means the entire process and the experiences that users make when they enter in the conversation with the bot, like welcome messages or how the bot communicates what it can do.
• The default message is also very important. It is a kind of last resort which is always shown when the chatbot does not understand an utterance or cannot respond to it. It should be nice and offer help about continuing the dialogue at the same time.

A default message is also meant to guide users back into the flow of conversation. Hipmunk's bot responds with nice graphics if you send a visual element which it does not understand. Afterward, it gently pushes the situation back to what it can do best: providing travel advice.

However, interacting with a chatbot should not only “work,” but also be fun and create a positive experience. With chatbots, personality carries a long way in this regard. It makes the bot unique and creates an emotional connection to the bot, but should obviously match the needs of users, business requirements, and tasks that the bot has to perform. A skateboard shopping bot will thus act differently than a support bot for upset customers. The bot Mitsuku by Pandorabots, for example, is a virtual personality based on artificial intelligence aiming to create realistic and friendly conversations. Accordingly, it responds with humorous replies if you ask about its well-being or future events.

Age, gender, character, kind (e.g., human, animal, machine), behavior, and origin are essential aspects to consider when creating a personality for a bot. A good method is to create a chatbot persona: a fictional character that often helps when writing dialogues. It is also possible to create several of these bot characters, create the appropriate dialogues, and then test how people from the target audience react towards them.

Fake it till you make it

When you have the dialogues and the personality ready, it is time to test your ideas – ideally, with the least effort required. One way is to use a method named after the Hollywood movie “The Wizard of Oz” by Victor Fleming from 1939. In the film, Dorothy meets the powerful wizard of Oz, but then discovers – be careful, spoilers ahead – his true nature: He is nothing more than a mechanical figure, controlled by a puppet master.

Likewise, you can start your chatbot prototyping with illusions: People from the target audience interact with a so-called-bot via a chat interface (e.g., in a messenger), but on the other end of the communication, there is a colleague of yours answering with pre-defined text blocks. It is, however, important to keep on track with the dialogue flow from the prototyping. This is easier said than done, especially when you discover situations that you did not foresee in your plan. It is important to note these situations down so that you can work on solutions for the next iteration. Everything that people say is valuable feedback to find out how they will communicate with the chatbot. After the sessions, you should also interview the participants about their experiences to find out how they evaluate their exchange with the chatbot.

In practice, details differ quite a bit between various projects. One setup would be the following:

• First, you copy pre-formulated questions and answers in a Google Doc or in text snippets with a shortcut. The latter option is preferable because it is quicker. You can use a standard profile in a messenger as a fake bot and ask a colleague to mimic the chatbot. Obviously, humans will need longer to post the right answer into the chat, even with shortcuts – these are the kinds of constraints that you have to accept at this stage until you feel confident enough about your ideas to build a functional prototype.
• With these prerequisites in place, it is time to recruit participants from your target audience for the test. In a lot of cases, you might be interested in their immediate responses, so that you invite them to your lab or another room. This allows you to observe them during their interaction with the fake bot.
• A scenario can help participants imagine the situation in which they approach the bot, e.g., “It is 5 p.m., and you think about finish working for today. However, you also do not want to spend a lot of time stuck in traffic jams. A friend of yours told you about an automated tool that offers traffic information in a chat. You use this service for the first time now. Please comment on anything that comes to your mind.”
• In the session, you note down any significant detail about what the participants say, how they react, and what they do – often, registering the interaction in audio or video is useful.
• At the end of the session, you should interview the participants about their experiences. This is also the right time to inform your participant about the fake nature of the bot, explain the next steps of the bot development, and answer any questions they might have.
• You repeat this process with more participants until you feel confident that you do not get new insights about the current bot version. Then, you analyze the results and start the next iteration.

While this approach offers a vast range of valuable insights, it is obviously also quite artificial. Thus, an alternative would be to let the participants stay in their usual context and interact with the fake bot on their own devices – this is more realistic but also harder to organize, and you collect fewer direct insights about the effects of the bot on the users.

Like with other user experience evaluations, it is important to understand that you test the digital product, not the people who are using it. Therefore, you should explicitly stress this and make the product responsible for any errors that might occur. The final debriefing interview should also make sure that any uncertainties of participants are taken care of. Furthermore, participants deserve appropriate compensation for their time.

Chatbot mockups and remote tests

After a few tests and some iterating over solutions, you start feeling more confident about the most important aspects of the dialogue and develop a feeling whether there is a need for the envisioned chatbot. Therefore, the next iterations use more realistic prototypes which require more effort in creating but are also closer to the final product. One variant of this is a mockup – still a fake chatbot, but one that simulates how it looks and behaves a lot more concrete then during the Wizard of Oz tests.

Tools like Botsociety can help in creating and testing the chatbot mockup. They allow to design interactions with the bot and share those as a video or a click dummy. These videos are also an excellent way to convince stakeholders of the opportunities. Click dummies are a superb input for a remote usability test, where you share a link with your participants. If they click on this link, they are taken to a simulated chat interface and can pick a couple of pre-defined utterances. You define the reactions of the bot on these utterances inside the mockup tool and ask your human test-takers for feedback.

Functional chatbots and transition to development

Moving on in your user-centered bot design process, you will sooner or later reach a point where simulated chat interactions are not enough. This is the time for a functional prototype (high fidelity). The good thing is that you do not need to develop the bot from the ground up to test it. Frameworks like the ones mentioned in the box below offer typical chatbot functionalities, often in combination with analytics, natural language processing, and artificial intelligence. Using one of these frameworks can substantially speed up the development. Thus, the functional prototypes are very close to the final chatbot – often, your chatbot will be based on one of these frameworks, and the transition from functional prototype to implementation of the real bot is smooth.

An alternative is chatbot platforms which allow building a bot using a click interface without writing code. With these tools, you can concentrate on defining the dialogues, but you also have to rely on the functionalities they offer. Examples are Chatfuel, Textit, Dialogflow, Dexter or Chatbot Builder.

Involve humans in the chatbot interactions

As said, chatbots can give relevant information and answer a lot of questions, but at some point, they reach their limits. Users tend to value human input, especially in complex cases. Fortunately, chatbots can turn the dialogue over to a human service representative at any point in time. From a user' perspective, this hand-over should be flawless, without breaking the flow of communication – there is hardly anything worse than having to repeat everything. Also, users should not be confused about whether they are currently talking to a bot or a human. The bot should, thus, explicitly propose to consult a colleague from customer service and provide the chat protocol if the users agree on talking to a human chat partner.

Testing and enhancing bots

Strategic thinking and prototyping lay the groundwork of a successful chatbot. However, like with any digital product, you should continue to enhance the chatbot after it is public. A good way to evaluate the chatbot success and find opportunities for enhancement is analytics:

• There is a huge number of metrics that you might know from web analytics like the time users spend interacting with the bot or the number of returning users. These are equally important for chatbots.
• Furthermore, there are some metrics which specifically target chatbots. These include the frequency of individual intents, the kinds of utterances users make at various points (conversation funnel), or the emotions they communicate in their utterances. Special metrics like these are available in analytics providers specializing in chatbots, like Dashbot or Botanalytics, but also in a lot of the platforms mentioned in the box above.

It is also possible to automate some of the technical tests. This means that these tests become an integral part of every development step and are automatically performed to test basic functionalities. A tool like Botium can help in doing so. It should look familiar to anybody who knows Selenium, a similar tool to automate browsers. Also, you should try to implement a way to collect continuous feedback from users. As needs can change over time, a chatbot has huge potential to adapt to such changes.

Conclusion

The current chatbot boom is not solely based on technological change. Advantages of chatbots are that users do not need to install the bot, have little difficulties to understand the interface and learn about the bot's capabilities via a simple, speech-based onboarding. Thus, chatbot technology has huge potential to retrieve users exactly where they are. However, strategic planning and careful prototyping are prerequisites to do so. Assuring these leads to a good chance to succeed with your automated communicative helper.