Failure by Design — The Art of the Good Chatbot

The first thing you need to accept when it comes to planning, designing, training or even contemplating creating a chatbot is that you cannot possibly answer everything definitively. In my day-to-day job I deal with enthusiastic businesses who believe that they can plug their contact channels into some kind of benevolent version of HAL9000 (because it’s been 17 years after all) and every customer’s query will be miraculously dealt with, instantly and with an Oscar Wilde-esque turn-of-phrase.

The reality is that you’d be lucky if your bot started morbidly singing “Daisy, daisy” while it infuriated your customers and would likely drive them into the arms of human agents in greater numbers than ever before.

I believe that it’s this misconception over the simplicity of creating good bots that is driving the widespread disillusionment with conversational technology as a whole. Some are going so far as to say that the whole concept is practically dead. I don’t buy that.

So let’s get into it then: Failure by Design. It’s a term that seems incongruent with a solution that is intended to improve communication between brands and customers. The first thing you need to consider here is that, because a chatbot is a technology that is mediating an interaction between a human and a machine, it is a Human Computer Interaction (HCI), and therefore subject to the same appraisal as a graphical user interface (like a website or Microsoft Excel) or other forms of existing HCI. A bot is an interface with a system, and nothing more.

The trouble with chatbots (conversational interfaces) versus other types of digital user interfaces is that the expectations for conversation itself have been set over thousands of years of evolution while those for click or touch interfaces only thirty years or so.

So when it comes to designing a click or touch interface it’s relatively hard to fail because expectations have either been set low by poor experiences in the early days of graphical user interfaces, or have benefited from the decades of best practices that have developed during that period.

Conversational interfaces are different. Your conceptual model as a user of a chatbot is influenced by all the communication that you have been having with human beings for years and perhaps made even more discerning by the fact that chatbots are communicated with, largely, using the very same channels through which you communicate with human friends and family (and even human customer support) — like WhatsApp or Facebook Messenger.

So when you consider that your chatbot cannot possibly answer everything that is asked of it, and the fact that expectations of its experience are higher than would be for, say, a website or app, your chatbot is therefore doomed to failure, right? Right.

But there’s opportunity in failure. Let me explain.

You will naturally need to plan your bot in terms of both a success strategy and a failure strategy. The simplest approach is:

Success strategy: Respond to any precise queries or intents that we can predict up-front when we train the bot. I won’t get into the details of how we achieve this prediction. It can be derived from historical data (what customers ask) or more realistically and feasibly based on the anecdotal experience of the bot designer.


User: “How do I change my password?”

So, you create a nice catalogue of queries and answers. Depending on how your Natural Language Processing is set up you may also add some variations on those questions in order to give the bot a better breadth of vocabulary.

Failure strategy: The simplest way to do this is to simply respond with a dumb fallback if the bot doesn’t understand.


Bot: “I didn’t catch that. Can you rephrase it?”


Bot: “I didn’t catch that. Would you like to speak to a human?”

Chances are that with this approach your failure strategy will be applied more frequently than your success strategy and you’re going to, at best, drive more customers towards human agents; at worst drive them away from your business altogether after dealing with relentless hard failures.

At Twyla we take a more layered approach towards designing how our bots fail. Before we even begin to consider the precise questions that customers will ask, we evaluate the conceptual landscape of our clients’ subject matter to determine what the discrete concepts are within that domain, what the attributes of those concepts are and what actions might be performed against them.

This is far less exhaustive than trying to predict every granular thing that a customer might ask — and how they might ask it — and gives us a basis for a different kind of failure training.

So while the success strategy looks the same as before, the failure strategy now allows for the bot to detect contextual information like “password” and “account” and respond in a way that keeps the customer in the conversation.

Bot: “I’m not confident that I understood you. Do one of the following options apply?”

Buttons: Password — Log-in — Upload — Payment

The buttons can then flow the customer towards the specific answers that they seek.

Regardless of how you’re implementing — or planning to implement your bot technically, a solid, well-designed failure strategy should be a core part of your process. You should always hold the principle that the art of a good chatbot is keeping the customer engaged in the conversation. To do that well, you’re far better off planning on how your bot will fail in order for it to succeed and you will always learn more from failure.

To find out more about how Twyla tackles these issues technically, as well as our conversation design methodologies, feel free to get in touch with me directly, or contact us via our social channels.

Happy failing.