An Isotoma white paper

The chatbot ecosystem in 2016 Written by: Doug Winter Alex Holmes

// York Swinegate Court East Swinegate York, YO1 8AJ © Copyright Isotoma Limited 2016

// London Spitalfields, 1 Primrose Street London EC2A 2EX

// Manchester 5th Floor 20 Dale Street Manchester, M1 1EZ www.isotoma.com

// Introduction

Introduction “While chatbots have undoubtedly been overhyped, I am enthusiastic about their potential and the new styles of interaction they bring.”

We believe chatbots have significant potential alongside more traditional forms of applications, products and marketing. We are already talking to a number of customers about chatbotenabled products, and we expect more enquiries.

Doug Winter

This paper presents the results of this hands-on research into how the different chatbot platforms available right now shape up.

© Copyright Isotoma Limited 2016

There are a large variety of services, platforms and products in this emerging space, some clearly more mature and useful than others. We decided to do a deep-dive into the ecosystem around chatbots so we can help our customers make the right decisions for chatbot concepts, design and implementation.

2

// The chatbot ecosystem

The chatbot ecosystem Since the release of the Facebook Messenger Platform in 2016 chatbots have moved rapidly through the “hype” stage of Gartner’s Hype Cycle and are now entering the dreaded Trough of Disillusionment. Nevertheless they have significant potential and like many new technologies are likely to emerge from the Trough into something useful. This is an unusual market for something in the early stage of the hype cycle. One clear reason for this is the ongoing exponential compression of development times - market evolutions that might once have taken years can happen in weeks. The software services industry is gaining maturity rapidly and is far better at analysing possible opportunities and securing funding than it used to be. The emergence of IaaS and SaaS providers such as Amazon Web Services has made the implementation of these services much easier to bootstrap. This means we have a broad, relatively rich vendor landscape at a stage where most potential customers have not yet even identified a need, let alone gained an understanding of the problem domain, solution domain or grasped how these are relevant to them. Potential end users themselves are even more inexperienced, with the overwhelming majority of people never having used a conversation interface with a machine. Nevertheless the potential for use of chatbot technologies is very attractive, if you understand how and when to use them. Because of this high potential but low level of customer understanding, we felt it was worthwhile to conduct some more focused research into the area.

© Copyright Isotoma Limited 2016

3

// History

History

Even in the early nineties the primary way to interact with software was with a terminal, even if was running within a primitive GUI.

These include: // chef and its variants jive and valspeak (1986), which processed text input to produce barely amusing renditions of the input // Eliza, and Markov-chain based conversational systems, some of which can briefly fool the uninitiated into thinking they are conversing with something intelligent // Many, many bots created for IRC (Internet Relay Chat). There was a period where no self-respecting IRC channel would be without a bot // Text adventure games (aka interactive fiction) pioneered parsers that aimed to understand natural language text input, to variable degrees of success // MUDs (Multi-user dungeons, a type of text adventure) were often populated by more-or-less intelligent automated denizens. The more mature MUDs could have some quite rich interactions with (often buggy) non-player characters

Throughout that long period of dominance of the text interface a number of tools were developed which are the obvious predecessors of today’s emerging chatbots.

All of these would be merely a footnote in the already rich history of the Internet if it wasn’t for the massive consumer adoption of text message based interfaces.

Chatbots, in one form or another, have a rich heritage. Although the first GUI (Graphical User Interface) was developed at Xerox PARC in 1973, most computers had text-based interfaces until the GUI was adopted by Apple, Acorn, Amiga, Microsoft and others through the eighties and nineties.

© Copyright Isotoma Limited 2016

4

// History

The next big revolution Industry in the western world has been slow to capitalise on the emergence of this new channel, primarily because it seems expensive and it lacks the marketing opportunities of richer UIs. In Asia, however these channels have rapidly grown to dominate the way companies and consumers interact. This is for a range of reasons, some to do with the state of infrastructure (with poor fixed-line Internet availability but often excellent mobile penetration), some to do with the unique capabilities and restrictions of ideographic languages. The finest example of this is WeChat. Launched by TenCent in 2011, it has over one billion accounts with 700 million active users. The app requires the user to have a mobile phone. WeChat is widely used to conduct transactions that, in the west, would be conducted over the web: paying bills, shopping and money transfers. Businesses respond, using human agents, to chat requests on WeChat in a manner that is both obviously modern and strangely old fashioned. Now we come to 2016, where the Facebook Messenger Platform was launched early in the year. It has gained the capabilities for payment which are key to turning it into a true commercial channel. Facebook Messenger is accessible to all of the nearly 2 billion active users of the Facebook platform, and has sufficient penetration in the west to be plausible as a consumer channel in which to invest heavily.

© Copyright Isotoma Limited 2016

Messaging platforms are now very big business, with messaging app monthly users exceeding those for social networks. Many messaging platforms, particularly those that support enterprises such as Slack have very rich ecosystems with large numbers of integrations that can be used to support enterprise task completion already. This is an environment in which people are already aware of and using chat platforms on multiple devices. This will rapidly increase the penetration of Chatbots. In addition platforms such as Amazon’s Alexa, Google’s “OK Google”, Microsoft’s Cortana and Apple’s Siri have introduced voice chat as a viable means for task completion and information delivery. Chat is very much the next big revolution as a user interface.

5

// History

Messaging apps vs social networks “What were once simple services for exchanging messages, pictures, videos, and GIFs have evolved into expansive ecosystems with their own developers, apps, and APIs.” - Business Insider.

Fig 1: Messaging apps overtaking social networks in engagement, courtesy of Business Insider © Copyright Isotoma Limited 2016

6

// Conversational interfaces

Conversational interfaces In a nutshell chatbots provide a conversational interface to existing back-end IT services. This means they converse with the user in a natural language such as English. Some messenger platforms provide richer capabilities, and these are likely to grow over time. This includes:

© Copyright Isotoma Limited 2016

Take payment & send money.

Send & receive images.

Acquire photographs & video.

Show ‘cards’ or structured messages.

7

// Problem domain

Problem domain As with many new technologies, chatbots solve a problem that many people don’t even think they have. They aren’t a “faster horse” for other existing technologies, they have unique constraints and capabilities and are unlikely to replace existing channels in the short term. The three core characteristics we see in chatbots is that they are: // Domain specific. They are not like Siri, OK Google or Cortana which are in a sense general purpose. Chatbots exist to operate across a specific, probably quite narrow domain. // Focused on task completion. Users interact with a chatbot because they have a specific task they want to complete - which could be informational or transactional. // Dependent on existing APIs. Bots are, indirectly, about APIs - interfaces to other, existing, pieces of software. Without existing, useful software behind them for the execution of tasks they have little purpose.

“

Ironically the one thing that chatbots do not provide is chat. They aren’t there for small talk or casual conversation!

There are three types of task that are most likely to be conducted using a chatbot: // Informational. The user wishes to find something out which is within the domain of this specific chatbot // Consumer transactions. The user wishes to buy something, or gain some support around a pre-existing transaction. // Enterprise transactions. The user wishes to use one or more enterprise IT systems as part of their work.

© Copyright Isotoma Limited 2016

8

Natural language processing One of the core techniques to facilitate this sort of interface is Natural Language Processing (NLP – not to be confused with Neurolinguistic Programming). By “natural language” we mean languages humans use with each other, rather than programming languages. NLP is a blanket term for any kind of interpretation of utterances in natural languages. NLP has been used for all sorts of things – a good example is Sentiment Analysis – an automated process for determining how people feel about products, services, brands and other things. NLP for chatbots has a very specific use however – because of the task focus of a chatbot, what they generally need to determine from their human counterpart is the human’s intent. This needs to be selected from a predetermined list of functionality available to the bot, and the bot also needs to extract from an utterance some variables which will be passed to the chosen function. Bots will therefore generally exhibit a command-driven decision-tree structure (see example), with NLP interpretation of some utterances, then moving into more traditional option selection when options are sufficiently narrow.

© Copyright Isotoma Limited 2016

Example:

“I would like to buy 3 tickets to see Finding Dory tonight.” Would need to become: purchase_cinema_tickets(movie=“Finding Dory”, number=3, time=None, date=today) The bot then needs to ask further questions to narrow down the missing information “Finding Dory is showing at 17.00, 19.00 and 21.00 tonight, which viewing would you like?” - before finally executing the desired functionality.

9

// Platforms

Platforms The software industry has a far more mature understanding of what a good platform looks like than it did even a few years ago. In particular the success of developer focused features in platforms like GitHub and Amazon Web Services has made “developer evangelism” an obvious marketing approach for new Platform-as-a-Service (PaaS) solutions. We’ve spent time using these platforms to explore their features and to be able to make a qualitative judgement on their maturity and readiness for serious use.

Core considerations Integration with messaging platforms

For all of them the key value is that they abstract the underlying protocols and configuration of the messaging platforms, allowing you to concentrate on the conversational code itself.

Natural Language Processing

The vast majority of platforms come with their own NLP solution, or are designed to be integrated with separate specific NLP services such as Facebook-owned http://wit.ai

Pricing model

How the platform proposes to charge for usage.

Workflow management

Orchestrating the various pieces of functionality to create a coherent overall conversation.

Rich message support

Support for Images, sound, attachments and structured messages.

The following are some of the things we considered in our evaluation.

© Copyright Isotoma Limited 2016

10

// Platforms

Ancillary features Code hosting

Support for hosting current versions of your chatbot code, to support the development process.

Online code editing

Editing chatbot code directly in the browser with IDE-like features.

Code testing

Testing your code against the platforms.

Tooling

Version control, code import/export.

Operability features

Monitoring, alerting, logging and analytics.

Built-in corpus

Does the platform provide ready made corpuses of words and phrases?

Enterprise integration

Integration with standard enterprise services.

Built-in responses

Some platforms ship with canned code for simple responses such as the weather and news.

Other considerations

© Copyright Isotoma Limited 2016

Production readiness

How ready we think the platform is for being used in real production.

Documentation and training materials

Quality of tutorial and reference material for developers.

Multi-user capabilities

Allowing multiple people to be involved in a single transaction is a so-far underexplored capability of chatbots. We evaluate whether the platform has the capabilities to support this.

11

// Vendors

Vendors Api.ai Originally a spinoff of some of the features of “assistant.ai” from Speaktoit, api.ai has now been acquired by Google. Of all the services we reviewed, api.ai had the most solid abstraction for the intent-generation problem space. They have broken it down into easily-understood primitives such as agents, entities, intents, and contexts that can be used for a very wide range of different use-cases. Cleverly without a steep learning curve or forcing the user to understand the complexities of NLP. Also, crucially, users do not need to know how to code to create bots with api.ai, although the output of the bot would very likely be a separate application. An entity is comprised of one or more lexemes and an optional list of their synonyms, e.g. “direction” might include the lexeme “forward” with synonyms “forth, forwards”, and © Copyright Isotoma Limited 2016

“backward”, with synonyms “backward, back”. Composite entities then support a combination of these entities, for example an entity called “move” might be built as follows: “@sys.number [steps] ‘steps’ @direction:direction”, where “[steps]” is a variable, ‘steps’ is plain text, and when the user says “Move five steps forward”, the resulting parameters to an intent would be: “parameters”: { “move”: { “direction”: “forward”, “steps”: 5 } }

api.ai also offers lots of predefined entity lists for a variety of domains such as ordering a taxi, ordering a pizza, etc.

api.ai Phrases can be added for each parameter so that if a user conversationally provides most but not all of the required parameters, the bot can specifically ask for the rest until the intent requirements are fulfilled, e.g. “I see, you’d like a margherita pizza. Would you like it to be 8, 12, or 14 inch?”. The user can next train the bot by speaking to it and checking that the parameters are correctly identified. Once an intent is fulfilled, users can configure various methods (including webhooks) to send that information to their backend systems.

A user can then create intents which define a list of parameters, each of which are mapped to a predefined entity.

12

// Vendors

converse.ai Converse.ai seems to be focused on filling the gap in the market for heavily workflow-centric chat bots, which it has achieved by focusing the bot design around a visio-style visual workflow designer. Although the visual designer tool is very technically ambitious and overall converse.ai ticks most of the boxes, it feels clunky and very immature and needs a lot more work to make it as easy to use as many of its competitors. Aside from the workflow creation tool, converse.ai also has fully-featured NLP, analytics, and integration offerings.

chatfuel

converse.ai

The designers of the chatfuel application made the very pragmatic decision to aim their offering squarely at non-developers, and avoid tackling the more complex state and workflow features that make many of the chatbot platforms more difficult to use. In keeping with this decision, unlike many of the chatbot platforms, the primary means of getting information out of the application is by email. Overall chatfuel is a well-polished platform offering decent NLP that does an excellent job at covering the simpler bot use-cases. chatfuel

© Copyright Isotoma Limited 2016

13

// Vendors

chymebot Chyme is heavily focused on enterprise integrations (e.g. salesforce) and is currently available only to early-access users.

gupshup Gupshup is a bot development platform aimed squarely at javascript developers. Although it doesn’t have any native NLP features, it has decent integration with the popular wit.ai service. Gupshup’s creators have focused on code hosting, emulation, data storage (in a JSON data store) and an abstraction layer across multiple platforms. By focusing solely on these areas however, they have left some potentially core features to developers to implement, such as the creation of a state machine to manage workflows and intent parameter tracking and fulfilment.

chymebot

For more complex functionality, it is likely that you would use the included http library to integrate gupshup with separate application code hosted elsewhere using gupshup for the “glue code” to integrate your application with the various messenger services. One unique feature is the data storage facility, where using the context.simpledb. roomleveldata and context.simpledb.botleveldata you can store object data for as long as required and optionally reuse it later on, potentially even in a conversation with a different user.

© Copyright Isotoma Limited 2016

gupshup

14

// Vendors

Kore

meya.ai

Kore’s primary offering is their messaging application, a Slack competitor heavily focused on the enterprise.

Meya fulfils is a bot platform aimed at developers offering code hosting and an elegant workflow system.

Their bot platform integrates solely with their messaging application but boasts a huge number of integrations with common enterprise applications such as Zendesk, Salesforce, Jira, Google Apps, Twitter, NewRelic, Travis CI, and more, so might be enough to tempt people across from Slack.

Meya has an excellent YAML format for defining workflows called a “flow schema”. This defines a series of intents that map to states, which have predefined transitions. When a state is entered, its “component” is run, which is a coroutine written in javascript or python and hosted on the meya platform. The flow schemas offer a powerful but easy-to-understand convention for developers to plug together their components.

Demos of the NLP often seem to be inelegant, requiring users to word their input in a very specific way. A handful of screenshots and demos also clumsily showed off bugs with particular bots.

© Copyright Isotoma Limited 2016

One downside is that the hosted code has limited access to standard libraries so would have to HTTP to communicate with a separate application and puts a severe limitation on the bots that could be built solely using meya. Like gupshup, developers would likely find themselves using Meya only for the glue code (but with the added benefit of also hosting the state machine).

kore

meya.ai

15

// Vendors

recast.ai Recast is an immature NLP platform aimed at developers targeting the Microsoft Bot Framework. Borrowing from github’s visual design and forking mechanism, recast aims to create a community around NLP entities and intents, letting people copy and modify existing “bots”. Much of the work involved in creating an NLP-driven bot is in defining a corpus and training the bot on it until it can reliably determine intent from a wide range of different conversational inputs. While platforms like api.ai offer a limited number of predefined domain-specific corpuses, recast instead sees the value in building a community and having them collaboratively build up a wide range of bot definitions.

recast.ai

On the other hand, recast is still a very immature application. User interfaces are often awkward, the workflow designer tab shows a coming soon message and by integrating only with Microsoft Bot Framework getting a bot up and running in the wild is a much more laborious process than with most of Recast’s competitors.

© Copyright Isotoma Limited 2016

16

// Vendors

Feature matrix Core considerations gushup

chatfuel

converse.ai

chymebot[1]

kore

meya.ai

api.ai

init.ai[2]

recast.ai

Integration with messaging platforms

wit.ai

[3]

Natural Language Processing

[4]

Pricing model

[5]

Workflow management

[8]

[7]

[6]

[10]

[9]

[11]

Rich message support

[12]

Other considerations gushup

chatfuel

converse.ai

chymebot

kore

meya.ai

api.ai

init.ai

recast.ai

wit.ai

Production readiness Documentation and training materials Multi-user capabilities [1: Currently in early access], [2: Not yet released], [3: Microsoft Bot framework only], [4: NLP functionality available via an external service (wit.ai integration well supported and documented)], [5: No native NLP but functionality available via wit.ai and luis.ai integration], [6: Zendesk pricing], [7: chatfuel pricing], [8: Recast pricing], [9: Free tier only, owned and championed by facebook so unlikely to suddenly introduce pricing], [10: By leveraging per-user and per-room state, a workflow can be created, however there is no built-in state machine functionality], [11: Flowchart builder coming soon], [12: Feature is currently in limited beta for Converse AI users]

© Copyright Isotoma Limited 2016

17

// Vendors

Ancillary features gushup

chatfuel

converse. ai

chymebot

kore

meya.ai

api.ai

init.ai

recast.ai

wit.ai

Code hosting Online code editing Code testing Tooling

[13]

Built-in corpus Enterprise integration Built-in responses Analytics Broadcasts Webhook support Emulation support Database

[13: Import via CSV, export via JSON HTTP endpoint only]

© Copyright Isotoma Limited 2016

18

// Other options

Other options Of course, you don’t need to use one of these bot platforms, there are other options. All of the features these platforms provide are relatively easy to implement individually in the trivial case, except for Natural Language Processing. NLP can be outsourced to services such as wit.ai or luis.ai or you can use mature NLP libraries like Python’s spaCy, NLTK or ParseyMcParseFace. Of these we’d recommend wit.ai or spaCy. Whether you use a platform or build it yourself will depend on a number of factors: // // // //

Stage of development Scale Number of messaging platforms to support Privacy and security concerns

If you are at an early stage of development these platforms can significantly accelerate development and reduce the cost of change. They remove the need to write a lot of boilerplate, so allow you to explore your problem and solution directly. However at scale they can greatly increase cost, so for more mature developments you might want to consider insourcing the entire delivery chain, or moving to a lower-level PaaS such as generic AWS services.

© Copyright Isotoma Limited 2016

DIY Bots aren’t very difficult to write in themselves - you just need an asynchronous programming framework of some kind. There are many options for this from environments designed for asynchronicity such as node.js or Erlang, up to libraries such as Python’s Twisted. If you only wish to support a single messenger platform then you won’t need a huge amount of interfacing code either. However, supporting all of the messenger platforms will be a significant effort, particularly in testing. Tracking these interfaces over time, and updating your code to continue working with them will also require effort. This is the core advantage a bot platform offers. Serverless cloud services Alternatively, bots are well-suited to a “serverless” approach because of their flexibly latency requirements, asynchronous interface and variable scale. Amazon Web Services provides a number of services, such as their API Gateway and Lambda, which seem well suited to functionality of this kind. In particular latency is not massively important and data quantities are light which suit chatbots well. We expect a lot of addons to these serverless services to assist in chatbot construction.

19

// Recommendations & the future

Recommendations & the future “Whether you are new to the concept of chatbots or have already deployed them I hope this document has been useful. If you’d like to talk to us about a chatbot project, or anything else, please do get in touch” Doug Winter

Of all of these platforms we found api.ai to be the most feature rich, mature and reliable. During the writing of this paper api.ai was purchased by Google, so someone there clearly agrees. We found these platforms to be very well suited to early stages of development, or where you wish to support a large number of messaging platforms, and we’d encourage you to use them. If your chatbot becomes so successful that you want to bring your project in-house entirely, then you should be in a very strong position to do that. None of them force sufficient lock-in that you will be trapped. Looking further to the future, one of the apparently emergent properties of our future interfaces is that they become more human. Many applications that once we might have imagined would have complex “computerlike” interfaces now use the same things humans do - although in very different ways. The promise of the complex “semantic web” has been largely delivered by Google’s search without any semantic markup. Driverless cars do not require special tracks on the road, or special satellite signals - they read the road signs and watch for pedestrians with cameras. Spotify recommends music to me without asking me anything, it just observes my behaviour. In the same way, strangely, software may one day converse in Natural Languages, not only with humans but with other software. Natural Languages provide a scope and power that is entirely absent from computer languages, protocols and APIs and tolerates error and change in a flexible manner. At computer speeds even Natural Languages can become high bandwidth. What we’re observing with chatbots right now may well form the foundation of the next revolution in communication.

© Copyright Isotoma Limited 2016

20

// Authors

Authors Doug Winter Director | @winjer Doug has always been passionate about what makes the Internet tick. He has been developing web applications since 1996, and now provides sage advice, software architecture and war stories to help Isotoma’s larger and more complex projects.

Alex Holmes Developer | @ignorethecrane Alex has been a developer for the past 7 years and specialises in writing Python and Javascript. Over the last two and a half years Alex has focused on medical education and curriculum mapping.

© Copyright Isotoma Limited 2016

21

A big thanks to.. Our teammates and proofreaders: Francois Jordaan Sarah Birtles Andy Theyers

For further information please contact: Doug Winter [email protected] +44 (0) 1904 313 969

// York Swinegate Court East Swinegate York, YO1 8AJ © Copyright Isotoma Limited 2016

Alex Holmes [email protected]

// London Spitalfields, 1 Primrose Street London EC2A 2EX

// Manchester 5th Floor 20 Dale Street Manchester, M1 1EZ www.isotoma.com