Top 5 steps for making your bot smarter with the Chatbase API

on
By Sean Pearson, Chatbase Tech Support

Adding your bot to Chatbase will deliver a limited set of reports for tracking bot health in aggregate. However, taking the following 5 steps with the Chatbase API will unlock everything the service offers to make your bot smarter, including the automatic message clustering and suggested intents (currently offered via Early Access Program) uniquely made possible by Google’s machine learning (ML) capabilities.

The following steps usually take less than 20 minutes to do, and we’re working with our platform partners to make bot/Chatbase integration even easier. (See recent announcements from SnatchBot and Meya.ai on this subject.)

Step 1: Set intents

Why:
  • View metrics per intent such as frequency, user exit rate, and bot response time (Messages report)
  • View common conversation paths -- i.e., sequences of intents (Session Flow report)
  • Create funnels with intents as steps (Funnels report)
  • Enable ML that finds missed requests that map to existing intents (suggested intents) -- requires setting not_handled and message fields as well (currently in EAP)
  • Have messages labeled with intents in transcripts (Transcripts report)
How: A foundational concept in Chatbase, an intent is a label that describes a unique step within a conversation. That label is used to group different user messages that should cause the same bot response. (For example, “order car” and “send car” could be different requests with the same intent label, and which both cause the bot to dispatch a car.) Intents are set in the intent field in the API request (like so for this example):

{
  "intent": "order-car"
}

Setting clear, specific intents is crucial for understanding how users interact with your bot and whether they are following conversation paths your bot understands. Without doing that, you’ll give up considerable insight about user experience that is really valuable for optimizing your bot.

In cases when an intent is fulfilled over many steps (often called a slot-filling intent), set each intent to a step name in the format intent+user-selection or intent+entity-name. Returning to our car-service example, you could add strings for different requested car types:

{
  "intent": "order-car-suv"
}

And:

{
  "intent": "order-car-compact"
}

Or, if you’re only interested in the fact that a car type was selected, use the entity name:

 {
  "intent": "order-car-type-selected"
}

Filling this field is also important for decision-tree bots that don’t track intents. For those bots, set intent to a label that names the conversation step -- for example, "order-car" -- or better yet, to one containing either the entity name ("order-car-type-selected") or user selection ("order-car-suv", "order-car-compact,” etc). Doing that will add structure to conversation flow for cleaner analytics.

The intent field can also be set for bot messages, allowing creation of funnels using bot messages and labeling of bot messages in transcripts. For example:


Step 2. Set not_handled property

Why:
  • Have not-handled messages grouped by number of users affected (Messages report)
  • View common not-handled messages at each step of the conversation (Session Flow report) 
  • Enable ML to find missed requests that map to existing intents (suggested intents) -- requires setting intent and message fields as well (currently in EAP)
How: Not-handled messages are those for which your bot failed to recognize intent (and consequently may have defaulted to a catch-all fallback intent). Such messages are the main cause of bad user experiences.

It’s important to set the not_handled property to true for:
  • Messages for which your bot failed to recognize intent
  • Messages that have an error or “fallback” intent (so that Chatbase can analyze whether such messages should be labeled with more specific intents)
  • Messages that have a new intent not yet supported by the bot
By doing that, you’ll activate key features that provide deep insights into optimization opportunities.

Step 3. Ensure consistency between intents and not_handled settings 

Why:
  • Enhances report accuracy
How: Some developers conflate the intent and not-handled concepts. For example, we’ve seen users send error or fallback intents (which inherently imply a not-handled message) while contradictorily setting not_handled to false. For best accuracy, set not_handled to true if a fallback or error intent was also set.

Step 4. Send message content along with intents 

Why: 
  • View handled messages per intent (Messages report)
  • View common not-handled messages (Messages report) -- requires setting not_handled field as well
  • View conversation transcripts -- e.g., for not-handled messages (Transcripts report)
  • Enable ML to find missed requests that map to existing intents (suggested intents) -- requires setting intent and not_handled fields as well (currently in EAP)
  • Enable ML to cluster unsupported requests (currently in EAP)
How: If Chatbase doesn’t see raw content in the API's message field, it can’t identify all the not-handled user messages that belong to unrecognized intents. (This info is reflected in the Messages report.) Including that content also enables you to drill-down into specific conversations via the Transcripts feature, which can be very helpful in the bot-design phase. (Note: This step also requires the setting of intents as explained above but be sure to do so in the intent field; we see intents erroneously placed in the message field and vice-versa.)

Also, keep in mind that Chatbase automatically removes sensitive personally identifiable information (PII) such as social security numbers, phone numbers, and addresses from your reports.

Step 5. Set time_stamp field in correct format

Why:
  • Enables reception of messages
How: The time_stamp field requires milliseconds in Unix time. Otherwise, Chatbase won’t receive messages.

Note: The time_stamp value must be within the previous 30 days. Also, messages with a time_stamp over 2 days old are processed on a 24-hour interval, so it may take up to a day to see these messages reflected in your reports.

Getting help

Doing the above will make all the difference between a smart bot that offers a good user experience, and one that isn’t. If you have any questions about this advice, let us know via Stack Overflow (using the “chatbase" tag) or contact form.

About Chatbase 

Chatbase gives builders of conversational interfaces (or bots) sophisticated tools for creating better, and stickier, consumer experiences than ever before--leading to better conversion rates and retention. Chatbase is a cloud service that easily integrates with any bot platform and type, voice or text, and is free to use.

 Among other features, Chatbase uniquely relies on Google’s machine learning capabilities to automate the identification of  bot problems and opportunities that would otherwise take a lot of time, leading to faster optimizations and better bot accuracy.

 Chatbase is brought to you by Area 120, an incubator for early-stage products operated by Google.

Comments

Post a Comment