Using Logs to Improve Actions
If you have an Action published on Google Assistant, you should use your logs to improve intent accuracy, evaluate user journeys and more. By going through your logs, you can see the different paths real users are taking and address roadblocks or confusion that are keeping them from completing their task/goals or causing them frustration.
To views log in Dialogflow, go to the Dialogflow Console and in the left navigation menu, you can find items labeled as training, history, and analytics.
For this blog, we’ll focus on the training tab (seen in the screenshot above). This section contains all conversations your Action has had with users.
Each row is a conversation with a user (i.e., a session). The Request column has a number and that is how many turns of conversation that session had. The next column is No Match and that number is the number of times the conversation session that the Action didn’t have an appropriate response for the users’ request. No Matches happens when a user’s utterance does not match with any of your intents and triggers your Fallback response.
You can click on a row to open a window that shows the conversation — what the user said and which intent was triggered.
The no matches section is where I want to focus on for this analysis and how these no matches can identify desired functionality from users.
User desired functionality
One of the most fun things to find in no matches is what users want to be able to do with your Action. I find this type of data very valuable because these are potential features that I can implement.
For example, below is a conversation with the GDG San Francisco Action, a meetup group that has technical talks and workshops focused on Google technologies. In the conversation below, someone asked if they could give a talk.
Since my conversation design didn’t consider this, the Default Fallback Intent was triggered, which isn’t the best experience. It even appears that they tried to say it in a different way with “submit talk.”
This is great information! As an organizer, it would be ideal to provide information about how to become a speaker. I’ll address this request with a new intent, using the two phrases that the user said as my training phrases. Since this particular Action gets it’s data from Meetup.com I can use a BasicCard to let people who are interested in giving talks the information that they should reach out to the organizers and add a button to link to the meetup page that lists the organizers.
Great! With this new intent, if a user ever triggers this they can get the right information.
- — -
There will be times that you cannot provide the information that the user is asking. For those situations, you’ll want to acknowledge the user’s request.
Let’s say for the example above that people couldn’t submit a talk or there wasn’t a way to contact the organizers. I would have still created the intent and provide responses to the effect of “Thanks for your interest in sharing with our group, but unfortunately we do not accept presentation requests” along with a suggestion of other things they can do with the Action.
Just like how it’s important to let the user know what they can do, it’s important to acknowledge what the Action can’t do.
Learn more about Dialogflow’s Training feature check out the docs.
Thanks for reading! To share your thoughts or questions, join us on Reddit at r/GoogleAssistantDev.
Follow @ActionsOnGoogle on Twitter for more of our team’s updates, and tweet using #AoGDevs to share what you’re working on. Can’t wait to see what you build!