Actions On Google - How? Why?
•
6 likes•5,054 views
How to develop your actions on google? In these slides we covered the main aspects: API.AI, Voice UI, google assistant and more.
Report
Share
More Related Content
Viewers also liked
Viewers also liked (8)
Similar to Actions On Google - How? Why?
Similar to Actions On Google - How? Why? (20)
More from Ido Green
More from Ido Green (20)
Recently uploaded
Recently uploaded (20)
Actions On Google - How? Why?
- 1. Developing Conversation Actions for the Google Assistant +GreenIdo @greenido Developer Advocate June 2017
- 3. Google Home — voice-activated speaker powered The Google Assistant — A conversation between you and Google that helps you get more done in your world. Actions on Google — How developers can extend the assistant (via Conversation Actions)
- 6. “Ok Google, talk to personal chef” Conversation API, Actions SDK Invoke “personal chef” action “Sure, here’s personal chef. Hi, I’m your personal chef, what are you in the mood for?” Speech to Text “What protein would you like to use?” “Well, it’s kind of cold outside, so I’d like something to …” Text to Speech “Sure, here’s your personal chef” Speech to Text, NLP, Knowledge Graph, ML Ranking, User Profile, Text to Speech
- 9. Webhook (optional) “Ok Google, talk to personal chef” api.ai Invoke “personal chef” action “Well, it’s kind of cold outside, so I’d like something to …” “Sure, here’s your personal chef” Speech to Text, NLP, Knowledge Graph, ML Ranking, User Profile, Text to Speech “Sure, here’s personal chef. Hi, I’m your personal chef, what are you in the mood for?” “What protein would you like to use?” Speech to Text Text to Speech
- 12. Intent Triggered via a series of “user says” phrases Share a common set of entities - Create new Intent otherwise
- 13. Intent name Name to use in API.AI console Only used by the developer, not referenced in your code
- 14. Action name String passed to the webhook that indicates what action is being performed Can share the same action name across different Intents API.AI specific use of “Action” compared to Google Assistant “Conversation Action”
- 15. Input and Output Contexts You can require a context to be available before an Intent is enabled Intent can set context to enable other Intents Implement flow control between some Intents
- 16. Entities Values that we are trying to capture from the user phrases Can specify a parameter name and a type of value Values can be optional Values can be a list of fixed values
- 17. Text response Spoken when the Intent is complete Embed entity values if desired - “I found a recipe for $protein and $vegetable!” Webhook can dynamically adjust this response
- 20. Prompts Specify follow up questions if a user doesn’t specify certain values Read out in random order to make it more natural
- 21. Webhook with HTTPS POST request
- 23. Node.js client library https://developers.google.com/actions/apiai/fulfillment 'use strict'; // Boilerplate setup let ApiAiAssistant = require('actions-on-google').ApiAiAssistant; let express = require('express'); let bodyParser = require('body-parser'); let app = express(); app.set('port', (process.env.PORT || 8080)); app.use(bodyParser.json({type: 'application/json'})); // Create an instance of ApiAiAssistant app.post('/', function (request, response) { const assistant = new ApiAiAssistant( {request: request, response: response}); }); // Create functions to handle requests here // Start the server let server = app.listen(app.get('port'), function () { console.log('App listening on port %s', server.address().port); console.log('Press Ctrl+C to quit.'); });
- 24. Node.js client library https://developers.google.com/actions/apiai/fulfillment const assistant = new ApiAiAssistant({request: request, response: response}); const WELCOME_INTENT = 'input.welcome'; // the action name from the API.AI intent const NUMBER_INTENT = 'input.number'; // the action name from the API.AI intent const NUMBER_ARGUMENT = 'input.mynum'; // the action name from the API.AI intent function welcomeIntent (assistant) { assistant.ask('Welcome to action snippets! Say a number.'); } function numberIntent (assistant) { let number = assistant.getArgument(NUMBER_ARGUMENT); assistant.tell('You said ' + number); } let actionMap = new Map(); actionMap.set(WELCOME_INTENT, welcomeIntent); actionMap.set(NUMBER_INTENT, numberIntent); assistant.handleRequest(actionMap);
- 25. Facts About Google sample (context and deep links) https://developers.google.com/actions/apiai/fulfillment
- 26. Requesting user’s name and location https://developers.google.com/actions/assistant/helpers Q: "Recommend me a local bookstore" A: "To find bookstores near you, I'll just need to get your zip code from Google. Is that okay?" NAME (Registered device user’s full name) DEVICE_PRECISE_LOCATION (Coordinates and street address) DEVICE_COARSE_LOCATION (Zip code and city) // assistant is an instance of ApiAiAssistant or ActionsSdkAssistant let permission = assistant.SupportedPermissions.NAME; assistant.askForPermission('To address you by name', permission);
- 27. Account linking with OAuth 2.0 https://developers.google.com/actions/identity/account-linking "It looks like your <invocation name> account is not linked yet. You can link <invocation name> to your Google Account from the Google Home app." Google Home app provides a link to your login page. OAuth token is presented to your webhook. Will only connect to a login page on your domain, not a 3rd party login.
- 28. Speech Synthesis Markup Language (SSML) & Earcons https://developers.google.com/actions/reference/ssml <speak> <!-- Must be at the start of the string --> <say-as interpet-as="characters">SSML</say-as> <break time="3s"/>. <audio src="https://example.com/file.mp3"></audio> <say-as interpret-as="cardinal">10</say-as>. <say-as interpret-as="ordinal">10</say-as> <say-as interpret-as="characters">10</say-as>. <sub alias="World Wide Web">WWW</sub>. <p><s>This is one.</s><s>This is two.</s></p> </speak> <!-- Must be at the end of the string --> “S S M L” [3 second pause] [audio file plays] “Ten” “Tenth” “One Oh” World Wide Web [two sentences]
- 31. Action naming and policies https://developers.google.com/actions/policies/general-policies ● Policies about what Conversation Actions are allowed ● See the guidelines for more information ● Invocation names must be unique since it is a global name ● Direct Actions not widely available to build and deploy, register for interest
- 34. GUI :: VUI What’s the difference?
- 35. Top 3 Design Tips
- 37. Persona is conveyed through: ● Tone ● Word and phrase choices ● Functional design ● Style ● Technique ● Voice ...and based on: 1. Your user population and their needs 2. The imagery & qualities associated with your brand
- 38. Preserve and reinforce your persona by engaging the user as a separate entity from the Google Assistant. Own it. Hey! This is ___ Welcome to ___ Ready to play ____ Hi! ___ here. Hello. I’m ___ Greetings, human. Welcome back to ___ Hey again. ___ Let’s play ___ Here’s your ___ Brought to you by ___ Hi there, ___ Let’s get started. Ready for your ___ ___, here to… Live from ___ This is ___ What’s up, ___ and more...
- 40. Confidential and proprietary code code code code code code code code code code code “ dialog string” code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code code code STRUCTURE + CODE
- 41. Confidential and proprietary DIALOG + STRUCTURE
- 42. In conversations, there are no “errors”. 3
- 44. “I don’t understand you”
- 45. When a so-called “error” occurs in a conversation, it should be treated simply as a new turn in the dialog, only with different conditions.
- 48. Actions on Google Developer Community https://g.co/actionsdev Actions on Google Developers https://developers.google.com/actions @ActionsOnGoogle