Enter Google Assistant

It was back at Google I/O 2018 that Google announced a redesigned Actions console, intended to simplify developing Actions. But what are they, and how do they relate to Android?

Essentially Actions represent a way to extend the functionality of an Android app to different contexts. Particularly conversations, and not necessarily to the same smartphone or tablet running the original app.

It’s a way to increase an Android app’s potential reach by creating a new entrypoint, from devices covered by the Google Assistant. The idea is that this lets you bring newly-created services to users without needing to install anything, via the medium of voice.

In Google’s definition, an Action is: “An interaction you build for the Assistant that supports a specific intent and has a corresponding fulfillment that processes the intent”.

More technically speaking, as we will come to:

Actions are entry points into your app that define the invocation and discovery model for your app. You declare Actions in a JSON file called an Action package, which you eventually upload to your developer project when you want to test or submit your Actions project for approval.

As well as Android phones and tablets, users can invoke Actions from devices such as smart speakers, Google Assistant-enabled headphones, Android TV and Chromebooks. And also – note – Wear OS devices and iPhones…

You build an Action that will be fulfilled by Google Assistant. This would usually be a voice interaction but could also be a “visual conversation”, says Google, if this is supported by the device.

A circle of comms begins. First, the Action will run entirely in the cloud, with the device sending the conversation to the Assistant, which routes it to the fulfillment service via HTTPS POST requests.

The service will then figure out a relevant response and sends that back to the Assistant, which then returns it to the user.

Android

And where does Android come in?

Check out the post How creating an Action can complement your Android app on Google’s Android Developers Blog.

It is probably a mistake to try to rewrite all of your Android app as a conversational Action, since voice is a different modality with different constraints and usage patterns. Instead, you should start by selecting the most important or popular features in your app that translate well into a voice context and can be more easily accomplished there. Then, you can create your conversational experience to offer these features on Google Assistant devices. Check out the Conversation design site, which has several articles and guides about how to create a great voice UI.

Building Actions

How do you build an Action?

The journey begins on the Google Developer Console, where you create a Project and work with the Actions SDK to build your own Actions Package, which will define the default Action and additional context-specific Actions. You then move on to Fulfillment and Intents.

This involves receiving the user requests from the Assistant, processing the request and responding suitably, as per your application. “This back-and-forth request and response process drives the conversation forward until you eventually fulfill the initial user request,” says Google.

Discovery Methods

But how do users discover the Actions that have been created?

You can find the accessible functions of Google Assist and listed in a directory.

For example, there’s the ‘Cleartrip’ assistance where you ask Google Assistant to check for flight prices anywhere. It requires an origin city, a destination city and the travel date. And then a conversation such as:

“Ok Google, talk to Cleartrip”
“Find flights from Bangalore to London tomorrow”

Note that you will also have to submit your Actions for approval to Google.

You can also establish a link to a specific Action in your app. “When a user clicks on your link on a web or mobile browser, they will be directed to the Assistant and your Action will be invoked”.

The Actions could also be listed on websites, or by Google Assistant referring the user to them when they request a similar funciton. This is done by defining Action phrases:

Users discover your Actions by saying things like “Ok Google, learn about the number 42”. The Assistant knows you can handle this action and says “For that, try saying ‘Ok Google, let me talk to Geek Num’”. Action phrases give you an easy and powerful way to increase usage of your Actions by supporting specific and differentiated tasks that your Actions excel at handling.

Actions console

You can read about Google’s new Actions console and there are five tips for creating the best Actions for Android-based content.

Check out this session from Google I/O 2018 – Integrating your Android apps with the Google Assistant, presented by Krishna Kumar and Mariya Nagorna