uniFlow: Dialogflow V2 Unity Plugin Prototype

GitHub Repository: github.com/ITP-xStory/uniFlow

• Introduction
• Getting Started
• Define the Problem
• Accessibility
• Team Members
• Mentoring
• Longer-Term Goals

Introduction

The Dialogflow V2 Unity Plugin is a continuation from Api.ai: Unity Plugin, which made it easy to integrate the Dialogflow(Api.ai) natural language processing API into your Unity project. It's a team project in Open Source Studio and not an official plugin from Google or Unity. The project will continue on 2019 as a part of ITP xStory.

Getting Started

Steps for Getting Started: Full YouTube Playlist

1. Get the Service Account Key As JSON
2. Set the Environment Variable to JSON File Path
3. Install the Latest Cloud SDK & Get the Access Token
4. Create JsonData Script Inside Plugins Folder of Unity
5. Create Dialogflow API Script Inside Assets of Unity
6. Start Creating Dialogflow & Unity Projects!

Get the Service Account Key As JSON

The following process is based on Getting the Service Account key.

1. Go to Dialogflow console page.
2. Click on the gear icon, to the right of the agent name.
3. Under the GOOGLE PROJECT section, click on the name of the Service Account.
4. Click on the Create Service Account button at the top of the page.
5. In the pop up, enter a name for the service account.
6. Click on Role. Under the Dialogflow category, select Dialogflow API Admin.
7. Check the Furnish a new private key option and make sure JSON is selected for Key type.
8. Click the Create button.

Set the Environment Variable to JSON File Path

The following process is based on Using the key.

1. Create the file system in any location, and open in a terminal.
2. Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the file path of the JSON file that contains your service account key using this command:
   export GOOGLE_APPLICATION_CREDENTIALS=YOUR_JSON_PATH
3. Optional. Confirm the variable using this command:
   echo $GOOGLE_APPLICATION_CREDENTIALS

Install the Latest Cloud SDK & Get the Access Token

The following process is based on Google Cloud SDK documentation.

1. Download the latest Cloud SDK version.
2. Extract the contents of the file to any location on your file system.
3. Run gcloud init to initialize the SDK using this command:
   ./google-cloud-sdk/bin/gcloud init
4. Finish initializing and run gcloud auth to get access token using this command:
   ./google-cloud-sdk/bin/gcloud auth application-default print-access-token

Create JsonData Script Inside Plugins Folder of Unity

1. Download or copy JsonData.cs from the GitHub repository.
2. Open your Unity project, and create Plugins folder inside Assets.
3. In the Plugins folder, create new C# Script and paste the JsonData.cs.

Create Dialogflow API Script Inside Assets of Unity

1. Download or copy DialogflowAPIScript.cs from the GitHub repository.
2. Open your Unity project, select the object to add component.
3. Inside Inspector, click Add Component and select New Script.
4. Paste DialogflowAPIScript.cs in the new script.
5. Optional. If you changed file names of JsonData.cs and DialogflowAPIScript.cs, update them.
6. Insert YOUR_PROJECT_ID.
• Go to Dialogflow console page.
• Click on the gear icon, to the right of the agent name.
• Under the GOOGLE PROJECT section, you can find Project ID.
7. Insert YOUR_ACCESS_TOKEN.
• Get the access token via running gcloud auth.
• More details in Install the Latest Cloud SDK & Get the Access Token.

Start Creating Dialogflow & Unity Projects!

Examples of sending audio query:

1. Download or copy MicrophoneCapture.cs and WavUtility.cs from the GitHub repository, and insert them in the same Unity Plugins folder (with other C# Script files).
2. Insert YOUR_PROJECT_ID and YOUR_ACCESS_TOKEN in MicrophoneCapture.cs.
3. STEP 1 & 2 will automatically create a button that uses your microphone to send audio query.

Learn more about basic Dialogflow concepts from:

• Build Actions for the Google Assistant (Level 1 | Level 2 | Level 3)
• Dialogflow official docs page

Define the Problem

• Api.ai: Unity Plugin is no longer working in Dialogflow V2, due to the difference in the way the basic system works (Dialogflow API v2 versus v1).
• Dialogflow API V1 will be shut down in one year – on October 23rd, 2019.
• The goal is to figure out ways to build similar plugin that works in Dialogflow V2 system.

Accessibility

All documentation is done with considerations of the W3C Web Content Accessibility Guide.

Team Members

• Ilana Pecis Bonder | Testing
• Alice Sun | Documentation & demo
• Lin Zhang | C# Scripts

Mentoring

• Nicole He
• Casey Pugh
• Roi Lev

Longer-Term Goals

• Make the process of getting access token simpler.
• Conduct larger scale of user testing.
• Improve interface design in Unity.
• Create an inviting community for contributors and contribution guideline.