Robot Engage Blog Buy Now
Blog

Misty Community Forum

Application Template: Misty Concierge Template

The Misty Concierge Template is an open-source, community-maintained JavaScript skill template for Misty II robots. As a Misty developer, you can use this template to quickly build and deploy skills that put Misty to work as a concierge for your business.

The repository includes the following resources:

  • conciergeFoursquareTemplate - The template for a JavaScript skill that allows Misty to answer questions about nearby places, food, and services. When this skill runs, users can activate Misty with the “Hey, Misty!” wake word to ask a question. Misty moves her body and head to look at the speaker and maintain eye contact while capturing speech. She sends her recording of the question to a custom Dialogflow agent, which processes the language and returns a data structure with extracted information. Misty uses this data to request a list of recommendations from the Foursquare Places API, and parses the response into a human-readable text string that she reads out loud via Google’s text-to-speech service.
  • conciergeBaseTemplate - A stripped-down version of the concierge template. This code file provides the framework for sending audio files to (and getting data back from) a Dialogflow agent. Use this template when you want to build all of your interactions with Dialogflow and don’t need another API, or when you want to use an API other than Foursquare.
  • dialogflow - Includes a .zip file with JSON data for configuring a Dialogflow agent for use with the conciergeFoursquareTemplate skill.

The JavaScript skill templates in this repository integrate with Dialogflow and the Foursquare Places API to give Misty the ability to answer questions about nearby food, services, and places. As she listens and responds, Misty moves her head and arms, blinks her LED, plays sounds, and changes her expression to invite engagement and create an interesting experience for the people who speak to her.

The README in the template repository provides all of the information you need to set up your Dialogflow agent, get Foursquare credentials, and update the templates for use with your own Misty II. As this is an open-source initiative, it also offers contribution guidelines. If you’d like to work on open issues, improve this documentation, add examples built on other services, or have ideas for other ways to advance the project, we hope you’ll submit a pull request.

We hope to see this thread become an active hub for brainstorming and collaboration on the future of the concierge template. This post is set as a wiki, so it can improve with contributions from the Misty community. Feel free to edit or add information as the template grows!

6 Likes

Hey Jonathan & team. Great job on this template. Was able to get the Concierge template up and running. In Google Cloud I combined this with the Conversation intents shared by @cameron It’s working 90% and I often have to say “hey misty” at least twice. Encountered one roadblock in the function _UpdateAuthToken where I was getting an undefined token. Replaced the misty.Set… statement with 2 lines from Cameron’s code. His JSON.parse refers to a response.accessToken instead of .authToken in the template. Not sure if anyone else will have an issue there. Look forward to sharing more and discussing in the Uplink session.

1 Like

Thanks for sharing this template! Was able to get through the README, create accounts, and load the skill. I did not make the change suggested above by @cgkocks. Tested and Misty responds better than half the time with proper response.

I did not have either a Dialogflow or Foursquare account, so had to set up. This was not too bad, but did take a little time. For the Dialogflow Google account a credit card is needed. I am worried that Misty will go crazy (because of my silly programming bug) and max out the card - but it appears that Google will warn me before any charges happen. When opening each account I had to go through a captcha test. Luckily they were asking me to verify “I am not a robot” and not “I am not doing this for my robot” !!! :grinning: :robot: !!!

One Question: Should the credentials be stored in the JSON file? In previous example skills they have been.

The first change I can think of is to have Misty read a GPS sensor (through the backpack serial port) so that the location does not need to be typed in and will change as Misty travels.

Again, thanks for the awesome template!

3 Likes

This is typically what I recommend, as I find it makes credentials management easier for me – but in this case the developer who created the template took a different route, and used the misty.Set method to store the credentials in Misty’s database, which offers the ability to make the credentials generally available to other skills that might use the Dialogflow / GCP resources.