Robot Engage Blog Buy Now
Blog

Misty Community Forum

Misty Skill: Conversation

Misty can have a conversation with you on any range of topics. It’s fully customizable!

Misty records your voice input, sends it off to Google DialogFlow which converts it to text, uses machine learning to match it to an appropriate intent, and sends back an audio file for Misty to play.

You can easily add training phrases and responses through DialogFlow so Misty can have a conversation about anything!

Full code, documentation and video can be found here:

9 Likes

Awesome. Thanks for sharing, @cameron!

This is amazing! Thanks @cameron !

Awesome !!

This is awesome!

sweet! thanks for sharing @cameron

Ha! This is great :clap: - Thank you for sharing, @cameron.

We tried the Misty Conversation Skill with pretty good success. We are running into problems some times and are not sure why. It runs most of the time but then it stops. I check connectivity and rechecked the program. Attached it the console screen shot of what happened. It looks to me like the file is coming back but not saving. I cannot find the audio file on robot if I look in Command Centre. Any thoughts?.

Hi All,

I have the latest build (from yesterday) and tried the skill as well. I am having the same issue at tcooper… See attached console log.

any thoughts on how to address this issue?

Hello I have some issues with Misty conversation. My misty listen to my command and looks like it is fetching data from dialogflow but not able to convert it to the audio file. I am not getting any response from it.
Here is the log file

j1jw4dg5mf38fawa1m7mms:detectIntent",“authorizationType”:“Bearer”,“token”:“ya29.c.KpUBtgfa9TYVTdtpnqs6w01TPXc5i05WX6yFlWynJusGqTo0yK601Yb-dhgW6rG9RWvCQecp8wJxk-EyJ6zLuS_6MzUqaox0DEOvwcgXPKcXcy91OPP2RVd13Ux8TCjZW-jT8GEhXPBhDNjmYAYsNmpyotos0znLTspUKUiwF8BAUQuxIX3fcf3X34woqf1d6PBM3G5ogM0”,“arguments”:"{“queryInput”:{“audioConfig”:{“audioEncoding”:“AUDIO_ENCODING_LINEAR_16”,“languageCode”:“en-US”,“sampleRateHertz”:48000}},“inputAudio”:“UklGRiQAAABXQVZFZm10IBAAAAABAAEAgLsAAAB3AQACABAAZGF0YQAAAAA=”,“outputAudioConfig”:{“audioEncoding”:“OUTPUT_AUDIO_ENCODING_LINEAR_16”,“synthesizeSpeechConfig”:{“speakingRate”:1,“pitch”:5,“volumeGainDb”:0,“effectsProfileId”:,“voice”:{“name”:“en-US-Wavenet-C”,“ssmlGender”:“SSML_VOICE_GENDER_FEMALE”}}}}",“save”:false,“apply”:false,“contentType”:“application/json”,“callback”:“ProcessDialogFlowResponse”},“message”:“Calling command ‘SendExternalRequest’”,“timestamp”:“2019-12-20T00:31:33.9750213Z”,“truncated”:false}

{“data”:{“data”:“DialogFlow response: {“queryResult”:{“languageCode”:“en-US”}}”},“message”:“Calling command ‘Debug’”,“timestamp”:“2019-12-20T00:31:34.2753416Z”,“truncated”:false}

{“data”:“DialogFlow response: {“queryResult”:{“languageCode”:“en-US”}}”,“message”:"‘conversation’ Debug => ",“timestamp”:“2019-12-20T00:31:34.2753416Z”,“truncated”:false}

{“data”:{“data”:“Input text: undefined”},“message”:“Calling command ‘Debug’”,“timestamp”:“2019-12-20T00:31:34.2753416Z”,“truncated”:false}

{“data”:“Input text: undefined”,“message”:"‘conversation’ Debug => ",“timestamp”:“2019-12-20T00:31:34.2753416Z”,“truncated”:false}

{“data”:{“data”:“Ouput text: undefined”},“message”:“Calling command ‘Debug’”,“timestamp”:“2019-12-20T00:31:34.2753416Z”,“truncated”:false}

{“data”:“Ouput text: undefined”,“message”:"‘conversation’ Debug => ",“timestamp”:“2019-12-20T00:31:34.2753416Z”,“truncated”:false}

{“data”:{“red”:0,“green”:173,“blue”:239},“message”:“Calling command ‘ChangeLED’”,“timestamp”:“2019-12-20T00:31:34.2753416Z”,“truncated”:false}

{“data”:{“fileName”:“27_temp_output_audio.wav”,“data”:“undefined”,“immediatelyApply”:true,“overwriteExisting”:true},“message”:“Calling command ‘SaveAudioAssetToRobot’”,“timestamp”:“2019-12-20T00:31:34.2753416Z”,“truncated”:false}

Thanks for posting about your issues here, @tcooper, @guptahome100, and @sp22. It’s great to see this skill getting so much traction :slight_smile:

I’ve not run this skill myself in some time, but as @cameron may not have a chance to chime in for a while, I wanted to point to a few things that could be causing the issues you’re seeing:

a) Misty’s 2019.12.17 system update changed the behavior of the misty.StartKeyPhraseRecognition() command, and so you’re likely running into a few deprecated API calls in this skill code
b) there’s a bug in the current software that sometimes causes Misty to record silent audio files. We’re hoping to have a fix out soon, but until then, you can read about the workaround in this post
c) a few different bugs snuck with some of the changes for Misty’s capture speech functionality. I doubt those are causing the issues here, but keep them in mind as you troubleshoot

If you do find the root cause, I hope you’ll fork the repo and share your workaround here. With so many projects on the forums (a good thing!), it’s unlikely that the Misty staff has the bandwidth to get to the bottom of issues with community skills ourselves. That said, we’re eager to help out in whatever ways we can!

1 Like

@tcooper @guptahome100 @sp22 @johnathan I update the conversation skill to work with the new capture speech functionality in the Misty API.

I tested with my robot here and it works again.

On a related note, sometimes when I run the skill and ask Misty a question, when she plays the response I hear a small hiccup in the audio - it’s like a 200 millisecond silence in the middle of playing the response. When this happens, the skill doesn’t work moving forward, and the only solution is to turn off my Misty and turn her back on again.

1 Like

Hi,
I have received the bot this week. Don’t have big wrtting skills. This topic is interresting. Does the script can enable french conversation? If so, does it need to be modified, where and how does it need to be modified?

Hello Cameron and the Misty Team,

Hope everyone had a great holiday and best wishes for the new year. I am back and the skill is working again. I am working on building the demo for CES – so far so good. Thanks a bunch.

Sanjay

1 Like

Thanks for your great work @cameron. Was getting no response data from DialogFlow so I tried your new code in your repo and all is working (including batteryLevel). Some changes I see are volume, new return properties, texttospeech URL for batteryLevel. Very helpful.

1 Like

Hi, Conversation skill users!

The audio recording issue discussed in this thread is resolved by the Android patch released on January 7, 2020. This patch will not be installed automatically as part of an over-the-air system update. Instead, it must be applied manually, from a computer connected to the same network as your Misty II robot.

It’s safe to consider this patch a requirement for the best experience with using the conversation skill (or for anything else that involves Misty’s microphones) :slight_smile:

You can download the patch and find installation instructions in this post:

Let us know if you have questions! :robot:

1 Like