Robot Engage Blog Buy Now
Blog

Misty Community Forum

Documentation for C# SDK

when will more documentation be available for the C# .NET SDK (i.e., describing methods and parameter values)? Or more sample C# code? For example, I am trying to use the “Hey Misty” Key Phase methods and can’t find any documentation.

I’m hoping to publish reference documentation for the SDK libraries within the next few months, but for now please feel free to post any questions you have here and we’ll do our best to answer them in a timely manner!

Key phrase recognition in the .NET SDK is expected to work in a similar fashion to the commands in the REST & JavaScript API. The parameters for each command should be the same, and they should behave in a similar way.

Using “wake word” functionality requires you to:

a) Register a listener for KeyPhraseRecognizedEvent event messages
b) Call the StartKeyPhraseRecognition command (REST API docs, for parameter descriptions)
c) Set up a callback function that handles data from KeyPhrasedRecognized events.

There are a few different ways to register events in the .NET SDK. Most event types let you use traditional C# events and event handlers, and you can also call an SDK library method to register a listener for each specific event type. The docs have examples that show the different ways to register events in the .NET SDK (those examples don’t use KeyPhraseRecognized events, but the process is similar for all event types.)

Here’s a bit of (untested) sample code:

// Registers your KeyPhraseRecognizedEvent
// The first argument is the name of the callback (KeyPhraseRecognizedCallback)
// 250 is the debounce
// The third arg is "keepAlive" (set to "true" to keep event registered after it triggers for the first time)
// fourth and fifth arguments are optional "eventName" and "commandCallback" params
_misty.RegisterKeyPhraseRecognizedEvent(KeyPhraseRecognizedCallback, 250, true, null, null);

// Starts key phrase recognition, but does not enable capture speech functionality
// See REST API docs on StartKeyPhraseRecognition for parameter definitions
_misty.StartKeyPhraseRecognition(false, false, null, null, null);

And then the KeyPhraseRecognizedCallback might look like this:

private void KeyPhraseRecognizedCallback(IRobotEvent eventResponse)
{
    // Logs a message with timestamp of KeyPhraseRecognized event
	_misty.SkillLogger.LogVerbose($"MySkill : KeyPhraseRecognizedCallback : Created= {FormatDate(eventResponse.Created)}");

    // Define what else should happen here! 
}

yes - I have similar code and that part works. But how do I record the audio after “Hey Misty” has been identified? I tried the SaveAudio() and PlayAudio() to hear what was recorded but could not get it to work. Do I need to use GetAudioFile() or other methods? Do I specify the name of the .wav file or is it hardcoded e.g., “hey_misty.wav”?

Any help would be appreciated. Thanks.

As long as you set the first arg in StartKeyPhraseRecognition to true, then Misty should start recording speech immediately after she hears the wake word.

The VoiceRecord event type provides information about the speech capture recordings that Misty creates with the CaptureSpeech or StartKeyphraseRecognition commands. VoiceRecord events trigger after the completion of a speech capture attempt. Here’s an (again, untested) sample:

_misty.RegisterVoiceRecordEvent(OnVoiceRecordReceived, 10, false, "VoiceRecordEvent", null);

And that callback might look something like this (which would have Misty play the recording out loud):

private void OnVoiceRecordReceived(IVoiceRecordEvent voiceEvent)
{
	string fileName = "";
	if (voiceEvent.ErrorCode == 0)
	{
		fileName = voiceEvent.Filename;
		_misty.PlayAudio(fileName, 100, null);
	}
}

Or you could call GetAudioFile command to retrieve the audio file data in cases where you want to send it off to a service like Dialogflow for additional processing.

1 Like

If you’re comfortable with JavaScript, there is an example of using VoiceRecord events with KeyPhraseRecognition in the JavaScript API docs. The general idea is pretty similar to how you’d do things in C# :slight_smile: