Robot Engage Blog Buy Now
Blog

Misty Community Forum

Wake Word Javascript Help

All,

I have a silly question… I am using the wakeword skill (from the examples). The wake word detection works one time only… I would think that it would continue to recognize wake words… I am sure, I dont have the behavior understood correctly.

function detectWakeWord() {
   misty.Debug("Starting key phrase recognition...");
   misty.StartKeyPhraseRecognition();
   misty.ChangeLED(0, 0, 255); 
   misty.RegisterEvent("heyMisty","KeyPhraseRecognized", 10, false);
   misty.Debug("Started wake word detection. Misty will play a sound when she hears 'Hey Misty'.");
}

function _heyMisty() {
   misty.Debug("Key phrase recognized!");
   misty.Debug("Starting wake word detection again...");  
   detectWakeWord();
}

did you try misty.RegisterEvent("heyMisty","KeyPhraseRecognized", 10, true); ?

This is a good suggestion, although I tested it on my own robot – along with the original snippet that @guptahome100 shared – and it results in the same behavior, where Misty only reacts to the first wake word. And I would still expect the code in the original snippet to work – the _heyMisty() callback should register another heyMisty wake word event listener after the first listener is unregistered.

There have been a few behind-the-scenes changes to wake word handling in preparation for some features that are coming with the next system update, and it’s not impossible that you’ve uncovered a bug here, @guptahome100. I’ll share your discovery with the team and we’ll get back to you when we learn more.

I tried that as well and had no luck. Seems like some plumbing changed… Glad I asked :slight_smile:

Just to follow up – the changed plumbing does appear to be the culprit here. That plumbing in the current software is causing Misty to try and record speech after recognizing the wake word – and because her microphones are already being used by the speech capture mechanism, they aren’t available to start listening for the wake word again.

You can expect this to be resolved with the new parameters that will be available for the StartKeyPhraseRecognition command when end-of-speech detection becomes generally available with next week’s system update, on 2019-12-17T07:00:00Z. Because much (though not all, hence it’s omission from the last release announcement) of the plumbing for end-of-speech detection was put into place with Misty’s last update, you may actually be able to get your snippet to work by using misty.StartKeyPhraseRecognition(false);, instead of just misty.StartKeyPhraseRecognition();. It was a miss on our part that we failed to observe how laying the groundwork for speech capture in the current software would alter existing skill code. Thanks very much for catching this and bringing it to our attention, and apologies for any frustration it’s caused on your end!

Thanks for the update. Looking forward to the new system release.

sanjay

Jonathan,

Got the new build and I am still having issues. Even though on the console I can see that key phrase recognition has been restarted, Misty does not recognize “hey misty”. Any thoughts on what could be going on?

sanjay

Thanks for the update, @guptahome100. So that i can be sure I am using the same code as you to help troubleshoot, are you testing with the same snippet as shared in the original post to this thread?

here it is

detectWakeWord();

// Sets up wake word detection
function detectWakeWord() {
   misty.Debug("Starting key phrase recognition...");
   misty.StartKeyPhraseRecognition();
   misty.ChangeLED(0, 0, 255); 
   misty.RegisterEvent("heyMisty","KeyPhraseRecognized", 10, false);
   misty.Debug("Started wake word detection. Misty will play a sound when she hears 'Hey Misty'.");
}

function _heyMisty() {
   misty.Debug("Key phrase recognized!");
   misty.ChangeLED(255, 0, 0);
   misty.Pause(2000);
   misty.Debug("Starting wake word detection again...");
   misty.StartKeyPhraseRecognition();
   misty.ChangeLED(0, 0, 255); 
   misty.RegisterEvent("heyMisty","KeyPhraseRecognized", 10, false);
   detectWakeWord();
}

I’m still validating this, but it looks like you’ve discovered a bug :bug:

The latest update added new params to the StartKeyPhraseRecognition command, so that you can start speech capture after Misty recognizes the wake word. If you just want to activate the wake word without using speech capture, you’d set the CaptureSpech argument to false, like this:

misty.StartKeyPhraseRecognition(false);

However, when I try updating your snippet with this new parameter, Misty always tries to capture a voice recording, no matter what value I pass in for the CaptureSpeech argument. We’re actively investigating this and I’ll keep you updated here with our learnings.

1 Like

Just as a bit of follow up, this a confirmed bug in the current release. I will add an entry to the Known Issues section of these forums, so you can track our progress on a fix.

For now, you can work around the issue by accounting for the fact that StartKeyPhraseRecognition always triggers Misty to capture speech. Here is an example from the JavaScript API documentation that shows how to use the speech capture functionality of this command of a JavaScript skill:

// The following illustrates how to use key phrase recognition in a
// JavaScript skill for Misty II

/* Event Listeners */

// Registers a listener for VoiceRecord event messages, and adds return
// properties to event listener so that we get all this data in the
// _VoiceRecord callback.
misty.AddReturnProperty("VoiceRecord", "Filename");
misty.AddReturnProperty("VoiceRecord", "Success");
misty.AddReturnProperty("VoiceRecord", "ErrorCode");
misty.AddReturnProperty("VoiceRecord", "ErrorMessage");
misty.RegisterEvent("VoiceRecord", "VoiceRecord", 10, false);

// Registers a listener for KeyPhraseRecognized event messages
misty.RegisterEvent("KeyPhraseRecognized", "KeyPhraseRecognized", 10, false);

misty.StartKeyPhraseRecognition();

/* Callbacks */

// Triggers when misty hears the key phrase
function _KeyPhraseRecognized() {
   misty.Debug("Key phrase recognized! Now listening for speech.");
}

// Triggers when Misty starts listening for the key phrase 
function _StartKeyPhraseRecognition() {
   misty.Debug("Now listening for the key phrase.");
}

// Triggers when Misty finishes capturing a speech recording
function _VoiceRecord(data) {
   // Get data from AdditionalResults array
   var filename = data.AdditionalResults[0];
   var success = data.AdditionalResults[1];
   var errorCode = data.AdditionalResults[2];
   var errorMessage = data.AdditionalResults[3];

   // If speech capture is successful, tell us and play the recording
   if (success = true) {
      misty.Debug("Successfully captured speech! Listen closely...")
      misty.PlayAudio(filename);
   }
   // Otherwise, print the error message
   else {
      misty.Debug("Error: " + errorCode + ". " + errorMessage);
   }
}

Other workarounds could be as simple as calling another StartKeyPhraseRecognition command in the callback function for the VoiceRecord event (psuedocode: “Hey, Misty” → starts listening → “Start listening again!” → triggers VoiceRecord, with callback that invokes StartKeyPhraseRecognition → repeat).

Thanks Jonathan… So I was using this to help debug what I am really trying to do, which is use the conversation skill set. I am wondering how to fix the issue with that. It seems like the issue I have is what at least one other individual is having. I has submitted a request earlier in the day on that… Anyway, you can take a look?

PS: I will be on vacation for the next 10 days starting tomorrow morning :slight_smile:

Then off to CES!