Home Community Blog Buy Now
Blog

Misty Community Forum

Help with images saving and facial recognition

Hi, this is my first post here. I’m currently working on a project with Misty for my senior project. I currently have two big snags that are impeding progress.

  1. When I save a picture it seems to only actually take and save the image if I change the resolution each time I run the skill, even if it is using a new file name.
    code --> misty.TakePicture("intruder", 320, 240, true, true);

  2. Using facial recognition through the command center in the browser misty seems to have no trouble recognizing me, however trying to use facial recognition in my skill code misty has only recognized me two or three times out of the dozens the skill has been ran.

code for event listener

function registerFaceDetection() {
    misty.AddPropertyTest("FaceDetect", "PersonName", "exists", "", "string");
    misty.RegisterEvent("FaceDetect", "FaceRecognition", 1000, true);
}

I access the data using data.PropertyTestResults[0].PropertyValue and it almost always returns “unknown person”
Again, using facial recognition through the command center works fine, so I’m unsure why using it in the skill seems to be different. Does the sdk use a separate model that needs to be trained separately from the api, or am I missing something in my code?

Thanks

Did you check logs? If so, were there are any errors? For example, compare with the log messages in the bug report at Issues with `Set` and `Get` after 2020.04.07 System Update

Thanks for the quick reply! I ran the skill again today and there are no logs generated for 2020/4/15. I checked yesterday’s logs and I think they are talking more about external api’s I was trying to access as well, and I’m not seeing any similarity to the logs you linked. With everything but the facial recognition and camera code stripped from my skill I am not getting any logs generated when I run the skill.

When you say there are no logs, where are you checking for these logs? You can use the Download Log File button on the Command Center Page or if for some reason that doesn’t work you can access the log directory on the robot by going to \\YOUR_ROBOT_IP\c$\Data\Misty\Logs

You can also change the robot log level by using SetLogLevel command.

Otherwise, without seeing your whole skill it’s hard to tell what you’re doing wrong.

I was checking the logs using ‘http://mistyip/api/logs?date=2020/4/17
The logs I get from that seem to be the same as the one’s I downloaded.
I don’t see anything that blatantly seems like it is a camera or facial recognition failing or erroring.

Here is the code for my skill, much of which is taken from misty dev blogs.

// Starts Misty's face detection process, so we can register for
// (and receive) FaceRecognition event messages.
misty.StartFaceDetection();

// Sets up our FaceRecognition event listener.
function registerFaceDetection() {
    misty.AddPropertyTest("FaceDetect", "PersonName", "exists", "", "string");
    misty.RegisterEvent("FaceDetect", "FaceRecognition", 1000, true);
}

// Defines how Misty should respond to FaceDetect event messages. Data
// from each FaceDetect event is passed into this callback function.
function _FaceDetect(data) {

    if (data.PropertyTestResults[0].PropertyValue == "unknown person"){
        misty.TakePicture("intruder", 800, 600, true, true);
        misty.ChangeLED(255, 0, 0);
        misty.MoveArmDegrees("both", -80, 100);
        misty.DisplayImage("e_Anger.jpg");
    } else {
        // Prints a debug message with FaceDetect event data
        misty.Debug(JSON.stringify(data));
        misty.ChangeLED(0, 255, 0); // Changes LED to green
        misty.PlayAudio("s_Joy.wav", 100);
        misty.DisplayImage("e_Joy.jpg"); // Displays happy eyes
        misty.MoveArmDegrees("both", -80, 100); // Raises both arms
    }

    misty.StopFaceDetection();
    
}

registerFaceDetection();

Hi Jacob,

It looks like you’ve crossed up face recognition and face detection. Face detection just checks if it sees ANY face and the name returned is always “unknown person”. What you want here is face recognition.

If you make that switch do things work as expected?

Yes! Thank you so much. It’s so easy to get tunnel vision working on it for so long. Do you have any idea why the camera would be acting how it does though? The next goal is to take a picture, save it, and send it using an api request to twilio. It at least takes the photo the first time and each time I change the resolution, so I can go ahead with the api request, however, I would like to have the photo functionality working completely.

I’m not sure what could be happening with TakePicture. I tried this myself with the following code and it worked as expected; I ended up with 4 photos on the 410 in the \c$\Data\Users\DefaultAccount\Pictures\Images\ folder. If you share your relevant code we might be able to see the problem.

for (var i = 1; i <= 4; i++) {
misty.PlayAudio(“s_SystemCameraShutter.wav”, 100);
misty.TakePicture(“photoSaveTest” + i + “.jpg”, 320, 240, true, true);
misty.Pause(5000);
};

Thanks Adam! My code is the same as above:

misty.TakePicture("intruder", 800, 600, true, true);
misty.ChangeLED(255, 0, 0);
misty.MoveArmDegrees("both", -80, 100);
misty.DisplayImage("e_Anger.jpg");

Your example gave me the idea of adding a misty.Pause after taking the photo, and now it seems to work fine. the code is now:

misty.TakePicture("intruder", 800, 600, true, true);
misty.Pause(900);
misty.ChangeLED(255, 0, 0);
misty.MoveArmDegrees("both", -80, 100);
misty.DisplayImage("e_Anger.jpg");

For future reference, should I always pause after taking a photo? I assumed you paused because it’s in the loop, and the other actions seem to run fine concurrently with no pausing.

This got interesting :slight_smile: I think that the problem here is that TakePicture is called and then before it completes, StopFaceRecognition is called. StopFaceRecognition shuts down continuous image acquisition which aborts the attempt to take a picture. The following code works for me with the pause and without the pause I get no picture.

So I think that in general you don’t need to have pause after calling TakePicture, but you do need a pause in between TakePicture and StopFaceRecognition.

I’ll ask people more familiar with this code and/or dig in deeper myself on Monday and confirm.

misty.StartFaceRecognition();

function registerFaceRecognition() {
    misty.AddPropertyTest("FaceRec", "PersonName", "exists", "", "string");
    misty.RegisterEvent("FaceRec", "FaceRecognition", 1000, true);
}

function _FaceRec(data) {

    if (data.PropertyTestResults[0].PropertyValue == "unknown person"){
		misty.TakePicture("unknown", 800, 600, true, true);
		misty.Debug("unknown");
    } else {
		misty.TakePicture("known", 800, 600, true, true);
        misty.Debug("known");
    }
	
	misty.Pause(1000)

    misty.StopFaceRecognition();
}

registerFaceRecognition();
1 Like