Home Community Blog Buy Now
Blog

Misty Community Forum

Person Pose Estimator - How to get image at time of Pose Event?

We have been able to test the person pose estimator using the following C# code. How do we get the image at the time the Pose Event occurred? Also any documentation on the Person Pose Estimator would be very helpful. Thanks.

============================================
private void TestPersonPoseEstimator()
{
_misty.RegisterPoseEstimationEvent(1000, false, “PoseEstimationEvent”, null);
_misty.StartPoseEstimation(0.20, 0, null);
_misty.PoseEstimationEventReceived += ProcessPoseEstimationEvent;
}

private void ProcessPoseEstimationEvent(object sender, IPoseEstimationEvent e)
{
IList humanPoseKeyPoints = (IList)e.KeyPoints;
foreach (HumanPoseKeyPoint hpkp in humanPoseKeyPoints)
{
//HumanPoseBodyPart.LeftAnkle;
//HumanPoseBodyPart.LeftEar;
//HumanPoseBodyPart.LeftElbow;
wdm.WriteDebugMessage(ProcName, "Body Part: {hpkp.BodyPart} "); wdm.WriteDebugMessage(ProcName, "Image X: {hpkp.ImageX} ");
wdm.WriteDebugMessage(ProcName, "Image Y: {hpkp.ImageY} "); wdm.WriteDebugMessage(ProcName, "Confidence: {hpkp.Confidence} ");
wdm.WriteDebugMessage(ProcName, "Pitch: {hpkp.Pitch} "); wdm.WriteDebugMessage(ProcName, "Yaw: {hpkp.Yaw} ");
}
}

SAMPLE OUTPUT:
skillrunner.js:511 "Misty.Skill.ErgonomicsCoach: IN ProcessPoseEstimationEvent: – Body Part: Nose " “2020-09-14T14:52:31.0482832Z”
skillrunner.js:511 "Misty.Skill.ErgonomicsCoach: IN ProcessPoseEstimationEvent: – Image X: 209 " “2020-09-14T14:52:31.0482832Z”
skillrunner.js:511 "Misty.Skill.ErgonomicsCoach: IN ProcessPoseEstimationEvent: – Image Y: 76 " “2020-09-14T14:52:31.0482832Z”
skillrunner.js:511 "Misty.Skill.ErgonomicsCoach: IN ProcessPoseEstimationEvent: – Confidence: 0.8645661 " “2020-09-14T14:52:31.0639061Z”
skillrunner.js:511 "Misty.Skill.ErgonomicsCoach: IN ProcessPoseEstimationEvent: – Yaw: -0.0870705 " “2020-09-14T14:52:31.0639061Z”
skillrunner.js:511 "Misty.Skill.ErgonomicsCoach: IN ProcessPoseEstimationEvent: – Pitch: -0.217829 " “2020-09-14T14:52:31.0639061Z”

any update on this question? how do we get the image based on the person pose estimator values? documentation? :slight_smile:

@wspeerbrecker thank you for trying out this new feature.
I’m excited to see what you build and thank you for being patient with us while we work on the documentation.

Right now there is, unfortunately, no way to get the image that we used to get the person pose estimator values. However, we’ve had some success sending a TakePicture as part of the event call back, it would be ~100 ms behind what was used in the pose event. Hopefully, that will fit your requirement for now.

Also, the imageX and imageY values are based on a 320x320 grid of the picture, so you’ll want to adjust the values based on the size of the image you request from the camera.