Misty fails to obtain pose (always)

[is-bibih]

Ever since Misty performed a system update last week (on Wednesday, Nov. 4th), it has been impossible for it to obtain pose for mapping or tracking. Currently, it has these software versions:

• Robot Version: 1.20.2.10706
• Sensory Services Version: 1.20.2
• Windows OS Version: 10.0.17763.253

Prior to the update, the robot was usually able to obtain pose, so I think the issue might be related to it. I have been mostly using the Navigation Module to attempt making a map, but I also tried doing it with the JS API and the API Explorer. Misty fails to get pose either way.

I’ve made sure the areas are appropriately lit using pictures from the fish-eye lens, and they do have different textures and depths, so I don’t think the spaces are the cause of the problem (especially since Misty was able to obtain pose previously). I’ve tried the suggestions regarding mapping and obtaining pose (spinning slowly in place, resetting the sensor, etc.), but nothing seems to have worked so far.

Is there a way I can fix this? Alternatively, is there a way to revert to the previous software versions? Any help would be greatly appreciated!

[Vlad]

Hi, sorry we missed this.

Are you familiar with adb and do you know how to get adb logs? Take a look at this post for some info about connecting via adb Misty does not obtain POSE on an existing map

and this:

Trouble AV streaming to VLC

It’s possible the streaming service is hanging on something. Do you know how to use adb? Here’s how to install it: https://www.xda-developers.com/install-adb-windows-macos-linux/ The most useful thing would be to grab the adb logs for the streaming service and send to me. Here’s how: Open up a command line or terminal adb connect <ROBOT_WIFI_IP>:5555 adb shell ps -ef | grep misty You should see something like this: root 2900 1 0 18:30:32 ? 00:01:46 sh /data/misty/scripts/mi…

If you can get the logs please email to vlad (at) mistyrobotics and I will take a look.

I wonder if the SLAM service is disabled somehow… are you able to get images from Fisheye camera? What does the endpoint GET <robot-ip-address>/api/services/slam return?

[is-bibih]

Thank you!

Currently, I don’t have an Ethernet-USB adapter, so I’ll get back to you about the adb logs (hopefully by tomorrow).

Regarding the fisheye camera, I am indeed able to get images. The enpoint returns

{"result":true,"status":"Success"}

[Vlad]

If you go to the Navigation page (Misty Robotics / SDK / Mapping) and connect to the robot, what are the Sensor, Navigation, SLAM status codes?

[is-bibih]

Before starting mapping or tracking:

• Sensor: Connected
• Navigation: Uninitialized
• SLAM: Ready

After selecting “Start Mapping”:

• Sensor: Streaming
• Navigation: Exploring
• SLAM: Exploring, Streaming

After selecting “Start Tracking”:

• Sensor: Streaming
• Navigation: Relocalizing
• SLAM: Tracking, Streaming

When mapping, Error occurred attempting to get map. undefined appears in the web console. Is this related?

[Vlad]

Do you have a LOT of maps? Just curious if you’ve deleted any or just kept making new maps. It’s possible you may low on space and that’s creating some issues. Just checking.

Also, mapping and tracking are mutually exclusive. The system should handle that, but just something to be aware.

Just curious, how are you checking that you are getting pose or not?

[is-bibih]

I deleted all the maps at some point when I was trying to use the API Explorer to map, but I do currently have a lot of maps (around 28). However, I think they’re all mostly empty? They usually end up looking something like this:

I’ll try deleting all of them again, just in case.

And I check if Misty has pose with this part of the Navigation Module. From the tutorials, I understood that it’s supposed to turn white and display values for the pitch, roll, and yaw.

I kind of remember that it also shows up in the SLAM status, but I’m not sure. When I used the JS API and the API explorer to try mapping, I checked whether the SLAM status said Misty had pose or not.

[Vlad]

28 maps is not a lot. If you had hundreds with stuff in them then maybe it would be an issue.

Reason I asked about how you check for pose is that I’ve seen in the past where the website sometimes doesn’t update properly, but usually a reload fixes it. Can you also expand the Show Diagnostics button and see what it says?

Once you get adb access and we can get logs I should be able to get more info to ascertain what’s happening.

[is-bibih]

Here are the diagnostics from the Navigation Module.

According to me, I sent an email with the SLAM log yesterday, but my email has been known to be unstable. I just sent them again through a different account. Thanks!

{
 "Diagnostics": {
  "Navigation Info": {
   "numKeyFrames": 1,
   "numKeyPoints": 1001,
   "numMapPoints": 7,
   "numTrackedPoints": 7,
   "occupancyGridSize": [
    0,
    0
   ],
   "usingImuProcessModel": false
  },
  "Performance Info": {
   "Accel IMU": {
    "avg FPS:": "125.107705",
    "latency stats:": {
     "max": "19.870415",
     "mean": "1.620770",
     "min": "0.441906"
    }
   },
   "Depth": {
    "avg FPS:": "30.054192",
    "latency stats:": {
     "max": "30.417124",
     "mean": "21.228808",
     "min": "19.990865"
    }
   },
   "Gyro IMU": {
    "avg FPS:": "99.854406",
    "latency stats:": {
     "max": "15.776558",
     "mean": "1.633886",
     "min": "0.394606"
    }
   },
   "IR": {
    "avg FPS:": "0.000000",
    "latency stats:": {
     "max": "0.000000",
     "mean": "0.000000",
     "min": "0.000000"
    }
   },
   "Visble": {
    "avg FPS:": "29.974759",
    "latency stats:": {
     "max": "24.086490",
     "mean": "12.455764",
     "min": "10.486148"
    }
   }
  }
 }
}

[Vlad]

Sorry, I did get your previous email, I just somehow missed it.

Looking at the logs and the navigation diagnostics above it seems like the sensor and software is working ok but it’s not seeing a lot of features to track. This either means the scene is not great for mapping, somehow the calibration is off, or something is wrong with the laser on the sensor.

Can you do the following in order of difficulty:

• Check that when the robot is mapping or tracking the laser on Misty’s StructureCore sensor is on. You should see a faint red sparkle on it. You can also look at it with a phone camera which should see the near IR laser.
• If the laser is on then move the robot to a nice open space with good texture and light around and try mapping again
• Lastly, I need you to do a recording of the sensor to see if the calibration is off. You can do this with the following REST commands (don’t map at same time):
  • To start recording: POST <robot-ip-address>/api/slam/record/start
  • To stop recording: POST <robot-ip-address>/api/slam/record/stop
  • Don’t make a long recording. A few seconds is fine. Have the robot pointed at a flat wall from about 1m out. I just want to see that the depth map is good.
  • Afterwards you will have to pull the recording directory using adb:
    • adb shell ls /sdcard/occ/mapping_sessions/
    • note the name of the latest directory starting with Occ
    • then adb pull /sdcard/occ/mapping_sessions/Occ..
    • zip that up and send it to me

[is-bibih]

Thanks for your help!

I got around to doing this today and somehow mapping works fine now? I thought it might be the lighting since I changed the lightbulb recently, but now it works even with the lights out (there’s some natural light). I have no idea what else could have changed, so I sent the zip just in case.

I don’t know if this is specific to my robot, but just in case it helps someone, the POST requests that worked for me were POST <robot-ip-address>/api/slam/record/start and POST <robot-ip-address>/api/slam/record/stop (without the dev directory).

[Vlad]

Hmmm, the recording you sent me does indeed have basically no depth information. It sounds like sometimes you get good depth and other times you don’t.

I also see some low voltage on the StructureCore USB bus errors in the log. This may be causing the sensor to malfunction sometimes giving you bad data which then can’t be used to map. My guess would be that the laser isn’t turning on sometimes

When you get a chance, please take a longer recording and maybe hand turn the robot a few degrees left and right just for some motion.

[is-bibih]

Yeah, it makes sense it would be something like that. Today Misty was unable to get pose as well.

I hope the new recording I sent can be useful. Is there anything I can do about the issue?

[Vlad]

I think there is something wrong with the StructureCore sensor on your robot. I think you should get it replaced. Reach out to support (at) mistyrobotics to get that replaced.

[is-bibih]

Alright, I’ll look into that. Thank you for your help!