Robot Not a Dev? Pre-order Now
Not a Dev?

Misty Community Forum

New Command Center Moving Map

Although it’s still a work-in-progress, I figured this would be a good time to post the work I’ve been doing on creating a new moving map within the Command Center. It’s still pretty basic at this point, but definitely fun to play with.

Command Center: Moving Map

How to use it now
Follow this link to access an updated version of the Command Center with the new Moving Map section:

http://misty.thewebpro.com/cc.html

Just scroll down to the Moving Map section and click the Toggle Map button to get started.

IMPORTANT: If you do not see Misty moving in the map when driving, you’ll need to refresh your browser (maybe more than once). This happens because the map initialized before Misty’s websocket connection was made…I should have a fix for this in a future release.

What is does
The moving map pulls in real-time sensor data from Misty using the REST API and displays it on a map in the Command Center. This currently includes data from the Time of Flight (ToF) and Bump sensors, the IMU, Actuators (just head yaw for now), and drive encoders.

With this data, you can watch Misty build an idea of her surroundings while you drive her around your space. With enough driving time and exploration, you can build a decent map of the space around Misty. You can then save that map data and load it in (coming soon) for later use when you want to explore the same area.

Current features

  • Real-time movement and rotation following of Misty on map
  • Adds ToF data to map as soft black markers (valid data only)
  • Adds bump sensors to map as hard blue markers
  • Adds initial location marker (to allow you to find your way back home)
  • Decent memory management
  • Save your map to JSON format
  • Reset your map and start with a clean canvas

Other information
This moving map was built on top of the Phaser desktop and mobile HTML5 game framework that uses Canvas and WebGL to power browser-based games. Specifically I use Phaser version 2 here since I’m more familiar with that version. For more info on Phaser, visit https://phaser.io/.

Future roadmap (near-term)

  1. Update the ToF sensor dots in the map to self-update if better or contradictory data comes in. This could also help detect dynamically moving objects (e.g. people, pets, doors).

  2. Build the “Load Map” function to bring in previously saved locations. This is great for moving your robot from, for example, the office to home and using a pre-saved map to move around in. This is part of the reason I built in the “parking space” feature.

  3. With both new and saved maps, be able to click a point on the map and instruct Misty to go there via path finding (easy now that the map is built on top of a gaming platform). Multiple could also be added to make waypoints enabling a quick and dirty way to create a smart wander skill. Waypoints would also save with the “Save Map” feature to be reusable.

Have fun and let me know what you think! I’d love to hear about your experience with it.

4 Likes

Awesome to see what you’ve been working on! Thanks for sharing :slight_smile:

@michaelrod Just came across you post and this is fascinating! It looks like you are increasing the opacity of a object with multiple measurements from the sensor. Are you simply plotting this on the web page? Or also building a map? Constructing a map in this way could be really exciting.

How do you deal with multiple measurements of the same location in space? I wonder how you could synthesize measurements from each of the sensors in a given pass and/or new measurements with an old map. As measurements accumulate for a given point in space, the probability of an object goes up, so it seems like the map could then effectively become a 2D probability space that gets refined over time. In this case the robot could plan routes by optimizing for the minimum gradient path between it and the desired location.

I’m not a software engineer, maybe I’m behind the times and your program already does this?

Hey @miles…I’m pulling in data from the 4 front and back ToF sensors with a very fast debounce (from 10ms to 140ms, depending on the sensor). Then on every hit, I plot it with a very transparent black marker. As more ToF hits appear, the spot will get darker basically drawing a spot of the object it hit. In this way, you can pretty much guarantee there’s an object in that spot if it’s dark enough to see on the map.

As for multiple measurements in the same location, they help darken that feature on the map. For memory purposes in the future, I’ll limit the “darkness”, but for now it can stack pretty infinitely. I did add a slow decay to these markers, so they will fade out and die over time, but I’ll probably exchange that for a smarter decay int he future. In the update I just released today (details coming very shortly), the ToF markers can also be over-written if another ToF maker farther down Misty’s focus is detected. This lets the data update itself…great for if someone crosses Misty’s path temporarily so the ToF markers don’t show permanently show something that was there temporarily.

About planning routes, I’m already on it…stay tuned for my next post here in a few minutes.

1 Like

Just in before the weekend, I’ve got a little update for the Moving Map feature.

Here’s a quick list of the important items in this update:

  • Fixed “robot ready” connection issues - The map now waits for a successful connection to Misty before trying to initialize and will warn if Misty (and the map) isn’t connected.

  • Updated gridwork - grid lines now represent 10cm (lighter gridlines) and 1m (darker gridlines)

  • Resized map to 8000 x 8000 pixels - You can now map out an area of ~33 sq. meters or 100 sq. ft. (this will get bigger once I get better memory management in)

  • Added ToF bullets - the ToF pulses (only valid ones with a status of 0) are now being represented in real-time

  • Added manual panning & camera following - Camera now follows robot movement as you move around the map or can be panned manually with the arrow keys

  • Better memory optimization - without this, the map would use a large amount of memory when used for long periods

  • ToF marker updating - the black ToF markers that are left by a valid ToF hit will now overwrite if a farther ToF hit is detected (in other words, they die when hit by one of our new ToF bullets mentioned above)

  • Added tile mode - allows a different perspective on the same data

  • New map display modes - The previous grid map has now been joined with a mode to see just tiles, or for seeing both at the same time (default).

  • Updated and optimized saving of map - map clears out any “dead” sprite (ToF marker) data before saving

  • Allow loading of map - Now pull in a map you’ve previously saved. Important: if you’re Misty is in the same exact spot and pointing the same direction as when the map was saved, you can pick up right where you left off.

  • New draw modes and pen colors - In grid display mode, fill your map up with ToF data by scanning with Misty. Then, switch between tile and grid/tile display modes and use the pen to draw in walls manually to clean things up, add in virtual barriers for Misty, and make your map easier to view.

  • Added initial visual path finding (work in progress) - You can now add drop a “path marker” on the map and Misty will map out a path, avoiding obstacles and with enough clearance to fit Misty through.

Much more work still ahead, but for what’s right next up, along with some bug fixes and finishing touches, I’m working on 1) getting Misty to physically follow the path created when dropping a “path marker”, and 2) a fullscreen mode to use up the browser with the much larger map…great for larger-scale mapping in a house or office.

How to use it
Follow this link to access the latest version of the Moving Map embedded in the Command Center:

http://misty.thewebpro.com/cc.html

1 Like

This update is awesome! Deleting objects where the ToF “bullets” pass through is an interesting way of updating the map. It seems that requires the assumption that the latest measurement is more accurate than the previous measurement which is problematic for ToF sensors as well as the error in the position of the robot itself within the map. In your gif it seems to update the map to more accurately represent the wall, have you noticed that is always the case?

You mention dropping a marker in the location of every ToF “hit” and that they can stack. In the case of shooting through a previous marker, I wonder if you could subtract a marker from the stack in the old location in addition to adding one in the new location. With enough samples this may account for the error in each measurement. Alternatively a Kalman filter might help estimate the true position.

Last question, I hadn’t thought about the map decaying, what’s the reasoning behind that?

2 Likes

this is fun!

orthogonal question: why not make it a separate website instead of embedding and forking the (is it open source?) Command Center? in particular, it is quite confusing to see a Misty-branded website that is not officially part of Misty Robotics, or is it?

my question is not motivated by legal trolling, but rather, this motivates us to return to the question of security and lack of HTTPS support for the Command Center.

1 Like

Thanks, @miles. As you mentioned, the way I setup the ToF marker updates helps toss out less accurate ToF measurements in real-time as long as a higher percentage of the data is valid.

In your gif it seems to update the map to more accurately represent the wall, have you noticed that is always the case?

Absolutely. This updated version is constantly updating the ToF marker positions as Misty moves and only leaving those markers which represent the most accurate data. We also get the bonus of killing off less valid data and freeing up browser memory on a constant basis.

I wonder if you could subtract a marker from the stack in the old location in addition to adding one in the new location

This is what’s happening…I was looking for the lowest cost way of updating the ToF markers and the “bullets” seemed perfect. If the ToF data says something previously marked is no longer there, then the bullet kills it. But it works in the same way as when the ToF markers stack on top of each other strongly marking an object…the bullets slowly kill off an object, working their way top-down.

Alternatively a Kalman filter might help estimate the true position.

Thanks for the suggestion…I’ll look into it. As long as I can keep the per-frame processing low as it is now and present the most accurate data, that’s the method I’ll stick with. Otherwise it’d quickly become a memory hog and run too slow or, worse, kill the browser.

Last question, I hadn’t thought about the map decaying, what’s the reasoning behind that?

It was an initial way of lowering the memory usage before I added the bullets. I’ll likely take that out or greatly reduce the decay on a future update.

1 Like

I built the map into the existing Command Center since it already had all the manual driving controls and the websockets built-in…didn’t need to recreate everything all over again. As the initial version started taking shape, I realized how it fit pretty perfectly with the other tools available (sensor data, expression, head movement controls, etc…) in the Command Center and I was using them enough in tandem with the map that I just never thought of moving it out.

Hopefully I’m not stepping on anyone’s toes over at Misty though…if they ask, I’d happily create a standalone version. In fact, the fullscreen version which I’ve got in the works is just that.

We (Misty) perfectly expected people to take the web-based tools and re-mix them in this way. If this starts to get popular, it might be good to change up how it looks a bit. But we are probably a ways away from that mattering :slight_smile: