Data Projects

NASA Space Robotics Challenge

Image Processing

Problem

Given a small set of training images, create an object recognition algorithm. Given an image, your algorithm should be identify regions or piexels as rovers, the two different stations, the Moon's surface, or space.

Resources

Overview

This project requires some amount of programming ability. In the actual NASA SRC, this problem was solved using Python and the image processing library OpenCV.

Background

The NASA Space Robotics Challenge (SRC) is a virtual robotics challenge which has completed two phases. In 2017, Phase 1 involved programming a humanoid Valkerie robot to semi-autonomously prepare a base on Mars for the arrival of human travelers. In 2021, Phase 2 involved programming a small fleet of moon rovers to coordinate in the locating, mining, and hauling of resources from the lunar regolith.

Both phases relied on combining several technologies and algorithms to create semi-autonomous and fully autonomous robots. One important technology used was image processing for computer vision.

The video below was one of many test runs from SRC phase 2. Three robots collaborate to find, excavate, and haul resources that could be used to support operations on the Moon.

The robot interactions are entirely vision based, using cameras mounted on top of each robot chassis. Around 1:00 in the video above, notice that the hauler executes a docking procedure with the excavator. The subwindow which appears temporarily in the top left is actually a feed from the excavator's camera, which has turned around to face the hauler. All of the hauler's movements are based on the location of its red-orange bucket in the frame of that camera!

In addition to robot interactions, navigation is a critical component in autonomous robotics. If a robot on the Moon crashes or flips over, we can't just go pick it up! Navigation is also important when we want to haul resources to a particular location like a processing plant. Computer vision was essential in navigation in the SRC.

Being able to identify objects in images can help with navigation. Sometimes a robot can triangulate its own position using relative positions of known objects, or it can infer distance to a target object.