Cool New Technology Will Allow Scientists to Work Virtually on Mars in 3-D

A screen view from OnSight, showing how scientists can meet together in a virtual 3-D simulation on the Martian surface. Such technology will allow scientists to better study features of interest and plan rover or lander activities. Image Credit: NASA/JPL-Caltech

A screen view from OnSight, showing how scientists can meet together in a virtual 3-D simulation on the Martian surface. Such technology will allow scientists to better study features of interest and plan rover or lander activities. Image Credit: NASA/JPL-Caltech

The rover and lander missions on Mars have been cool enough with their high-tech cameras, lasers, and on-board laboratories, but now they are about to get even better, thanks to a new partnership between NASA and Microsoft. New technology being developed, called OnSight, will allow scientists to work virtually on Mars within 3-D simulations, changing how they interact with the machines and conduct science operations.

How it's done now: scientist Katie Stack Morgan examines images from the Curiosity rover on Mars on regular computer screens. Image Credit: NASA/JPL-Caltech

How it’s done now: Scientist Katie Stack Morgan examines images from the Curiosity rover on Mars on regular computer screens. Photo Credit: NASA/JPL-Caltech

As Dave Lavery, program executive for the Mars Science Laboratory mission at NASA Headquarters in Washington, explained: “OnSight gives our rover scientists the ability to walk around and explore Mars right from their offices. It fundamentally changes our perception of Mars, and how we understand the Mars environment surrounding the rover.”

It will first be used with the Curiosity rover in Gale crater, as the rover is entering the scenic foothills of Mount Sharp, a region of buttes and mesas much like the American southwest.

So how does it work? OnSight will create a 3-D simulation of the surrounding Martian environment using data from the rover. The technology will allow scientists from around the world to meet together virtually at the rover’s worksite(s), where they can plan activities and preview their work, regardless of their actual physical location. It sounds like something out of a science fiction movie, but it’s not; OnSight uses holographic computing to overlay visual information and rover data into the user’s field of view. The result is a hybrid of the real physical world on Mars and computer-generated imagery.

Like other Virtual reality technology, OnSight will allow scientists to "be" with the rovers on Mars or other missions. Photo Credit: Wikimedia

Like other Virtual reality technology, OnSight will allow scientists to “be” with the rovers on Mars or other missions. Photo Credit: Wikimedia

“We believe OnSight will enhance the ways in which we explore Mars and share that journey of exploration with the world,” said Jeff Norris, JPL’s OnSight project manager.

Up until now, scientists have had to make do with using imagery from Mars on regular computer screens. OnSight will add a more natural sense of depth to the human visual experience.

Similar to other virtual reality technology, scientists will wear a Microsoft HoloLens device, which places them “within” images of the Martian terrain sent back by the rover. They can then virtually walk around on the surface and closely examine rock outcrops or other interesting features, almost like actually being there. Rover activities can be planned by simply looking at a target and using gestures to select menu commands. Rather than just looking at images, viewers will be able to feel more like they are there on the surface of Mars themselves. How cool is that?

As Norris noted: “Previously, our Mars explorers have been stuck on one side of a computer screen. This tool gives them the ability to explore the rover’s surroundings much as an Earth geologist would do field work here on our planet.”

New "selfie" image of the Curiosity rover at the Book Cliffs outcrop in Pahrump Hills, taken on sol 868 (January 14, 2105). Image Credit: NASA / JPL / MSSS / Elisabetta Bonora / Marco Faccin

New “selfie” image of the Curiosity rover at the Book Cliffs outcrop in Pahrump Hills, taken on sol 868 (Jan. 14, 2105). Image Credit: NASA/JPL/MSSS/Elisabetta Bonora/Marco Faccin

OnSight was developed out of the growing partnership between NASA/JPL and Microsoft, which has focused on advances in human-robot interaction. The JPL team which developed OnSight specializes in technology for controlling robots and spacecraft. OnSight is now the next logical step in that advancement.

Also, OnSight should not be confused with InSight (Interior Exploration using Seismic Investigations, Geodesy and Heat Transport), the next Mars lander to be launched next year, which will examine the interior of the planet to help scientists understand the evolutionary history of Mars.

OnSight will first be tested with the Curiosity rover later this year, and then might be extended to the upcoming Mars 2020 rover mission and others (and not limited to just Mars, of course). The mesas and buttes in the foothills of Mount Sharp, which Curiosity is now very close to, would provide a fantastic location for testing this kind of technology.

While it still isn’t the same as actually and physically being there, until humans do land on the Red Planet, OnSight will probably be the next best thing.

 

Want to keep up-to-date with all things space? Be sure to “Like” AmericaSpace on Facebook and follow us on Twitter: @AmericaSpace

1 comment to Cool New Technology Will Allow Scientists to Work Virtually on Mars in 3-D