GSFC Data Science Group Releases Innovative Foundation Model: SatVision Top-Of-Atmosphere (TOA)
At Goddard Space Flight Center, the GSFC Data Science Group has completed the testing for their SatVision Top-of-Atmosphere (TOA) Foundation Model, a geospatial foundation model for coarse-resolution all-sky remote sensing imagery. The team, comprised of Mark Carroll, Caleb Spradlin, Jordan Caraballo-Vega, Jian Li, Jie Gong, and Paul Montesano, has now released their model for wide application in science investigations.
Foundation models can transform the landscape of remote sensing (RS) data analysis by enabling the pre-training of large computer-vision models on vast amounts of remote sensing data. These models can be fine-tuned with small amounts of labeled training and applied to various mapping and monitoring applications. Because most existing foundation models are trained solely on cloud-free satellite imagery, they are limited to applications of land surface or require atmospheric corrections. SatVision-TOA is trained on all-sky conditions which enables applications involving atmospheric variables (e.g., cloud or aerosol).
SatVision TOA is a 3 billion parameter model trained on 100 million images from Moderate Resolution Imaging Spectroradiometer (MODIS). This is, to our knowledge, the largest foundation model trained solely on satellite remote sensing imagery. By including “all-sky” conditions during pre-training, the team incorporated a range of cloud conditions often excluded in traditional modeling. This enables 3D cloud reconstruction and cloud modeling in support of Earth and climate science, offering significant enhancement for large-scale earth observation workflows.
With an adaptable and scalable model design, SatVision-TOA can unify diverse Earth observation datasets and reduce dependency on task-specific models. SatVision-TOA leverages one of the largest public datasets to capture global contexts and robust features. The model could have broad applications for investigating spectrometer data, including MODIS, VIIRS, and GOES-ABI. The team believes this will enable transformative advancements in atmospheric science, cloud structure analysis, and Earth system modeling.
Examples of image reconstruction by SatVision-TOA. Left: MOD021KM v6.1 cropped image chip using MODIS bands [1, 3, 2]. Middle: The same images with randomly applied 8×8 mask patches, masking 60% of the original image. Right: The reconstructed images produced by the model, along with their respective Structural Similarity Index Measure (SSIM) scores. These examples illustrate the model’s ability to preserve structural detail and reconstruct heterogeneous features, such as cloud textures and land-cover transitions, with high fidelity.
NASA Performs First Aircraft Accident Investigation on Another World
5 min read
Preparations for Next Moonwalk Simulations Underway (and Underwater)
NASA’s Ingenuity Mars Helicopter, right, stands near the apex of a sand ripple in an image taken by Perseverance on Feb. 24, 2024, about five weeks after the rotorcraft’s final flight. Part of one of Ingenuity’s rotor blades lies on the surface about 49 feet (15 meters) west of helicopter (at left in image).
NASA/JPL-Caltech/LANL/CNES/CNRS
The review takes a close look the final flight of the agency’s Ingenuity Mars Helicopter, which was the first aircraft to fly on another world.
Engineers from NASA’s Jet Propulsion Laboratory in Southern California and AeroVironment are completing a detailed assessment of the Ingenuity Mars Helicopter’s final flight on Jan. 18, 2024, which will be published in the next few weeks as a NASA technical report. Designed as a technology demonstration to perform up to five experimental test flights over 30 days, Ingenuity was the first aircraft on another world. It operated for almost three years, performed 72 flights, and flew more than 30 times farther than planned while accumulating over two hours of flight time.
The investigation concludes that the inability of Ingenuity’s navigation system to provide accurate data during the flight likely caused a chain of events that ended the mission. The report’s findings are expected to benefit future Mars helicopters, as well as other aircraft destined to operate on other worlds.
NASA’s Ingenuity Mars Helicopter used its black-and-white navigation camera to capture this video on Feb. 11, 2024, showing the shadow of its rotor blades. The imagery confirmed damage had occurred during Flight 72.
NASA/JPL-Caltech
Final Ascent
Flight 72 was planned as a brief vertical hop to assess Ingenuity’s flight systems and photograph the area. Data from the flight shows Ingenuity climbing to 40 feet (12 meters), hovering, and capturing images. It initiated its descent at 19 seconds, and by 32 seconds the helicopter was back on the surface and had halted communications. The following day, the mission reestablished communications, and images that came down six days after the flight revealed Ingenuity had sustained severe damage to its rotor blades.
What Happened
“When running an accident investigation from 100 million miles away, you don’t have any black boxes or eyewitnesses,” said Ingenuity’s first pilot, Håvard Grip of JPL. “While multiple scenarios are viable with the available data, we have one we believe is most likely: Lack of surface texture gave the navigation system too little information to work with.”
The helicopter’s vision navigation system was designed to track visual features on the surface using a downward-looking camera over well-textured (pebbly) but flat terrain. This limited tracking capability was more than sufficient for carrying out Ingenuity’s first five flights, but by Flight 72 the helicopter was in a region of Jezero Crater filled with steep, relatively featureless sand ripples.
This short animation depicts a NASA concept for a proposed follow-on to the agency’s Ingenuity Mars Helicopter called Mars Chopper, which remains in early conceptual and design stages. In addition to scouting, such a helicopter could carry science instruments to study terrain rovers can’t reach.
One of the navigation system’s main requirements was to provide velocity estimates that would enable the helicopter to land within a small envelope of vertical and horizontal velocities. Data sent down during Flight 72 shows that, around 20 seconds after takeoff, the navigation system couldn’t find enough surface features to track.
Photographs taken after the flight indicate the navigation errors created high horizontal velocities at touchdown. In the most likely scenario, the hard impact on the sand ripple’s slope caused Ingenuity to pitch and roll. The rapid attitude change resulted in loads on the fast-rotating rotor blades beyond their design limits, snapping all four of them off at their weakest point — about a third of the way from the tip. The damaged blades caused excessive vibration in the rotor system, ripping the remainder of one blade from its root and generating an excessive power demand that resulted in loss of communications.
This graphic depicts the most likely scenario for the hard landing of NASA’s Ingenuity Mars Helicopter during its 72nd and final flight on Jan. 18, 2024. High horizontal velocities at touchdown resulted in a hard impact on a sand ripple, which caused Ingenuity to pitch and roll, damaging its rotor blades.
NASA/JPL-Caltech
Down but Not Out
Although Flight 72 permanently grounded Ingenuity, the helicopter still beams weather and avionics test data to the Perseverance rover about once a week. The weather information could benefit future explorers of the Red Planet. The avionics data is already proving useful to engineers working on future designs of aircraft and other vehicles for the Red Planet.
“Because Ingenuity was designed to be affordable while demanding huge amounts of computer power, we became the first mission to fly commercial off-the-shelf cellphone processors in deep space,” said Teddy Tzanetos, Ingenuity’s project manager. “We’re now approaching four years of continuous operations, suggesting that not everything needs to be bigger, heavier, and radiation-hardened to work in the harsh Martian environment.”
Inspired by Ingenuity’s longevity, NASA engineers have been testing smaller, lighter avionics that could be used in vehicle designs for the Mars Sample Return campaign. The data is also helping engineers as they research what a future Mars helicopter could look like — and do.
During a Wednesday, Dec. 11, briefing at the American Geophysical Union’s annual meeting in Washington, Tzanetos shared details on the Mars Chopper rotorcraft, a concept that he and other Ingenuity alumni are researching. As designed, Chopper is approximately 20 times heavier than Ingenuity, could fly several pounds of science equipment, and autonomously explore remote Martian locations while traveling up to 2 miles (3 kilometers) in a day. (Ingenuity’s longest flight was 2,310 feet, or 704 meters.)
“Ingenuity has given us the confidence and data to envision the future of flight at Mars,” said Tzanetos.
More About Ingenuity
The Ingenuity Mars Helicopter was built by JPL, which also manages the project for NASA Headquarters. It is supported by NASA’s Science Mission Directorate. NASA’s Ames Research Center in California’s Silicon Valley and NASA’s Langley Research Center in Hampton, Virginia, provided significant flight performance analysis and technical assistance during Ingenuity’s development. AeroVironment, Qualcomm, and SolAero also provided design assistance and major vehicle components. Lockheed Space designed and manufactured the Mars Helicopter Delivery System. At NASA Headquarters, Dave Lavery is the program executive for the Ingenuity Mars helicopter.
Spacesuits and Space Biology Fill Day Aboard Space Station
Astronaut Suni Williams displays science hardware housing bacteria and yeast samples for a biomanufacturing experiment that may enable the production of food and medicine in space.
Spacesuits and space biology were the main focus on Tuesday as the Expedition 72 crew aboard the International Space Station gets ready for a spacewalk and keeps up critical research.
Flight Engineers Alexey Ovchinin and Ivan Vagner from Roscosmos worked throughout the day inside the Poisk module where they are getting ready for the year’s final spacewalk. Vagner first conducted leak checks on Poisk’s airlock hatch where the cosmonauts will exit into the vacuum of space at 10:10 a.m. EST on Thursday, Dec. 19. Next, the duo paired up activating and inspecting their individual Orlan spacesuits, installing batteries, and removing excess gasses and liquids from the suits they will wear during the spacewalk. The cosmonauts are expected to spend about six hours and 40 minutes removing external science experiments and relocating European robotic arm hardware.
Station Commander Suni Williams was back inside the Quest airlock on Tuesday checking out and replacing components on a U.S. spacesuit. Afterward, she joined NASA Flight Engineer Butch Wilmore for eye exams with the Ultrasound 2 device. The duo took turns scanning each other’s eyes as doctors on the ground monitored in real-time the condition of each astronaut’s cornea, lens, and optic nerve. Wilmore also transferred cargo between the Kibo and Columbus laboratory modules then installed air quality sensors inside Quest.
NASA Flight Engineer Don Pettit started his day reconfiguring and cleaning a pair of research incubators, one of which can generate artificial gravity, inside the Kibo lab module. He later joined Wilmore who scanned Pettit’s eyes with the Ultrasound 2 then assisted him with the cargo transfers between Kibo and Columbus. Pettit also reviewed procedures for an upcoming cancer and disease detection study.
Flight Engineer Nick Hague of NASA explored the potential of micro-algae as a way to remove carbon dioxide, produce oxygen, and grow food on spacecraft. Hague processed sample containers for installation inside the BioLab research facility to understand the effects of radiation and microgravity on the micro-algae and learn how to sustain crews on missions to the Moon, Mars, and beyond.
Hague, who is also the commander of the SpaceX Crew-9 mission, joined his crewmates Williams, Wilmore, and cosmonaut Aleksandr Gorbunov inside the Dragon crew spacecraft for a departure training session at the end of the day. Hague and Gorbunov launched together on Dragon to the orbital outpost on Sept. 28. They will return to Earth in the spring with Williams and Wilmore.
Space Meets Sound: NASA Lands in 2024 Spotify Wrapped
NASA’s podcasts let you experience the thrill of space exploration without ever leaving Earth.
Credit: NASA
NASA’s audio storytelling reached new frontiers in 2024, with Spotify Wrapped revealing the agency’s podcasts as a favorite among listeners worldwide. In celebration of the milestone, NASA astronaut Nick Hague spoke with Spotify about what space sounded like this year.
“Music is one of those things that connects us to the planet,” said Hague, in the video released on Spotify and NASA social accounts. “Music is a vital part of life up here. The soundtrack up here, it’s just going all the time. Everybody’s got their own flavor of music. Every Friday night the crew gets together, we turn on music and we stream things that we like. Whether they’re into pop or hard rock, it’s an international mix. When I think of space walks, I think of classical music, slow, methodical tunes, because that is the way that we conduct spacewalks. Slowly and methodically. Classical music captures the essence of, just floating in space.”
With listeners in more than 100 countries, NASA podcasts reached new audiences and inspired people around the world on Spotify this year. Other 2024 highlights included:
Ranked as a top choice for thousands of listeners seeking to learn about science and space.
Spent a combined 37 weeks in Spotify’s top charts for science podcasts.
The top streamed podcast was “NASA’s Curious Universe”, and the top streamed episode was “A Year in Mars Dune Alpha.”
“We’re thrilled to have our space-centric content featured in Spotify Wrapped 2024,” said Brittany Brown, director of digital communications, NASA Headquarters in Washington. “Our collaboration with Spotify is a testament to NASA’s commitment to producing innovative and engaging content. We’re excited to see how audiences continue to respond to this unique listening experience only NASA can provide.”
The agency’s podcasts cover a wide range of topics, including in-depth conversations with NASA astronauts, stories that take audiences on a tour of the galaxy, and Spanish-language content.
“Music, just like space, connects us all,” said Katie Konans, audio program lead, eMITS contract with NASA. “Our partnership with Spotify has allowed NASA to share the wonder and excitement of space with music and podcast lovers globally. This year, we’re thrilled to take this connection to new heights by bringing the Spotify Wrapped 2024 conversation beyond planet Earth.”
NASA released its collection of original podcasts on Spotify in 2023, furthering the agency’s mission to engage the Artemis Generation in the science, space exploration, and discovery.
In addition to Spotify, users may find NASA podcasts on Apple Podcasts, Google Podcasts, and Soundcloud.
NASA Astronauts O’Hara and Moghbeli Explore Earth Data
NASA/Joel Kowsky
On Dec. 4, 2024, NASA astronauts Loral O’Hara, left, and Jasmin Moghbeli spent a moment in part of the Earth Information Center, an immersive experience combining live NASA data sets with innovative data visualization and storytelling at NASA Headquarters in Washington.
O’Hara and Moghbeli spent six months in space as part of Expedition 70 aboard the International Space Station. On Nov. 1, 2023, they performed a spacewalk together that lasted 6 hours and 42 minutes.
![Examples of image reconstruction by SatVision-TOA. Left: MOD021KM v6.1 cropped image chip using MODIS bands [1, 3, 2]. Middle: The same images with randomly applied 8 × 8 mask patches, masking 60% of the original image. Right: The reconstructed images produced by the model, along with their respective Structural Similarity Index Measure (SSIM) scores. These examples illustrate the model’s ability to preserve structural detail and reconstruct heterogeneous features, such as cloud textures and land-cover transitions, with high fidelity.](https://www.nasa.gov/wp-content/uploads/2024/12/reconstruction-v4.png)
