How to clean up terabytes of topographic data

I’ve spent this summer playing with geophysical models at Lawrence Livermore National Lab.

The lab has great computing resources. They make it possible for me to do rigorous, statistically meaningful numbers of model runs, and to quickly explore new ideas.

They also let me generate data. Terabytes of data. Most of it is topographic data, showing the evolution of ripples and dunes (see example image at top of page).

Over the last month, I’ve worked with LLNL interns Aaron Robeson and Carlos Downie to design two algorithms that reduce this data into useful parameters. We extract the frequencies, shapes, and movement velocities of the features using the data reduction workflow below: