September update

Ahoy weatherheads! It’s been a busy few months for us as we shift gears from dreaming to building.

Catalog updates

• Our dataset for NOAA GFS analysis has getting some awesome usage! Thanks to everyone who has tested it out.
• We are prepping to launch NOAA GEFS forecasts as the second entry to dynamical.org/catalog. It will include a large historical forecast record and update in real-time. Dealing with GEFS in its standard format has, in the past, broken my spirit; and I suppose it’s part of why we are so passionate about making weather data easier to work with! So… thank you GEFS? Excited for this one. GEFS ensemble average downward long-wave radiation flux.
• We plan to fast-follow GEFS with GFS (forecasts vs our current analysis) and HRRR from NOAA as well as GDPS and HRDPS from ECCC (Environment and Climate Change Canada). As part of these releases we will be announcing our first catalog sponsor!

GFS analysis feedback

A hearty thank you to everyone who filled out our GFS survey. Common themes included the value of live updating and forecast data, and a clear preference for reading data with python + xarray. That aligned with our expectations (wonderful) and is reflected in our roadmap above. We did see more desire for long history (re)analysis data than we had anticipated -- we'll be moving up a true reanalysis dataset in priority thanks to this feedback. Reply to this email if there's a specific reanalysis you'd like!

Assembling our Steering Committee

We are officially launching our steering committee, which will be comprised of industry users and experts in the technical and scientific domain. The committee will meet quarterly to evaluate and influence the catalog and open-source roadmap and advise potential collaborations and partnerships.

If you’re interested in joining or nominating someone else, please respond to this email or email [email protected]!

What we’ve been reading

My souls, how the wind did scream along! And every second or two there’d come a glare that lit up the white-caps for a half a mile around, and you’d see the islands looking dusty through the rain, and the trees thrashing around in the wind; then comes a h-whack!—bum! bum! bumble-umble-um-bum-bum-bum-bum—and the thunder would go rumbling and grumbling away, and quit—and then rip comes another flash and another sockdolager.

I’m reading Adventures of Huckleberry Finn, which has some stellar meteorological descriptions. Should we petition NOAA to make their detailed forecasts use “h-wack!-bum! bum!” and “sockdolager?” I think yes.

We enjoyed Dr. Jack Kelly’s post on the vision this community shares, where he outlined a complimentary route to solving major pain points. Thanks for the dynamical.org shoutouts, Jack! Definitely check it out here: https://openclimatefix.org/post/lazy-loading-making-it-easier-to-access-vast-datasets-of-weather-satellite-data

We were thrilled to see our pals at Brightband announce themselves to the world. The field of AI meteorological forecasts is progressing at quite the clip - three cheers for more open source models! We also enjoyed Zeus AI's cool work synthesizing multiple, sparse data sources into a single gap filled analysis state.

Enjoy the weather!

Let there be zarr: GFS hourly analysis is live

Last week, we announced dynamical.org to the world. The response was unexpected*: in just seven days, over a hundred nerds registered themselves as testers with many more following along for the ride.

Apparently, we are not alone in our frustration with yesterday's weather data. Together we can create a future of well-organized data about our most beloved chaotic system!

Today, we are excited to announce the first entry into dynamical’s catalog: an hourly GFS analysis from 2015-01-15 to present.

Testers! Fire up your rigs - in the catalog entry above, we've included documentation and an example notebook. If you run into any issues, email us at [email protected]. As our very first release, we’re expecting there will be lots of additions and refinement based on community input.

Our pals at Source Cooperative are generously providing storage for this first catalog entry. Their team is wonderful and their mission is badass - check them and the broader Radiant Earth organization out.

Enjoy the weather^!

MM

* or expected, depending on how long you've been working with yesterday's weather data

^ courtesy of Alex M.

Introducing dynamical.org

Introducing dynamical!

Thank you for being the first to follow our journey making weather and climate data more accessible. We are humbled by the response. Our mission clearly strikes a chord - weather data today is a pain to work with.

Over one hundred people (!?) signed up to test the first versions of the data we endeavor to release. And that’s without having spilled the beans on the best parts 😉

So what’s the plan? First, check out Alden’s fantastic introduction to dynamical.org (video link). Second, read about how we are designing dynamical to be a long-term steward and rock-solid foundation on which to build.

We are aiming to release our first public test next week in collaboration with Source Cooperative: a GFS analysis with a long historical record. If you are interested in testing this out and haven’t already told us, reply to this email. We plan to fast-follow with documentation and a data catalog with our roadmap.

Enjoy the weather!

MM + AKS

P.S. Please forward this email to folks working with weather data and consider amplifying our announcement on linkedin / fedi!