Check out the Icechunk example usage in our docs for IFS ENS, GFS, and HRRR forecasts to get started or browse these datasets on the Registry of Open Data on AWS.

Why Icechunk?

• Consistent reads every time. Updating a standard Zarr while users are reading it requires extreme care. Icechunk handles atomic updates correctly, eliminating rare race conditions that can occur during standard Zarr updates.
• Reliability and throughput. Icechunk is developing features to offer long-term stable dataset URLs. This provides direct access to underlying chunk data for high-throughput reads while allowing us to provide a permanent URL. Note: This feature will be part of the upcoming Icechunk 2.0, so when released dynamical.org will require a change to the access URL.
• Performance with default Python tooling. While the storage formats are similar, the Icechunk client is currently faster than default libraries used for traditional Zarrs in python. You get maximum efficiency (comparable to specialized libraries like Obstore) by default.

When Icechunk 2 is released we will make two small breaking changes (and notify you about them here):

• URLs will be updated to stable, long-term URLs
• Icechunk client version 2 will be required

Special thanks to the AWS Open Data Sponsorship Program for supporting the storage and distribution of these datasets.