Hierarchical Data Format, commonly abbreviated HDF, HDF4, or HDF5 is a library and multi-object file format for the transfer of graphical and numerical data between computers. It was created by the NCSA, but is currently maintained by The HDF Group The freely available HDF distribution consists of the library, command-line utilities, test suite source, Java interface, and the Java-based HDF Viewer (HDFView)
HDF supports several different data models, including multidimensional arrays, raster images, and tables. Each defines a specific aggregate data type and provides an API for reading, writing, and organizing the data and metadata. New data models can be added by the HDF developers or users.
HDF is self-describing, allowing an application to interpret the structure and contents of a file without any outside information. One HDF file can hold a mixture of related objects which can be accessed as a group or as individual objects. Users can create their own grouping structures called "vgroups."
The next version of NetCDF, version 4, is based on HDF5.
Because it uses B-trees to index table objects, it works well for Time series data like stock market ticks or network monitoring data. The bulk of the data goes into straightforward arrays (the table objects) that can be accessed much more quickly than the rows of a SQL database. But you still have B-Tree access for non-array data. If you find yourself designing a Star schema to fit your data into SQL, then you might want to investigate HDF5 as a simpler, faster alternative storage mechanism.