HDF5 I/O

h5io.py

Convenience functions/classes for working with hdf5 files and saving the outputs from MCMC sampling.

The HDF5Backend class manages an hdf5 file which stores the setup of a model along with the outputs of sampling from that model.

HDF5Backend instances can be created from existing hdf5 files, by providing a MeasurementModel instance and dictionary of settings, or through the parse_yaml function, which will attempt to construct a model from a properly formatted yaml file.

class modds.h5io.HDF5Backend(filename=None, model=None, settings=None, overwrite=False)

Manages an hdf5 file and defines the structure of saved MCMC runs.

Calling the constructor with a filename will load the filename and make convenience functions available to operate on the file, including sampling.

Calling the constructor with a filename along with a model and a dictionary of settings will allow you to create a new hdf5 file in addition to loading it as an HDF5Backend instance.

The actual structure of the hdf5 file should not be necessary to know to interact with the instance, but should you wish to manually interact with it, is specified as follows:

/model : bytestring representation of MeasurementModel instance /version : str, version of colossus associated with the file /ndim : int, number of dimensions of parameter space /observables/R : radial locations of observed quantities /observables/q : observed quantity /observables/q_err : observational uncertainty

/state/chain : (nwalker, niter, ndim)-shaped array of posterior samples /state/niter : int, number of iterations /state/nacc : (nwalker,)-shaped array of acceptance counts for walkers /state/ppc : (nwalker, niter, ngrid)-shaped array of posterior predictive

checks

/state/lnp : (nwalker, niter)-shaped array of log posterior values /state/lnl : (nwalker, niter)-shaped array of log likelihood values

/settings/cosmo : Colossus cosmology (str, e.g., “planck15”) /settings/quantity : one of {‘density’, ‘enclosedMass’, ‘surfaceDensity’,

‘deltaSigma’}

/settings/r_grid : grid of radii for posterior predictive checks /settings/nwalkers : number of walkers in the ensemble

Parameters:
  • filename (str) –
  • model (MeasurementModel instance) –
  • settings (dict) – Keys should include cosmo, nwalkers, and r_grid.
append_state(pos, lnp=None, ppc=None, lnl=None, Mvir=None, cvir=None, rsp=None, gamma_min=None, acceptances=None)

Append sampling results to the existing datasets.

Parameters:
  • pos (array_like) – shape of (nwalkers, ndim) of parameter positions
  • lnp (array_like) – shape of (nwalkers,) of log posterior values
  • ppc (array_like) – shape of (nwalkers, ngrid) of posterior predictive checks
  • lnl (array_like) – shape of (nwalkers,) of log likelihood values
  • Mvir (array_like) – shape of (nwalkers,) of virial mass values
  • cvir (array_like) – shape of (nwalkers,) of concentration values
  • rsp (array_like) – shape of (nwalkers,) of splashback radius values
  • gamma_min (array_like) – shape of (nwalkers,) of minimum gamma values
  • acceptances (array_like) – shape of (nwalkers,) of acceptance counts (1 if accepted, 0 if not)
dump(filename, model, settings, overwrite=False)

Stores the model/settings and initializes arrays.

load(filename)

Sets the filename, reads the model, and sets the cosmology.

sample(niter, pool=None)

Sample from the model for niter iterations, optionally with a user provided pool of workers.

modds.h5io.parse_yaml(filename, overwrite=False)

Read in a yaml file and construct an hdf5 output file.