Material Detail

A comparison of hypothesis testing methods for ODE models of biochemical systems

A comparison of hypothesis testing methods for ODE models of biochemical systems

This video was recorded at Workshop on Probabilistic Modelling of Networks and Pathways, Sheffield 2007. In this talk we present a comparison of different methods of testing alternative hypotheses expressed using ODE models of biochemical systems. We investigated applicability, limitations and stability of a range of hypotheses testing methods including maximum likelihood based information criteria, local deterministic approximations around maximum a posteriori estimates (Laplace approximations) for computing marginal likelihoods, importance sampling based marginal likelihood estimators, and a path sampling estimator built upon the principles of thermodynamic integration. We demonstrate that in the cases where models are linear in the parameter space, Laplace approximations provide a fast and stable estimate of the marginal likelihoods required for computing Bayes factors. This estimate, however, fails when the models have non-trivial parameter posteriors. We reject common importance sampling estimators as they produce very unstable estimates in practical cases (relative errors of the estimates vary from 40% to 600% depending on the particular example used). We demonstrate that the annealed importance sampling estimator of the marginal likelihoods and path sampling methods produce very good estimates even in non-trivial cases (relative error within 1%-8%). Maximum likelihood information criteria often produce the correct ordering of the hypotheses. These methods, however, do not produce a quantitative measure of model preference (odds) and sometimes even fail, preferring a more complex model over the true one, and there is no general method to detect such a failure. The study is performed over realistically sized ODE models of biochemical systems using simulated data sets.

Quality

  • User Rating
  • Comments
  • Learning Exercises
  • Bookmark Collections
  • Course ePortfolios
  • Accessibility Info

More about this material

Browse...

Disciplines with similar materials as A comparison of hypothesis testing methods for ODE models of biochemical systems

Comments

Log in to participate in the discussions or sign up if you are not already a MERLOT member.