Approximations of the solutions of a stochastic differential equation using Dirichlet process mixtures and Gaussian mixtures
Abstract
Stochastic differential equations arise in a variety of contexts. There are many techniques for approximation of the solutions of these equations that include statistical methods, numerical methods, and other approximations. This article implements a parametric and a nonparametric method to approximate the probability density of the solutions of stochastic differential equation from observations of a discrete dynamic system. To achieve these objectives, mixtures of Dirichlet process and Gaussian mixtures are considered. The methodology uses computational techniques based on Gaussian mixtures filters, nonparametric particle filters and Gaussian particle filters to establish the relationship between the theoretical processes of unobserved and observed states. The approximations obtained by this proposal are attractive because the hierarchical structures used for modeling are flexible, easy to interpret and computationally feasible. The methodology is illustrated by means of two problems: the synthetic Lorenz model and a real model of rainfall. Experiments show that the proposed filters provide satisfactory results, demonstrating that the algorithms work well in the context of processes with nonlinear dynamics which require the joint estimation of states and parameters. The estimated measure of goodness of fit validates the estimation quality of the algorithms.References
Y. Ait-Sahalia, Maximum likelihood estimation of discretely sampled diffusions: a closed form approximation approach, Econometrica, vol. 70, no. 1, pp. 223–262, 2002.
B. Anderson, and J. Moore, Optimal Filtering, Prentice-Hall Information and system sciences series, Englewood Cliffs, New Jersey, 1979.
D. Alspach, and H. Sorenson, Nonlinear Bayesian Estimation Using Gaussian Sum Approximations, IEEE transactions on automatic control, vol. 17, no 4, pp. 439–448, 1972.
E. Allen, Modeling with Itbo Stochastic Differential Equations, Springer, Texas Tech University, USA, 2007.
C. Archambeau, D. Cornford, M. Opper, J. Shawe-Taylor, Gaussian Process Approximations of Stochastic Differential Equations, JMLR. Workshop and Conference Proceedings, vol. 1, pp. 1–16, 2007.
S. Arulampalam, S. Maskell, N. Gordon and T. Clapp, A Tutorial on Particle Filters for On-line Non-linear/Non-Gaussian Bayesian Tracking, IEEE Transactions on signal processing, vol. 50, no. 2, pp. 174–188, 2002.
D. Blackwell, and J. MacQueen, Ferguson distributions via Polya urn schemes, The annals of statistics, pp. 353–355, 1973.
O. Calin, An Introduction to Stochastic Calculus with Applications to Finance, Ann Arbor, 2012.
C. Chui, G. Chen, Kalman Filtering with Real-Time Applications, Springer Science & Business Media, 2008.
A. Doucet, J. De Freitas and N. Gordon, An introduction to sequential Monte Carlo methods, Sequential Monte Carlo methods in practice. Springer New York, pp. 3–14, 2001.
A. Doucet, S. Godsill and C. Andrieu, On sequential Monte Carlo sampling methods for Bayesian filtering, Statistics and computing, vol. 10, no. 3, pp. 197–208, 2000.
Dan Crisan and Arnaud Doucet, A Survey of Convergence Results on Particle Filtering Methods for Practitioners, IEEE Transactions on signal processing, vol. 50, no. 3, pp. 736–746, 2002.
O. Elerian, S. Chib, N. Shephard, Likelihood inference for discretely observed non-linear diffusions, Econometrica, vol. 69, no. 4, pp. 959–993, 2001.
L. Evans, An Introduction to Stochastic Differential Equations, American Mathematical Soc., 2012.
M. Escobar, and H. West, Bayesian density estimation and inference using mixtures, Journal of the American Statistical Association,vol. 90, no. 430, pp. 577–588, 1994.
T. Ferguson, A Bayesian analysis of some nonparametric problems, The annals of statistics, vol. 1, pp. 209–230, 1973.
W. Fong, S. Godsill, A. Doucet, M.West, Monte Carlo smoothing with application to audio signal enhancement, IEEE Transactions on Signal Processing, vol. 50, no. 2, pp. 438– 449, 2002.
A. Gelfand, A. Kottas, and S. MacEachern, Bayesian nonparametric spatial modeling with Dirichlet process mixing, Journal of the American Statistical Association, vol. 100, no. 471, pp. 1021–1035, 2005.
S. Godsill, A. Doucet and M. West, Monte Carlo smoothing for nonlinear Time series, Journal of the American statistical Association, 2012.
N. Gordon, D. Salmond and A. Smith, Novel approach to nonlinear/non-Gaussian Bayesian state estimation, IEE Proceedings F-Radar and Signal Processing. IET, pp. 107-113, 1993.
A. Guidoum, and K. Boukhetala, Quasi Maximum Likelihood Estimation for One Dimensional Diffusion Processes, In submission (The R Journal), 2014.
A. Hurn, K. Lindsay and V. Martin, On the efficacy of simulated maximum likelihood for estimating the parameters of stochastic differential equations, Journal of Time Series Analysis, vol. 24, no. 1, pp. 45–63, 2003.
A. Hurn, K. Lindsay, and A. McClelland, A quasi-maximum likelihood method for estimating the parameters of multivariate diffusions, Journal of Econometric, vol. 172, no. 1, pp. 106–126, 2013.
K. Ito, and K. Xiong, Gaussian Filters for Nonlinear Filtering Problems, IEEE Transactions on Automatic Control, vol. 45, no. 5, pp. 910–927, 2000.
A. Jazwinski, Stochastic Processes and Filtering Theory, Academic Press, Inc. New York, USA, 1970.
B. Jensen, and R. Poulsen, Transition Densities of Diffusion Processes: Numerical Comparison of Approximation Techniques, Journal of Derivatives, vol. 9, no. 4, pp. 18–32, 2002.
C. S. Jones, A simple Bayesian method for the analysis of diffusion processes, Available at SSRN 111488, 1998.
S. Julier and J. Uhlmann, A new extension of the Kalman filter to nonlinear systems, AeroSense’97. International Society for Optics and Photonics, pp. 182–193, 1997.
S. Julier and J. Uhlmann, Unscented Filtering and Nonlinear Estimation, Proceedings of the IEEE, vol. 92, no. 3, pp. 401-422, 2004.
S. Julier, J. Uhlmann, and H. Durrant-White, A New Method for the Nonlinear Transformation of Means and Covariances in Filters and Estimators, IEEE Transactions on Automatic Control, vol. 45, pp. 477–478, 2000.
E. Kalnay, Atmospheric modeling, data assimilation and predictability, Cambridge University Press, 2003.
G. Kitagawa, Monte Carlo Filter and Smoother for Nonlinear Non Gaussian State Models, Journal of computational and graphical statistics, vol. 5, no. 1, pp. 1–25, 1996.
K. Kokkala, A. Solin, S. Sarkka, Sigma-Point Filtering and Smoothing Based Parameter Estimation in Nonlinear Dynamic Systems, arXiv preprint arXiv:1504.06173, 2015.
J. Kotecha, and P. Djuric, Gaussian sum particle filtering, IEEE Transactions on signal processing, vol. 51, no. 10, pp. 2602–2612, 2003.
P. Kloeden, and E. Platen, Numerical Solution of Stochastic Differential Equations, Springer Verlag, 1995.
P. Kloeden and E. Platen, Numerical Solution of Stochastic Differential Equations, Springer-Verlag, 1992.
J. Liu and R. Chen, Sequential Monte Carlo Methods for Dynamical Systems, Journal of the American statistical association, vol. 93, no. 443, pp. 1032–1044, 1998.
J. Liu and M. West, Combine Parameter and State Estimation in Simulation Based Filtering, Sequential Monte Carlo methods in practice. Springer New York, pp. 197–223, 2001.
M. Lin, J. Zhang, Q. Cheng, and R. Chen, Independent Particle Filters, Journal of the American Statistical Association, vol. 100, no. 472, pp. 1412–1421, 2005.
E. Lorenz, Deterministic Nonperiodic Flow , Journal of the atmospheric sciences, vol. 20, no. 2, pp. 130–141, 1963.
M. Luo, State estimation in high dimensional systems: the method of the ensemble unscented Kalman filter, Inference and Estimation in Probabilistic Time-Series Models, 2008.
S. MacEachem, and P. Muller, Estimating Mixture of Dirichlet Process Models, Journal of Computational and Graphical Statistics, vol. 7, no. 2 pp. 223-238, 1998.
P. Maybeck, Stochastic models, estimation, and control, Academic Press, vol. 3, 1982.
I. Mbalawata, S. Sarkka, H. Haario, Parameter estimation in stochastic differential equations with Markov chain Monte Carlo and nonlinear Kalman filtering, Computational Statistics, vol. 28, no. 3, pp. 1195-1223, 2013.
P. Muller, F. Quintana, Nonparametric Bayesian data analysis, Statistical science, pp. 95–110, 2004.
R. Neal, Markov chain sampling methods for Dirichlet process mixture models, Journal of Computational and Graphical Statistics, vol. 9, no. 2, pp. 249–265, 2000.
J. Neddermeyer, Sequential Monte Carlo Methods for General State-Space Models, Tesis Doctoral. Diplomarbeit (Master thesis), University of Heidelberg, 2007.
J. Neddermeyer, Computationally Efficient Nonparametric Importance Sampling, Journal of the American Statistical Association,vol. 104, no. 486 788–802, 2009.
J. Nicolau, New Technique for Simulating the Likelihood of Stochastic Differential Equations, The Econometrics Journal, vol. 5, no.1, pp. 91–103, 2002.
B. ∅ksendal, Stochastic Differential Equations, An Introduction with Applications, Springer, 5th edition, 2000.
B. ∅ksendal, Stochastic Differential Equations: An Introduction with Applications, Springer, New York, 6th edition, 2003.
B. ∅ksendal, An Introduction to Malliavin Calculus with Application to Economics, Lecture Notes, Norwegian School of Economics and Business Administration, 1997.
A. Pedersen, A new approach to maximum likelihood estimation for stochastic differential equations based on discrete observations, Scandinavian journal of statistics, vol. 22, no. 1 pp. 55–71, 1995.
J. Pitman, M. Yor The two-parameter Poisson Dirichlet distribution derived from a stable sub-ordinator, The Annals of Probability, vol. 25, no. 2, pp. 855–900, 1997.
M. Pitt and N. Shephard, Filtering via simulation: Auxiliary particle filters, Journal of the American statistical association, vol. 94, no. 446, pp. 590-599, 1999.
A. Rabaoui, N. Viandier, J. Marais, E. Duos, P. Vanheeghe, Dirichlet Process Mixtures for Density Estimation in Dynamic Nonlinear Modeling: Application to GPS Positioning in Urban Canyons, IEEE Transactions on Signal Processing, vol. 60, no. 4, pp. 1638–1655,
A. Ruttor, P. Batz, M. Opper, Approximate Gaussian process inference for the drift function in stochastic differential equations, Advances in Neural Information Processing Systems, pp. 2040–2048, 2013.
L. S´anchez, S. Infante, V. Griffin and D. Rey, Spatiotemporal dynamic model and parallelized ensemble Kalman Filter for precipitation data, Brazilian Journal of Probability and Statistics (in press).
L. S´anchez, S. Infante, J. Marcano, V. Griffin Polinomial Chaos based on the parallelized ensamble Kalman filter to estimate precipitation states, Statistics, Optimization and Information Computing, vol. 3, no. 1, pp. 79–95, 2015.
S. Sarkka, Bayesian filtering and smoothing, Cambridge University Press, 2013.
H. Sorenson, D. Alspach, Recursive Bayesian Estimation Using Gaussian Sums, Automatica, vol. 7, no. 4, pp. 465–479, 1971.
J. Sethuraman, A constructive definition of Dirichlet priors, Statistica Sinica, pp. 639–650, 1994.
Z. Schuss, Nonlinear Filtering and Optimal Phase Tracking Applied, Springer Science & Business Media, vol. 180, 2011.
M. Stefano, Simulation and inference for stochastic differential equations: with R examples, Springer Science & Business Media, 2009.
M. Stefano, sde: Simulation and Inference for Stochastic Differential Equations, R package version, vol. 2, no. 10, 2009.
O. Stramer and M. Bognar, Bayesian inference for irreducible diffusion processes using the pseudo-marginal approach, Bayesian Analysis, vol. 6, no. 2, pp. 231–258, 2011.
O. Stramer, M. Bognar and P. Schneider, Bayesian Inference of Discretely Sampled Markov Processes with Closed-Form Likelihood Expansions, The Journal of Financial Econometrics, vol. 8, no. 4, pp. 450–480, 2010.
H. Tanizaki, Nonlinear Filters Estimation and Applications, Springer Science & Business Media, 2013.
G. Terejanu, T. Singh, P. Scott A Novel Gaussian Sum Filter Method for Accurate Solution to Nonlinear Filtering Problem, Information Fusion, 2008 11th International Conference on. IEEE, pp. 1–8, 2008.
C. Yau, O. Papaspiliopoulos, G. Roberts, C. Holmes, Bayesian non-parametric hidden Markov models with applications in genomics, Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 73, no. 1, pp. 37–57, 2011.
M. West, Mixtures models, Monte Carlo, Bayesian updating and dynamic models, Computing Science and Statistics, pp. 325–335, 1993.
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
