Estimation of mixture-distributions using the Direct Look-up method

Abstract

Mixture distributions and models are useful methods of describing data that cannot be estimated with a single probability distribution. Estimating mixture models based on samples from unknown distributions is a highly iterative process, prone to issues like non-convergence, high runtime or local optima. It is therefore an interesting area to develop a method of parametric estimation without some of these issues using a direct approach without iteration. Isaeva et al. [2011a,b] developed a method to avoid these problems regarding estimation of non-linear mathematical functions called the Direct Look-Up (DLU) method. To implement this method for estimating mixture models is the main goal of this master thesis. The idea is to compute a wide range of possible solutions to the parametric estimation problem of mixture distributions prior to observing data. Within a standardized range, a set of possible combinations of two normal distributions are chosen according to an experimental design, and the mixture distributions are computed. Using Principal Component Analysis (PCA) the generated library of distribution curves is reduced to a number of basal curves, storing vast amount of information into a few vectors and values. Finding the best solutions to a new observation is then a matter of linear projection onto the curve library to identify the nearest curves. The method is to be compared to traditional estimation methods for mixture models such as Expectation Maximization (EM) and Markov Chain Monte Carlo (MCMC). Results are evaluated and compared using log likelihood values. The general performance of the DLU for mixture models is acceptable. Many observations can be estimated quickly, with a predictable time frame. Results in comparison with other methods show that it is not as optimized as the EM for this problem, while it works better than MCMC. Differences in prediction error and log likelihood evaluations of the estimated parameters are discussed and analysed. Possibilities for taking advantage of the generalizable nature of the DLU method is discussed. Using the DLU for mixture models is a method that may at some point work well, even though the method needs some tweaks to make it better. Generalising the method for a larger collection of mixture models is discussed, and ideas for improvement are presented.