Optimal allocation of testing resources for statistical simulations
Estimation of statistical moments from simulation, i.e., mean and standard deviation of the output (or response), may involve large uncertainty caused by the variability (or lack of confidence) in the input random variables. Investigations have been done on the quantification of the uncertainty in the output model, but little effort has been expended towards methods to reduce the variability of the output moments.
Theoretically, it is known that allocating resources to obtain more experimental data of the input variables to better characterize their probability density functions (PDFs), can reduce the variance of the output moments. The methodology proposed and executed here, used an optimization method combined with a nested-loop arrangement. The internal loop (or inner-loop) was used to obtain the mean and the standard deviation of the output; the external loop (or outer-loop) was used to generate the probability distribution of the output moments, thus quantifying the uncertainty of the output moments based on the variation of the input random variables. An optimization model was implemented to determine the optimal number of experiments required to minimize the variance of the output moments given a constraint. The constraint could be any available resource, such as cost of material, money available, time, etc. The optimization proposed was constrained by the money available and used a penalty method that kept the search within the feasible design space. Since the scenario included integer variables and involved a high computational cost, the optimization used a modified particle swarm optimization (PSO) method since it has a small number of user parameters and it is easily implemented.
A method to generate the output moments based on the moments of the input variables was implemented. The method used the multivariate t-distribution and the Wishart distribution to generate realizations of the population mean and population covariance of the input variables, respectively. This method is sufficient to handle independent and correlated random variables. Using a probabilistic code written in MATLAB and Monte Carlo sampling, the distribution of the output moments was generated. The statistics of these distributions were then processed by the optimization code to determine the optimal additional experiments. Numerical examples included illustrative problems with linear and nonlinear output functions; however, the methodology was not limited to such models. The results showed that additional experimental data depend upon several factors, such as the number of initial data points, the importance of the input variables, and the cost of each additional experiment.