Experimentation, Validation, and Uncertainty Analysis for Engineers

Hugh W. Coleman and W. Glenn Steele
Third Edition, John Wiley & Sons, New York, (2009) 317 pp Hardbound, $140.00
ISBN 978-0-470-16888-2

This book reveals to practitioner engineers and technicians involved in experimentation, modeling, and simulation, an approach based on uncertainty analysis concepts for the validation of simulation processes. This new edition incorporates new material: two methods for uncertainty propagation evaluation (Taylor Series Method, TSM, and the Monte Carlo Method, MCM) and a new chapter on “Validation of Simulations”, where the contribution of the authors to the new ASME “Standard for Verification and Validation in Computational Fluid Dynamics and Heat Transfer” V& V20-2009 is presented. The topic of uncertainty evaluation follows the ISO Guide: “Evaluation of Measurement Data—Supplement 1 to the “Guide to the expression of the uncertainty in Measurement”—Propagation of Distributions Using a Monte Carlo Method”.

The first chapter traces the leading line of the book, by detailing the importance of the uncertainty evaluation from the early stages of experiment planning through the validation of the simulations. The important definitions, the basic concepts of random and systematic errors and of measurement uncertainty are introduced, and the goal of the validation of simulations by quantitative evaluations is presented. Chapter 2 introduces the statistical distributions most encountered in uncertainty analysis, the estimators for their parameters, the confidence intervals for the mean value, and the Chauvenet’s criterion for rejection of outliers from a sample. Finally, the uncertainty of one measured variable is evaluated using both TSM and MCM. Additional exercises are proposed at the end of this and every chapter.

Chapter 3 continues the topic of the evaluation of the measurement uncertainty for the multivariate parameters using the TSM and MCM. For the TSM, an extrapolation of the relation of composing partial uncertainties for the limits of the confidence intervals is deduced for particular cases. The authors are unclear here because this can confuse some readers, as the relation may be seen as a method of composing confidence intervals deriving from different distributions—which has no mathematical justification. The advantage of the MCM method in evaluation of the uncertainty intervals when the uncertainty values are large and the resulting distribution is skewed is discussed through examples, along with the method of determination of the confidence intervals.

Chapter 4 deals with application of the uncertainty analysis in experiment planning and validation. Through comparative, detailed examples, the uncertainty analysis is presented as a decision element in the method selection and for adjustments of experiments in the planning phase. The need of considering the domain of values of the variables and their evolving uncertainty values is discussed. The authors give advice to avoid pitfalls related to interpretation of specifications. In comparing the experimental techniques, the use of the uncertainty magnification factors and the uncertainty percentage contribution is recommended for decision making process.

The fifth chapter is the most extended of the book. Initially it lists the uses of the detailed uncertainty analysis in the more advanced phases experiment design, construction, debugging, execution, data analysis and reporting the results. The chapter then focuses on detailed uncertainty analysis, i.e., determination of the uncertainty components from random or systematic sources. Type A and type B methods are used to evaluate the random components. Practical knowledge is shared about determining the uncertainty from systematic sources. The concept of time frame for experiments is discussed in relation with the long term evolution of the measurand, and the test conditions. The correlation of uncertainty components is separately treated, the measurement methods that use it for the purpose of diminishing the measurement uncertainty are analyzed and the restrictions explained. Multiple real-life experiment examples are used to illustrate each topic, and the pitfalls of poor estimations are shown for each step, with a detailed analysis of the sources of all apparent discrepancies. In the debugging stage of the experiment, the uncertainty analysis based on the results can reveal unforeseen influences and help adjust the experiment. Even the choice of the test points and the test sequence is not left to the hazard, but documented recommendations are given, along with running balance checks for the overall adjustment and validation of an experiment.

Chapter 6 presents the models verification and validation (V&V) methodology from the ASME V&V202009 standard, procedures which can be applied in all areas of computational engineering and science. All steps of V&V are listed and briefly addressed. The topic of quantification of the degree of accuracy of simulation at a specified validation point through comparison with experimental data is further developed. Several cases are analyzed, in which the validation uncertainty is evaluated while the validation variable is directly measured, determined by calculation based on measurement of other variables (indirect measurements) or from other models. TSM and MCM approaches are exemplified. The interpretation of results points out the restrictions and the additional assumptions needed to estimate the confidence interval of the simulation result.

The last chapter of the book presents methods of data treatment for the purpose of reporting the results in a most effective manner. The regression analysis detailed here allows determining the mathematical expression representing the experiment result, which it is recommended as a compact way of presenting the results. Evaluating the uncertainty associated to such an expression is the main topic of this chapter. Additional topics are provided in the six Appendices: A- tables for popular distributions, B- the TSM and MC detailed presentation, C- calculations meant to justify the large-sample uncertainty propagation relation (3.22), D- the coverage intervals for MCM, E- the asymmetric systematic uncertainties and their propagation in indirect measurements, and F- the dynamic characteristics of measuring instruments.

The book stands out as very rich in practical information, presented through comprehensive examples, general enough for allowing specialists in any area to find responses to his or her particular questions. It is an invaluable source of information for the specialists involved in designing, verification and validation of simulations, providing a didactic, goal oriented systematic approach, for these activities.

Mariana Buzhuga
Scantek, Inc.
Columbia, MD