DOE Overview

Much of our knowledge about products and processes in the engineering and scientific disciplines is derived from experimentation. An experiment is a series of tests conducted in a systematic manner to increase the understanding of an existing process or to explore a new product or process. Design of experiments, or DOE, then is the tool to develop an experimentation strategy that maximizes learning using a minimum of resources. Design of experiments is widely used in many fields with broad application across all the natural and social sciences. It is extensively used by engineers and scientists involved in the improvement of manufacturing processes to maximize yield and decrease variability. Often engineers also work on products or processes where no scientific theory or principles are directly applicable. Experimental design techniques become extremely important in such studies to develop new products and processes in a cost effective and confident manner.

Why DOE?

With modern technological advances, products and processes are becoming exceedingly complicated. As the cost of experimentation rises rapidly it is becoming impossible for the analyst, who is already constrained by resources and time, to investigate the numerous factors that affect these complex processes using trial and error methods. Instead, a technique is needed that identifies the "vital few" factors in the most efficient manner and then directs the process to its best setting to meet the ever increasing demand for improved quality and increased productivity. The DOE techniques provide powerful and efficient methods to achieve these objectives.

Designed experiments are much more efficient than one-factor-at-a-time experiments which involve changing a single factor at a time to study the effect of the factor on the product or process. While the one-factor-at-a-time experiments are easy to understand, they do not allow the investigation of how a factor affects a product or process in the presence of other factors. The relationship where the effect that a factor has on the product or process is altered, due to the presence of one or more other factors, is called an interaction. Many times the interaction effects are more important than the effect of individual factors. This is because the application environment of the product or process includes the presence of many of the factors together instead of isolated occurrences of one of the factors at different times. Consider an example of interaction between two factors in a chemical process where increasing the temperature alone increases the yield slightly while increasing the pressure alone has no effect on the yield. However, in the presence of both higher temperature and higher pressure the yield increases rapidly. Thus, an interaction is said to exist between the two factors affecting the chemical reaction.

The DOE methodology ensures that all factors and their interactions are systematically investigated. Therefore, information obtained from a DOE analysis is much more reliable and complete than results from one-factor-at-a-time experiments that ignore interactions and may lead to incorrect conclusions.

Introduction to DOE Principles

The design and analysis of experiments revolves around the understanding of the effects of different variables on other variable(s). In terms of mathematical jargon, the objective is to establish a cause-and-effect relationship between a number of independent variables and a dependent variable of interest. The dependent variable, in the context of DOE, is called the response, and the independent variables are called factors. Experiments are run at different factor values, called levels. Each run of an experiment involves a combination of the levels of the investigated factors. Each of the combinations are referred to as a treatment. [Note] When the same number of response observations are taken for each of the treatments of an experiment, the design of the experiment is said to be balanced. Repeated observations at a given treatment are called replicates.

The number of treatments of an experiment is determined on the basis of the number of factor levels being investigated in the experiment. For example, if an experiment involving two factors is to be performed, with the first factor having x levels and the second having z levels, then x x z treatment combinations can possibly be run, and the experiment is an x x z factorial design. If all x x z combinations are run, then the experiment is a full factorial. If only some of the x x z treatment combinations are run, then the experiment is a fractional factorial. In full factorial experiments, all the factors and their interactions can be investigated, whereas in fractional factorial experiments, certain or all interactions are not considered because not all treatment combinations are run.

It can be seen that the size of an experiment escalates rapidly as the number of factors, or the number of the levels of the factors, increases. For example, if 2 factors at 3 levels each are to be used, 9 (3x3=9) different treatments are required for a full factorial experiment. If a third factor with 3 levels is added, 27 (3x3x3=27) treatments are required and 81 (3x3x3x3=81) treatments are required if a fourth factor with three levels is added. If only two levels are used for each factor, then in the four factor case, 16 (2x2x2x2=16) treatments are required. For this reason, many experiments are restricted to two levels, and these designs are given a special treatment in this reference. Fractional factorial experiments further reduce the number of treatments to be executed in an experiment.

DOE Types

For Comparision: One Factor Designs

These are the designs where only one factor is under investigation, and the objective is to determine whether the response is significantly different at different factor levels. The factor can be qualitative or quantitative. In the case of qualitative factors (e.g. different suppliers, different materials, etc.), no extrapolations (i.e. predictions) can be performed outside the tested levels, and only the effect of the factor on the response can be determined. On the other hand, data from tests where the factor is quantitative (such as temperature, voltage, load, etc.) can be used for both effect investigation and prediction, provided that sufficient data is available. In DOE++, predictions for one factor designs can be performed by transferring the data to the Multiple Linear Regression tool available in the Other Tools in the Project Tree.

For Factor Screening: Factorial Designs

In factorial designs, multiple factors are investigated simultaneously during the test. As in one factor designs, qualitative and/or quantitative factors can be considered. The objective of these designs is to identify the factors that have a significant effect on the response, as well as investigate the effect of interactions (depending on the experiment design used). Predictions can also be performed when quantitative factors are present, but care must be taken since certain designs are very limited by the choice of the predictive model. For example, in two level designs only a linear relationship can be used between the response and the factors, which may not be realistic.

• General Full Factorial Designs

In general full factorial designs, each factor can have different number of levels, and the factors can be quantitative or qualitative or both. In this version of DOE++, the software always converts all the factors into a qualitative space, therefore no predictions can be performed for this design.

• Two Level Full Factorial Designs

These are factorial designs where the number of the levels for each of the factors is restricted to two. Restricting the levels to two and running a full factorial experiment reduces the number of treatments (compared to a general full factorial experiment), and allows for the investigation of all the factors and all their interactions. If all factors are quantitative, then the data from such experiments can be used for predictive purposes, provided a linear model is appropriate for modeling the response (since only two levels are used, curvature cannot be modeled).

• Two Level Fractional Factorial Design

This is a special category of two level designs, where not all factor level combinations are considered, and the experimenter can choose which combinations are to be excluded. Based on the excluded combinations, certain interactions cannot be determined.

• Plackett-Burman Design

This is a special category of two level fractional factorial designs, proposed by R. L. Plackett and J. P. Burman, where only a few specifically chosen runs are performed to investigate just the main effects (i.e. no interactions).

• Taguchi's Orthogonal Arrays

Taguchi's orthogonal arrays are highly fractional designs, used to estimate main effects using only a few experimental runs. These designs are not only applicable to two level factorial experiments, but also can investigate main effects when factors have more than two levels. Designs are also available to investigate main effects for certain mixed level experiments where the factors included do not have the same number of levels.

For Optimization: Response Surface Method Designs

These are special designs that are used to determine the settings of the factors to achieve an optimum value of the response.

For Product or Process Robustness: Robust Parameter Designs

The famous Taguchi Robust Design is one of the design type for robust parameter design. This design is used to design a product or process to be insensitive to noise factors.

For Life Tests: Reliability DOE

This is a special category of DOE where traditional designs, such as the two level designs, are combined with reliability methods to investigate effects of different factors on the life of a unit. In Reliability DOE, the response is a life metric (e.g. age, miles, cycles, etc.), and the data may contain censored observations (suspensions, interval data). One factor, and two level factorial designs (full, fractional, and Plackett-Burman) are available in DOE++ to conduct a Reliability DOE analysis.

Stages of DOE

Designed experiments are usually carried out in five stages -- planning, screening, optimization, robustness testing, and verification.

Planning

It is important to carefully plan for the course of experimentation before embarking upon the process of testing and data collection. A thorough and precise objective identifying the need to conduct the investigation, assessment of time and resources available to achieve the objective and integration of prior knowledge to the experimentation procedure are a few of the considerations to keep in mind at this stage. A team composed of individuals from different disciplines related to the product or process should be used to identify possible factors to investigate and determine the most appropriate response(s) to measure. A team-approach promotes synergy that gives a richer set of factors to study and thus a more complete experiment. Carefully planned experiments always lead to increased understanding of the product or process.

Screening

Screening experiments are used to identify the important factors that affect the system under investigation out of the large pool of potential factors. These experiments are carried out in conjunction with prior knowledge of the system to eliminate unimportant factors and focus attention on the key factors that require further detailed analyses. Screening experiments are usually efficient designs requiring a few executions where the focus is not on interactions but on identifying the vital few factors.

Optimization

Once attention is narrowed down to the important factors affecting the process, the next step is to determine the best setting of these factors to achieve the desired objective. Depending on the product or process under investigation this objective may be to either maximize, minimize or achieve a target value of the response.

Robustness Testing

Once the optimal settings of the factors have been determined, it is important to make the product or process insensitive to variations that are likely to be experienced in the application environment. These variations result from changes in factors that affect the process but are beyond the control of the analyst. Such factors as humidity, ambient temperature, variation in material, etc. are referred to as noise factors. It is important to identify sources of such variation and take measures to ensure that the product or process is made insensitive (or robust) to these factors.

Verification

This final stage involves validation of the best settings of the factors by conducting a few follow-up experiment runs to confirm that the system functions as desired and all objectives are met.

Inside This Reference

The following list provides a summary of the chapters in this reference book.

Statistical Background on DOE Chapter:
Statistical Background provides a review of the principles and terminology used in this reference. The chapter gives brief descriptions of the distributions used in calculations related to experiment analysis, and it explains the concept of hypothesis testing. Hypothesis testing finds direct application in the analysis of experiments to conduct significance tests and is widely used in later chapters.

Simple Linear Regression Analysis Chapter:
Simple Linear Regression Analysis introduces regression analysis using linear regression models with a single factor. Gaining a clear understanding of regression analysis is important in interpreting the results obtained from the analysis of designed experiments. The procedure to conduct significance tests using the analysis of variance (ANOVA) is explained in this chapter.

Multiple Linear Regression Analysis Chapter:
Multiple Linear Regression Analysis expands on the analysis of simple linear regression models and discusses the analysis of linear regression models with more than one factor, namely, multiple linear regression models. Significance tests on individual factors are discussed in this chapter. The chapter sets the stage to gain an understanding of ANOVA models, which are a category of multiple linear regression models used to analyze data obtained from experiments.

ANOVA for Designed Experiments Chapter:
This chapter uses the concepts covered in the previous two chapters to illustrate the analysis of single factor and factorial experiments (multiple factors) using ANOVA models. The chapter explains how the analysis objective affects the choice of the underlying models, namely regression or ANOVA. The concepts of randomization and blocking are also covered in this chapter.

Two Level Factorial Experiments Chapter:
Two Level Factorial Experiments concentrates on a special case of the factorial experiments, where each factor under investigation is run at two levels. The concept of blocking, introduced in Chapter ANOVA for Designed Experiments, is expanded in this chapter to include experiment designs that use incomplete blocks. Unreplicated two level experiments are also covered in this chapter. The later portions of the chapter discuss fractional factorial experiments and related concepts such as aliasing, folding over and design resolution.

Highly Fractional Factorial Designs Chapter:
This chapter covers the Plackett-Burman designs and Taguchi's orthogonal arrays.

Response Surface Methods for Optimization Chapter: Response Surface Methods discusses methodologies and experiment designs that help achieve an optimum response value. Optimization of single and multiple responses is covered in this chapter.

Robust Parameter Design Chapter:
Taguchi's Robust Parameter Design Method presents a brief discussion of Taguchi's philosophy to achieve a robust design.

Reliability DOE for Life Tests Chapter:
Reliability DOE introduces a new concept in DOE, where the disciplines of reliability and design of experiments are integrated, and illustrates how design of experiments can be used to "build" reliability into the products. Reliability DOE analysis is an essential portion of the methodology of Design for Reliability, which is a process to design robust products that operate with minimal failures. The chapter illustrates the analysis of experiments with censored observations and introduces the use of the Weibull, lognormal and exponential distributions as the underlying models of the response.