DOE Overview

Much of our knowledge about products and processes in the engineering and scientific disciplines is derived from experimentation. An experiment is a series of tests conducted in a systematic manner to increase the understanding of an existing process or to explore a new product or process. Design of experiments (DOE), then, is the tool to develop an experimentation strategy that maximizes learning using a minimum of resources. DOE is widely used in many fields with broad application across all the natural and social sciences. It is extensively used by engineers and scientists involved in the improvement of manufacturing processes to maximize yield and decrease variability. Often engineers also work on products or processes where no scientific theories or principles are directly applicable. Experimental design techniques become extremely important in such studies to develop new products and processes in a cost effective and confident manner.

Why DOE?

With modern technological advances, products and processes are becoming exceedingly complicated. As the cost of experimentation rises rapidly, it is becoming increasingly difficult for the analyst, who is already constrained by resources and time, to investigate the numerous factors that affect these complex processes using trial and error methods. Instead, a technique is needed that identifies the "vital few" factors in the most efficient manner, and then directs the process to its best setting to meet the ever increasing demand for improved quality and increased productivity. DOE techniques provide powerful and efficient methods to achieve these objectives.

Designed experiments are much more efficient than one-factor-at-a-time experiments, which involve changing a single factor at a time to study the effect of the factor on the product or process. While one-factor-at-a-time experiments are easy to understand, they do not allow the investigation of how a factor affects a product or process in the presence of other factors. An interaction is the relationship whereby the effect that a factor has on the product or process is altered due to the presence of one or more other factors. Oftentimes interaction effects are more important than the effect of individual factors. This is because the application environment of the product or process includes the presence of many of the factors together instead of isolated occurrences of one of the factors at different times. Consider an example of interaction between two factors in a chemical process, where increasing the temperature alone increases the yield slightly while increasing the pressure alone has no effect. However, in the presence of both higher temperature and higher pressure the yield increases rapidly. In this case, an interaction is said to exist between the two factors affecting the chemical reaction.

The DOE methodology ensures that all factors and their interactions are systematically investigated. Therefore, information obtained from a DOE analysis is much more reliable and complete than results from one-factor-at-a-time experiments that ignore interactions and thus may lead to incorrect conclusions.

Introduction to DOE Principles

The design and analysis of experiments revolves around the understanding of the effects of different variables on another variable. In technical terms, the objective is to establish a cause-and-effect relationship between a number of independent variables and a dependent variable of interest. The dependent variable, in the context of DOE, is called the response, and the independent variables are called factors. Experiments are run at different factor values, called levels. Each run of an experiment involves a combination of the levels of the investigated factors, and each of the combinations is referred to as a treatment. When the same number of response observations are taken for each of the treatments of an experiment, the design of the experiment is said to be balanced. Repeated observations at a given treatment are called replicates.

The number of treatments of an experiment is determined on the basis of the number of factor levels being investigated. For example, if an experiment involving two factors is to be performed, with the first factor having m levels and the second having n levels, then m x n treatment combinations can possibly be run, and the experiment is an m x n factorial design. If all m x n combinations are run, then the experiment is a full factorial. If only some of the m x n treatment combinations are run, then the experiment is a fractional factorial. In full factorial experiments, all the factors and their interactions can be investigated, whereas in fractional factorial experiments, at least some interactions are not considered because some treatments are not run.

It can be seen that the size of an experiment escalates rapidly as the number of factors (or the number of the levels of the factors) increases. For example, if 2 factors at 3 levels each are to be used, 9 (3x3=9) different treatments are required for a full factorial experiment. If a third factor with 3 levels is added, 27 (3x3x3=27) treatments are required, and 81 (3x3x3x3=81) treatments are required if a fourth factor with three levels is added. If only two levels are used for each factor, then in the four-factor case, 16 (2x2x2x2=16) treatments are required. For this reason, many experiments are restricted to two levels, and these designs are given a special treatment in this reference. Using a fractional design further reduces the number of required treatments.

DOE Types

For Comparison: One Factor Designs

With these designs, only one factor is under investigation, and the objective is to determine whether the response is significantly different at different factor levels. The factor can be qualitative or quantitative. In the case of qualitative factors (e.g., different suppliers, different materials, etc.), no extrapolations (i.e., predictions) can be performed outside the tested levels, and only the effect of the factor on the response can be determined. On the other hand, data from tests where the factor is quantitative (such as temperature, voltage, load, etc.) can be used for both effect investigation and prediction, provided that sufficient data is available. (In Weibull++ DOE folios, predictions for one factor designs can be performed using the multiple linear regression folio or free form folio.)

For Factor Screening: Factorial Designs

In factorial designs, multiple factors are investigated simultaneously during the test. As in one factor designs, qualitative and/or quantitative factors can be considered. The objective of these designs is to identify the factors that have a significant effect on the response, as well as investigate the effect of interactions (depending on the experiment design used). Predictions can also be performed when quantitative factors are present, but care must be taken since certain designs are very limited by the choice of the predictive model. For example, in two level designs only a linear relationship can be used between the response and the factors, which may not be realistic.

• General Full Factorial Designs

In general full factorial designs, the factors can have different number of levels, and they can be quantitative or qualitative.

• Two Level Full Factorial Designs

With these designs, all factors must have only two levels. Restricting the levels to two and running a full factorial experiment reduces the number of treatments (compared to a general full factorial experiment), and it allows for the investigation of all the factors and all their interactions. If all factors are quantitative, then the data from such experiments can be used for predictive purposes, provided a linear model is appropriate for modeling the response (since only two levels are used, curvature cannot be modeled).

• Two Level Fractional Factorial Design

This is a special category of two level designs, where not all factor level combinations are considered, and the experimenter can choose which combinations are to be excluded. Based on the excluded combinations, certain interactions cannot be investigated.

• Plackett-Burman Design

This is a special category of two level fractional factorial designs, proposed by R. L. Plackett and J. P. Burman [1946], where only a few specifically chosen runs are performed to investigate just the main effects (i.e., no interactions).

• Taguchi's Orthogonal Arrays

Taguchi's orthogonal arrays are highly fractional designs, used to estimate main effects using only a few experimental runs. These designs are not only applicable to two level factorial experiments, but also can investigate main effects when factors have more than two levels. Designs are also available to investigate main effects for certain mixed level experiments where the factors included do not have the same number of levels.

For Optimization: Response Surface Method Designs

These are special designs that are used to determine the settings of the factors to achieve an optimum value of the response.

For Product or Process Robustness: Robust Parameter Designs

The famous Taguchi robust design is for robust parameter design. It is is used to design a product or process to be insensitive to noise factors.

For Life Tests: Reliability DOE

This is a special category of DOE where traditional designs, such as the two level designs, are combined with reliability methods to investigate effects of different factors on the life of a unit. In reliability DOE, the response is a life metric (e.g., age, miles, cycles, etc.), and the data may contain censored observations (suspensions, interval data).

For Experiments with Constraints: Optimal Custom Design

The optimal custom design tool can be used to modify the above standard designs to plan an experiment that meets any or all of the following constraints: 1) limited availability of test samples, 2) factor level combinations that cannot be tested, 3) factor level combinations that must be tested or 4) specific factors effects that must be investigated.

Stages of DOE

Designed experiments are usually carried out in five stages: planning, screening, optimization, robustness testing and verification.

Planning

It is important to carefully plan for the course of experimentation before embarking upon the process of testing and data collection. A thorough and precise objective identifying the need to conduct the investigation, an assessment of time and resources available to achieve the objective and an integration of prior knowledge to the experimentation procedure are a few of the goals to keep in mind at this stage. A team composed of individuals from different disciplines related to the product or process should be used to identify possible factors to investigate and determine the most appropriate response(s) to measure. A team-approach promotes synergy that gives a richer set of factors to study and thus a more complete experiment. Carefully planned experiments always lead to increased understanding of the product or process.

Screening

Screening experiments are used to identify the important factors that affect the system under investigation out of the large pool of potential factors. These experiments are carried out in conjunction with prior knowledge of the system to eliminate unimportant factors and focus attention on the key factors that require further detailed analyses. Screening experiments are usually efficient designs requiring a few executions where the focus is not on interactions but on identifying the vital few factors.

Optimization

Once attention is narrowed down to the important factors affecting the process, the next step is to determine the best setting of these factors to achieve the desired objective. Depending on the product or process under investigation, this objective may be to either maximize, minimize or achieve a target value of the response.

Robustness Testing

Once the optimal settings of the factors have been determined, it is important to make the product or process insensitive to variations that are likely to be experienced in the application environment. These variations result from changes in factors that affect the process but are beyond the control of the analyst. Such factors as humidity, ambient temperature, variation in material, etc. are referred to as noise factors. It is important to identify sources of such variation and take measures to ensure that the product or process is made insensitive (or robust) to these factors.

Verification

This final stage involves validation of the best settings of the factors by conducting a few follow-up experiment runs to confirm that the system functions as desired and all objectives are met.