Overcoming barriers to Design of Experiments (DOE)

• Design of experiments (DOE) helps you perform the right investigation, at the right time, using the right conditions. DOE is more of a philosophy than it is a standard operating procedure. 
• A one-factor-at-a-time (OFAT) design may sometimes be suitable, but you need to be sure that there really is only one variable, that conditions are optimal, and that there are no controllable interactions.
• Design of experiments is appropriate when: more than one factor could influence the outcome; you want to test many factors, even with limited resources; when you need to screen for important factors; and when you want to understand the interactions between factors. 
• This blog looks at how software and hardware solutions, and collaboration between biologists, bioinformaticians, statisticians, and automation engineers overcome the traditional hurdles that have hindered the implementation of DOE as the ‘default’ approach in the life sciences. These solutions bring the approach within the reach of every life scientist.
• If you’d like to keep learning about DOE after you're done with this article, make sure to check out our other DOE blogs, download our DOE for biologists ebook, or watch our DOE Masterclass webinar series.

Design of experiments (DOE) isn’t just for biologists. Agriculturists (as we saw in our last blog), chemists, engineers, and even marketers use DOE^1-3 despite not being statisticians. Market researchers, for example, use DOE (which they call conjoint analysis) to explore the influence of various components of a marketing campaign on consumer behavior.³

Design of experiments is about making sure you perform the right investigation at the right time in your experimental sequence using the right experimental conditions. So, DOE is more a philosophy for getting the most out of your experiments than a standard operating procedure, as we'll see in this blog. 

The complexity threshold

Design of experiments is a powerful, robust research tool. However, there is a ‘complexity threshold’ below which one factor at a time (OFAT) protocols are appropriate, such as when there really is only one variable, conditions are fixed and there are no controllable interactions. 

It’s easy to kid yourself that you’re below the complexity threshold. But are you sure that only one variable affects the outcome? Are you sure that there are no interactions? Are you sure that the conditions are optimal and that there are no unknown unknowns?

Design of experiments is appropriate when more than one factor could influence the experiment’s outcome. Design of experiments is also ideal when you want to test many factors, even when resources are limited, and can be used regardless of how much you already know about the system - from a lot to almost nothing. In addition, DOE can screen for the most important factors and uncover hidden interactions. In other words, DOE, rather than OFAT, should be the ‘default’ method in the life sciences. So, why isn’t DOE the standard approach?

Overcoming barriers

There are many perceived barriers to adopting DOE as the standard experimental approach in life sciences: the seeming complexity of the statistical foundation; the need for more complex protocols; and difficulties modeling and visualizing the large amounts of data. The right blend of tools (software and hardware) and expert support can surmount these barriers.

DOE Barrier 1: the statistics are too hard to understand

At first sight, DOE’s statistical foundation can appear daunting to non-mathematicians. Traditionally, applying DOE was possible only with a deep knowledge of statistics or access to a bioinformatician or statistician. Thankfully, this is no longer the case. Advances in software make DOE more accessible to non-mathematicians by offering protocols and functionality that take on much of the mathematical and statistical burden. Indeed, as we’ve mentioned, a very broad range of disciplines use DOE without being expert statisticians.

Nevertheless, bioinformaticians and statisticians can use DOE software to help biologists, such as when introducing DOE into a busy laboratory or tackling a particularly difficult problem (see below). Bioinformaticians and statisticians understand DOE theory, but might lack some knowledge about the science and application. Biologists wanting to implement DOE often lack a deep understanding of DOE theory, but have strong knowledge of their domain space. Working together allows scientists to make the best use of statistical tools to study different systems.

DOE barrier 2: the experiments are too hard to plan and execute

Translating a DOE design into manual pipetting instructions often involves considerable time using spreadsheets to plan liquid stock concentrations, transfer volumes, plate layouts, and so on, even before you get into the lab.

The more complex the problem you’re trying to solve, the more experiments you need to perform. The size and complexity of experiments can rise rapidly. Manually executing complex experiments is extremely challenging: there’s usually a lot of work and it’s very hard to avoid mistakes. 

The many sophisticated and powerful lab automation solutions, including on-line data acquisition and analysis, overcome this hurdle. However, this presents a new challenge: biologists need to integrate the output of the DOE software with the software that controls their lab automation. Often, laboratory automation software can be a barrier to use for biologists. However, collaboration with Automation Engineers can help make the transition from manual to automated liquid handling.

It should still be noted that every DOE is different. So a script written for one automated liquid handling device for one DOE will be very different from the script for the next DOE. This can often mean that the scripts written to execute a DOE are not easily repurposed for other contexts meaning that the automation engineer can become the bottleneck needing to continually script new protocols for each DOE.

DOE barrier 3: the experiments are too hard to model

DOE can explore multiple factors simultaneously. However, this produces highly multidimensional data that can be difficult to visualize and interpret. Fitting models to your data to analyze and interpret your results can require sophisticated statistics. Again, this stage often lends itself to the collaboration between bench scientists and the statisticians and bioinformaticians that helped design the DOE. 

General-purpose data analysis software has several solutions including guiding biologists through the available modeling techniques, multidimensional plotting, contour plots, and ways to filter and cluster the data from which you can derive insight. Biology-specific applications often add implementation-specific features, such as heatmaps in plate format to draw the relationship between the physical setup and the data generated. 

Set them up, knock them down

Given that DOE involves statistical understanding and automation engineering skills, where does the biologist fit? Biologists own the science which means they can ask the right questions in the first place—so you don't waste time relearning stuff you know—and make sure the planned experiments make scientific and practical sense. But, after all, science is about collaboration. By using the right tools and getting the right help from the right places you can overcome all these barriers and access the power of DOE. To paraphrase Isaac Newton: to see further, you should stand on the shoulders of statisticians.

Interested in learning more about DOE? Make sure to check out our other DOE blogs, download our DOE for biologists ebook, or watch our DOE Masterclass webinar series.

References

1. Tye H. Application of statistical ‘design of experiments’ methods in drug discovery. Drug Discovery Today 2004;9:485-491
2. Murray PM, Bellany F, Benhamou L et al. The application of design of experiments (DoE) reaction optimisation and solvent selection in the development of new synthetic chemistry. Organic & Biomolecular Chemistry 2016;14:2373-84
3. Almquist E and Wyner G. Boost your marketing ROI with experimental design. Harvard Business Review 2001;Available at https://hbr.org/2001/10/boost-your-marketing-roi-with-experimental-design Accessed November 2022