PhD tips – Dealing with “failed” experiments
“PhD tips” is an ongoing series of blog posts written by postdocs and aimed at graduate students at the University of Oxford (Department of Zoology). I wrote this in April 2021.
I hope you are all doing well. This week, I want to talk about dealing with “failed” experiments.* I put “failed” in quotes because, as I will argue below, only a very small fraction of experiments perceived as failed have actually failed in the sense that they were completely pointless.
The first thing to realize is that the core of experimental science is an iterative process: you do a thing, it does something (or not), you think about it, you do it a little different next time. Crucially, this process works the same way whether the results made you happy or not (for whatever reason). You think about what happened, you change stuff, you do it again the same way, you do it again with a twist, or you do something else entirely. A very normal experience as a scientist is to – on average – be a little disappointed in the outcome of your experiment. See below for a list of outcomes that tend to make people unhappy. Again: this is normal and part of the process. The most “successful” projects are built on a foundation of “failed” or semi-failed attempts at doing something.**
The second thing to realize is that learning is at the core of this iterative process: a thing has to happen for us to better understand what we’re dealing with. This means that the essential goal of each experiment is to learn something, anything! And this “lesson” doesn’t usually come in the shape of a perfect plot that’s publication-ready on the first try. Instead, the next learning item usually comes from a mangled mess of an experiment or dataset.
What exactly you are learning from each attempt depends of course on the details of your project, but below I am giving a few real-life examples of typical experimental outcomes that tend to make people unhappy, and some suggestions on what one could learn from them.
- Treatment repeatedly has no effect compared to control
- Not failed at all, this is a valuable negative result.
- Outcome: solid scientific insight.
- Noise / differences between independent biological replicates very high
- Not failed at all, this is valuable data we can learn from.
- Potential next step: figure out if noise is biological variance or technical measurement error, or both.
- Outcome: a better understanding of your system, solid scientific insight
- Mistake made in experiment and you noticed it
- Example: forgot to wash cells before treating them.
- Semi-failed, this is valuable data we can learn from. By comparing the washed cells with the unwashed cells we can learn how this step in the experiment influences the final results. Can be useful for future troubleshooting.
- Outcome: an experimental protocol where we understand better what each step does, or where we realize one step is less important than previously thought; honing of lab skills.
- Unexpected/”impossible” result in experiment but you don’t remember making a mistake
- Example: positive/negative control didn’t yield the expected result, even though it has always worked before and the experiment was (to the best of your knowledge) executed as usual.
- Semi-failed. An opportunity to evaluate the reliability of the protocol/equipment/material.
- Potential next steps: introduce checklists and more note-taking to ensure little details are adhered to; evaluate assumptions e.g. is the control genotype actually this genotype, is something contaminated, is the machine still working, etc.
- Outcome: a more reliable experimental protocol, honing of lab skills
- Experiment got contaminated in some way
- Example: algae grow over all your aquatic plants and kill them
- Semi-failed. An opportunity to evaluate “housing” conditions for your organism.
- Potential next steps: pilot experiment to optimize housing conditions before doing “actual” experiments; investigate whether the contaminant did something interesting
- Outcome: a more reliable experimental protocol; chance of a random cool discovery with the contaminant
- Sample was accidentally destroyed/lost
- Example: dropped tube with the sample on the floor. Cannot be recovered.
- Close to “actually failed”, but still an opportunity to introduce better safety measures / risk assessment.
- Potential next steps: introduce safety procedures to ensure better sample preservation, labelling etc.
- Outcome: a more reliable experimental protocol
The third thing to realize is that feelings of frustration, anger, sadness, fear, are all very common in the wake of a “failed” experiment, especially if it took a long time. And trying to think about the next steps while still frustrated can turn into a vicious cycle where you make mistakes because frustrated brains are bad at solving problems. So, instead, allow yourself to feel these feelings, get a good night’s sleep, and then start troubleshooting the next day with a fresh and relaxed mind.
So, to summarize: doing your PhD, and science in general, is about learning. Every experiment, no matter the outcome, can teach you something. Some of the lessons are perhaps a bit more appealing and easier to swallow than others, but they are all useful and necessary. So next time an experiment “fails”, try taking a nice little break if you can, and then list all the things you still learned about your experimental setup, or organism of interest.
Finally, I would like to share a little mental trick I developed over the years that has helped me with dealing and accepting experimental setbacks. I like to imagine that any given project will take me X hours to complete, including setbacks, trials by fire, random distractions, dead-ends etc. In science, X is usually unknown. But importantly, there is always an X. Every time I conduct an experiment that “fails”, and let’s say it took me 6 hours, I have now reduced X by 6 hours. Not bad! I am one step closer to finishing the project. It’s a bit tongue-in-cheek of course, but it still helps me with evaluating my work for what it is: I gave these 6 hours my best and the outcome doesn’t change that.
If you would like to chat to me further about dealing with “failed” experiments, please email me at elisa.granato [at] zoo.ox.ac.uk.
*If you work on theory you can probably replace “failed experiments” here with “theoretical approaches that didn’t work out” or something along those lines and maybe some of this might still be helpful.
**This is the science version of “The master has failed more times than the student has even tried”.