When professionals from fields like healthcare talk to me about polling at MIT, they often assume that designing patient surveys will be easy. They have questions, patients have answers, and getting from Point A to Point B seems like it should be a simple matter of handing off a clipboard and a pen.
But as these professionals quickly realize, surveying people can be extraordinarily tricky, even when (maybe especially when) the topic is their own behavior and experiences. By watching out for these six common survey pitfalls, healthcare organizations can gather accurate and reliable data that improves both operations and patient care.
Mistake #1: Ignoring Respondent Psychology
If you’ve ever seen an episode of House, you know that “everybody lies.” When patients are asked about topics like drug use, drinking, or their sexual history, they may be tempted to provide the answers that they wish were true, rather than the ones that actually are. On a patient survey, five beers a night can shrink to two, and that particularly reckless junior year of college might be conveniently erased from history. To gather more accurate data, survey designers may find that they need to write questions with forgiving wording. A simple acknowledgement that certain behaviors are fairly common—and don’t define a person’s entire life—can go a long way toward getting a patient to open up.
Mistake #2: Assuming Perfect Memory
Similarly, patients may simply have difficulty remembering information. You know that expression about not even being able to remember what you had for breakfast yesterday? Well, that’s more than just an expression, and it can heavily skew survey data. By giving patients bounded timeframes (the last week or month, rather than year), and by providing mental joggers (say, asking patients to think back to their last appointment), survey designers can reduce guesswork on the part of respondents.
Mistake #3: Writing Ambiguous Questions
A survey is essentially a conversation between the researcher and the respondent. But unlike an in-person conversation, a survey doesn’t offer the opportunity for clarification. The meaning of questions must be crystal clear. (For example: Does “drugs” mean prescription medications or illicit substances?) Generative AI tools like ChatGPT offer a quick, inexpensive way for researchers to refine their question-wording. By posing their survey questions to these large language models, designers can identify—and fix—potential points of confusion that they might otherwise miss.
Mistake #4: Failing to Get Specific
Our experiences are subjective, to the point that our feelings can distort the way we report hard facts. For instance, if we arrive early to an appointment and then we get in to see the doctor a few minutes late, we might report that we waited “a long time.” Even if we’re asked to provide an exact amount of time, we might say that we were left waiting for thirty minutes, rather than seven—again, based more on our feeling about the situation than anything else. Consider asking patients for very specific information: What time did you arrive at the office? What time were you seen by the doctor? The more specific you are in your questions, the better data you’re likely to gather.
Mistake #5: Oversimplifying Satisfaction Rates
If I don’t love my Uber driver, I simply opt out of rating them. Why? We’re in an era of rate-inflation, and anything under five stars is seen as nuclear, potentially even getting a driver booted off of the app. Survey designers should be aware of this dynamic when asking about patient satisfaction. One workaround is to ask patients to rate their experience on a more expansive scale—maybe 1 to 100, instead of 1 to 5.
Mistake #6: Neglecting Sample Quality
Writing good survey questions is only half the battle. You also need to get people to answer them—and ensure that the people who do represent a proper cross-section of the population you’re surveying. Consider surveying patients before they leave the doctor’s office or hospital. This will not only improve patients’ recall of information, but it will likely produce response rates far higher than surveys conducted by mail, email, or phone. Survey length can also play an important role in response rates. Some data is better than no data, and it’s preferable to get many responses to a four-question survey than no responses to a twenty-question survey. I always tell my students: Consider how long you think your survey should be, and then cut that in half.
The Impact of Effective Surveys
Data gathered from patient surveys can yield powerful insights about population-wide risks, patient satisfaction trends, and potential improvements to care. But this data is only as good as the surveys that produce it. There’s no one right way to conduct a survey, but there are plenty of wrong ones. By avoiding these pitfalls, healthcare organizations can make sure they’re hearing the voice of their patients loud and clear.
About Adam Berinsky
Adam Berinsky is the Mitsui Professor of Political Science at MIT and serves as the director of the MIT Political Experiments Research Lab (PERL). In addition, he is a Faculty Affiliate at the Institute for Data, Systems, and Society (IDSS) and lead instructor of the MIT Professional Education course, “Effective Communication through Surveys and Market Research”.
