Kristen Cibelli Hibben, a 2014 winner of the Charles Cannell Fund in Survey Methodology, is studying how the right prompts in a web survey can get responders to provide more complete and accurate answers.

Kristen Cibelli Hibben, photo courtesy of the Michigan Program in Survey Methodology.

Kristen Cibelli Hibben was just out of college in 1999 when she conducted her first survey. She and a classmate from Tufts talked to “the right people at the right time” and transformed an independent research project into an 18-month study examining what local NGOs in Bosnia and Herzegovina thought of the International Criminal Tribunal for the former Yugoslavia (ICTY).

What they found was that NGOs were not well informed about the process of international justice and had many misconceptions about it. The ICTY formally received the results, and the findings added to a growing awareness of the importance of outreach by the Tribunal in the region.

“We had some wonderful people advising us,” recalls Hibben. “But I think that in a way it was our naiveté that enabled us to just go ahead and do it.”

Hibben went on to work several years at the Human Rights Data Analysis Group (HRDAG), a non-profit that teams with human rights advocates around the world to collect data on human rights abuses in countries such as Liberia, Sierra Leone, Chad, and Iran. But having to rely on the expertise of others to do the rigorous survey work eventually convinced Hibben to return to school. “It was hard for me to leave that work, but I knew that it was important,” she says.

Now a 5th-year doctoral student at the Program in Survey Methodology at the University of Michigan (U-M), Hibben is examining ways to get better responses from survey participants, whether located in areas of conflict or not. “How can we effectively communicate the goal and the purpose of the survey, and how can we effectively motivate respondents to think hard about the questions we’re asking and provide us with accurate and complete information?” she asks.

To answer these questions, Hibben will analyze the effectiveness of three tactics in an online survey. To improve respondent commitment, Hibben will include language asking participants to commit to providing complete and accurate information (see sample commitment statement below).

The survey will provide tailored feedback by prompting respondents who skip questions to do their best to answer every one. Similarly, if respondents check off answers too quickly, the program will remind them to consider each question individually.

“This is based on research done decades ago by [Institute for Social Research researcher] Charlie Cannell here,” Hibben says. “But what I discovered going through that old literature is that those are great ideas and they haven’t been examined in decades.”

Hibben will add one more twist—contextual recall cues built into the survey that encourage respondents to improve the accuracy of their memories, for example, by thinking about what happened, where it happened, and who else was there.

Hibben will try out the techniques alone and in different combinations in a web survey given to about 1,000 parents of child patients at U-M Pediatric Clinics. And because she will be able to check respondents’ answers against actual records, she can clearly measure the impact of each approach.

Appropriately, Hibben’s research is backed by financial support from the Charles Cannell Fund in Survey Methodology, awarded last year. “The symbolism of having funding from the Cannell Fund, given that I am carrying forward some of Charlie Cannell’s ideas … I feel very proud of that,” she says.

Kristen Cibelli Hibben with her colleagues at the Institute for Social and Environmental Research – Nepal. Photo courtesy Kristen Cibelli Hibben.

Hibben will begin analyzing data this fall. If the techniques are as effective as she expects, they could become low-cost ways for surveyors to improve their results—as long as the prompts aren’t overused. “If you can play up the credibility and importance of the survey for policy purposes,” she explains, “that is where commitment may help respondents differentiate it from marketing surveys.”

Meanwhile, Hibben is still in touch with friends and colleagues doing human rights work, and she envisions returning as a contractor after her graduation in 2016. “Survey methodology is a very applied type of degree,” she says. “I like to think about how this can inform the work that we actually need to do out in the world.”
[stextbox id=”grey”]
Sample Commitment Statement

Hibben designed this commitment statement to appear at the beginning of the web survey given to parents of child patients at U-M:

You have been selected to represent families whose children receive care at the University of Michigan Pediatric Clinics. In order for the information from this research to be the most helpful it is important that you try to be as accurate, complete, and honest as possible with your answers.