How Human Analysis Arms the Treasury Department

And the survey says…

With the intention of providing better online customer service, the Department of Treasury surveys website visitors about their experience. In addition, they want to know if the visitor chose an online experience first versus another channel like the call center.

The survey is a mix of “closed” (yes/no) and open-ended questions. The agency quickly realized that many answers didn’t directly correspond to the question asked.

Using text analytics software alone did not work. They needed contextual analysis by someone who could make sense of the responses that did not match the questions. To get this complete picture, the agency deployed PublicRelay’s hybrid solution that combines top technology with expert human analysis.

Human looking at website survey on phone

Creating a Measurement Model

The goal of the survey was three-fold:

  • Categorize general visitor concerns by what they were trying to accomplish
  • Find product areas on the site needing usability improvements (forms, instructions, data retrieval, etc.)
  • Identify all requested improvements submitted

The human analysis provides the ability to:

  • Correlate the free text answers with the yes/no answers for a complete view of the submission
  • Correctly provide a topic for the open-ended answers that are not tied to the question asked
  • Provide analysis of both the quantitative and qualitative survey data

An Action Plan Every Month

The agency reviews the site survey analysis monthly. This allows the team to prioritize efforts on areas that need the most attention.

Whether it’s making changes to the site’s navigation to make it easier to find forms or instructions, or more rigorous testing of data retrieval, the agency now has an action plan each month.

“PublicRelay’s analysis helped us make targeted decisions on the whole picture, not just what was immediately obvious.”

Executive, Department of Treasury