Test Results

Improvements made to viewing test results to make it easier to read and digest for non research professionals. Furthermore, put features in the hands of the customer that enabled filtering and changing views of their data, historically only available to Feedback Loop admins.

Wireframes • Visual Design

 
 
 
 
 
 

History of project

  • All charts were PNGs generated by the Feedback Loop research consultants before being published to customers. 

    • If customers wanted additional slices or wanted to remove respondent, the customer had to request it and the charts had to be republished. 

    • Customers wanted to be able to hover over small slices of graphs to see what the percentage was (since when the slice was too small there were no numbers in it).

  • Charts were stacked graphs, which were difficult for many users to understand or quickly digest. 

  • Customers could not manage their own respondents or responses. Sometimes they wanted to hide a response from the open ends (specifically in cases where it had cursing and they were showing to executives), but also had to request Feedback Loop to do that.

  • Results that were statistically significant were hidden beneath a lot of layers and people didn’t understand the significance. 

  • Reports feature was not used because it looked bad, was hard to use, and was not easy to update information to share with people again.

 
 

Research

Researched about chart readability: specifically in relation to stacked vs grouped bar charts, as well as other chart options to figure out the best configuration for the layout of charts. Found that people were more quickly about to determine data from grouped bar charts. Specifically referenced this paper.

Competitor analysis (for companies such as Maze, Suzy, Zappi and Survey Monkey) to see how they were representing charts - especially the more complex ones such as matrix questions. 

Had conversations with one of our company advisors, Alex Genov, a researcher at Zappo to learn more about what different charts meant and what was seen as industry standard. Additionally, just got insight into what he would expect to see as a research and what was industry best practice. 

Also researched what was industry best practice for visualizing different types of charts, in terms of choosing colors for the default graphs. Referenced this article to choose our default color sets. As a later feature we added the ability for customers to be able to choose their own chart colors if they wanted (especially for branding purposes).


 

Live charts vs PNG

All charts and verbatim responses will update in real time based on changes to the respondent list. Additionally there was added functionality around slicing data and data analysis. 

After doing the initial research I really just worked closely with the developers experimentations on HighCharts, the graph library we were using and modified the charts in real time. 

  • Edited the hover area for charts of 0% to be the whole column, since there was no place for the customer to physically hover on the chart. 

  • Edited the style of the hover state

  • Filtered out options if they clicked on the group itself (ex. clicking on 3 at the axis would remove those options from the chart). This was because many people usually just wanted to show the extremes (1s and 5s) since that was where the interesting insights were to them.

Additionally we had to figure out how to show context of questions - such as variants how to see what those variants were and then how to sift through charts for the various variants, so they could dive deeper into how each performed.

One of the PMs used to work as a Market Researcher, so we used a format similar to what she used to use in her presentations that resonated well with people.

 

Findings more in context

Edited the layout of the page to have an overall test finding section, and surfacing the automatic insights (those generated by our data science team as being statistically significant).

 

Shared link vs report

Went with a more current model of sharing providing a generated sharable link that people could turn off or on to share. The link defaulted to off until a customer decided to turn it on for that test. In the live page they could decide what they wanted to be visible in the shared link by controls under each chart. Furthermore, this link was able to be accessed by anyone with the link, and you did not have to sign in to reach the link. We found once we provided these options many more people started sharing the results with others, vs having to make their own charts in excel.

 

Customer facing managing response capability

By adding the ability for customers to manage respondents, Feedback Loop was able to provide more defensible data. This allowed customers to see all the responses and choose to hide open ended responses from view (but not taking their data out of the charts) and/or removing the respondent all together (if it was spam, a bot, etc).