In-Person Usability Testing at the RaiseMore Beta in Oklahoma City
In September of 2015, We ran a two day feedback summit in Oklahoma City with the entire RaiseMore team. We went through 4 feedback sessions. The rounds of one-on-one interaction with real people was incredibly helpful. The Group discussions On Day 3 discussing the product roadmap was very insightful. The weekend would inform a lot of the next several months of product features and iteration.
We brought in both tech and fundraising people included large organizations like M.D. Anderson as well as smaller orgs like Carry the Load. We went through rounds of feedback on both the actual products and also our roadmap going forward. The insights from that session were insanely helpful and surprising. Among our team, there was a feeling that we really had our finger on the pulse of the industry and in the end, that was a bit of a liability. While we did have a good handle on software design - we did not, in fact - have a good handle on the needs of our fundraising users.
I wanted to create this post to document some of the things we looked at in case those things are at all helpful for others. Some of it’s dated but it might have some helpful things to at least think about when you begin a software project. Especially one for users that you’re new to working with.
Here are a Few Takeaways*:
- Up to 80% of our users would be using Windows computers. Up to 30% of those might use touchscreen technology in some capacity.
- Our collective use of Apple hardware caused us to miss some glaring usability issues. We missed hover states, color inconsistencies and responsive issues on smaller screens. We missed these things because we were seeing the product through our tech focused lens.
- Visual design inside the dashboard caused a lot of confusion. Defaulting to ghost buttons and a flat UI pattern was often and easily missed by our testers.
- Our internal bias toward aesthetics vs. experience was exposed. This surfaced some internal frustration on our team between UX and Visual design imperatives. The dialogue we had here was tense but positive. If people can’t accomplish the product tasks, we will have a hard time making them customers.
- The things we thought were going to be super important were a lot less important. One of our assumptions was that the person using the app would start at the feed page. This would allow people to see the status and relevant data points of all their teammates. During testing, our groups let us know they would opt for a view based on the health of their event(s) instead.
- An interesting pattern emerged here. The money raised is important but not on such a granular level. The overall event health was a more desirable metric. From there our partners could dig deeper and address individual coaching needs. There was a very “leader-leader” structure to the way our partners managed events.
In addition to the findings above and our notebooks full of information after the beta, we had a crew on hand to film the days. Enjoy.
RaiseMore Beta Day 1 & 2
RaiseMore Beta Day 1
RaiseMore Beta Day 2
*Numbers extrapolated to identify total user-base from user-base who was able to attend the beta retreat.
Keaton Taylor is a designer and front-end developer moving through the world asking lots of questions and being wrong about no less than 50% of his ideas at any given time. He is an IC at Apartment Therapy Media and all his thoughts and ideas are his own.