OPEN CHECKBOOK DETROIT
TEAM PROJECT FOR UNIVERSITY OF MICHIGAN'S CITIZEN USER TESTING GROUP
USER EXPERIENCE TEAM:
LILLIAN LI, MUZI LIN, MAI NAKHALA, JAMIE NEUMANN, BONNIE WHITE
SEPTEMBER - OCTOBER 2019
STAKEHOLDER INTERVIEWS, USER INTERVIEWS, USABILITY TESTING
THE FULL STORY
As part of the Citizen User Testing Group (CUTGroup), a volunteer organization at the University of Michigan, I worked with the City of Detroit and Data Driven Detroit (D3) to assess the Open Checkbook mobile app. This tool aims for the transparent presentation of vendor spending by the City of Detroit in an intuitive and informative way. My team created a protocol; proctored short, structured interviews; and ran rapid usability tests with Detroit citizens to uncover technical and content issues within the tool.
Based off stakeholder interviews with Kat Hartman and Juan Carlos Angeles from Detroit's Department of Innovation and Technology, and Meghin Mather from D3, my team compiled three focus points for assessing the mobile app.
The tool’s intuitiveness and ease on a mobile device, with an emphasis on the “Chart” section of the prototype
The users’ perceptions of the government—specifically transparency regarding vendor spending—and their overall trust in the City of Detroit.
The tool’s use of jargon and charts, and how this affects the users’ ability to use and understand information provided by the tool.
A NEED FOR SPEED
Rapid research and a bi-weekly deadline pace challenged my team, inspiring us to work fast and revise often. After multiple lightning drafts, we had our protocol and were ready for testing.
USER TESTING DAY
Our testing space was at the Wilder branch of the Detroit library. In the role of proctor, I ran through interview questions, and guided each user through usability tasks, probing when I sensed extra emotion, and forming a genuine connection. As notetaker, I was diligent about prompting the user when s/he was no longer verbalizing actions, and made sure to not only synthesize what I observed, but also record direct quotations.
The main challenge we ran into was that while fifteen volunteers had signed up, only six came. Once I realized we would have a smaller sample size, I started asking volunteers if they had any extra time to spare so that we might go more in-depth with their answers.
FROM RAW DATA TO RECOMMENDATIONS
Even with six volunteers, we had a lot of raw data. The main challenge was finding themes across the users. We made good use of all the whiteboards in the building! By being able to visualize and synthesize user answers, my team was able to identify the major pain points. We could then formulate recommendations for the next iteration of the mobile app.
We gave our presentation to our stakeholders and answered their questions. I found that the quotations we gathered were especially useful in illustrating our recommendations. Later that month, Juan Carlos, the front-end developer of Open Checkbook, presented our final report to the office of Detroit's chief financial officer.
Below is a selection of our presentation artifacts, and here is the full presentation.
Fail in order to succeed! Even with the time crunch, my team went through multiple protocol drafts to make sure we had the right idea.
Get the most out of the users you have. When I realized fewer users were showing up, I spent more time with the ones who came.
When in doubt, write it out. We could have gotten lost in the forest of raw data, but by throwing everything up on a whiteboard, we could see patterns better.