athenahealth QuickPay: A Heuristic Evaluation and Usability Study
Athenahealth QuickPay allows patients and organizations to view and pay their medical bills through an app using an access code received in the mail.
In partnership with Athenahealth, I conducted a heuristic evaluation and usability study on an updated design of their bill payment portal and user flow.
This project was completed as part of my MS in Information Studies degree program.
January 2021, 5 months
Tools & Methods
Unmoderated usability test
Qualitative & Quantitative results
Thank you to athenahealth's Consumer Financials UX team for all the help and guidance as I complete this project.
Overall, the current iteration of athenahealth's QuickPay is very well crafted. Many participants and evaluators praised its consistency and clarity in style and the level of detail provided for the cost breakdown a specific medical bill. A user testing participant also noted that this iteration of QuickPay is one of the easiest insurance interface they have seen (P10).
Based on the findings above, here are some recommendations for athenahealth as they move forward in the development of QuickPay:
Users expect certainty, especially regarding medical bills. Although the interview participants were aware that a bill was due, this sentiment was typically accompanied by a statement or inquiry regarding the specific date it is due. Including a pay-by date that notes at what point a bill changes from "due now" to "overdue" can help alleviate this uncertainty.
Users expect clarity in the steps they need to take during the payment process, and this includes a clear guide/indicator on how to go back to previous payment steps. The numbered steps to pay provides clarity to how many steps are involved in the payment process, and adding indicator arrows or a button that simply says "back" and/or "next" provides a clear "backdoor' for users as they go through the payment process.
Based on interview observations, the participants tend to be unsure about the location of the "Make Payment" button which would start the payment process for all open bills. Consider a way for a user to pay a specific bill in full from the same page as that specific bill's details (i.e. add a "pay this bill" within the specific bill details page that would bring the user to the payment flow but only for that one specific bill).
Users want to be certain that they have paid their bill in full or how much left they need to pay in order to pay their bills in full. At the end of the payment process, it is highly recommended that the interface shows a notification or a message that indicates how much left they have to pay or zero balance if they have already paid their bill in full.
Recommendations & Next Steps
Recommendations & Next Steps
The user testing portion was originally intended to be a moderated user testing. However, due to unrelated and unforeseen circumstances (one of which was the once-in-a-century freak winter storm in Texas), the original study schedule was pushed back. Later rounds of testing should consider a moderated test in order to gain better insights from users.
Although the focus of the participant pool was an even distribution between the age brackets, later rounds of testing should also consider a more even split of participant genders.
When providing clickable prototype links through Figma for user testing, whether they are moderated or not, make sure to hide the Figma toolbar as it could get in the way of buttons necessary to complete a task.
This experience has been extremely valuable to me as I got a glimpse of how an in-house UX research team functions in a large organization. Although the collaborative aspect of research was a little limited due to the constraints of the project, I enjoyed the process and getting to know the UX team members.
Limitations & Lessons Learned
Athenahealth provides various cloud-based services for the healthcare industry, including electronic health records (EHR), revenue cycle management & medical billing, and Epocrates and other point-of-care mobile apps.
In this project, I worked with the Consumer Financials team and conducted a usability study for a proposed design update of their QuickPay experience. QuickPay allows patients and organizations to access, view, and pay medical bills using a unique access code from a paper statement received in the mail.
QuickPay has recently undergone a redesign process, and, together with the Consumer Financials team, I conducted a heuristic evaluation and usability study of the redesigned system and user task flow.
I conducted a screen-by-screen heuristic evaluation of prototype with proposed redesign based on Nielsen's 10 Heuristics, and I also utilized Athenahealth's in-house Design Quality Score study in which 4 UX subject matter experts (SMEs) across different product teams within Athenahealth conducted their own heuristic evaluations.
I also conducted an unmoderated usability study using UserTesting. 15 participants were required to complete 4 tasks within QuickPay, and a SUS rating was calculated at the end of each session.
A heuristic evaluation serves as a good way to diagnose usability issues while iterating on a design. It can provide a benchmark score on how the design's usability, but it should not be used as the only method in identifying potential usability issues and should ideally be paired with user testing.
Using Nielsen's 10 Heuristics, I conducted a screen-by-screen evaluation of the QuickPay prototype, specifically screens that would later on be a part of the user task flow in the usability testing. For each screen, I identified both good experiences and heuristic violations using the following rating scale:
Catastrophe refers to issues that will likely contribute to task failure or abandonment and should be addressed immediately
Major problem refers to issues that could lead to task failure or workarounds and are high-priority fixes
Minor problem refers to issues that are impediments to efficiency and satisfaction and are low-priority fixes
Acceptable/Cosmetic refers to minor flaws that do not impede task success
Good experience refers to positive/non-issue observations
Following the evaluation, the scores are averaged based on the number of observations. The higher the score, the better the system is.
athenahealth's Design Quality Score (DQS)
DQS is an in-house research tool for athenahealth. In it's essence, it is a heuristic evaluation conducted by 4 subject matter experts with different backgrounds, namely:
UX SME - The designer involved in the project or problem space;
Product SME - Someone that understands the product space, the prioritization of development, and key goals of the user product; and
General SME - Someone that can be a point person with specific expertise about the problem space.
Like the individual evaluation, the scores are tallied and averaged. The higher the resulting score, the better the system is.
After all evaluations were completed, I compiled the data into a spreadsheet and tagged them based on the severity rating. I also went through every observation and gave each observation an "area of note" tag. This "area of note" tag allows the observations to be grouped based on the following:
General - Observations regarding the design as a whole/global design items
Payment flow - Observations that specifically calls out the payment flow process
Verbiage - Observations related to wording, word choice, jargon, etc.
Info hierarchy design - Observations related to presentation of information
Layout design - Observations related to screen layouts
Prototype design - Observations related to prototype functionality, error, etc.
Style design - Observations related to style.
To support the results from the heuristic evaluation and generate user insights, I conducted a usability test to both check the system usability as well as to identify user pain points. The test produced qualitative and quantitative feedback to guide recommendations and further iterations of the system design.
User Tasks & Test Script
For internal user demo of the system, I worked with a member from the Consumer Financials UX team in developing a script and a video of the tasks a hypothetical user may take. Based on this demo script and the overall function of QuickPay, I identified four main tasks that a user may undergo when using the system:
Accessing the portal using the code received in a paper statement
Viewing overall summary of all medical bills
Viewing detail summary of a specific medical bill
Paying the medical bill as due
After discussion with my field supervisor to gain insight on the procedure athenahealth teams would have to take to start and complete an unmoderated usability test, I determined that the usability test would need 15 participants evenly spread across different age ranges.
I chose to focus on the even distribution of the age range as it would allow me to gain insights from a wider audience range as athenahealth's QuickPay is a payment system that can be used by anyone as long as their healthcare provider uses athenahealth's billing system.
After the heuristic evaluation and usability test sessions were completed, I compiled the data gathered into a spreadsheet to better identify common problem areas, including user pain points during the test, most commonly violated heuristic, and most commonly identified problem areas.
Combining the results from athenahealth's DQS and the evaluation I completed on my own, there were five evaluators that took part in the process and generated and combined 78 observations. From the 78 observations, 26 are minor issues, 31 are acceptable/cosmetic issues, and 21 are good experiences. There were no catastrophic or major issues observed.
The calculated scores are simply an average of all the scores given for the observations; the higher the value, the better the design quality. At a combined average of 2.94 out of 4.0, this current design is considered well done by athenahealth's standards.
Based on the compiled observations, the most common problem areas were related to Verbiage (9 acceptable/cosmetic, 6 minor issues) and Payment Flow (3 acceptable/cosmetic, 8 minor issues).
Payment Due vs. Due now: Do they mean different things? Not clear.
In step 3 in Make a Payment (payment review), in the "show details" breakdown, each bill says "amount paid" but the user hasn't paid that amount yet (they will once they confirm this payment).
Problem area: Verbiage
Multiple evaluators noted that some of the verbiage used in the redesign could be confusing or potentially be misleading.
As a user, I want to just go and pay that bill from the bill breakdown page once I look at the details. But I'm forced to go back to the first page to "make a payment"
The end of the payment step felt a bit like a dead end -- maybe there is a CTA to return to the homepage or something?
Problem area: Payment Flow
At certain steps in the payment process, some evaluators noted that there seems to be a lack of or limited user freedom in choosing how to approach and complete payment.