Open Site Navigation

athenahealth QuickPay: A Heuristic Evaluation and Usability Study

Athenahealth QuickPay allows patients and organizations to view and pay their medical bills through an app using an access code received in the mail.

Overview

In partnership with Athenahealth, I conducted a heuristic evaluation and usability study on an updated design of their bill payment portal and user flow.

This project was completed as part of my MS in Information Studies degree program.

Role

UX Researcher

Duration

January 2021, 5 months

Tools & Methods
  • Figma

  • usertesting.com

  • Heuristic evaluation

  • Unmoderated usability test

  • SUS Score

  • Qualitative & Quantitative results

Thank you to athenahealth's Consumer Financials UX team for all the help and guidance as I complete this project.

Let's chat!

Based on the results of the test, all 15 participants found that the QuickPay system pleasant to use, and the four tasks were easy to complete. At the end of each task, participants rated the ease of completing the task; results show that on average, the four main tasks I identified were quite easy to complete.

At the end of a test session, participants were asked a series of 10 questions based on the System Usability Scale questionnaire and would provide a rating response to each question. The calculated score can then be used to determine the usability/ease of use of the system. Based on the calculated score, QuickPay can be considered to be highly usable with an average SUS score of 83.67. The lowest score calculated was 70, and the highest was 90. For reference, the industry average SUS Score is around 68. With a score of 83.67, QuickPay can be considered an "Excellent" system.

Usability Test

Despite the high scores, participants still experienced hiccups and/or expressed uncertainties during the test, most notably regarding:

  • The specific date of when a bill is due (Task 2); 

  • How to go back to previous steps during the payment process (Task 4).

"It says payment due, these are the dates (service dates). I don't know if these bills are overdue or not or what the timeframe is because there's nothing that tells me that. I've got bills from three different months so obviously something is past due." - P13

"It says due now, does that mean they're past due? To me due now means they are past due." - P10

"It does say you owe, so I'm assuming that it's due now, but there's nothing specific that says when you're supposed to pay." -P15

"The only thing I can think of is to X out and that might take me to the previous page." -P12

"Is there a back arrow? No, there is not. Usually you would click on the steps up here but they don't work, so I don't now how to go back...I didn't know how to go backwards unless there was something that said 'change.'" -P14

"There's no back button... I would prefer if the check mark is linked [to the previous payment step]." -P4

Bill due date

Payment steps

Overall, the current iteration of athenahealth's QuickPay is very well crafted. Many participants and evaluators praised its consistency and clarity in style and the level of detail provided for the cost breakdown a specific medical bill. A user testing participant also noted that this iteration of QuickPay is one of the easiest insurance interface they have seen (P10). 

Based on the findings above, here are some recommendations for athenahealth as they move forward in the development of QuickPay:

  • Users expect certainty, especially regarding medical bills. Although the interview participants were aware that a bill was due, this sentiment was typically accompanied by a statement or inquiry regarding the specific date it is due. Including a pay-by date that notes at what point a bill changes from "due now" to "overdue" can help alleviate this uncertainty.

  • Users expect clarity in the steps they need to take during the payment process, and this includes a clear guide/indicator on how to go back to previous payment steps. The numbered steps to pay provides clarity to how many steps are involved in the payment process, and adding indicator arrows or a button that simply says "back" and/or "next" provides a clear "backdoor' for users as they go through the payment process.

  • Based on interview observations, the participants tend to be unsure about the location of the "Make Payment" button which would start the payment process for all open bills. Consider a way for a user to pay a specific bill in full from the same page as that specific bill's details (i.e. add a "pay this bill" within the specific bill details page that would bring the user to the payment flow but only for that one specific bill).

  • Users want to be certain that they have paid their bill in full or how much left they need to pay in order to pay their bills in full. At the end of the payment process, it is highly recommended that the interface shows a notification or a message that indicates how much left they have to pay or zero balance if they have already paid their bill in full

Recommendations & Next Steps
Recommendations & Next Steps
  • The user testing portion was originally intended to be a moderated user testing. However, due to unrelated and unforeseen circumstances (one of which was the once-in-a-century freak winter storm in Texas), the original study schedule was pushed back. Later rounds of testing should consider a moderated test in order to gain better insights from users.

  • Although the focus of the participant pool was an even distribution between the age brackets, later rounds of testing should also consider a more even split of participant genders.

  • When providing clickable prototype links through Figma for user testing, whether they are moderated or not, make sure to hide the Figma toolbar as it could get in the way of buttons necessary to complete a task.

This experience has been extremely valuable to me as I got a glimpse of how an in-house UX research team functions in a large organization. Although the collaborative aspect of research was a little limited due to the constraints of the project, I enjoyed the process and getting to know the UX team members.

Limitations & Lessons Learned
Background

Athenahealth provides various cloud-based services for the healthcare industry, including electronic health records (EHR), revenue cycle management & medical billing, and Epocrates and other point-of-care mobile apps.

In this project, I worked with the Consumer Financials team and conducted a usability study for a proposed design update of their QuickPay experience. QuickPay allows patients and organizations to access, view, and pay medical bills using a unique access code from a paper statement received in the mail.

Research Plan

QuickPay has recently undergone a redesign process, and, together with the Consumer Financials team, I conducted a heuristic evaluation and usability study of the redesigned system and user task flow.

I conducted a screen-by-screen heuristic evaluation of prototype with proposed redesign based on Nielsen's 10 Heuristics, and I also utilized Athenahealth's in-house Design Quality Score study in which 4 UX subject matter experts (SMEs) across different product teams within Athenahealth conducted their own heuristic evaluations.

I also conducted an unmoderated usability study using UserTesting. 15 participants were required to complete 4 tasks within QuickPay, and a SUS rating was calculated at the end of each session.

Heuristic Evaluation

A heuristic evaluation serves as a good way to diagnose usability issues while iterating on a design. It can provide a benchmark score on how the design's usability, but it should not be used as the only method in identifying potential usability issues and should ideally be paired with user testing.

Individual Evaluation

Using Nielsen's 10 Heuristics, I conducted a screen-by-screen evaluation of the QuickPay prototype, specifically screens that would later on be a part of the user task flow in the usability testing. For each screen, I identified both good experiences and heuristic violations using the following rating scale:

  • Catastrophe refers to issues that will likely contribute to task failure or abandonment and should be addressed immediately

  • Major problem refers to issues that could lead to task failure or workarounds and are high-priority fixes

  • Minor problem refers to issues that are impediments to efficiency and satisfaction and are low-priority fixes

  • Acceptable/Cosmetic refers to minor flaws that do not impede task success

  • Good experience refers to positive/non-issue observations

Following the evaluation, the scores are averaged based on the number of observations. The higher the score, the better the system is.

athenahealth's Design Quality Score (DQS)

DQS is an in-house research tool for athenahealth. In it's essence, it is a heuristic evaluation conducted by 4 subject matter experts with different backgrounds, namely:

  • UX SME - The designer involved in the project or problem space;

  • Product SME - Someone that understands the product space, the prioritization of development, and key goals of the user product; and

  • General SME - Someone that can be a point person with specific expertise about the problem space.

Like the individual evaluation, the scores are tallied and averaged. The higher the resulting score, the better the system is.

After all evaluations were completed, I compiled the data into a spreadsheet and tagged them based on the severity rating. I also went through every observation and gave each observation an "area of note" tag. This "area of note" tag allows the observations to be grouped based on the following:

  • General - Observations regarding the design as a whole/global design items

  • Payment flow - Observations that specifically calls out the payment flow process

  • Verbiage - Observations related to wording, word choice, jargon, etc.

  • Info hierarchy design - Observations related to presentation of information

  • Layout design - Observations related to screen layouts

  • Prototype design - Observations related to prototype functionality, error, etc.

  • Style design - Observations related to style.

Usability Test

To support the results from the heuristic evaluation and generate user insights, I conducted a usability test to both check the system usability as well as to identify user pain points. The test produced qualitative and quantitative feedback to guide recommendations and further iterations of the system design.

User Tasks & Test Script

For internal user demo of the system, I worked with a member from the Consumer Financials UX team in developing a script and a video of the tasks a hypothetical user may take. Based on this demo script and the overall function of QuickPay, I identified four main tasks that a user may undergo when using the system:

  1. Accessing the portal using the code received in a paper statement

  2. Viewing overall summary of all medical bills

  3. Viewing detail summary of a specific medical bill

  4. Paying the medical bill as due

Test Participant

After discussion with my field supervisor to gain insight on the procedure athenahealth teams would have to take to start and complete an unmoderated usability test, I determined that the usability test would need 15 participants evenly spread across different age ranges.

I chose to focus on the even distribution of the age range as it would allow me to gain insights from a wider audience range as athenahealth's QuickPay is a payment system that can be used by anyone as long as their healthcare provider uses athenahealth's billing system.

Results

After the heuristic evaluation and usability test sessions were completed, I compiled the data gathered into a spreadsheet to better identify common problem areas, including user pain points during the test, most commonly violated heuristic, and most commonly identified problem areas.

Heuristic Evaluation

Combining the results from athenahealth's DQS and the evaluation I completed on my own, there were five evaluators that took part in the process and generated and combined 78 observations. From the 78 observations, 26 are minor issues, 31 are acceptable/cosmetic issues, and 21 are good experiences. There were no catastrophic or major issues observed.

The calculated scores are simply an average of all the scores given for the observations; the higher the value, the better the design quality. At a combined average of 2.94 out of 4.0, this current design is considered well done by athenahealth's standards.

Based on the compiled observations, the most common problem areas were related to Verbiage (9 acceptable/cosmetic, 6 minor issues) and Payment Flow (3 acceptable/cosmetic, 8 minor issues).

Quotation mark

Payment Due vs. Due now: Do they mean different things? Not clear.

Quotation mark

In step 3 in Make a Payment (payment review), in the "show details" breakdown, each bill says "amount paid" but the user hasn't paid that amount yet (they will once they confirm this payment).

Problem area: Verbiage

Multiple evaluators noted that some of the verbiage used in the redesign could be confusing or potentially be misleading.

Quotation mark

As a user, I want to just go and pay that bill from the bill breakdown page once I look at the details. But I'm forced to go back to the first page to "make a payment"

Quotation mark

The end of the payment step felt a bit like a dead end -- maybe there is a CTA to return to the homepage or something?

Problem area: Payment Flow

At certain steps in the payment process, some evaluators noted that there seems to be a lack of or limited user freedom in choosing how to approach and complete payment.

❤️ 🐻| Ayu Larasati