Simplii Superbowl Offer Usability Test
Simplii Superbowl Offer Usability Test
Evaluating Simplii's Superbowl offer redemption page.
Evaluating Simplii's Superbowl offer redemption page.
Date: December 2022
Role: User Experience Researcher
Methodology: Moderated usability test with qualitative coding analysis.
Research Goal: Evaluate the usability of the offer page's copy and layout to determine reasons for client dropoff.
Process:
Project Onboarding
Study Planning and Conducting
Study Analysis
Results
Next Steps
Date: December 2022
Role: User Experience Researcher
Methodology: Moderated usability test with qualitative coding analysis.
Research Goal: Evaluate the usability of the offer page's copy and layout to determine reasons for client dropoff.
Process:
Project Onboarding
Study Planning and Conducting
Study Analysis
Results
Next Steps
Simplii Superbowl Offer
Simplii Superbowl Offer
01 Project Onboarding
Working alongside Simplii's Senior UX Researcher, we conducted an intake meeting with the bank's Sales and Offers team to determine:
The reason the Sales and Offers team requested a research study
Their goals for their campaign
Their expectations for the study
The business objectives the study results would support
Through this meeting, we discovered that the team was encountering an obstacle with one of their newest campaigns; a Superbowl campaign that encouraged clients to apply for a credit card with the bank.
After the offer's release, there was lower than expected engagement from clients, and the Sales and Offers team were hoping to gain insight into potential causes. The team hypothesized that there were either usability issues with the offer layout or confusion arising from instructive wording used.
The team was hoping to be informed of specific areas that could be causing the issue in order to make edits and increase client redemption of the offer.
01 Project Onboarding
Working alongside Simplii's Senior UX Researcher, we conducted an intake meeting with the bank's Sales and Offers team to determine:
The reason the Sales and Offers team requested a research study
Their goals for their campaign
Their expectations for the study
The business objectives the study results would support
Through this meeting, we discovered that the team was encountering an obstacle with one of their newest campaigns; a Superbowl campaign that encouraged clients to apply for a credit card with the bank.
After the offer's release, there was lower than expected engagement from clients, and the Sales and Offers team were hoping to gain insight into potential causes. The team hypothesized that there were either usability issues with the offer layout or confusion arising from instructive wording used.
The team was hoping to be informed of specific areas that could be causing the issue in order to make edits and increase client redemption of the offer.
Beginning of the Rates & Fees information on the offer page. More information is accessible by scrolling inside the container.
During the intake meeting, we also wanted to understand the Sales and Offers team's existing questions and assumptions about what may be causing the issue. The following questions guided our study planning:
What do clients expect to happen when they click "Remind me later"?
Is the large amount of consent text scary to users?
Would clients only want to see this on desktop due to less scrolling? Or is it better on mobile since it's easier to click submit on their phone?
Is the offer page too long?
Does the offer have too much text?
Can we do anything to improve overall experience rather than focusing on one specific thing?
Does the offer flow make sense? Should the sections be rearranged, like having the rates and fees higher up?
Beginning of the Rates & Fees information on the offer page. More information is accessible by scrolling inside the container.
During the intake meeting, we also wanted to understand the Sales and Offers team's existing questions and assumptions about what may be causing the issue. The following questions guided our study planning:
What do clients expect to happen when they click "Remind me later"?
Is the large amount of consent text scary to users?
Would clients only want to see this on desktop due to less scrolling? Or is it better on mobile since it's easier to click submit on their phone?
Is the offer page too long?
Does the offer have too much text?
Can we do anything to improve overall experience rather than focusing on one specific thing?
Does the offer flow make sense? Should the sections be rearranged, like having the rates and fees higher up?
02 Study Planning and Conducting
With the Sales and Offers team's goals, questions, and assumptions in mind, I assisted the Senior UX Researcher in planning a moderated usability study utilizing the think-aloud protocol. We created a discussion guide using Microsoft Excel, and I built an interactive prototype of the offer page in Figma.
Our discussion guide examined the following themes:
Text and copy pain points
Usability pain points
User Expectations of outcomes of the "remind me later", "submit", and "no thanks" buttons.
User suggestions
We then created a study on User Interviews and recruited 8 participants who had gotten a credit card in Canada in the past 90 days.
I moderated 3 interviews, and was present to take notes for the remaining sessions. In addition, we invited team members from the Sales and Offers and Design teams to sit in on the interviews, providing an opportunity for them to see clients provide feedback themselves and to provide the opportunity for them to ask their own questions.
03 Study Analysis
After conducting the interviews, I worked on analyzing the results from the qualitative notes taken.
Using deductive qualitative coding, I tagged the results into themes directly based around the original questions and goals from the Sales and Offers team. The tag groups and individual tags used included:
Copy
Pain Points
Suggestions
Wins
UX
Pain Points
Suggestions
Wins
UI
Pain Points
Suggestions
Wins
Expectations
"Remind me later" button
"No thanks" button
"Get started" button
After tagging all notes taken, I created summaries for findings for each tag group in order to visualize and compare each participant's thoughts on each theme and summarize each actionable or relevant point.
04 Results
When compiling my report for the study, I broke down the information into the following sections:
Results
The most significant and actionable takeaways from the study centred primarily around the copy the offer contained rather than the usability of the page.
The most recurring theme brought up by participants surrounded the lack of information on the card itself. Clients indicated that they were interested in knowing about the annual fees, additional card holder options, card benefits and requirements, and why the card stands out.
Clients also remarked that there was an overwhelming amount of text, and that the Terms and Conditions were long.
Some language was confusing to the client audience, such as the term "pre-approval". Some participants mentioned they didn't know what that would entail, and wouldn't want to pursue an offer they didn't understand.
There was little information on the odds of winning the Superbowl prize and how the guaranteed $150 cash offer would work.
These pain points acted as blockers to adoption, as clients prioritized the long-term guaranteed outcomes of signing up over the potential chance at winning a contest.
These were indicated as suggestions from the UX Research team as an area to examine for improvement. As a result, the text on the page was edited in the updated release.
Superbowl Offer Information—Mobile, no high-level highlights of the card itself or contest details are included.
Options for client next steps: "Get started" would begin the application, "Remind me later" would trigger the popup to return next time, and "No thanks" would dismiss the offer.
Questions Revisited
I also included a section in the final report directly answering the questions established in the Project Onboarding Phase. The key findings related to these questions included:
Clients assumed they would receive an email if they selected "remind me later", which was inconsistent with the actual behaviour of the button.
Participants did not find the consent text scary.
Participants found it more convenient to view the offer on mobile, and did not find it required too much scrolling.
Participants did not find the page too long, contrary to the original hypothesis.
Options for client next steps: "Get started" would begin the application, "Remind me later" would trigger the popup to return next time, and "No thanks" would dismiss the offer.
Questions Revisited
I also included a section in the final report directly answering the questions established in the Project Onboarding Phase. The key findings related to these questions included:
Clients assumed they would receive an email if they selected "remind me later", which was inconsistent with the actual behaviour of the button.
Participants did not find the consent text scary.
Participants found it more convenient to view the offer on mobile, and did not find it required too much scrolling.
Participants did not find the page too long, contrary to the original hypothesis.
05 Next Steps
After summarizing the results of the study, we compiled recommendations for next steps for the Sales and Offers team to consider.
Refocus the emphasis on the offer itself rather than the contest, as clients are more motivated by guaranteed offers.
Add clear next steps clients should expect after accepting the offer in order to ease discomfort clients may experience due to lack of information.
Ensure all terms are easy to understand by an average client. For example, many participants did not know what pre-approval meant.
We then scheduled a discussion with the Sales and Offers team to share the study findings and suggestions. We discussed the pain points found, how the copy and lack of product information was likely acting as a deterrent for clients (and the reasons behind why they were acting as deterrents), and specific areas that could be edited to assist in improving adoption.
After the Sales and Offers team released their edited offer page, adoption rates made a slight increase until the end of the offer period. The offer campaign was also used as a successful case study by Braze, the platform used to build the offer page and host the campaign on Simplii's website and mobile app. To learn more about the campaign, Braze's case study can be found here.
Want to see more projects?
DFO Design System UX Studies
Simplii Offers Page Usability Test
All Projects
Want to see more projects?
Simplii Superbowl Offer Usability Test
Evaluating Simplii's Superbowl offer redemption page.
Date: December 2022
Role: User Experience Researcher
Methodology: Moderated usability test with qualitative coding analysis.
Research Goal: Evaluate the usability of the offer page's copy and layout to determine reasons for client dropoff.
Process:
Project Onboarding
Study Planning and Conducting
Study Analysis
Results
Next Steps
Simplii Superbowl Offer Usability Test
Evaluating Simplii's Superbowl offer redemption page.
Date: December 2022
Role: User Experience Researcher
Methodology: Moderated usability test with qualitative coding analysis.
Research Goal: Evaluate the usability of the offer page's copy and layout to determine reasons for client dropoff.
Process:
Project Onboarding
Study Planning and Conducting
Study Analysis
Results
Next Steps
Simplii Superbowl Offer Usability Test
Evaluating Simplii's Superbowl offer redemption page.
Date: December 2022
Role: User Experience Researcher
Methodology: Moderated usability test with qualitative coding analysis.
Research Goal: Evaluate the usability of the offer page's copy and layout to determine reasons for client dropoff.
Process:
Project Onboarding
Study Planning and Conducting
Study Analysis
Results
Next Steps
01 Project Onboarding
Working alongside Simplii's Senior UX Researcher, we conducted an intake meeting with the bank's Sales and Offers team to determine:
The reason the Sales and Offers team requested a research study
Their goals for their campaign
Their expectations for the study
The business objectives the study results would support
Through this meeting, we discovered that the team was encountering an obstacle with one of their newest campaigns; a Superbowl campaign that encouraged clients to apply for a credit card with the bank.
After the offer's release, there was lower than expected engagement from clients, and the Sales and Offers team were hoping to gain insight into potential causes. The team hypothesized that there were either usability issues with the offer layout or confusion arising from instructive wording used.
The team was hoping to be informed of specific areas that could be causing the issue in order to make edits and increase client redemption of the offer.
During the intake meeting, we also wanted to understand the Sales and Offers team's existing questions and assumptions about what may be causing the issue. The following questions guided our study planning:
What do clients expect to happen when they click "Remind me later"?
Is the large amount of consent text scary to users?
Would clients only want to see this on desktop due to less scrolling? Or is it better on mobile since it's easier to click submit on their phone?
Is the offer page too long?
Does the offer have too much text?
Can we do anything to improve overall experience rather than focusing on one specific thing?
Does the offer flow make sense? Should the sections be rearranged, like having the rates and fees higher up?
Beginning of the Rates & Fees information on the offer page. More information is accessible by scrolling inside the container.
01 Project Onboarding
Working alongside Simplii's Senior UX Researcher, we conducted an intake meeting with the bank's Sales and Offers team to determine:
The reason the Sales and Offers team requested a research study
Their goals for their campaign
Their expectations for the study
The business objectives the study results would support
Through this meeting, we discovered that the team was encountering an obstacle with one of their newest campaigns; a Superbowl campaign that encouraged clients to apply for a credit card with the bank.
After the offer's release, there was lower than expected engagement from clients, and the Sales and Offers team were hoping to gain insight into potential causes. The team hypothesized that there were either usability issues with the offer layout or confusion arising from instructive wording used.
The team was hoping to be informed of specific areas that could be causing the issue in order to make edits and increase client redemption of the offer.
During the intake meeting, we also wanted to understand the Sales and Offers team's existing questions and assumptions about what may be causing the issue. The following questions guided our study planning:
What do clients expect to happen when they click "Remind me later"?
Is the large amount of consent text scary to users?
Would clients only want to see this on desktop due to less scrolling? Or is it better on mobile since it's easier to click submit on their phone?
Is the offer page too long?
Does the offer have too much text?
Can we do anything to improve overall experience rather than focusing on one specific thing?
Does the offer flow make sense? Should the sections be rearranged, like having the rates and fees higher up?
Beginning of the Rates & Fees information on the offer page. More information is accessible by scrolling inside the container.
01 Project Onboarding
Working alongside Simplii's Senior UX Researcher, we conducted an intake meeting with the bank's Sales and Offers team to determine:
The reason the Sales and Offers team requested a research study
Their goals for their campaign
Their expectations for the study
The business objectives the study results would support
Through this meeting, we discovered that the team was encountering an obstacle with one of their newest campaigns; a Superbowl campaign that encouraged clients to apply for a credit card with the bank.
After the offer's release, there was lower than expected engagement from clients, and the Sales and Offers team were hoping to gain insight into potential causes. The team hypothesized that there were either usability issues with the offer layout or confusion arising from instructive wording used.
The team was hoping to be informed of specific areas that could be causing the issue in order to make edits and increase client redemption of the offer.
During the intake meeting, we also wanted to understand the Sales and Offers team's existing questions and assumptions about what may be causing the issue. The following questions guided our study planning:
What do clients expect to happen when they click "Remind me later"?
Is the large amount of consent text scary to users?
Would clients only want to see this on desktop due to less scrolling? Or is it better on mobile since it's easier to click submit on their phone?
Is the offer page too long?
Does the offer have too much text?
Can we do anything to improve overall experience rather than focusing on one specific thing?
Does the offer flow make sense? Should the sections be rearranged, like having the rates and fees higher up?
Beginning of the Rates & Fees information on the offer page. More information is accessible by scrolling inside the container.
02 Study Planning and Conducting
With the Sales and Offers team's goals, questions, and assumptions in mind, I assisted the Senior UX Researcher in planning a moderated usability study utilizing the think-aloud protocol. We created a discussion guide using Microsoft Excel, and I built an interactive prototype of the offer page in Figma.
Our discussion guide examined the following themes:
Text and copy pain points
Usability pain points
User Expectations of outcomes of the "remind me later", "submit", and "no thanks" buttons.
User suggestions
We then created a study on User Interviews and recruited 8 participants who had gotten a credit card in Canada in the past 90 days.
I moderated 3 interviews, and was present to take notes for the remaining sessions. In addition, we invited team members from the Sales and Offers and Design teams to sit in on the interviews, providing an opportunity for them to see clients provide feedback themselves and to provide the opportunity for them to ask their own questions.
03 Study Analysis
After conducting the interviews, I worked on analyzing the results from the qualitative notes taken.
Using deductive qualitative coding, I tagged the results into themes directly based around the original questions and goals from the Sales and Offers team. The tag groups and individual tags used included:
Copy
Pain Points
Suggestions
UX
Pain Points
Suggestions
UI
Pain Points
Suggestions
Expectations
"Remind me later" button
"No thanks" button
"Submit" button
After tagging all notes taken, I created "Highlights" in Dovetail for each tag group in order to visualize and compare each participant's thoughts on each theme and summarize each actionable or relevant point.
After conducting the interviews, I worked on analyzing the results from the qualitative notes taken.
Using deductive qualitative coding, I tagged the results into themes directly based around the original questions and goals from the Sales and Offers team. The tag groups and individual tags used included:
Copy
Pain Points
Suggestions
UX
Pain Points
Suggestions
UI
Pain Points
Suggestions
Expectations
"Remind me later" button
"No thanks" button
"Submit" button
After tagging all notes taken, I created summaries for findings for each tag group in order to visualize and compare each participant's thoughts on each theme and summarize each actionable or relevant point.
After conducting the interviews, I worked on analyzing the results from the qualitative notes taken.
Using deductive qualitative coding, I tagged the results into themes directly based around the original questions and goals from the Sales and Offers team. The tag groups and individual tags used included:
Copy
Pain Points
Suggestions
UX
Pain Points
Suggestions
UI
Pain Points
Suggestions
Expectations
"Remind me later" button
"No thanks" button
"Submit" button
After tagging all notes taken, I created summaries for findings in order to visualize and compare each participant's thoughts on each theme and summarize each actionable or relevant point.
04 Results
When compiling my report for the study, I broke down the information into the following sections:
Results
The most significant and actionable takeaways from the study centred primarily around the copy the offer contained rather than the usability of the page.
The most recurring theme brought up by participants surrounded the lack of information on the card itself. Clients indicated that they were interested in knowing about the annual fees, additional card holder options, card benefits and requirements, and why the card stands out.
Clients also remarked that there was an overwhelming amount of text, and that the Terms and Conditions were long.
Some language was confusing to the client audience, such as the term "pre-approval". Some participants mentioned they didn't know what that would entail, and wouldn't want to pursue an offer they didn't understand.
There was little information on the odds of winning the Superbowl prize and how the guaranteed $150 cash offer would work.
These pain points acted as blockers to adoption, as clients prioritized the long-term guaranteed outcomes of signing up over the potential chance at winning a contest.
These were indicated as suggestions from the UX Research team as an area to examine for improvement. As a result, the text on the page was edited in the updated release.
Superbowl Offer Information—Mobile, no high-level highlights of the card itself or contest details are included.
Options for client next steps: "Get started" would begin the application, "Remind me later" would trigger the popup to return next time, and "No thanks" would dismiss the offer.
Questions Revisited
I also included a section in the final report directly answering the questions established in the Project Onboarding Phase. The key findings related to these questions included:
Clients assumed they would receive an email if they selected "remind me later", which was inconsistent with the actual behaviour of the button.
Participants did not find the consent text scary.
Participants found it more convenient to view the offer on mobile, and did not find it required too much scrolling.
Participants did not find the page too long, contrary to the original hypothesis.
05 Next Steps
After summarizing the results of the study, we compiled recommendations for next steps for the Sales and Offers team to consider.
Refocus the emphasis on the offer itself rather than the contest, as clients are more motivated by guaranteed offers.
Add clear next steps clients should expect after accepting the offer in order to ease discomfort clients may experience due to lack of information.
Ensure all terms are easy to understand by an average client. For example, many participants did not know what pre-approval meant.
We then scheduled a discussion with the Sales and Offers team to share the study findings and suggestions. We discussed the pain points found, how the copy and lack of product information was likely acting as a deterrent for clients (and the reasons behind why they were acting as deterrents), and specific areas that could be edited to assist in improving adoption.
After the Sales and Offers team released their edited offer page, adoption rates made a slight increase until the end of the offer period. The offer campaign was also used as a successful case study by Braze, the platform used to build the offer page and host the campaign on Simplii's website and mobile app. To learn more about the campaign, Braze's case study can be found here.