UX Case Study: A New Purchase Experience for Course Hero
Timeline Oct 2022 - Jan 2023
Role Product Designer II, partnered with Lead Design
Skills UX-UI design iterations, responsive design, prototyping, usability research, A/B testing
Role Product Designer II, partnered with Lead Design
Skills UX-UI design iterations, responsive design, prototyping, usability research, A/B testing
Overview
Course Hero provides study help to high school and college students through its document library, access to tutors, and other content for premium subscribing members. It also offers earn-as-you-go access through a basic membership when students add their own study documents to the Course Hero content library.
Students commonly discover Course Hero by searching the web for questions or keywords related to their course assignments, and landing on a document preview in the Course Hero library. To see the rest of the document, students must register and either pay to subscribe or upload their own study docs to gain access.
The Problem
Course Hero offers a Premier paid subscription plan, with monthly, quarterly, or annual options for subscription duration. The problem with the plan purchase experience is that it would be difficult to scale to include new product offerings. Additionally, the visual style needed to reflect new branding standards.
Prior research showed users found the amount of information on the page to be overwhelming, and they were often unclear on how the features differed for each option. Because of how they were presented for comparison, many students were disappointed to find out all of the options were charged by the month, as the pricing comparison suggested.
How might we improve the plan selection and purchase experience and provide additional value to students in a way that is transparent, builds trust, and leads to long term retention?
(Above) Legacy plan selection and purchase page
Legacy purchase form
Design Strategy
Since the vision for the new purchase flow was such a radical change from the original, and the monetization funnel was very expensive to test, the changes were divided into incremental phases in order to ensure any negative impacts were understood & accounted for before proceeding. The strategy was to implement the changes in 3 phases:
Phase 1: replace the purchase form in the legacy page
Phase 2: separate the flow into 2 steps (Plan selection, and Checkout)
Phase 3: add the new product to the Plan selection page
Phase 1: New Purchase Form
In the first phase we introduced a new purchase form. We tested 2 designs against the original:
Design 1
For this design, I simply updated the UI style and some legal copy, keeping the dropdown selection in the form.
For this design, I simply updated the UI style and some legal copy, keeping the dropdown selection in the form.
Design 2
For the 2nd, I clarified the total price (calculating a discounted base price multiplied by 12 months) just above the call to action, and eliminated the dropdown field. Selecting a different plan duration above (annual, quarterly, or monthly) would simply cause the total to update.
For the 2nd, I clarified the total price (calculating a discounted base price multiplied by 12 months) just above the call to action, and eliminated the dropdown field. Selecting a different plan duration above (annual, quarterly, or monthly) would simply cause the total to update.
Phase 2: Plan Selection & Checkout steps
This phase was concerned with testing separate plan selection and checkout screens against a checkout-only page.
The lead designer and I made the decision to separate the functions of choosing a plan and selecting a billing cycle checkout page. There were several reasons for this: having the plan selection as a first step allowed for more room to compare the features between the paid and unpaid plan, and the extra space was necessary to add another product.
Once a user decided to continue with a Premier plan, the next screen would show 3 options for the plan duration. Users could also see a short list of the features in the plan, with specific limits noted for the different subscription lengths.
We tested 2 different layouts for the Plan page: a table that compared the Premier with Basic, and a stacked card, showing the Premier features as a list, with a subtle callout at the bottom directing users to share their study content as another way to gain access to the platform.
Design 1: Feature Table
Design 2: Card
Checkout page
Our plan was to add the new product to each of the two layouts, and also test a
checkout-only page.
checkout-only page.
Testing the plan page for mobile web
I also created mobile versions to test for each the 3 design options.
The first design, containing a table, presented some challenges in presenting all of the important information on a smaller screen. I considered whether to make the page responsive or adaptive.
• The limited space made it challenging to get enough information on the screen at once
• It also made it difficult to ensure large enough touch targets
• Combining horizontal swiping, vertical scrolling, and touch targets was challenging
• Too many table interactions made it difficult to ensure that the screen was accessible
• It also made it difficult to ensure large enough touch targets
• Combining horizontal swiping, vertical scrolling, and touch targets was challenging
• Too many table interactions made it difficult to ensure that the screen was accessible
Together, we decided to conduct usability testing for the first design option for mobile web, to find out which of these were the easiest for the users to understand and easily compare the features between the Premier and Basic plans. To conserve space on the small screen and allow for users to focus on the comparisons, I chose to include an information popup window to list out the feature details for all of the mobile web options.
Design 1: Table
(Above) For Design 1: 3 different treatments were tested for the mobile web versions of the table.
(Above) Early iteration of table on mobile web with toggle button for Phase 2
(Above) Early iteration of table variation on mobile web in a list format for Phase 2
Design 2: Cards
(Above) For Design 2, we tested a subtle callout to try uploading vs. a comparison card for Basic.
Insights
Usability testing feedback showed:
Of the 2 designs, the small table and stacked cards showing the basic option resonated most with users and were the easiest for them to understand the availability of and comparison between the paid and non-paid options
Not everyone noticed the small text “Share your docs to earn access” on the Plan page, which led some to believe that Basic was simply free.
The features popup information were discoverable thanks to the info icon, but the design in the list format (fig. 1a above) were discovered more quickly than in the other designs.
The product pricing did not match user’s mental model of what subscription tiers should look like. Users expected Premier options to be monthly plans starting from $9.95 per month with different feature limits for unlocks and tutor questions. Many got upset upon learning $9.95 per month is an annual subscription and considered it to be dishonest pricing.
Research pointed out that users were confused by the way the limits changed for each plan, likely because:
1) the limits didn’t increase in proportion to the plan duration
2) feature limits weren’t all visible at once to aid comparison
Insights from the Phase 2 mobile web usability research informed us how to test a multi-product comparison table on mobile web:
• Use a toggle button to show the 3 items
• Make the text "Share your docs" be more informative
• Use a carousel animation to list feature limits
• Add more visual emphasis to the feature popup link
• Make the text "Share your docs" be more informative
• Use a carousel animation to list feature limits
• Add more visual emphasis to the feature popup link
(Above) Bundled plan in mobile list table with toggle button for Phase 3.
Phase 2 iteration of list table with toggle button, chosen for phasing in the bundled version for Phase 3.
Based on the recommendations of our UX researcher, we made several design decisions from the usability test:
We decided to include the Basic tier as a plan in the comparison table so users could clearly see the difference in feature limits.
We made the decision to test a card comparison of the new product with the Premier plan, with less emphasis on the Basic plan, to see if this was enough information for users to understand all of their options and make an informed decision.
And we also decided to test a single page version with a toggle switch to show all features and adjusted pricing on the same page, to see if the features and pricing were compelling enough to influence a purchase
Our business stakeholders did not want to emphasize the basic option too much, for fear of converting too many paying customers to becoming content-contributors. However our UX researcher recommended including a list of Basic features to make comparing easier for the user.
Phase 3: Adding New Product Bundle
Due to variations of the plan lengths (monthly, quarterly, and yearly), it remained challenging to show the prices when introducing the plans. Showing the differences in limitations of features also complicated the messaging somewhat. Those options were also explored, as well as allowing users to select the subscription plan length at the same time as the plan in a checkout-only version.
Below: #1 Table with bundle, for Phase 3.
Below: #2 Cards with bundle
Below, #3: Checkout-only single page design
At this stage, we had a good idea for how to present designs #1 and #2 for testing in mobile web. The checkout-only experience, on the other hand, was getting more complicated with the addition of a third product with a significantly longer feature list, and agreed to do another round of usability testing just on that page, to narrow our focus.
Outcome
Test results for Phase 2 showed that the first two Plan page options did not do as well as the 3rd single-page selection & checkout flow.
Option 1 was a loss, and option 2 did not do as well as option 3. The 3rd option saw a 7% CVR increase, which was countered by a 30% drop in students choosing to upload content, resulting in a net-net 3% CVR increase.
Our conclusion was that the results may have been different had we included the pricing on the plan pages.
I learned much from this experience that pushed me to think about how different strategies or more research could have streamlined this exercise and moved the designs forward in a way that best met both the business and customer's needs.
Final winner for Phase 2: Checkout Only
The winning experience for Phase 2 was a straight-to-checkout payment screen, with a separate page explaining the separate product benefits. The bundled plan was not included in this experience.
Final winner for Phase 3: Checkout with bundled plan
The final experience for Phase 3, with the bundled "Hero Trio" plan, selectable via a toggle switch.
Learnings
Keep end goal in sight
Although it was important to iterate from our starting point, I believe it would have saved some time and effort if we started out exploring designs with the new product included to test what worked well with the new bundled plan, since that was our goal, then remove the new product and A/B testing more narrowly focused options for Phase 2.
(1) Start concept work and usability testing with the new product included in designs. Learning early how best to compare and present a third product option, especially on mobile screens, would have eliminated some of the explorations that only work best for comparing 2 products, as the phase 2 testing explored.
(2) Include pricing tests outcome in concept work. At the start of this initiative, pricing tests were underway, however we had no way to know what they would be. Not having pricing to refer to in usability tests may affect participants perception of value.
Build the right thing, build the thing right.
Carving out an interim phase to get the pricing and content right may have saved time on the layouts and messaging in the 3rd phase. Since the new packaged product did not offer a quarterly subscription, we risked losing customers when they would find out that it was not an option after coming to the purchase page. A/B test results showed the design going straight to checkout (the only one that immediately showed the price) suggested that knowing the pricing up front was an equally important consideration as knowing features in making the decision to move forward to the purchase stage.
Be transparent, and balance business needs with user needs
Changing the pricing structure to match the user's mental model to that of increasing features for an increased monthly rate would have made it much easier to include pricing up front, something the users indicated that they wanted. By being up-front with the pricing comparisons (showing the prices in full instead of comparing them as if they were all monthly) would have likely caused a temporary drop in CVR, but the increased trust would help elevate the brand and create more loyalty over the long term.
