ScreenRecording2024-04-07at7.33.51PM-ezgif.com-video-to-gif-converter.gif

Introduction

Enhancing user retention and navigation with eye-tracking research on Smarthistory mobile

Smarthistory, partner of Khan Academy, is a non-profit providing art history resources to students and instructors.

I conducted on-site user research and redesigned SmartHistory’s mobile site to improve navigation and accessibility within its pages and menu items, increasing retention past the homepage from 200+ monthly visitors.

Skills

Web analytics

A/B testing

Eye tracking

Information architecture

WCAG accessibility

Outcome

An updated mobile site with a reordered navigation menu, accessible text and spacing, and visual guides for reading through long history content.

Slide deck

Figma prototype

Design challenge

How can we redesign Smarthistory so that students, teachers, and art history can easily find information?

Project overview

Client's problem

My approach

Smarthistory is a nonprofit institution providing art history resources for students and teachers of AP programs and introductory college courses. This is a UX consulting project with a Smarthistory client where I self-directed a combination of UX research methods on their site to identify usability problems and design feasible recommendations of solutions using Figma.

The client noticed decreasing trends on mobile user engagement on Smarthistory’s homepage and landing pages through Google Analytics and Hotjar, but they were unable to identify specific reasons.

Smarthistory also wants to organize their content in a more flexible and robust manner to optimize visibility for all global content, as most of their content centers around the Americas and Europe.

I started the project by consulting with the client on their perceived pain points with their website and the final deliverables they were expecting. Afterwards, I ran web analytics on their Google Analytics and Hotjar data to inform the focus of my research and design. I proceeded to conduct eye-tracking tests with Tobii Pro software to evaluate how users navigated in real time. With this information, I crafted initial designs on Figma and used A/B tests to validate my final designs.

Key recommendation

Add introductory guiding content to the homepage and content pages

New homepage with introduction and visual previews grouped by art history topic

Table of contents shortcut for long content pages

Adding an introductory paragraph to the homepage would help users learn more about Smarthistory and its offerings. Similarly, a table of contents with key points at the top of content pages would allow users to overview the page's structure and skip to desired sections.

Informed by research

88% of users felt overwhelmed by the text length on the screen and expected to see a summary of key points at the top of the page, leading to extensive scrolling and missing key information.

Key recommendation

Relabel the hamburger menu and group into new sections

Grouping subsections by collapsible world regions and time periods within the hamburger menu creates shorter lists that reflect the categories users expect to see, rather than mixing all contents in a long list.

Additionally, distinguish course-specific content under “Course Preparation” section as not all visitors are in specific course programs.

Informed by research

Participants misinterpreted links due to a lack of clarity in wording. 50% of participants mistakenly believed “Start here” would redirect them to introductory guides, when these were actually under “Beginners’ guides”.

Key recommendation

Refine menus to meet accessibility standards

Redesigned menu with larger padding

WCAG 2.1 AA level states that the minimum accessible button height is 24x24 pixels, and 44x44px for the AAA level. Increasing all touch targets to at least 24 x 24 px will reduce missed taps, especially for navigation menus and buttons.

Informed by research

Sufficient padding is crucial for mobile interfaces as 60% of test participants faced difficulties with responsiveness after tapping on narrow padding.

Key recommendation

Move the original footer menu to the top

The navigation bar is more discoverable and remains pinned to the header. It includes a hierarchical breadcrumb to the current article.

Increases in page views and session duration will indicate if the new menu is successful.

Informed by research

In A/B tests, moving the menu to the top reduced task failure rates from 60% to 25%. The footer menu highlights important times and sections that are difficult to identify in text-heavy pages.

Research overview

Since I was given access to Smarthistory's data on Google Analytics and Hotjar, I started by observing current behaviors on the Smarthistory website and performing web analytics to track the performance of each page. However, while this evaluation narrowed the scope of my research to pinpointing which pages had lower engagement, I could only extract quantitative research metrics and I needed to dive deeper into the more qualitative behaviors explaining the reasons behind low engagement. I therefore followed up with eye-tracking usability tests to have real-time observations on how and why was the placement of content driving users to quickly disengage and exit Smarthistory pages.

Web analytics with Google Analytics

Given data from Google Analytics between February 2022 to February 2023, I reviewed metrics on Smarthistory’s site to better understand how users were accessing and discovering pages.

Screen Shot 2024-04-10 at 4.27.07 PM.png

30% of visitors accessed Smarthistory via mobile devices, yet mobile visitors had the highest bounce rate at 80.25% compared to 52.29% for desktop.

This means that most mobile visitors exit Smarthistory after visiting just one page without interacting further!

Most users arrive via search engines, direct links, and referral sites, but 81.5% of users visit only 1 page and 81.5% visit for less than 10 seconds.

Need to prioritize redesigning Smarthistory's mobile site

With this, I decided to redesign Smarthistory’s mobile site. In low-income households, a larger percentage of students will only own mobile devices rather than desktop computers due to affordability (U.S. Census Bureau).

Behavioral analytics with Hotjar

I rewatched mobile screen recordings and activity heatmaps to understand how users were interacting with the pages and determine where clicks and engagement could be optimized.

Content pages, including essays, videos, images, and additional resources for artworks, are the most visited pages. However, landing pages, which provide introductions to art history through geography and time periods, are not frequently visited.

Within landing pages, users typically only read the first two sections before skimming through the remaining sections on the list, likely indicating that it’s difficult to discover specific content due to the volume of information.

Screen Shot 2024-04-10 at 4.27.17 PM.png

Screen Shot 2024-04-10 at 4.38.43 PM.png

Then, I decided to use eye-tracking tests to take a further look into how the placement of content was driving users to quickly disengage and exit the Smarthistory mobile site.

Here are the research questions I sought to answer with the eye-tracking tests:

→ What are the main pathways users take to find the information they need?

→ How effectively and intuitively do pages inform users of existing content?

→ What are the characteristics of a typical user visit?

Scenario: You are a student preparing to write a paper on Leonardo da Vinci’s works and his impact.

Task 1: Find where a copy of the Mona Lisa was found.

Task 2: Next, find when da Vinci returned to France.‍

Task 3: Next, find an artistic technique that characterizes European Neoclassicism in the 1700s.

Scenario: You are working on a project about Modern art (before 1980).

Task 4: Within the Modernisms 1900-1980 section, find the name of one influential photographer.

Task 5: Find a definition of Dada Readymades.

I conducted moderated eye-tracking usability tests on 8 participants with an interest in art history or who previously took an introductory art history course.

Following each test, I showed participants replays of their eye movements and asked them about where their eyes fixated on certain points, navigation choices, and interactions with the content.

Finally, I asked participants to complete a post-test questionnaire and a System Usability Scale assessment.

Sequence01-ezgif.com-video-to-gif-converter (3).gif

Findings: Eye-tracking tests and questionnaires

88% of users were confused by the organization and labels of the hamburger menu and content pages

Users reported feeling overwhelmed by the text length on the screen and expected to see a summary of key points at the top of the page. This led to excessive scrolling and missing key pointers to complete their tasks.

Lack of accessible padding and responsiveness in menu items interrupted navigation for 60% of users.

The touch heatmap shows that users frequently missed the hamburger menu due to the lack of padding. By WCAG 2.0, the minimum target should be 44x44 px, while the current target only is 20x20 px.

The height of the navigation menu sections could benefit from more padding to meet the 44px minimum. 5 users were concerned about tapping on the wrong section due to their lack of separation.

75% of users enjoyed the bottom navigation menu but struggled to find it.

While 75% of participants used the bottom navigation menu, only 12.5% (one user) thought it was easy to find.

Heatmap of eye-tracking focus points

6453cda07e3f7cc1471d223c_Visualization3 2.png

Screen Shot 2024-04-10 at 9.00.56 PM.png

menubottom-ezgif.com-video-to-gif-converter.gif

42.5 out of 100 was the average reported System Usability Score of the Smarthistory mobile site.

Participants thought the content was captivating, but the website was overwhelming and confusing.

The most common words from the questionnaire were "informative" and "text-heavy", although one participant mentioned "user-hostile".

6452dcb1efd904ce224305aa_word-cloud 1-p-1600.png

Low-fidelity mockups and annotations

Reordering information architecture in menus and content pages

88% of eye-tracking participants were confused by the organization and labeling of the hamburger menu and landing pages. This indicates that reordering of the information architecture is necessary, both through rewording of sections and adding more page guides to organize content pages better. For instance:

Original menu structure

Proposed menu changes

Screen Shot 2024-04-10 at 10.32.35 PM.png

Original content page scrolling

Proposed table of contents shortcut

Screen Shot 2024-04-10 at 10.40.41 PM.png

A/B testing: determining optimal navigation menu placement

50% of participants preferred moving the footer menu to the top of the page to increase its discoverability.

I decided to conduct A/B testing on 6 new participants to see which menu placement led to more effective navigation. The control version was the original footer placement, tested with 2 participants given previous eye-tracking tests. The larger group of 4 participants performed the tasks with the new menu placement at the header.

Original footer menu

Proposed menu at the header

Screen Shot 2024-04-10 at 10.58.27 PM.png

Participants using the original menu at the footer faced the same problems found from the eye-tracking tests, where they “didn’t realize there was such a helpful menu until they scrolled to the bottom” and felt overwhelmed by the volume of text. Task failure rate was 60%.

Participants using the new navigation menu at the header had a lower task failure rate at 25%. They reported feeling more secure during the test and started by browsing through the navigation menu and scanning key words rather than scrolling through long content pages.

Therefore, the navigation menu should be moved to the top.

Takeaways and opportunities

The client reported a growth of 200+ monthly mobile visitors after June 2023 and a decreased mobile bounce rate from 80% to 59%.

The Smarthistory client received my presentation positively; they were unaware of the current problems with information architecture and accessibility for touch targets on mobile. The team has already made changes, starting from a top navigation menu with ample padding for mobile users.

Given more time to conduct research, I would have included more art history students rather than users with limited backgrounds in art history. Participants felt overwhelmed due to unfamiliar terms crucial to an art history site, affecting their success with tasks, i.e. recognizing art movements.