Designing with Data

Improving a Travel Product with an A/B Test

Project Details

My Role: Senior Designer
Year: 2015
Tools: Sketch, HTML & CSS

About the Project

In 2015, I joined’s Travel Guides team to look at the product with fresh eyes. 

This was exciting because achieving statistical significance on A/B tests post-booking is difficult. Doing so requires large improvements that shift customer behavior in a major way.

To find the area of greatest opportunity, I analyzed existing qualitative research, Google Analytics data, and the overall design.

I gravitated toward the transport section because—according to Google Analytics data—it was the most visited content. From experience, I knew travelers often book properties based on how easy they are to get to—especially for city trips.

For these reasons, I knew the transport info was a big opportunity.


I analyzed the product through the lens of of Human Computer Interaction (HCI) and usability best practices.

Potential Problems → Proposed Solutions

First, I identified some potential problems.

Then, I pivoted the list of problems into a list of solutions.

The problems were:

  • Ambiguous title
  • Weak color contrast
  • Multiple instances of H1 tags
  • Partially-hidden content
  • Missing useful information
  • Slow page load time
  • Poor information architecture
  • Slow-to-scan layout
  • Weak typographical hierarchy
  • Insufficient line length

Some solutions could be:

  • Create descriptive titles
  • Increase color contrast
  • Improve HTML semantics
  • Remove JavaScript
  • Add directions to property
  • Mobile-optimize images
  • Create content groupings
  • Use one-column layout
  • Strengthen hierarchy
  • Improve line-length


Once I understood why customers gravitated to the content and what the problems were, I defined the customer problem.

The problem to solve:

“The transport information is difficult to use and lacks crucial information.”

The customer problem became the foundation for my hypothesis:

“The transport info provides valuable customer reassurance.

However, it’s hard to use and understand, which prevents customers from engaging with it.

Therefore, customers don’t feel reassured, bounce from the content, and cancel their booking.

If the transport info’s ease-of-use and content quality are improved, customers will feel reassured, won’t bounce, and will not cancel their booking.

We will know this is true when we observe a decrease in bounces and a reduction in booking cancellations.”


The aesthetics simply needed to enhance the product’s ease-of-use.

So, I visualized solutions to my list of problems.

I focused on:

  • Color contrast
  • Line length
  • Hierarchy
  • Scannability
  • Iconography
  • Interaction

The Test

To ensure the product was better and not just different, I validated the changes with an A/B test.

50% of traffic saw “A” (base)

50% of traffic saw “B” (variant)

The Result

The result validated my hypothesis and uncovered additional positive benefits for both customers and the business.

Primary Metrics

  • Reduced cancellations
  • Decreased bounce rate

Secondary Metrics

  • Decreased CS inbound
  • Decreased page load time
  • Increased customer loyalty

To be sure the secondary metrics results were real, I got a second opinion from our data science team. They confirmed the results were real.

Cue high-fives.

Final Product

Because there was strong evidence the new design was better than the old, the redesign went “full on”.

Thank you to…

Phil Hammel for being my go-to person for design feedback.
Lukas Vermeer for being my go-to guy for data-related questions.