Delivering Better Content


2017 - 2018


Product Designer



A Brief Introduction

In a cross-functional conversation with our Audience Development and Editorial partners, we were informed our content recommendation API ( was not consistently serving content related to its parent content.

 This meant that related metrics were underperforming as well. Some of that could be due to a lack of iteration which saw teasers stagnate after carryover into the recent AT Media redesign (2017) The bottom line was that it was imperative that we figure out what's happening and how to solve for it.

What Problems Were We Solving?

We believed that iterating on our content recommendation tools and processes would give readers more access to better content. This increased content quality could lead to both an increase in reader engagement and a net increase in ad impressions - which could help to positively affect revenue. By implementing and testing data-driven recommendation APIs, we thought we could deliver better content to our readers, more consistently.

Beginning with Data

We already used for editorial analytics, so pulling recommendations from their API was a relatively low hurdle. In addition to, we planned to test LiftIgniter, a machine learning API for making recommendations. We had already been using LiftIgniter for a limited number of users on an adjacent feature and saw an upward trend in related metrics. The busy work of defining goals and KPIs came next.

Editorial Goal: Deliver better content recommendations on article pages.
Metrics: Time on Site and Pageviews.

Business Goal: Raise viewability numbers for advertising impressions.
Metrics: Viewability, Impressions and CTR for ads in DFP.

User Goals: Make it easier and more fun to read AT Media properties by making relevant content easier to find.
Metrics: Social sharing and Unique sessions.

The Work

Our cross-functional team came together to define a project scope. What came out of that were a few ideas on how we could make this engaging and fun for readers to navigate the three categories of content we want to deliver. Some questions started to crop up:

  • Could we make it feel magical?
  • A choose your own adventure game?
  • A survey of some sort?
  • A simple tabbed solution?
A sketch for a custom content experience with yes no questions
Idea 1: Choose Your Own Adventure!
A sketch for a custom content experience with personal preference options
Idea 2: Choose Another Adventure!
A sketch for a custom content experience with user selectable options
A Stand Alone Component That Changes State From Questions to Content Present
A sketch for a custom content experience with tabbed selections
Idea 4: Components That Could Be Tab-Based

Early explorations

Lots of other fun possibilities were discussed. We compiled research both on what had worked for us in the past and what other media outlets were doing to make related content more visible. The options kind of ran the gamut or what was possible: text links, teaser rolls, simplified teasers, images, stylized images. It was kind of all over the place. Based on our research and some of the notes we took, I started playing around in Sketch trying to see what some of those ideas might look like.

concepts for choose your own content adventure.
Quick Mockups for a Choose Your Own Content Component.

After reviewing what we had in Sketch, we moved the mockups to InVision to test them out. Through some simple user testing (both in-person and through’s PEEK feature) we realized that the fun ideas we had were kind of terrible for a reader. We tested the concepts with readers inside our ideal demographics range: Two female and one male, between 24-34, 2 with children, one without. The overwhelming response by these readers was that we had made the interface more complex than it needed to be. Some specific comments were, “I might use this once, but it’s pretty cumbersome,” and “My only problem… is that this makes me have to think a lot more than I feel like I should about what I’m going to be seeing.” It was clear that these early concepts created roadblocks for readers and were not conducive to skimming headlines and clicking into articles. This qualitative data early on was a guiding light for the concepts that came after.

During our regroup with one of our Product Managers, it became more and more clear that we were starting too big. We needed to scale back and start again. What does that look like? How does that feel?

We refocused and simplified our original scope:

  • We have three content types we need to deliver to all readers
  • We want all three to be optimized across devices
  • We need an easy-to-use interface that doesn't feel clunky

  • Descoping the cruft

    Based on these learnings it was clear that initial ideas didn’t meet the goals of the feature. So we stripped everything back to a significantly lower common denominator for testing. We still wanted readers to feel like they were in control of the experience, so we tested the concept with 3 clickable tabs correlated to our three content categories.

    Simpler content delivery.
    A More Clear Look at the Five 3-option Concepts

    Implementation and data gathering

    In the implementation stage, we ran an initial test comparing to LiftIgniter apples-to-apples and saw a marked difference in engagement. LiftIgniter was clearly making more actionable recommendations. Over time we would come to traffic most of our reader recommendations through LiftIgniter because of the staggering increase in engagement.

    What we found after a few days of monitoring analytics numbers was that very few people (<2%) were switching through the tabbed content on larger screens. Smaller screens were doing only slightly better. After talking through the results and ramifications, we decided to split the three content types out into three columns on larger screens. We kept the tabbed interface on smaller screens in hope that adoption was taking more time than expected.

    It wasn't. We got analytics numbers back and "desktop" click through rate had gone up from 3.6% to 5.8% and "mobile" had gone down to 3.3% with 99% of clicks coming from the default active tab. Up to this point, we had been using the API to drive recommendations in the "Related" and "Popular" tabs. We decided to follow some suggestions from and LiftIgniter to test a “back to the drawing board” option. Their data suggests that a simple 4x3 grid of article teasers might increase overall interactions.

    The four post by three column.
    The 4x3 grid of teasers on a large screen.

    Based on and LiftIgniter suggestions we did, in fact, remove all tabs on smaller screens and went to a 3 row, 4 column view on larger screens. Switching to this layout yielded growth in teasers clicked across screen sizes. As time has gone on we’ve tested and implemented more changes. Moving to LiftIgniter to power most of our machine learning-driven content recommendations. Splitting recirculation into a mix of plain text links after post content and a highly scrollable new horizontal teaser design has been an unexpected win for us. Our design has again evolved so that most of our content on the home page as well as the below-post section follow a more traditional layout.

    The four by three column.
    Highly Scrollable new teasers

    Click through rates on this new design seem to hover around 8-10% - which is incredibly high.

    Site Wide Teaser Iteration

    At the end of 2017 our team continued to be more experimental and take more risks with how we were showing content. With the results from below-post updates in hand, we set out to test content delivery in other places. A piece of low-hanging fruit related directly to the personalization tests was our home page teasers. We saw a 35%+ increase in clicks on teasers on the home page by updating our layout to closely resemble our below-post tests. I worked with our Lead PM and my Design Manager on a couple of iterations. Using our A/B testing framework and Google Optimize, we tested multiple variants. What we landed on was simple and scrollable and was a big win for engagement metrics.

    2 screens showing two different teaser styles, one vertical and one horizontal. Indicators show the horizontal version was better.
    Moving from vertically arranged teasers to horizontal ones increased clicks on the home page by just under 36%