Case Study: The Cost of Not Listening to Your Customers

Key Takeaways

• Customer-Centric Approach is Crucial: PantryEssentials failed to put the customer first and made significant changes based on assumptions rather than actual customer feedback. This led to increased confusion, dissatisfaction, and a perceived decline in value. Customers were primarily drawn to PantryEssentials for its variety and personalized selection, which was compromised by the new credit-based pricing model and automated recommendations.
• Importance of Thorough Testing and Data-Driven Decisions: Despite conducting some beta testing, PantryEssentials ignored critical feedback that highlighted issues with the new pricing model and product recommendation algorithm. Instead of listening to what the data was telling them, they proceeded with a flawed system, assuming they knew better than their customers. This approach resulted in a negative user experience and increased customer complaints
• Clear Communication and Value Proposition: PantryEssentials's failure to effectively communicate the benefits of the new pricing model and the value proposition to customers led to confusion and skepticism. Customers were not convinced of the advantages, and the messaging appeared opaque and potentially deceptive. A clear and compelling marketing campaign could have mitigated some of these issues, but the lack of one contributed to customer churn and revenue decline.

Introduction

New Coke, Qwikster, Apple Newton, Microsoft Bob, and Mircrosoft Bob—what do these all have in common? They're all epic product failures that remind us why you should never cover your ears when your customers are talking or stop paying attention to who your customers are.

Most companies strive to strike the perfect balance of monetizing their products and services while also giving customers a stand-out, high-quality user experience thereby creating brand loyalty. Achieving that “perfect balance” however, is not an easy feat, and far too often when a grand idea grips hold of the CEO or senior leaders, it is difficult to change hearts and minds to course correct. In this article we examine how one business was unable to strike this equilibrium because they didn’t 1) put the customer first and 2) started listening too late when the train had already left the station. 

The path to success is often littered with well-intentioned missteps

We partnered with a company, let’s call it “PantryEssentials” (the real name of the company has been hidden for privacy purposes) that catered to the home cook and specialized in sending customers subscription boxes of the latest kitchen gadgets, spices from across the globe, recipe/cook books, canned goods, small batch/boutique products, etc. Customers paid a flat monthly fee and were able to select products of their choice online up to a certain dollar amount. Although this subscription model was generally profitable, PantryEssentials wanted to make more revenue and build something even easier for their customers to use. PantryEssentials made two fundamental shifts:

1. Pricing model: Instead of a flat monthly fee, PantryEssentials introduced a credit-based monetization approach. For example, if a customer paid $50, they would receive 60 credits to redeem for products. The more credits a customer bought, the more “free credits” they would also receive as an incentive to keep spending. 
2. Increased automation: PantryEssentials also introduced a new product feature that would recommend items based on the customer’s historic purchasing behavior. The thought was that this would alleviate the need to scroll through pages of products online. Customers had the opportunity to reject or switch products that the algorithm initially recommended.

What actually ended up happening was an increase in confusion, more questions from customers, and a perceived decline in value of the subscription box, driven by:

1. A lack of clear communication around the advantages of the new pricing model followed by not highlighting the “what’s in it for you” proposition in marketing and branding collateral.
2. Customers did not understand how the “credits" translated to dollar amounts. For instance, they were left wondering why sardines from Portugal cost 25 credits and how this translated into dollars.
3. The algorithm not matching customers’ specifications; for example, customers who cited being gluten free, having food allergies, or niche foodie interests were receiving product recommendations that didn’t match their needs whatsoever
4. Surprise billing – customers assumed that when they swapped product recommendations, there would be no difference in credits spent, but this was not the case. Instead, customers were often surprised that the cost of their box was higher than what they were used to spending every month with PantryEssentials. 

Key lessons from this case study

Don’t make assumptions about what your customer's pain points are 

PantryEssentials assumed that customers’ number one pain point was “decision fatigue” as there are so many food brands and product options to choose from on a day-to-day basis. Because of the matching algorithm PantryEssentials rolled out, dissatisfaction increased. The top complaints about the boxes became, “This isn’t what I want or would have selected” and, “Your selections don’t align with my cooking values, needs, or desires.” Sure, decision fatigue is a pain point customers have, but customers were coming to PantryEssentials because of the variety of unique, global products and the ability to select products for themselves based on their own preferences and lifestyles.

Examine your assumptions and listen to what the data is telling you

PantryEssentials conducted tests where a beta version of this new model was exposed to a few top customers who were asked what they liked or didn’t like about the user experience and the perceived value of the new pricing model. This is best practice and a step we would recommend to all customers designing a new product or iterating on an existing version. However in the case of PantryEssentials, while customers conceptually liked the product matching algorithm, they were confused by the complicated credit system and didn’t think the user experience design was compelling or practical enough to shift away from a process that had already been working. 

PantryEssentials however, chose to listen to what they wanted to hear, and it became the classic case of “we know what the customers want better than they do” and as the age-old saying goes, don’t fit a square peg into a round hole - if the data isn’t telling you the story you want, don’t enhance or force the lesser storyline to fit an agenda that could be costly. 

Test, test, and retest to get products ready for primetime launch

One of the most problematic issues with PantryEssentials rolling this new structure out was that the complicated algorithm on the back-end didn’t fully work. The haste to move to the new pricing model and the hype of being in the game of “AI” overrode the quality of the actual user experience. PantryEssentials did not ensure that the algorithm was properly trained and failed to do extensive testing in order to see if the algorithm recommendations were matching customers' basic, practical needs before they went live. While a product doesn’t need to be 100% perfect when it’s rolled out, it should achieve the basic goals you attend to.

Communicate a clear and powerful value proposition to your customers

PantryEssentials did not come up with a powerful enough marketing and branding campaign that would excite and delight customers to adopt and want to invest in these changes. Instead messaging was opaque and created confusion, skepticism, and a feeling of “you’re ripping me off.”

Conclusion

The cost of churn or losing customers and winning new ones is high, and PantryEssentials found this out the hard way. The implementation of the confusing new pricing model and faulty matching algorithm, resulted in five consecutive quarters of revenue decline, costing PantryEssentials dearly. While there are always negative signals in launching a new product, it is imperative to investigate the drivers of these signals deeper and to genuinely listen, internalize, and operationalize customer feedback within major product changes. PantryEssentials leadership became singularly focused on increasing revenue along with the perception of needing to get into the AI race, and ignored the canary in the coal mine during initial beta testing shifting away from customer centric design and strategic thinking.

Send DataSend DataSend DataSend DataSend DataSend Data