Condiment bottles arranged in infinity symbol

Federico Gastaldi

Do Shoppers Have Too Many Choices?

US consumer goods are proliferating rapidly, with implications for consumers and companies.

Walk down the aisles of any US convenience store and you could easily feel assailed by rows of similar—yet different—products competing for attention. Bags of Tostitos Scoops! tortilla chips share shelf space with bags of Tostitos Scoops! Multigrain or Tostitos Hint of Lime, while cans of Diet Coke vie with those of Coca-Cola California Raspberry and Coca-Cola Cherry Vanilla Zero Sugar. The seemingly endless options stretch beyond the food and drink aisles to shelves offering diapers, detergents, stationery, soaps, coffee, cosmetics, and more.

“We see all these new brands or new varieties appearing in people’s shopping carts over the past decade or so, which speaks to the fact that companies are putting them out there for shoppers to buy,” says Chicago Booth’s Joseph Vavra. The number of “niche” alternative products increased by 4.5 percent a year from 2004 to 2016, according to a study of consumer packaged goods Vavra conducted with Booth’s Brent Neiman in which they analyzed data on almost 700 million transactions involving 118 different product groups.

US households appear to have welcomed this product explosion. As new flavors fill store shelves, consumers are buying them, often at the expense of the original product’s market share. There has also been a growing diversity in household consumption. Among Tostitos buyers, one family prefers Crispy Rounds, another Scoops!, and another the Artisan Recipes Baked Three Cheese Queso variety.

Difference itself may be the one thing that people have in common. “When you look at the data, this increase in niche consumption in the US is happening across all kinds of demographic groups, across all ages, incomes, races, and education levels,” says Vavra.

He sees all this choice as good for consumers. If what you really like is a chip flavored with lime, your life is better when you can choose that rather than having to settle for plain or nacho flavor instead. Some delight at the red, white, and blue Oreo (with a layer of popping candy) that is released around Independence Day, or generally prefer that their Cheez-It crackers be both puffed and delivered with “scorchin’ hot cheddar” flavor.

But this segmentation represents a significant step away from how marketers have traditionally operated, and it comes with risks. Among them, there must be a limit to the amount of variety that companies are willing and able to supply consumers. For brands, niche products are great, unless they become a drag on profits. Will that ever happen?

A price paradox

Growing variety, when all else is equal, creates what economists call positive welfare effects for consumers. As companies fragment their products more and more, consumers are able to buy the things that they really like—getting closer to their optimal choice.

Even better for consumers, we haven’t been paying more for the additional choice. Tailoring products to preferences should give companies the potential to charge higher prices. But Vavra and Neiman find that product markups remained largely flat over the period of their study.

What’s even weirder about this is that the CPG industry has been consolidating, so a handful of big manufacturers now have the majority of market share. Stanford’s C. Lanier Benkard and Ali Yurukoglu and Chicago Booth’s Anthony Zhang looked at concentration in product markets in the US from 1994 to 2019 and find strong evidence that small groups of megacorporations are effectively enjoying monopolies at the industry or ownership level—much more so, in fact, than previously thought.

Exploiting data from MRI-Simmons, a provider of attitudinal and behavioral US consumer insights, the researchers determine that almost half of US product markets—including food and beverages, health care, apparel, and electronics, as well as a slew of nonmanufacturing markets, such as insurance and financial services—were “highly concentrated” or dominated by two or three multinationals. Over time, the likes of General Mills, Nestlé, Procter & Gamble, and Unilever have systematically acquired and subsumed other consumer brands. This trend notionally gives them not only market share but also the lion’s share of market power by which to influence or even set prices.

But the researchers find that this trend is being offset by another: an upswing in competition at the individual product level. Although there are fewer companies offering products in a certain sector (say, food products), there are more companies offering them in a specific market (such as chips). That is creating more competition at the level of individual products, which keeps prices low.

The Herfindahl-Hirschman Index is a standard measure of market concentration, and it’s typically used before and after mergers and acquisitions to determine the level of competitiveness that exists in a market. A market’s HHI is the sum of the squared share of every competitor in the market. If there are three companies dominating a market with shares of 20 percent, 30 percent, and 50 percent, respectively, the HHI would be 3,800 (202 + 302 + 502). Markets that have an HHI of 2,500 or higher are considered highly concentrated, or noncompetitive. HHIs that are lower—between 1,500 and 2,500—are considered to be moderately concentrated, or more competitive.

Rather than look solely at the sector-level situation—as in, how many companies are making food products—Benkard, Yurukoglu, and Zhang applied the measure to specific product markets. They find that the median HHI among US consumer product markets fell from about 2,250 in 1994 to about 1,950 in 2019, indicating an upswing in competition. The rivalry seems to be robust across the board and for all of the 330 individual product markets in the US that the researchers studied.

More for your money

Competition among new varieties has kept product markups mostly unchanged.

This competition is translating into benefits for US wallets and pocketbooks—and may still be good for companies too. Says Vavra, “We know that people have complex and differentiated tastes and preferences, so adding new brands to your product portfolio means you can meet that demand while carving out a more loyal customer base for your organization.”

It helps that as companies grow, it costs less to meet demand. Larger companies may leverage economies of scale as they acquire smaller players and ramp up new products, brands, and varieties. Economies of scale lower production costs which, in turn, can be passed through to consumers as lower prices.

“These large conglomerates are able to consolidate their presence and integrated know-how and expertise for the firms they acquire,” says Zhang. “And in doing so, they’re likely benefiting from all the advantages that accrue to size—enhanced capabilities and efficiency gains, from production to logistics and inventory management to marketing—which reduces their costs.”

Shopper navigating a maze of store shelves

A break with the past

Variety seems like a win for companies and consumers alike, and yet this is—to some extent—uncharted territory. All this choice could be less an elegant example of the market supplying what consumers want and more a reflection of shifts in the tectonic plates underlying consumer markets.

Unpacking the dynamics underpinning product fragmentation, Chicago Booth’s Jean-Pierre Dubé looked at the US beer industry, which in 1980 was dominated by just two players and their pale lagers: Miller Brewing and Anheuser-Busch.

But today, shopping carts are stocked with pale ales, wheat beers, porters, stouts, red ales, and more from a variety of brewers, some of which have been bought up by bigger players. (Miller Brewing and Anheuser-Busch have themselves been acquired.) Craft brews made up just 5 percent of the US take-home market and about the same in terms of total revenue in 2005. By 2018, this sector had more than doubled its share to 12 percent, with revenues quadrupling to 20 percent. “It literally went from being something tiny to being a significant portion of the industry in just 15 years,” says Dubé.

To understand how that happened, Tilburg University’s Bart J. Bronnenberg, Dubé, and University of Texas at Dallas’s Joonhwi Joo used the NielsenIQ Datasets at Booth’s Kilts Center for Marketing to look at the beer purchases made between 2004 and 2018 by about 100,000 US households. They find that craft’s growing market share was driven by one demographic group, millennials, who accounted for more than a third of all craft-beer purchases. Baby boomers and the Greatest Generation constituted just 20 percent and 13 percent of craft-beer consumption, respectively.

Is this all an issue of taste? As these younger people came of age and exercised their spending power, did they have new preferences and choices that created demand for new types of beer? Dubé says it was actually the other way around: the upswing in demand for craft beer was driven by supply-side forces, namely the availability of craft beer.

“When you look at the timeline, you see a convergence of factors,” he says. “In the 1980s, we had the sudden deregulation of craft beer, and by the end of the 1990s, we had the internet as an increasingly viable advertising and distribution platform. These two factors made it possible for a raft of new players to enter the market. As more craft beers entered the market during the early 2000s, a new generation—the millennials—turned 21 and were legally allowed to buy beer. What happened is that these new consumers came into the market and started forming their beer brand preferences at the exact point that product variety increased. They had more choice than their predecessors and therefore formed different beer-buying habits and preferences.”

Habits are built over time and hard to break, says Dubé. Older generations stuck with the national brands of pale lagers that they grew up drinking. “What the craft beer example shows us is that shopping habits are formed not because one generation is intrinsically different from another,” he adds. “Rather, these habits are being driven by suppliers’ ability and strategic decision to offer variety.” (For more on the topic, read “Why craft beer’s rise is a warning flag for all sorts of big brands.”)

This makes sense in the context of a company’s growth strategy. Catering to more tastes, as Vavra suggests, can help a brand secure a greater number of loyal customers. Historically companies haven’t supplied variety because they’ve been held back by organizational and operational capabilities, not to mention budgets. But Dubé says that has changed in the 21st century, thanks in large part to technology.

Advertising and online distribution

At the time that Americans favored Budweiser and Miller, only one or two brands also dominated many other product categories. People who wanted cereal for breakfast generally reached for a box with a Kellogg’s label. When they wanted carbonated beverages, they chose between Coke and Pepsi. Historically, the budget required to build a new brand through television and other mass media created barriers to entry, sustaining the dominance of the established national names.

And the internet has helped change that, with Google and keyword searches, and then social media and targeted advertising. Using Facebook, companies developed the ability to identify the consumers who liked content corresponding to a particular product, and perhaps liked a product other than what had long dominated. Digital advertising was cheaper and more efficient than mass advertising, making it an accessible and effective tool for smaller players looking to get a foothold at a time when a new generation of consumers also wanted their brands to better align with their own political and social views. All this allowed companies to find their market.

And just as barriers to entry in advertising were eradicated, so too were many of the restrictions and challenges related to distributing goods. Suddenly consumers were not only hearing about new brands, but they also had access to them because they weren’t limited to the stores around them.

Where market power is growing

While concentration has decreased overall in US product markets since 1994, it has increased in certain areas (such as car rentals) and declined in others (such as glue).

“There was a time when the idea of having your laundry detergent FedExed to you was ridiculous,” says Dubé, who has published extensive research on the historically persistent structure of consumer product industries. “Today’s consumers can go to a website, click on a laundry product, and have it shipped to them within 24 hours—sometimes within a couple of hours, thanks to Amazon Fresh and other grocery delivery platforms. And by the way, that detergent no longer has to be Tide. It can be a regional brand that is locally sourced and has sustainable packaging.”

The upshot is that it got easier for newcomers to disrupt markets. The internet eradicated the need for a huge marketing budget or even a physical store. Warby Parker started advertising and selling affordable glasses online, then upturned the eyeglass industry. In apparel, Bonobos set about redefining the retail experience in men’s apparel by blending physical and online platforms; consumers can secure a good fit in a store but do their actual shopping online. (Walmart took notice and bought the company in 2018.) Dollar Shave Club, known for its viral video advertisements on social media, destroyed Gillette’s stranglehold by creating an alternative to expensive razors and selling them online, giving users the option to purchase upgrades or add-ons. Unilever purchased it in 2016. Diapers.com, founded in 2005, was scooped up by Amazon five years later.

These brands were able to start up fast, buy a website, and bring consumers to the point of sale online, says Dubé. The savviest players exploited social media to identify their customers, determine what they needed, and offer products and experiences that created pull-through demand (drawing customers in with search engine optimization, social media marketing, and word of mouth). Lacking a network of physical stores and dedicated sales staff, their overheads remained low, even as they offered more things that consumers wanted.

“Startups with new, differentiated brands are offering us more and more variety—more than we’ve had before,” says Dubé. “We’re being sold choice and we’re buying it.”

Brands buy innovators. Could it be a win-win?

As consumers bought up choice, the CPG conglomerates were busily buying up the competition. Management consulting firm Kearney’s 2021 Consumer and Retail M&A Report catalogs $58 billion in M&A deals made in the first quarter of 2021 alone, and its poll of 100 of the world’s largest CPGs indicates that 60 percent of them were looking to acquire smaller ventures (worth $500 million or less) heading into 2022.

Possibly the “new normal,” says Dubé, is for large CPG companies to buy newcomers to stay ahead of disruption and grow their product portfolio. They may also look to startups for innovative ideas.

Many big CPGs have tried and failed to launch viable competitor brands to their best-selling products, he notes. In the US, neither Miller nor Anheuser-Busch was able to launch a pale ale alternative or a craft beer on its own, instead pursuing newcomers. Certainly the incentives are clearer for an outsider to take on a best-selling product than for its manufacturer to set out to cannibalize itself.

The system that has developed seems to satisfy the needs of big and small companies—and entrepreneurs—while consumers get more choice at lower cost. Large CPGs use their resources to maintain and grow the companies they buy, while smaller companies focus on innovation and are busy “searching for the magic,” as Dubé puts it. For entrepreneurs, selling their business offers the double drawcard of a handsome payday and the chance to move quickly to another project. It’s an attitude that Dubé has picked up among his MBA students. Younger entrepreneurs are less tied to the idea of starting long-term businesses and are more interested in having “fast impact” before pivoting to something different, he says.

This would seem to be good news all around, except that it’s at odds with the late economist Joseph Schumpeter’s notion of creative destruction, as he described it in his 1942 book, Capitalism, Socialism, and Democracy. Schumpeter held that as companies get big, they become complacent, and flabby incumbents are replaced by hungry newcomers. What seems more so the case now is that incumbents are acquiring newcomers, which in turn divest their innovation, expertise, and know-how and move on to the next idea or challenge.

Consumers have become accustomed to having more choice, and the demand for variety is likely to remain robust. 

This doesn’t bother Dubé, particularly if there are clear benefits for both parties. And he says that when products with the same parent company are fighting for dominance, “it could also indicate creative destruction in the sense that the incumbent brand is getting supplanted after a long period of dominance.” But it still challenges much of the established thinking around the risks—and the potential advantages—of monopolies. US legislators have historically sought to limit the market power of large corporations. Congress passed three major antitrust laws in the past century, all aimed at preventing monopolies, prohibiting price-fixing, and driving free competition. The discussion about concentration has traditionally centered on the number of companies operating and competing in different segments—and that remains a focus for regulators, including the current head of the Federal Trade Commission and for one industry in particular, which a New Yorker headline describes as “Lina Khan’s Battle to Rein in Big Tech.”

Zhang argues that his findings raise the question of whether regulators should pay more attention to competition at the level of individual products and services. “There is some subtlety required to understand the big picture of what’s going on in product markets right now, and to see things through the lens of consumers, who are enjoying greater choice and more competitive product pricing from American manufacturers today than they were 20 years ago in certain markets,” he says.

But he stops short of saying that Schumpeter needs a rethink and the current situation is a win-win-win. The risks attached to big market power still prevail, and much depends on the intentions of incumbent players.

The biopharma and tech industries are highly prone to innovation and disruption, he adds. “What we’re seeing in these industries in particular is a lot of ventures getting into the early stages of innovation—phases one and two, where it’s all about concept and prototyping.” But when a new pharma or tech idea gets developed, in phase three, these products—unlike many other consumer goods or services—face specific barriers to entry. Getting new pharma products to market costs a lot, and app developers have to navigate a market dominated by a handful of megaplayers. Users tend to flock to a product only once it has been integrated into an incumbent platform—think YouTube or Instagram, which saw their user bases grow exponentially after they’d been acquired by Google and Facebook, respectively.

In tech and pharma, then, a lot of startups sell at phase three to incumbents with the resources and financial clout to take their products to market, says Zhang, who agrees that this is at odds with the Schumpeterian idea of creative destruction. And in these industries particularly, phase-three acquisitions posit a risk to innovation.

London Business School’s Colleen Cunningham and Yale’s Florian Ederer and Song Ma looked at 20 years of acquisitions data from the pharmaceutical industry and find evidence that the big pharma corporations are buying up newcomers, in certain instances in order to destroy them. These so-called killer acquisitions may account for as much as 7 percent of all buyouts in the pharmaceutical space in any given year, and this is a conservative estimate, according to the study.

When it comes to the tech sector, Columbia PhD student Sai Krishna Kamepalli and Booth’s Raghuram G. Rajan and Luigi Zingales argue that when the likes of Google and Facebook acquire new companies, they can effectively use their power of incumbency and the tremendous importance of network externalities in the tech space to prevent entrants from gaining an adequate foothold. This allows them to reduce what they need to pay to acquire the entrant. This is a worry, write Kamepalli, Rajan, and Zingales. In tech, entry is hard to finance, and the prospect of acquisition is a major draw. By lowering the likely acquisition price, Google, Facebook, and other incumbents are effectively creating a kill zone in the startup space, they argue. (For more, read “Why big-tech mergers stifle innovation.”)

Killer acquisitions and kill zones in tech and pharma represent significant risks to competition and to innovation in those areas, says Zhang. They are risks that are very much front of mind for the FTC, and they may apply to other product sectors, including CPGs.

“On the one hand, we do see that the proliferation of variety at the product level translates into choice and price benefits for consumers—even if we have more monopolies and less competition at the market level,” Zhang says. “But if a lot of product choice is coming from acquisitions, what’s to stop big CPGs doing the same as the pharma and tech giants and using their market power to cap buyout prices for startups or kill off competitor brands? And what might that mean in terms of markups for consumers?”

Why more choice can be better, or not

More product choice can be great for consumers, in theory, but also overwhelming. Consider inexperienced investors trying to pick between hundreds of stock and bond funds. Without a clear path, some could get fed up and put their retirement accounts in a slow-growing money market fund instead.

More on the risks

Long-term market competition could be in peril, but CPGs run some shorter-term risks too. Yes, they’re enjoying economies of scale, synergies, and innovation that they’ve acquired—but it’s challenging to preserve the credibility and authenticity of the brands they buy. Take Nantucket Nectars, a craft beverage brand acquired by Ocean Spray in the 1990s. The appeal was in its identity, says Dubé. Company founders Tom First and Tom Scott originally sold drinks from their boat in Nantucket Harbor, Massachusetts, and in recycled bottles, and the story of the brand was unique, local, small, and fun. But it lost its luster as part of a larger company. Ocean Spray ended up selling it a few years later at a discount.

A judge in 2016 dismissed a potential class-action lawsuit against Molson Coors over its marketing of Blue Moon. Described on its website as a craft beer, Blue Moon was developed not in an independent craft brewery but at one owned by Molson Coors Brewing.

“A big part of these craft brands—whether it’s a beer, a pair of eyeglasses, or a pair of pants—is that people believe these products to be more natural, and that’s especially true of the food and beverage industry,” says Dubé. Consumers are increasingly tuned into locally sourced ingredients and similar considerations, and even if a company moves in that direction, shoppers are skeptical. “When things are mass produced, it’s hard to be credible about where ingredients are sourced. And this is a global trend, so the question becomes: Does the person who’s been managing Oreo cookies have the right expertise to market a cookie that is the ‘anti-Oreo cookie?’ A cookie made with no palm oil, or with natural sweeteners and sustainable packaging?”

It’s also hard to measure authenticity, until that’s necessary in court. There has been an upward trend in class-action consumer lawsuits in the US, with businesses from food and beverages, household cleaning products, cosmetics, personal-care products, and more coming under fire for alleged unethical marketing practices mostly related to the labeling of their products, for instance about a brand bestowing certain health benefits, deriving from a desirable origin, or being “all natural.” This trend hit an all-time high in 2021, when 325 class-action lawsuits were filed against food and beverage companies, according to law firm Perkins Coie.

And while choice may be good, it has costs for consumers, who might spend more time, effort, and in some cases money in order to find what they want. Companies run the risk of overwhelming consumers with too many options. University of California at Berkeley’s Olivia R. Natan (a graduate of Booth’s PhD Program) studied a restaurant delivery platform as more and more options were added. The additional variety was intriguing and brought more people to the platform for the first time, but they ended up ordering less often as sorting through options became more difficult.

“Choice overload is real—we just don’t fully understand when and why it happens yet,” says Chicago Booth’s Daniel Bartels, who explains that human beings have cognitive limitations that complicate decision-making. “Say we have to choose between two options, and each option has three salient attributes or characteristics, so there are six pieces of information that we will need to take into account. Now increase those options to five or more, and suddenly there are 15 or more pieces of information that need our attention. The chances are suddenly much higher that I will make a suboptimal decision, or that I simply fail to make one at all.”

There is no “magic number” with choice overload, so it’s unclear to a marketer whether that next new offering is the one that will end up frustrating the customer. “We don’t know for sure how many options are just enough or how many are too many,” says Bartels. “A lot, too, depends on the complexity of the choice facing us, that and the costs we incur in getting it wrong.”

The stakes are relatively low if you face too many options when choosing a bag of chips or a six-pack of soda—if the flavor turns out not to be for you, you haven’t wasted all that much time or money on it. But say you’re shopping for a laptop, a car, a health-insurance plan, or an investment fund for retirement savings. In these instances, you might find more choices to be intimidating, and making the wrong choice could be costly.

Tools can help make decisions simpler. Aggregators or screeners sort or filter options in terms of key features or characteristics. KAYAK, Booking.com, Autolist, LappyList, and other online platforms use algorithms to process large quantities of variables, whittling them down to just a few choices.

But these aggregators are themselves multiplying. “We may get to a point where we have too many tools,” says Bartels, listing Yelp, Thrillist, Eater, and Tripadvisor as only some of the sources of information people turn to. “It’s not just the choice options themselves, but the array of places to go for input into those choices that is also proliferating.” (See “Why more choice can be better, or not,” above.)

Offerings are set to keep on expanding, at least in the CPG industry. Pharma and tech aside, as long as the barriers to market entry for certain products and services remain low—or lower than they were in the 1990s—we are likely to see brands continue to fragment and variety proliferate even more, argues Dubé.

“Back in the ’90s, mass media was all we had. Today, CPG brands have alternative means of engaging with customers that are cheaper and more efficient. Marketers have all kinds of online and social tools at their disposal to create communities of engagement and loyalty around their products,” he says.

The resulting landscape may work for entrepreneurs, consumers, and CPG manufacturers—but it’s evolving. Many entrepreneurs get their start by circumventing stores, but consumers might get fed up with all the cardboard and packaging involved in getting products sent to their doorsteps. Regulation or taxes could emerge that change the current approach. Small brands could once again have to compete for shelf space in physical stores.

But consumers have become accustomed to having more choice, and the demand for variety is likely to remain robust, Dubé says. In that case, the question may not be whether there can be too many flavors of tortilla chips but whether your favorite flavor has yet been produced—and if it hasn’t been, whether the stars will remain aligned long enough for the market to deliver the exact product you didn’t even know you needed.

More from Chicago Booth Review

Related Topics

More from Chicago Booth

Related Topics

Your Privacy
We want to demonstrate our commitment to your privacy. Please review Chicago Booth's privacy notice, which provides information explaining how and why we collect particular information when you visit our website.