In fashion, one day you're in and the next day you're out. That's Heidi Klum's tagline on "Project Runway." It also describes the management shakeup at Ralph Lauren. The company's CEO, who joined Ralph Lauren less than two years ago, is already out.

Stefan Larsson will be leaving Ralph Lauren in May due to differences he had with the company's namesake founder and chairman.

Shares of Ralph Lauren (RL) plunged nearly 12% in early trading on the news, even though the company also reported quarterly earnings that topped Wall Street's forecasts and reaffirmed its outlook for the year.

But sales continue to fall, and that's a problem that Larsson has been unable to fix.

Larsson, formerly the head of Gap's Old Navy unit, left Gap (GPS) in September 2015 to become CEO of Ralph Lauren. At the time, this was viewed as a huge blow for Gap since Old Navy was the one division of the company that consistently did well.

Ralph Lauren investors were thrilled too because they felt that hiring Larsson would free up Lauren, who had been CEO, to focus more on designing and the company's long-term plans. Lauren is still chairman of the company that bears his name.

Related: Ralph Lauren laying off 8% of its workers

But it appears that Larsson and Lauren could not see eye-to-eye on where the company should go next.

"Stefan and I share a love and respect for the DNA of this great brand, and we both recognize the need to evolve," Lauren said in a statement. "However, we have found that we have different views on how to evolve the creative and consumer-facing parts of the business."

Last year, the company announced plans to close about 50 stores and lay off 8% of its workforce. This was part of what Larsson called a "Way Forward" plan for Ralph Lauren.

At the time, Lauren said he was on board with Larsson's vision for the company.

"I'm trusting my baby with him, and my baby has to grow up," Lauren said.

Clearly, something happened since then that led Lauren to take back custody of his child.

Ralph Lauren CFO Jane Nielsen will lead the company as it searches for a new CEO. Interestingly though, Lauren said that the company's "Way Forward" plan remains on track.

And Larsson said in a statement that he is "proud of the progress the whole team has made and I am committed to ensuring its uninterrupted execution" before he departs.

Still, the shakeup at Ralph Lauren comes at a tumultuous time for many brick-and-mortar retailers, and apparel companies in particular.

Amazon (AMZN) has taken a bite out of the entire retail industry, as more and more consumers shop online -- even for clothes.

So-called fast fashion retailers, most notably H&M, Forever 21 and Zara, are also becoming increasingly popular with younger consumers.

And this has taken its toll on many retail icons. Larsson's former company, Gap, continues to struggle.

Related: Layoffs in aisle 4! Retailers are big job killers

So has Abercrombie & Fitch (ANF), which once was the king of mall-based retailers. A&F announced Wednesday that it was promoting its president and chief merchandising officer Fran Horowitz to be its CEO to try and turn around the company.

She is the first permanent CEO since former chief Mike Jeffries was forced out after making a series of embarrassing comments about customers.

And department stores Macy's (M) and Kohl's (KSS) both reported dismal holiday sales.

There is now even speculation that Macy's could be looking for a buyer.

Macy's stock rose Thursday morning following a New York Post report that said long-time Macy's CEO Terry Lundgren, who will step down later this year, might want to sell the company before his tenure ends.