The company said that "in light of Mr. Zimmer's termination," it is postponing its annual shareholders' meeting which had been scheduled for 11 a.m. today in California.
"Over the last 40 years, I have built The Men's Wearhouse into a multi-billion dollar company with amazing employees and loyal customers who value the products and service they receive at The Men's Wearhouse," Zimmer said in a statement. "Over the past several months I have expressed my concerns to the Board about the direction the company is currently heading. Instead of fostering the kind of dialogue in the Boardroom that has, in part, contributed to our success, the Board has inappropriately chosen to silence my concerns by terminating me as an executive officer."
The company, based in Fremont, Calif., was founded in 1973 by Zimmer. It is one of the country's largest specialty men's retailers with 1,143 stores across the country. The company operates Moores and K&G Stores in addition to its namesake stores.
Zimmer's famous tagline was, "You're going to like the way you look. I guarantee it."
Richard Jaffe, analyst with Stifel Nicolaus, wrote in a note that Zimmer had reduced his role at the company "significantly" over the past two years. Zimmer named a new CEO, Doug Ewert, two years ago while designer Joseph Abboud was named as chief creator director in Dec. 2012.
We believe that despite Zimmer's planned transition to a smaller role at the company, he had difficulty letting go of the reins and the leadership of the business," Jaffe wrote in his note to investors. "We believe that this led to a conflict with the board and his subsequent termination."
A spokesman for Men's Wearhouse declined to comment to ABC News.While Zimmer has been the face of the company for decades, accumulating 500 hours of footage, Jaffe said, "The use of Zimmer as spokesperson has, coincidently been under review as management has been evaluating his effectiveness, particularly with the millennial consumer." Zimmer is 64 years old.
The millenial consumer is typically described as "Generation Y", or people born between 1976 and 2000.Zimmer's most recent pay figure is $1,985,916, according to executive compensation data firm Equilar. According to the firm's most recent proxy, he has 1,771,625 shares, 3 percent of the company, that are worth about $66.4 million based on Tuesday's closing stock price of $37.47.
President and CEO Doug Ewert earned $2.1 million last year, down from $5.4 million in 2011. Executive vice president Charles Bressler earned $2 million, up from $1.8 million in 2011, according to Equilar.
The company said the annual meeting was postponed to re-nominate the existing slate of directors without Zimmer, and it will announce the meeting's rescheduled date "shortly."
The announcement didn't faze Jaffe from Stifel Nicolaus' "buy" rating for Men's Wearhouse.
"We believe current management has been running the business effectively and will continue to do so," Jaffe wrote. "While we believe near term uncertainty exists as Men's Wearhouse responds to a difficult macro-economic environment, we remain positive on the long term viability of the company, and at the current stock price, the risk/reward is attractive."
Men's Wearhouse stock was trading around $36.71 a share late Wednesday morning, down about 2 percent.
Last week, the company reported that first-quarter earnings increased by 23 percent.