The tea party certainly isn’t over, but after nearly five years is it starting to die down?

Tea party supporters will mark the movement’s fifth anniversary in Washington on Thursday. Scheduled speakers include politicians like Sens. Ted Cruz and Rand Paul, as well as conservative media stars such as Sean Hannity and Mark Levin.

Karlyn Bowman and Jennifer Marsico, two American Enterprise Institute scholars who study public opinion, write at Forbes.com that the tea party’s national popularity has held steady by some measures and waned by others. “But as more Americans have come to know the Tea Party movement, unfavorable views have risen sharply,” they write.

Senate Minority Leader Mitch McConnell of Kentucky Associated Press

Tea partiers continue to be a small but significant voting bloc. A 2013 poll found that 12 percent of registered voters were self-proclaimed members of the movement, just one point off from the 13 percent who gave that response in 2010. Similarly, polls that ask whether respondents “support” the tea party show remarkable steadiness. Surveys taken in 2010 and 2014 show that about one-quarter of voters consider themselves supporters, while about two-thirds do not.

But when Ms. Bowman and Ms. Marsico looked at polls that asked whether people had a favorable opinion of the movement, they found a dramatic drop-off in support. “At least six pollsters ask people whether they have a favorable or unfavorable opinion of the Tea Party,” write the authors. All of the recent surveys showed a rise in unfavorable sentiment since the question was first asked. “In Pew’s poll, 24 percent had an unfavorable opinion in February 2010; in October 2013, more than double that number, 49 percent, gave that response.”

It turns out that even Republicans are increasingly skeptical of the movement. “We would expect Democrats to have unfavorable views of the right-leaning Tea Party, but in several polls, many Republicans, especially those who call themselves moderate to liberal Republicans, now voice unfavorable opinions. These negative feelings were exacerbated during the government shutdown.” write Ms. Bowman and Ms. Marsico.

The tea party continues to receive strong backing from conservative Republicans—65 percent, according to a Pew poll taken in October—but its appeal has not spread much beyond those voters. Tea party supporters “look a lot like the conservative GOP base,” write the authors. That may partly explain why tea party candidates who are challenging conservative Republicans in Senate primaries this year are having a tough time of it. Matt Bevin is trailing Mitch McConnell in Kentucky by nearly 30 points. In Texas, John Cornyn leads his tea party challenger by 20 points. In Kansas, Pat Roberts is 26 points ahead of the tea party candidate among GOP primary voters. These tea party candidates are trying to depict the incumbents as too liberal, but GOP voters aren’t buying it. Nor should they. Mitch McConnell’s problem is not that he’s insufficiently right-wing—it’s that he’s in the minority.