A dramatic final ten minutes saw skipper Liam Perkins, Ryan Storer and Jonathan Whittall all shown yellow cards, while Bury St Edmunds scored two tries to seal a 23-20 victory at Roundstone Lane.
Raiders started the day 13th and despite picking up a losing bonus point dropped two places in the table.
Levett knows his team need to fight to turn things around and said: “We’ve got to start reflecting quite harshly on ourselves and saying these are the things we need to be doing because we are now in a proper battle.
“I think it’s six games we’ve been leading at half-time and we haven’t gone on to win any of those, we must start making pressure count.”
Raiders looked on course to end a run of five successive defeats but a late capitulation ensured they suffered a sixth consecutive loss.
Levett felt ending the game with a three player disadvantage was the ultimate factor.
He said: “The game should have been put to bed well before they got back into it.
“I think sometimes people make errors and mistakes, which happens at every level of sport, but at the moment we are not doing what we need to do on a Saturday. You can’t finish any game with 12 players and expect to win.
“I’ve got no complaints about any of the yellow cards, we’ve just got to look at what we can do better.”
Don’t miss out on all the latest breaking news where you live.
Here are four ways you can be sure you’ll be amongst the first to know what’s going on.
1) Make our website your homepage
2) Like our Facebook page
3) Follow us on Twitter
4) Register with us by clicking on ‘sign in’ (top right corner). You can then receive our daily newsletter AND add your point of view to stories that you read here.
And do share with your family and friends - so they don’t miss out!
Always the first with your local news.
Be part of it.