Brighton failed to get going throughout as the Toffees dominated much of the match and fully deserved their 2-0 win after a Gaetan Bong own goal and Cenk Tosun strike in the second half.
Albion fell a place to 11th in the Premier League table but remain seven points clear of the relegation zone, with eight matches to play.
Duffy played at Goodison Park for the first time since leaving Everton in 2014 and said afterwards: "It was a tough game. I don't think we turned up and got punished for it.
"It was a disappointing performance from us. It wasn't really us. When they are on it like that they can punish you and that's what happened.
"It's frustrating because we thought we'd got over the period of them having control. We started the second half pretty well but I just don't think it was there from the start today.
"It wasn't good enough but it's a blip. We've just got to pick ourselves up because we've been here before.
"The run's over but we'll go again."
Duffy admitted the manner of the defeat was the most disappointing aspect of the day and said: "We know we've done well. It's not over one game is a season, it's over 38.
"You can only lose so many but it's the manner in which we lost it today which is disappointing.
"I think we got dominated for the majority of the game. If you look back over most of the season we haven't really been dominated like that."
Keep up with all the top football news and transfer gossip from England, Scotland and beyond with footballwire.co.uk