Potter was jeered throughout his return to Brighton, where he spent three mostly successful seasons before joining Chelsea last month.
Trossard opened the scoring for the Seagulls with Pascal Gross and own goals from Trevoh Chalobah and Ryben Loftus-Cheek sealing the win for Roberto De Zerbi's men. Kai Havertz netted a second half consolation for the Blues.
Crooks, writing on the BBC website, said: "There seems to be some suggestion that Graham Potter owes Brighton fans an apology for leaving Amex Stadium for Stamford Bridge so abruptly," said Garth.
"An apology - what for? Brighton fans should think themselves very lucky to have had Potter for as long as they did. It was well known in football circles that he had turned down a number of offers before he succumbed to the lure of Chelsea.
"The problem the Seagulls have now is can they hang on to Leandro Trossard, who destroyed Chelsea, and made it very uncomfortable for Potter on his return to his old club. It is not unusual for good players to end up at bigger clubs - or managers for that matter."
Garth Crooks team of the week...
1. Ederson - Man City
Garth said: "Ederson, like all goalkeepers, is capable of making the odd mistake or having a poor performance but he wins more games for his team than he loses. His save from Leicester's Youri Tielemans on Saturday was another classic example." Photo: GEOFF CADDICK
2. Dalot - Man United
Garth said: " If there was a header to be made, he made it. He defended brilliantly and at times in very difficult circumstances." Photo: Ash Donelon
3. Mee - Brentford
Garth said: "The goal by Ben Mee against Wolves deserved recognition, that's why he's made my selection. Sometimes defenders have inspired moments in the opposition's penalty area and the finish by Mee was as intentional as it was brilliant." Photo: Clive Rose
4. Laporte - Man City
Garth said: "It has taken the Spaniard a while to find his feet in a team that does not carry any passengers. He now looks like the real deal and was instrumental in City's clean sheet against a very dangerous Leicester City." Photo: Michael Steele