Potter was jeered throughout his return to Brighton, where he spent three mostly successful seasons before joining Chelsea last month.
Trossard opened the scoring for the Seagulls with Pascal Gross and own goals from Trevoh Chalobah and Ryben Loftus-Cheek sealing the win for Roberto De Zerbi's men. Kai Havertz netted a second half consolation for the Blues.
Crooks, writing on the BBC website, said: "There seems to be some suggestion that Graham Potter owes Brighton fans an apology for leaving Amex Stadium for Stamford Bridge so abruptly," said Garth.
"An apology - what for? Brighton fans should think themselves very lucky to have had Potter for as long as they did. It was well known in football circles that he had turned down a number of offers before he succumbed to the lure of Chelsea.
"The problem the Seagulls have now is can they hang on to Leandro Trossard, who destroyed Chelsea, and made it very uncomfortable for Potter on his return to his old club. It is not unusual for good players to end up at bigger clubs - or managers for that matter."
Garth Crooks team of the week...
5. Summerville - Leeds
Garth said:: "Crysencio Summerville certainty had the sort of impact Liverpool never expected. The 21-year-old Leeds midfielder could not have had a better birthday present than scoring the winning goal." Photo: Nathan Stirk
6. Partey - Arsenal
Garth said: "Thomas Partey has been in outstanding form for the Gunners this season but has often found himself playing second fiddle in my selections to the likes of Granit Xhaka and Martin Odegaard. Well, not so against Nottingham Forest." Photo: Stuart MacFarlane
7. De Bruyne - Man City
Garth said: "It is time this most wonderful player won the Champions League and with it the Ballon d'Or." Photo: Shaun Botterill
8. Odegaard - Arsenal
Garth said: "Martin Odegaard strolled around Emirates Stadium conducting most of Arsenal's attacks and looking very composed and cultured while doing it." Photo: David Price