Dunk, 28, has been excellent for Albion this season and Potter believes he has a crucial role to play as Brighton fight to stay in the Premier League.
Albion are 14th but remain just three points above the drop. Their next four matches - starting with Villa at the Amex on Saturday - are crucial with lowly Bournemouth, West Ham and Watford all to come.
Dunk was heavily linked with a move to Leicester last summer but remained on the south coast and was installed as club captain following the retirement of Bruno.
Dunk signed a five-year-contract with the Seagulls in October 2018, which runs out in June 2023.
The 6ft 4in Brighton born centre back is however a reported Chelsea fan and if the £50m offer is genuine, the lure of European football and competing at the top end of the table could be hard to resist.
Potter, however, played down the January window transfer talk. "He signed a long term contract (July 2018). We are quite relaxed about the situation," said the Albion head coach.
"I have really enjoyed working with Lewis as a footballer and as a person. He is really important for us and we are really happy that he is with us.
"January is always a little bit different to every other month if we are being honest. "There is always a lot going on, a lot of talk from you guys (media) questions and interest.
"The window is open so there is more things going on in that regard but that is normal we know that is the case. When the window is open there is always more talk other than the football."