Potter has emerged as the favourite to take over at Spurs, should the Italian depart this summer.
Conte is yet to commit his future to the north London club beyond the current campaign amid links with French champions Paris St Germain.
Potter has guided the Seagulls to their highest Premier League points tally this term and underlined his credentials with Saturday’s 4-0 thrashing of Manchester United.
Yet the 46-year-old, whose Albion contract runs until 2025, insists he is happy at the Amex Stadium.
“You guys (the media) sometimes put stuff in the newspapers that’s not exactly true,” said Potter.
“What can I say? It’s got to the point now where there’s not even a vacancy and you’re getting linked to it. I don’t know what to say.
“I’ve never said I want to leave at all.
“It wasn’t so long back that I was probably getting criticism and if you’d gone on some local fan sites, I bet I wasn’t the flavour of the month.
“But I’ve always had the support of Tony (Bloom, Brighton owner) and the board and they’ve given me chance to work, so I’m grateful for that and I enjoy it here.”
Check out the latest Premier League transfer speculation below...
1. Ten Hag wants Sosa
Erik ten Hag wants to make Stuttgart star Borna Sosa his first Manchester United signing, as he aims to beat Chelsea to the £25m-rated left-back. (Kicker)
2. Kalajdzic open to Irons move
Sasa Kalajdzic is reportedly open to joining West Ham from Stuttgart in the summer transfer window. The Hammers could pick up the 6ft 7in centre-forward for around £17.2 million. (Kicker)
3. Palace want Pepe
Crystal Palace are interested in a move for FC Porto winger Pepe, but face competition from Arsenal. His current club could demand as much as £51m for his signature. (A Bola)
4. Vydra on way out?
Burnley striker Matej Vydra is expected to leave the club at the end of the season. (The Athletic)