Duffy, 28, has been linked with a move away from Albion throughout the summer with Celtic and West Ham the leading contenders for his signature.
Duffy made just 14 starts and seven substitute appearances for Albion last season as Potter often favoured defender Adam Webster to play alongside Brighton skipper Lewis Dunk.
The arrival of Dutch international defender Joel Veltman and the return of Ben White following his successful loan spell at Leeds has pushed him further down the pecking order.
Duffy has three years remaining on his contract with the Seagulls but the Ireland international is a boyhood Celtic fan and the Scottish Champions are keen to secure a deal for the defender.
The need for a player of Duffy's stature was highlighted last week as they exited the Champions League at the qualifying stages following their 2-1 home loss to Hungarian champions Ferencvaros.
"I think there's common knowledge that there has been interest that's been expressed," said Hoops boss Neil Lennon last week.
West Brom and West Ham are also in the market for the reliable and fully committed defender.
Celtic are said to be keen on a loan arrangement but so far a deal has not yet materialised.
Duffy was part of Brighton's training camp in Scotland earlier this week but it's unclear at this stage if he will feature for Albion in their pre-season friendly against Chelsea at the Amex Stadium on Saturday.
"He trained today with us. Trained hard and trained well," Potter said on Thursday night.
"We're having a pool and darts competition tonight as a bit of team building, so I imagine he'll be one of the favourites for that.
"He's a fantastic guy. Fantastic guy. I have an amazing relationship with him.
"He's a Brighton player and we're happy with that."