Roberts, 19, has been included in the Premier League match day squads this season but has yet to make an appearance in the top flight.
Roberts, who enjoyed a successful loan at Rochdale last season, has impressed in the Carabao Cup matches and benefitted from training with established Premier League players.
But Roberts is keen for game time and develop his career as he is behind the likes of Joel Veltman, Shane Duffy, Lewis Dunk, Adam Webster, Dan Burn and Marc Cucurella in the pecking order at the Amex.
Championship outfit Nttingham Forest are said to keen but League Sheffield Wednesday continue to have a long standing interest. Wednesday boss Darren Moore is keen to bolster the defensive ranks at Wednesday and previously worked well with another Albion loanee Taylor Richards while Moore was in charge at Doncaster.
Richards thrived under the guidance of Moore and has gone on to secure a further loan away from Brighton in the Championship with Birmingham.
Potter feels a loan at this stage on Roberts career could well be the right move.
"It is difficult for Haydon, he is competing in a really strong area of the pitch for us," said Potter. "I think he has to be patient, he probably has to keep working, might have to look at a loan situation for him to play some football.
"You have to look at our defensive record - one of the best in the Premier League. So the guys that have done it have done it well.
"So it is just unfortunate that he is at the start of his career and he has to probably be patient and wait for his opportunity."
Asked if any loan for Roberts could be delayed due to an ankle injury to Duffy, Potter added: "The (loan) decision wouldn't necessarily be taken on the back of that.
"We want him to enjoy his football, keep progressing. He's done that in the first part of the season, but it's important for him to play as well.
"We're looking at the right solution for him. It's not just a case of Haydon can go on loan, you've got to find the right option, make sure everything fits for him, for us and the club that wants him.
"We will continue to look at that, but we're not so stressed about it. We'll see. It's important that he continues to develop."
Roberts options were further limited at Albion as full back Cucurella roved he's more than capable of playing on the left side of a defensive trio during his impressive display in the 1-1 draw at Chelsea last month.
"It was Chelsea and he did it really well," said Potter of Cucurella. "Everyone loves him, he's a great guy around the place.
"A good personality, a good character, so he's added to that. And then he's just adjusted really well. It's basically just about how he goes about it every day.
"He wants to do his best. You can see the way he wants to play football. He's very fresh, wants to run, wants to fight for the team, he's got his quality.
"You look at his CV, you can understand why he's got what he has.
"He was an important signing for us. If you remember against Everton, we lost Solly, we didn't have anybody on the left. It was probably one of our worst performances, we lacked a bit of balance, so he's given us that option. I'm really pleased for him, because he's done really well."