A former bank in Newhaven High Street will be turned into a three storey house with a roof terrace.
The owner of the old Natwest building was given the green light for the scheme after it had remained empty for two years.
But the loss of another business outlet in the town centre was a cause for concern and criticism from councillors and members of the public.
Newhaven district councillor Steve Saunders said: “The loss of the bank itself was hard to fathom for local residents and councillors alike.
“The continued chiselling away at the commercial heart of the town centre businesses that the district council has allowed has frustrated many Newhaven residents along with me and my fellow councillors.
“With so much money promised to the town recently by way of regeneration funding and investment, it seems ludicrous that we can sustain so many new houses without banks, health, education and other such infrastructure to support them.
“The parking provision for such a scheme will exacerbate the burden already suffered by local residents and surrounding areas.
“Plumpton and Chailey continue to be allowed to refuse housing projects in their areas, whilst Newhaven is expected to repeatedly have it forced upon them.”
However a report to the planning committee said the building was outside the primary shopping area.
The report said the bank was marketed for a total of six months and a public auction was held in 2014 where no offers were received.
Officers said this showed a lack of interest in the use of the building as a bank or for other financial or professional services.
The report said that the committee would need to consider the proposal for converting the bank on its own merits, adding that given the small windows it would be unsuitable for residential or retail use.
And Newhaven district councillor Rod Main said: “It was the town council’s view that it should be possible to have some sort of retail/cafe/outlet shop on the ground floor while the upper floors could easily be residential.
“I am not convinced that this has been shown to be too difficult to achieve.
“Nonetheless, the planning committee felt that it would be difficult to sustain this point as the only ground for refusal.
“It will make an impressive residential property but I wonder who will be able or want to afford such a Chateau-esque house in Newhaven.”