An early Joshua Burgess try cancelled out New Milton’s opening score before the hosts went 12-5 up. A Paddy Burnett try was converted by Jeremy Newton Young – then the duo repeated the trick to carve open a 19-12 lead.
Tom Orchard was next to dot down but two converted home tries put them 26-24 up.
The sides swapped tries again, with Chris Hay touching down for Bognor, and it took a Darren bidwell try to give Bognor a one-point lead going into the final stages.
The New Milton 15 missed penalty that would have won the game with last play of game.
The win leaves Bognor level with Sandown – who visit them this weekend – on 64 points, well clear of fourth-placed Fordingbridge, who are 16 points behind.
Saturday’s big third-v-second clash at Hampshire Avenue kicks off at 2.30pm – all support welcome.