Ethan Robb's goal was cancelled out by a Tom Kavanagh penalty - then just when it looked like Jordy Mongoy had earned the Rocks victor,y Tom Richards popped up with an injury-time equaliser for the Hornets.
Horsham will feel it was a deserved point for their efforts whereas Bognor felt disappointed after holding their lead until the 94th minute.
Bognor were still without Jake Flannigan and Gavin McCallum due to injury but Charlie Bell was back from suspension and Calvin Davies from injury.
Eddie D'Sane won an early corner kick on the left but Lee Harding curled it into the near side netting. Joe Cook burst forward and won a corner on the right. Davies lofted it into the area and it was headed out by a Hornets defender before it fell directly to Robb who hit it on the volley, but it flew well wide.
Harding's cross from the right had to be dealt with by Davies and he managed to win a free-kick in his own team's penalty box.
D'Sane slipped Harding in with a through ball unmarked on 19 minutes. Harding had only Amadou Tangara to beat but the goalkeeper dived to his left to parry the ball away and then Joe Dandy gathered the ball before clearing.
Cook conceded a free-kick with a foul on D'Sane as he ran inside. Kavanagh's free-kick was headed away by Harvey Whyte on 22 minutes. Kavanagh then combined with D'Sane before hitting it inside the box but it came off Dandy before rolling to Tangara.
Mongoy won a corner after trying to spin round Chris Sessegnon. Davies' corner was played short to Whyte who played it back to Robb who, from about 25 yards out, hit it low into the bottom corner, beating the outstretched Sam Howes to give the Rocks the lead on 27 minutes.
Then Sessegnon sped down the right to float a cross in towards the back post. It was Cook who managed to head it away from Rob O'Toole. Ex-Rock Harvey Sparks got a cross in which was deflected into the arms of Tangara from the left. Harding went into the book on 34 minutes after having strong words with the referee. Then Harding ran across the defence before shooting it along the deck at goal. This forced Tangara into a low save, pushing the ball back out.
D'Sane did well to combine with Kavanagh before the striker played it out on the overlap to Sparkes, who sped on before smashing his cross along the face of goal and out for a goal kick. Robb played the ball down the line for Mongoy who won a corner when his cross ricocheted off a defender. Though Davies' corner was disappointing as it flew beyond the reach of Craig Robson in the area.
Robb's pass saw Dan Gifford through as he bamboozled his marker before eventually shooting. The ball came off the ankle of Howes and it bounced agonisingly wide of the left post on 41 minutes. Another long ball saw Gifford win possession again with the Hornets stranded at the back. He picked out Nathan Odokonyero who ran inside but was blocked bravely with his strike from the defender. D'Sane played it back to Kavanagh who hit it first time and it sailed over the bar following a deflection.
Horsham had the ball in the net inside stoppage time but it didn't count. O'Toole fired the ball low at goal and Tangara dived to his left to push it out. It fell directly to the onrushing Shamir Fenelon who knocked it in, but he was offside. To end the half, a long through ball saw Odokonyero set running on goal. But from outside the corner of the box he curled his shot into the side netting after cleverly passing a stranded Howes. HT 1-0
O'Toole and Robson clashed trying to gather a long ball on the right flank. A free-kick was played by Kavanagh but it was cleared away by the Bognor defence. Dandy made a rare run through the middle but Gifford was standing in an offside position following his pass towards Mongoy.
Sparks went down in the penalty area on 52 minutes following his advance but the referee waved it away. Bognor failed to clear the danger and lost possession on the edge of their own box. Cook's flailing leg saw D'Sane go to ground in the box and the referee pointed straight to the spot. Kavanagh stepped up to slam it high into net as Tangara dived to his right.
Horsham's tails were up after the equaliser. Kavanagh's corner found Harding but he directed his header spinning over the bar. A double substitution saw Kayne Diedrick-Roberts and Ashton Leigh replace Davies and Robson on 64 minutes in a much needed change. D'Sane went to ground in the box as Cook was marking him following a low Sessegnon cross but the referee waved that away as well.
Whyte went into the book on 70 minutes after a challenge on Fenelon, who was then substituted as he made way for Tom Richards in the same minute for the visitors. Bognor won a rare corner as Diedrick-Roberts' strike was deflected wide. Leigh loaded the ball in for Cook who headed it at goal. But he was denied by Sessegnon who headed it away off the goal line on 74 minutes.
Whyte went to ground following the corner but the referee ignored claims for a penalty as Howes gathered it in the air.
Bognor got their second goal on 77 minutes. A long ball forward by Leigh to Diedrick-Roberts saw him run on before cutting the ball back along the ground to Mongoy. He took his time to turn inside before superbly curling his shot high into the top corner of the net.
Horsham responded with a great chance of their own. D'Sane ran through and his powerful shot from a diagonal angle was confidently pushed away by Tangara who was making an outstanding difference to keep Bognor ahead. Then Whyte took a knock and had to be replaced by James Crane on 81 minutes.
Leigh combined well with Odokonyero who ran on before cutting inside onto his left foot before firing it over the bar on 84 minutes. A hard-working Cook carried the ball passed the halfway line before passing forward to Mongoy. He cut inside before firing it low and only just wide of the left post.
Cook was back to defend as well sliding in at the last moment to deny D'Sane a clear goal scoring opportunity with a fine tackle timed to perfection. Charlie Harris got a boot to another inswinging corner for Horsham. Tangara was there to deny him though as time was running outvfor the visitors.
Kavanagh spotted Tangara off his line and lobbed the goalkeeper. It flew over the bar though, much to the relief of the Bognor support.
But the game still wasn't over as Horsham got the equaliser deep into stoppage time. D'Sane hit the initial shot at goal following a forward pass that split the defence. Tangara managed to push it on to the post but that left the goal open for Richards to tap into the net. There was a suspicion of handall but the goal stood and the sides shared the points in an exciting encounter.
The Rocks now regroup for their next game this Tuesday when they welcome Worthing, this time in the Sussex Senior Cup. Kick off is 7.45pm.
Horsham head to Cheshunt next weekend.