Reece Myles-Meekums fired the hosts in front, after brilliant wingplay by Ola Ogunwamide, before James Hammond tucked his penalty past Millers keeper Harrison Chamberlain.
Former Hornet Tom Beere struck at the death to set up a nervy finale, but Horsham held on to claim a valuable three points.
The 2-1 win moves the third-placed Hornets six point clear of place-below Cray Valley, although the Millers have a game in hand.
Victory also moved Horsham to within a point of second-placed Dartford, who suffered a surprise defeat at Cheshunt, and three off leaders Billericay Town, who left it late to beat Carshalton Athletic at home.
The Hornets travel to Dartford this coming Saturday.
See some of John Lines’ pictures from the game on this page and the ones linked, and read Horsham manager Dominic Di Paola’s thoughts in the West Sussex County Times, out every Thursday.
53. Excellent first half display helps Horsham FC past promotion rivals Cray Valley Paper Mills – in pictures
A superb first half performance helped Horsham FC past Isthmian Premier promotion rivals Cray Valley Paper Mills Photo: John Lines
54. Excellent first half display helps Horsham FC past promotion rivals Cray Valley Paper Mills – in pictures
A superb first half performance helped Horsham FC past Isthmian Premier promotion rivals Cray Valley Paper Mills Photo: John Lines
55. Excellent first half display helps Horsham FC past promotion rivals Cray Valley Paper Mills – in pictures
A superb first half performance helped Horsham FC past Isthmian Premier promotion rivals Cray Valley Paper Mills Photo: John Lines
56. Excellent first half display helps Horsham FC past promotion rivals Cray Valley Paper Mills – in pictures
A superb first half performance helped Horsham FC past Isthmian Premier promotion rivals Cray Valley Paper Mills Photo: John Lines