County League Division 2 leaders EP went ahead after 22 minutes when a lovely team move, involving Matt Axell and George Gaskin, was finished off by Charlie Oatway.
The lead did not last long as Horsham levelled when Ray Freeman reacted quickest to tap home after Alex Bryant’s save just six minutes later.
EP regained the lead eight minutes into the second half when Joe Shelley headed home a Lewis Hamilton free kick but, again, Horsham quickly equalised as Wayne Clark levelled five minutes later, with EP appealing for offside.
Tony Nwachukwu gave the visitors the lead from the penalty spot after a foul by Bryant on 70 minutes but EP responded well and levelled three minutes later when Gaskin bundled home a Oatway cross.
Horsham went back ahead with 10 minutes to go through Nwachukwu, before late goals from Freeman and Clark put a gloss on the score for the visitors.
EP manager Dominic Di Paola said: “The scoreline makes it look like we got spanked but we probably could have won it, to be honest.
“We went ahead twice but didn’t stay ahead long enough to get a foothold in the game and we conceded some soft goals.
“Football-wise, we can play teams off the park but we keep conceding soft goals when we come up against the better teams, like Three Bridges, Pagham, Lingfield and now Horsham.
“A couple of decisions I made during the game cost us and I’m disappointed as it’s a cup that I thought we had a chance of winning.
“They just had that bit more athleticism and stamina towards the end as we ran out of gas a bit, but that’s because they’re four divisions above us.”