Shamir Fenelon’s 33-minute double had the hosts on course for the fourth qualifying round – but Finley Marjoram hit back for Invicta with three minutes to go until half-time.
And a disastrous start to the second period for Horsham saw the Seasiders turn the game on its head, Ben Mason and Josh Strouts netting within the opening eight minutes of the half to send the Hornets crashing out.
The result continues Folkestone's dominance against Horsham, with seven wins from the last eight meetings between the two sides.
The 13th-placed Hornets are back in National League South action this Saturday at top-of-the-table Hornchurch.
See some of Natalie Mayhew’s pictures from the game on this page and the ones linked – and you can read all the latest from Horsham FC in the West Sussex County Times, out every Thursday.
5. FA Cup: Horsham FC blow two-goal lead in third qualifying round defeat to Folkestone - the match in pictures
Horsham FC contrived to throw away a two-goal lead as division-below Folkestone Invicta roared back to beat the Hornets 3-2 in the FA Cup third qualifying round. Photo: Natalie Mayhew
6. FA Cup: Horsham FC blow two-goal lead in third qualifying round defeat to Folkestone - the match in pictures
Horsham FC contrived to throw away a two-goal lead as division-below Folkestone Invicta roared back to beat the Hornets 3-2 in the FA Cup third qualifying round. Photo: Natalie Mayhew
7. FA Cup: Horsham FC blow two-goal lead in third qualifying round defeat to Folkestone - the match in pictures
Horsham FC contrived to throw away a two-goal lead as division-below Folkestone Invicta roared back to beat the Hornets 3-2 in the FA Cup third qualifying round. Photo: Natalie Mayhew
8. FA Cup: Horsham FC blow two-goal lead in third qualifying round defeat to Folkestone - the match in pictures
Horsham FC contrived to throw away a two-goal lead as division-below Folkestone Invicta roared back to beat the Hornets 3-2 in the FA Cup third qualifying round. Photo: Natalie Mayhew