Heath started well before Chi settled and Lorenzo Dolcetti controlled things.
City keeper Ant Ender, in the form of his life in goal, made a string of good saves to keep Heath out.
Chi had good chances with Jimmy Wild and Dave Herbert linking up well and Rob Hutchings started to cause Heath problems with his energy and direct runs in behind.
It was goalless at half-time but City broke the deadlock on 57 minutes when Wild found Herbert on the edge of the box and he found the bottom corner.
Heath started to slow the game down and a very soft penalty converted by Callum Saunders let Heath back into it.
Chi were reduced to ten men when Kieran Hartley was shown his second yellow in the space of ten minutes.
Heath took advantage of their extra man and Max Miller headed the winner in the closing minutes.
Chi felt robbed and coach Danny Potter said: “The result is very disappointing, especially after we caused them so many problems early in the second half.
“We need to learn from our mistakes quickly and we must get back to winning ways in the league now.
“We’re still in the running for the league and we still have a cup final to look forward to. We wish Haywards Heath all the best in the next round and we will meet them again in the last game of the season when we play them in the league at home.”