Long nodded home Brynjar Gunnarsson's cross in the first period of extra time to secure the Royals' progression through to the fourth round.
Liverpool - held 1-1 in the original tie 11 days ago - looked to have booked their ticket to the next round as they lead 1-0 with seconds to go in the tie courtesy of Ryan Bertrand's first-half own goal.
But Gylfi Sigurdsson took the tie to extra-time from the penalty spot in injury time before Long's heroics in the 100th minute won it for Reading.
The visitors started well with Gunnarsson stinging the palms of stand-in keeper Diego Cavalieri and Simon Church also going close.
Grzegorz Rasiak missed a sitter from close range on 22 minutes, before Jobi McAnuff's deep cross almost found the head of Church.
But the Premier League side lead at half-time when Steven Gerrard's cross was turned into his own net by full-back Bertrand.
It was Gerrard's last involvement in the match as he was replaced at half-time complaining of a hamstring strain and joined Fernando Torres - a first-half departure with a knee injury - in the Liverpool treatment room.
The Reds dominated play after the break but gave Reading hope by missing a host of chances.
Substitute Brian Howard kept things interesting for the Royals, teasing in a deep cross which Ivar Ingimarsson should have got on the end of.
Jobi McAnuff should have levelled on 80 minutes when his stunning run put him through on goal - but his effort rolled agonisingly wide of the upright.
But there was another twist in the tale as Yossi Benayoun brought down Long in the third minute of stoppage time and Sigurdsson stroked home the penalty to take the tie to extra time.
Liverpool had a golden chance to take the match to a penalty shoot-out after Long's header but keeper Adam Federici kept out Benayoun.