James Morrison struck a last-gasp equaliser to spare West Bromwich Albion's blushes as Bristol City forced a draw at the Hawthorns.
The Premier League side led when Saido Berahino curled the opener within four minutes of coming off the bench.
But Jonathan Kodjia headed the Championship side level, before substitute Kieran Agard tapped in.
They looked set to progress, but Morrison turned in a Jonny Evans knockdown in the fifth added minute.
Reaction to Saturday's FA Cup third-round games
Morrison's fourth goal of the season capped a remarkable final quarter of the tie, as all four goals arrived late and Steve Cotterill's side will ultimately be distraught they are not guaranteed a fourth-round position.
With only six named substitutes and their squad stretched, City were forced to introduce 19-year-old goalkeeper Max O'Leary at half-time in what was his first senior appearance for any club.
West Brom manager Tony Pulis spoke of his affection for the tournament and his frustrations with fixture congestion ahead of kick-off.
He will no doubt breathe a sigh of relief, but the prospect of a replay sandwiched between a trip to Southampton and a home fixture against Aston Villa will not be too enticing.
City could have been forgiven for fearing the worst against higher-level opposition as they arrived with just one win in 10.
But, roared on by 5,300 travelling fans, they showed spirit and had 49% of the ball, with Kodjia missing the game's best chance shortly before Berahino struck.
Currently 22nd in the Championship, a replay between the visit of Middlesbrough and a trip to Leeds will fully test an already stretched squad.
But they can draw huge positives from their fightback and just seconds stood between them being the only lower-league side to beat Premier League opposition on FA Cup third-round Saturday.
Pulis' side will rue a slow start, with only half chances for James McClean and Morrison before the break, though McClean did see a header cleared off the line after the interval.
Pulis - who refused to talk about a reported Tottenham move for Berahino this week - will face more questions over the 22-year-old in January, though the pair shook hands as the opening goal was celebrated.
Berahino has made just eight league starts this season and Tottenham's reported will to let Andros Townsend move as part of any deal perhaps indicates they badly want the young striker.
At the other end, the return of Ben Foster will provide Pulis with less of an ongoing headache.
In his first appearance since overcoming a cruciate knee-ligament injury sustained in March, Foster looked in control until Kodjia headed home and Luke Freeman's shot forced a parry to give Agard a simple tap in.
The Baggies deserve credit for not panicking when faced with embarrassment and they seem intent on mounting a challenge for a sixth FA Cup success with a strong team on show.
But, with just three wins on the road this season, they will have to summon a big effort at Ashton Gate.
West Brom boss Tony Pulis: "It's tough on them but I thought between the two boxes we played some good football and got in some wonderful positions, but our quality in the final third was just lacking.
"I'm pleased that we kept going and the players are fantastic in that respect, they're a fit group of lads. But we will certainly have to manage them now in that respect with the number of matches in the coming weeks."
Bristol City manager Steve Cotterill: "It was a great performance from us but often you don't get two opportunities against a Premier League team and I felt for our second-half performance we did deserve to win the game.
"That would have done wonders for our season in the second half of the Championship, it's so disappointing. It was important for us."
West Brom travel to Chelsea on Wednesday, while Bristol City can move out of the Championship's bottom three when they host Preston on Tuesday.
Full Time Match ends, West Bromwich Albion 2, Bristol City 2.
90:00 +5:26 Full timeFull Time Second Half ends, West Bromwich Albion 2, Bristol City 2.
90:00 +4:16 Goal scoredGoal! Goal! West Bromwich Albion 2, Bristol City 2. James Morrison (West Bromwich Albion) right footed shot from the centre of the box to the top right corner. Assisted by Jonny Evans.
90:00 +4:08Stéphane Sessegnon (West Bromwich Albion) wins a free kick in the defensive half.
90:00 +4:08Foul by Kieran Agard (Bristol City).
90:00 +2:53Corner, West Bromwich Albion. Conceded by Joe Bryan.
90:00 +2:31Saido Berahino (West Bromwich Albion) wins a free kick in the defensive half.
90:00 +2:31Foul by Marlon Pack (Bristol City).
89:28Attempt missed. Callum McManaman (West Bromwich Albion) right footed shot from outside the box misses to the right.
88:42Craig Dawson (West Bromwich Albion) wins a free kick in the defensive half.
88:42Foul by Kieran Agard (Bristol City).
86:20Foul by Saido Berahino (West Bromwich Albion).
86:20Korey Smith (Bristol City) wins a free kick in the defensive half.
82:58 Goal scoredGoal! Goal! West Bromwich Albion 1, Bristol City 2. Kieran Agard (Bristol City) right footed shot from the right side of the six yard box to the centre of the goal.
82:01 SubstitutionSubstitution Substitution, Bristol City. Kieran Agard replaces Jonathan Kodjia.
81:36 SubstitutionSubstitution Substitution, Bristol City. Wes Burns replaces Aaron Wilbraham.
78:50Delay in match James Morrison (West Bromwich Albion) because of an injury.
77:05Chris Brunt (West Bromwich Albion) wins a free kick in the attacking half.
77:05Foul by Mark Little (Bristol City).
74:09 BookingBooking Jonathan Kodjia (Bristol City) is shown the yellow card for excessive celebration.
Goal! Goal! West Bromwich Albion 1, Bristol City 1. Jonathan Kodjia (Bristol City) header from very close range to the top right corner. Assisted by Luke Freeman with a cross.
72:35Foul by Jonny Evans (West Bromwich Albion).
72:35Jonathan Kodjia (Bristol City) wins a free kick in the defensive half.
71:27 BookingBooking Nathan Baker (Bristol City) is shown the yellow card for a bad foul.
71:21Saido Berahino (West Bromwich Albion) wins a free kick in the defensive half.
71:21Foul by Nathan Baker (Bristol City).
68:23Corner, West Bromwich Albion. Conceded by Nathan Baker.
68:11Attempt blocked. Saido Berahino (West Bromwich Albion) right footed shot from the left side of the box is blocked.
66:48 Goal scoredGoal! Goal! West Bromwich Albion 1, Bristol City 0. Saido Berahino (West Bromwich Albion) right footed shot from the centre of the box to the bottom right corner.
66:00Callum McManaman (West Bromwich Albion) wins a free kick in the defensive half.
66:00Foul by Jonathan Kodjia (Bristol City).
65:14Corner, West Bromwich Albion. Conceded by Luke Ayling.
62:33 SubstitutionSubstitution Substitution, West Bromwich Albion. Callum McManaman replaces James McClean.
62:30 SubstitutionSubstitution Substitution, West Bromwich Albion. Saido Berahino replaces Rickie Lambert.
62:25 SubstitutionSubstitution Substitution, West Bromwich Albion. Craig Dawson replaces James Chester.
60:50Hand ball by Jonathan Kodjia (Bristol City).
57:13Attempt missed. Jonathan Kodjia (Bristol City) right footed shot from the right side of the box is close, but misses to the right.
55:43Attempt missed. Salomón Rondón (West Bromwich Albion) right footed shot from the centre of the box is close, but misses to the left.
52:08Attempt blocked. James McClean (West Bromwich Albion) header from the centre of the box is blocked.
50:44Stéphane Sessegnon (West Bromwich Albion) wins a free kick in the defensive half.
50:44Foul by Marlon Pack (Bristol City).
49:07Attempt missed. Stéphane Sessegnon (West Bromwich Albion) right footed shot from the right side of the box is close, but misses to the right.
47:08Foul by James McClean (West Bromwich Albion).
47:08Mark Little (Bristol City) wins a free kick in the defensive half.
45:25Corner, West Bromwich Albion. Conceded by Nathan Baker.
45:00Second Half begins West Bromwich Albion 0, Bristol City 0.
45:00 SubstitutionSubstitution Substitution, Bristol City. Max O'Leary replaces Frank Fielding because of an injury.
45:00 +1:17 Half timeHalf Time First Half ends, West Bromwich Albion 0, Bristol City 0.
45:00 +1:09Jonny Evans (West Bromwich Albion) wins a free kick in the attacking half.
45:00 +1:09Foul by Aaron Wilbraham (Bristol City).
44:42Attempt missed. James Morrison (West Bromwich Albion) right footed shot from the centre of the box misses to the left.
41:32 BookingBooking James Morrison (West Bromwich Albion) is shown the yellow card for a bad foul.
41:28Foul by James Morrison (West Bromwich Albion).
41:28Korey Smith (Bristol City) wins a free kick in the attacking half.
38:26Attempt missed. Salomón Rondón (West Bromwich Albion) header from the centre of the box is close, but misses to the left.
35:15Attempt missed. Luke Freeman (Bristol City) right footed shot from outside the box misses to the right.
32:31Corner, West Bromwich Albion. Conceded by Mark Little.
32:26Attempt blocked. James McClean (West Bromwich Albion) left footed shot from the left side of the box is blocked.
29:04Darren Fletcher (West Bromwich Albion) wins a free kick in the attacking half.
29:04Foul by Luke Freeman (Bristol City).
20:39Attempt saved. Rickie Lambert (West Bromwich Albion) right footed shot from the right side of the six yard box is saved in the centre of the goal.
19:23Attempt missed. Jonathan Kodjia (Bristol City) right footed shot from outside the box misses to the left.
18:19Attempt saved. James McClean (West Bromwich Albion) left footed shot from outside the box is saved in the centre of the goal.
18:13Attempt blocked. Chris Brunt (West Bromwich Albion) left footed shot from outside the box is blocked.
17:13Stéphane Sessegnon (West Bromwich Albion) wins a free kick in the attacking half.
17:13Foul by Joe Bryan (Bristol City).
16:51Jonny Evans (West Bromwich Albion) wins a free kick in the defensive half.
16:51Foul by Jonathan Kodjia (Bristol City).
15:00James McClean (West Bromwich Albion) wins a free kick in the defensive half.
15:00Foul by Mark Little (Bristol City).
10:52Attempt saved. Joe Bryan (Bristol City) left footed shot from outside the box is saved in the centre of the goal.
8:28Foul by Salomón Rondón (West Bromwich Albion).
8:28Luke Freeman (Bristol City) wins a free kick in the defensive half.
7:28Attempt missed. Korey Smith (Bristol City) right footed shot from outside the box misses to the right.
6:37Hand ball by Darren Fletcher (West Bromwich Albion).
3:50Foul by Nathan Baker (Bristol City).
3:50Rickie Lambert (West Bromwich Albion) wins a free kick in the defensive half.
0:26Corner, Bristol City. Conceded by Chris Brunt.
0:00First Half begins.
0:00Lineups are announced and players are warming up.
Live data and text provided by our data suppliers