Adam Lallana's injury-time strike saw Liverpool edge a nine-goal thriller at Norwich in one of the most dramatic Premier League games of the season.
Roberto Firmino scuffed the Reds ahead but Dieumerci's Mbokani's backheel levelled and Steven Naismith's debut strike put the home side in front.
Wes Hoolahan added a penalty before Jordan Henderson and Firmino struck.
James Milner put the away side ahead and though Sebastian Bassong netted in added time, Lallana delivered a twist.
Relive Liverpool's incredible win at Norwich
The England midfielder's volley - which was fired into the ground and bounced agonisingly in for the home side - sparked chaotic scenes as Reds boss Jurgen Klopp raced along the touchline to celebrate with his players.
Just moments earlier, Klopp had been visibly incensed by his side's lapse in allowing Bassong to level - but the change in his mood was fitting in a game which ebbed and flowed like this one.
The German had his glasses knocked off by striker Christian Benteke during the celebrations and later said: "I have a second pair of glasses but I can't find them. It's really difficult looking for glasses without glasses."
The contrast in emotion on the touchline was as pronounced as anything the season has delivered so far and Norwich - who could be in the bottom three by the end of the day - will have to recover before Tuesday's trip to Tottenham.
Their record now stands at 32 Premier League games without a win in matches where they have fallen behind - but having been 3-1 up, few will sting like this.
The first five shots on target in this game found the net and neither side could claim to have deserved a win, with each guilty of lapses which at times led to chaos and resulted in just the fourth 5-4 result in Premier League history.
Russell Martin will feel worse than most after gifting Milner the chance to put Liverpool 4-3 up with a weak backpass.
And though Bassong's crisp strike from 20 yards appeared to have reprieved his fellow defender, substitute Lallana's first league goal of the season capped a frenzied finish.
The Reds are four points worse off than at this stage last season but could this win be a catalyst for a spell of consistency and a move higher than seventh place?
One wonders what reception Liverpool's players would have got from the German had it stayed 4-4, such was his demeanour, and his side masked some common inefficiencies in taking three points.
Mbokani's brilliant back-heeled finish arrived after failing to clear a corner, meaning the Reds have conceded eight goals from corners this season - a league high.
Their passing - at 75% accuracy - was not at its best, their defending was sometimes rash but they found key goals, despite having scored fewer times than relegation-threatened Sunderland before kick-off.
Much has been made of the need to sign a striker but after such a result, neglecting their defensive issues seems a big risk for Klopp who will surely be keen to have less stressful afternoons in the dugout.
"When you concede five goals you deserve to lose the game," concluded Norwich boss Alex Neil after the match.
The Scot admitted defensive errors have been his side's "downfall" this season and the 43 goals they have leaked in 23 games paints a damning picture - and only Sunderland (45) have let in more.
However, the performance of Naismith at the other end does offer something positive to take forward.
Making his debut alongside full-back Ivo Pinto - who had a tough afternoon - Naismith showed quality throughout.
He managed 18 goals in 52 league starts at Everton but may need to adopt the role of talisman rather than impact substitute at Norwich if they are to survive.
That said, they have now conceded 11 goals in three games - a stat which must improve if any Naismith contribution is to prove telling.
Norwich travel to Tottenham in the Premier League on Tuesday and Liverpool will be in action at the same time as they try to protect their 1-0 League Cup semi-final first-leg lead against Stoke City.
Full Time Match ends, Norwich City 4, Liverpool 5.
90:00 +5:33 Full timeFull Time Second Half ends, Norwich City 4, Liverpool 5.
90:00 +4:40 BookingBooking Adam Lallana (Liverpool) is shown the yellow card for excessive celebration.
90:00 +4:19 Goal scoredGoal! Goal! Norwich City 4, Liverpool 5. Adam Lallana (Liverpool) left footed shot from the centre of the box to the bottom right corner.
90:00 +4:12Attempt blocked. Steven Caulker (Liverpool) right footed shot from the centre of the box is blocked.
90:00 +1:57 Goal scoredGoal! Goal! Norwich City 4, Liverpool 4. Sebastien Bassong (Norwich City) left footed shot from outside the box to the bottom right corner. Assisted by Cameron Jerome following a set piece situation.
90:00 +1:33Offside, Liverpool. James Milner tries a through ball, but Christian Benteke is caught offside.
90:00 +0:59Attempt blocked. James Milner (Liverpool) right footed shot from outside the box is blocked. Assisted by Christian Benteke with a headed pass.
89:31 SubstitutionSubstitution Substitution, Liverpool. Steven Caulker replaces Alberto Moreno.
88:39Adam Lallana (Liverpool) wins a free kick in the defensive half.
88:39Foul by Martin Olsson (Norwich City).
88:05Foul by Lucas Leiva (Liverpool).
88:05Dieumerci Mbokani (Norwich City) wins a free kick in the defensive half.
84:11Attempt saved. Roberto Firmino (Liverpool) right footed shot from the left side of the box is saved in the centre of the goal. Assisted by Christian Benteke.
81:19 SubstitutionSubstitution Substitution, Norwich City. Cameron Jerome replaces Steven Naismith.
76:10Foul by Alberto Moreno (Liverpool).
76:10Matthew Jarvis (Norwich City) wins a free kick in the defensive half.
75:23 SubstitutionSubstitution Substitution, Liverpool. Christian Benteke replaces Jordan Henderson.
74:07 Goal scoredGoal! Goal! Norwich City 3, Liverpool 4. James Milner (Liverpool) right footed shot from the centre of the box to the centre of the goal.
72:40Attempt saved. Alberto Moreno (Liverpool) left footed shot from outside the box is saved in the centre of the goal. Assisted by Roberto Firmino.
Corner, Liverpool. Conceded by Dieumerci Mbokani.
70:25Corner, Liverpool. Conceded by Ivo Pinto.
69:45 SubstitutionSubstitution Substitution, Norwich City. Matthew Jarvis replaces Nathan Redmond.
69:26 SubstitutionSubstitution Substitution, Norwich City. Martin Olsson replaces Wes Hoolahan.
68:11Attempt saved. Sebastien Bassong (Norwich City) header from the centre of the box is saved in the bottom right corner. Assisted by Nathan Redmond.
67:40Foul by Alberto Moreno (Liverpool).
67:40Ivo Pinto (Norwich City) wins a free kick on the right wing.
66:50Foul by Kolo Touré (Liverpool).
66:50Steven Naismith (Norwich City) wins a free kick on the left wing.
65:07Offside, Liverpool. Lucas Leiva tries a through ball, but Adam Lallana is caught offside.
64:38Kolo Touré (Liverpool) wins a free kick in the defensive half.
64:38Foul by Dieumerci Mbokani (Norwich City).
63:36Offside, Liverpool. Jordan Henderson tries a through ball, but Adam Lallana is caught offside.
62:39 Goal scoredGoal! Goal! Norwich City 3, Liverpool 3. Roberto Firmino (Liverpool) right footed shot from the centre of the box to the bottom left corner. Assisted by Adam Lallana.
58:25 SubstitutionSubstitution Substitution, Liverpool. Adam Lallana replaces Jordon Ibe.
57:35Hand ball by Nathaniel Clyne (Liverpool).
54:46 Goal scoredGoal! Goal! Norwich City 3, Liverpool 2. Jordan Henderson (Liverpool) right footed shot from the centre of the box to the bottom left corner. Assisted by Roberto Firmino.
53:06 Goal scoredGoal! Goal! Norwich City 3, Liverpool 1. Wes Hoolahan (Norwich City) converts the penalty with a left footed shot to the centre of the goal.
52:11Penalty Norwich City. Steven Naismith draws a foul in the penalty area.
52:11Penalty conceded by Alberto Moreno (Liverpool) after a foul in the penalty area.
50:07Foul by Nathan Redmond (Norwich City).
50:07Jordon Ibe (Liverpool) wins a free kick in the defensive half.
49:24Foul by Roberto Firmino (Liverpool).
49:24Sebastien Bassong (Norwich City) wins a free kick in the defensive half.
48:50James Milner (Liverpool) wins a free kick in the defensive half.
48:50Foul by Wes Hoolahan (Norwich City).
48:35Delay over. They are ready to continue.
48:13Delay in match Roberto Firmino (Liverpool) because of an injury.
45:18Hand ball by James Milner (Liverpool).
45:00Second Half begins Norwich City 2, Liverpool 1.
45:00 +2:10 Half timeHalf Time First Half ends, Norwich City 2, Liverpool 1.
45:00 +0:07Offside, Liverpool. James Milner tries a through ball, but Roberto Firmino is caught offside.
44:19Foul by Robbie Brady (Norwich City).
44:19Jordan Henderson (Liverpool) wins a free kick on the right wing.
40:36 Goal scoredGoal! Goal! Norwich City 2, Liverpool 1. Steven Naismith (Norwich City) right footed shot from the right side of the box to the bottom left corner. Assisted by Wes Hoolahan.
39:48Corner, Norwich City. Conceded by Nathaniel Clyne.
39:25Dieumerci Mbokani (Norwich City) wins a free kick on the right wing.
39:25Foul by Alberto Moreno (Liverpool).
37:23Foul by Nathan Redmond (Norwich City).
37:23Mamadou Sakho (Liverpool) wins a free kick in the defensive half.
35:04Corner, Norwich City. Conceded by Lucas Leiva.
35:00Attempt blocked. Jonny Howson (Norwich City) right footed shot from the centre of the box is blocked. Assisted by Dieumerci Mbokani.
33:41Steven Naismith (Norwich City) wins a free kick on the right wing.
33:41Foul by Roberto Firmino (Liverpool).
31:02Delay in match Dieumerci Mbokani (Norwich City) because of an injury.
31:02Delay over. They are ready to continue.
30:32Dieumerci Mbokani (Norwich City) wins a free kick in the defensive half.
30:32Foul by Lucas Leiva (Liverpool).
29:58Offside, Norwich City. Robbie Brady tries a through ball, but Dieumerci Mbokani is caught offside.
29:37Attempt blocked. Nathaniel Clyne (Liverpool) right footed shot from outside the box is blocked.
28:04 Goal scoredGoal! Goal! Norwich City 1, Liverpool 1. Dieumerci Mbokani (Norwich City) right footed shot from the centre of the box to the centre of the goal. Assisted by Graham Dorrans with a headed pass following a corner.
27:04Corner, Norwich City. Conceded by Simon Mignolet.
26:18Dieumerci Mbokani (Norwich City) wins a free kick on the right wing.
26:18Foul by Mamadou Sakho (Liverpool).
25:28Foul by Dieumerci Mbokani (Norwich City).
25:28Alberto Moreno (Liverpool) wins a free kick in the defensive half.
24:33Attempt missed. Alberto Moreno (Liverpool) left footed shot from outside the box misses to the left. Assisted by James Milner following a set piece situation.
23:17Foul by Sebastien Bassong (Norwich City).
23:17Roberto Firmino (Liverpool) wins a free kick in the attacking half.
22:10Attempt blocked. Jordan Henderson (Liverpool) right footed shot from outside the box is blocked. Assisted by Jordon Ibe.
20:49Corner, Liverpool. Conceded by Robbie Brady.
17:50 Goal scoredGoal! Goal! Norwich City 0, Liverpool 1. Roberto Firmino (Liverpool) left footed shot from the left side of the box to the bottom right corner. Assisted by James Milner with a through ball.
17:01Foul by Ivo Pinto (Norwich City).
17:01Jordon Ibe (Liverpool) wins a free kick in the defensive half.
15:56Attempt missed. Roberto Firmino (Liverpool) right footed shot from the centre of the box is high and wide to the right. Assisted by Alberto Moreno.
12:23Steven Naismith (Norwich City) wins a free kick on the right wing.
12:23Foul by Mamadou Sakho (Liverpool).
4:45Foul by Steven Naismith (Norwich City).
4:45Lucas Leiva (Liverpool) wins a free kick in the defensive half.
1:36Dieumerci Mbokani (Norwich City) wins a free kick in the attacking half.
1:36Foul by Emre Can (Liverpool).
0:00First Half begins.
0:00Lineups are announced and players are warming up.
Live data and text provided by our data suppliers