Norris, 29, told BBC Radio Suffolk: "We won't be allowed to dwell on this, especially after the run we've had.
"We'll be back on it preparing for the next game because one result's not enough, we've only moved up one place, you need to fling a few together."
Roy Keane's side ran out 3-0 winners against the Foxes, who had scored five in their last outing against Doncaster.
And Norris says the Blues had to contend with mounting tension after a series of results that had seen them sink to 18th in the Championship.
"The run we've been on has been terrible, the pressure's been on the lads and the staff," he said.
"We needed to win, and I don't think many people gave us a chance against Leicester, who are flying, but that is a big win for us.
"A few days before the game you're thinking about the bad run, obviously it's on your mind and it's a bit more pressure."
Since mid-October Town have fallen from a play-off position to 17th in the Championship.
Despite the slump Norris believes his team-mates have shown glimpses of quality in the build up to the defeat of Leicester.
"Some of the games recently haven't been bad performances, overall we've had good spells in games and played well in halves maybe," Norris added.
"If we're really poor and getting hammered every week then I think we'd have to hold our hands up.
"It's a fine line in this league between winning and losing, at the end of the day we weren't good enough but hopefully this s a turning point."
Source: BBC Sport
Source: BBC Sport