It was a penalty from Barry Robson midway through the second half which resulted in the home side's winner.
Scott McDonald put Celtic ahead in the fourth minute, before goals from David Weir and Daniel Cousin gave Rangers the advantage.
McDonald drew the teams level just before half-time and it was the Celtic striker who won the penalty after being brought down by Kirk Broadfoot.
Title favourites Rangers still have three games in hand, but can only afford to lose one other match.
Plus, with the fixtures piling up, it will be a huge ask of the Ibrox side to keep their eye on the ball in the race for the title.
Celtic made an ideal start with McDonald driving home Jan Vennegoor of Hesselink's knock down from a suspiciously offside position.
Despite being one down, Rangers dominated the opening proceedings and drew level when Weir headed home a Steven Davis corner.
Celtic responded with a tremendous free-kick by Shunsuke Nakamura which keeper Neil Alexander touched over at full stretch.
Rangers took the lead with a goal almost identical to the first, but with Cousin glancing home Davis' corner off the underside of the bar.
Celtic drew level just before half-time when Aiden McGeady threaded a pass through to McDonald, whose strike went over Alexander with the help of a deflection off Christian Dailly.
Celtic nearly went ahead at the start of the second half but Davis took the ball off McDonald as the striker looked certain to score.
Vennegoor of Hesselink was lucky to stay on the park after a high tackle on substitute Amady Faye before Celtic took the lead.
A long through ball found McDonald at the edge of the area and he was brought down by a clumsy challenge from Broadfoot.
Robson drilled his penalty low past Alexander and, to round off a miserable afternoon for the visitors, Steven Whittaker was sent off in the last minute for a second yellow card.