In a remarkable end to the match, Rangers striker Dado Prso was red carded for a second yellow card and then Lithuanian Saulius Mikoliunas was sent off for barging linesman Andy Davies.
It was Mr Davies who awarded the penalty when it was ruled that Sotirios Kyrgiakos was pushed in the area.
Hearts could also find themselves in trouble with the SFA when missiles were thrown on to the pitch after the final whistle.
Until then, it looked as if an 87th minute strike from substitute Mark Burchill had given Hearts a point.
Rangers had led through a 49th minute goal from Nacho Novo and had looked comfortable before Burchill's strike.
The home side were without skipper Steven Pressley due to a virus and Rangers hit them hard early on.
Prso shot against a post but the offside flag was already up and Novo almost set up Ricksen, but Deividas Cesnauskis managed to make the crucial interception.
But Hearts hit back and won three corners in quick succession but the Rangers defence coped comfortably. Robbie Neilson came closest with a shot which was deflected over the top.
Paul Hartley had another effort on goal when he shot narrowly wide as the home side threatened the breakthrough.
A mistake by Ronald Waterreus gave Cesnauskis an opening after 18 minutes when he was just ten yards out, but the Lithuanian blasted the ball well over the top.
But the goalkeeper made amends shortly afterwards when the ball broke to Mikoliunas in the area and Waterreus managed to somehow touch his shot on to the post.
Rangers were under the cosh at this stage and Lee Miller was causing them problems with his ability in the air.
Cesnauskis and Mikoliunas kept switching wings to give Rangers fresh problems but, for all their possession, Hearts did not find too many openings.
Alex McLeish's side were always dangerous on the break and Novo shaved the post with a fine effort which had Craig Gordon beaten.
Rangers started the second half strongly and a slip by Lee Wallace almost let them in, but the alert Gordon came to his team-mate's rescue to clear his lines.
Then, Kyrgiakos headed wide from a free-kick when he should have done better.
The goal duly arrived in the 49th minute. Prso pounced on a long clearance from Marvin Andrews and pulled the Hearts' defence apart before releasing Novo, who scored with a precise finish.
Hearts had calls for a penalty turned down after Kyrgiakos was guilty of a clumsy challenge on Miller in a crowded area.
But the goal had rejuvenated Rangers and their passing was crisper and they looked increasingly dangerous going forward with Prso and Novo linking well.
Gordon produced a fine save to beat out a netbound shot from Ricksen and Andy Webster managed to clear before the lively Novo could take advantage.
The industrious Prso caused Hearts problems again with his running but his cutback found no takers in the area.
Hearts sent on Marius Kizys, Burchill and Joe Hamill in the closing stages to try and salvage it.
Waterreus did well to get his hand to an 87th minute Jamie McAllister free-kick, but he could not prevent the ball rolling to Burchill who had the simple task of stroking it home.
Then the game blew up with the late penalty award and Prso and Mikoliunas receiving their marching orders.