Manager Walter Smith made three changes from Parkhead with new signing Maurice Edu from Toronto making his debut, Damarcus Beasley and Kris Boyd replacing the injured Kevin Thomson, Charlie Adam and the departed Daniel Cousin.
Killie stunned the home side with the opening goal in just eight minutes when Garry Hay's hanging cross saw Allan McGregor bundled off the ball in mid-air by Frazer Wright who wasted no time in turning the loose ball into the net. It appeared to almost everyone inside the ground that the goalkeeper had been impeded - but Referee Calum Murray saw nothing amiss and awarded the goal.
Seven minutes later the scores should have been level when Edu released Beasley through on goal only for the American winger to hesitate as he rounded visiting 'keeper Alan Combe.
Rangers had the bulk of the pressure - and came close to restoring parity in 27 minutes when Kenny Miller's low cross was inches away from the inrushing Boyd.
The equaliser should have arrived seven minutes later when Miller's run across the edge of the area found Boyd in the clear, the centre's shot being deflected over the bar by Combe's outstretched leg.
Nacho Novo replaced the ineffectual Beasley at the interval, and the scores were levelled in 57 minutes when Boyd netted from the penalty spot after Manuel Pascali had hauled the substitute to the ground.
Pascali was yellow-carded for his troubles - but the greater mystery was how Combe managed to escape punishment after picking up the ball and throwing it at Boyd as he prepared to take the kick. Only the referee can explain that one...
It was the first League goal Kilmarnock had conceded this season - and four minutes later they lost another when Boyd turned to stab the ball home following excellent work by Steve Davis.
Charlie Adam replaced Edu in 64 minutes - but the Ayrshire men were far from finished for within sixty seconds Sasa Papac was clearing a Fernandez shot off the line.
Combe kept his side in the game in 73 minutes when he produced a superb one-handed save to turn wide Miller's low shot from a Novo cutback.
Three minutes later Papac was yellow-carded for a foul on Taouil.
Rangers almost sealed the points in 81 minutes when in a lightning break following a Killie corner Novo fed Pedro Mendes who in turn found Adam whose ferocious drive produced a spectacular save from Combe, who then surpassed even that three minutes later when he denied Novo after the substitute had fired in a 25-yard snap-shot.
Rangers were reduced to ten men in the closing stages when Papac received a second yellow and consequent red for a foul on Don Simmonds - but comfortably held out for the win.
Afterwards Walter Smith commented:
"I was disappointed to be behind at half-time. We had three or four good opportunities to score. We put pressure on Kilmarnock at the start of the second-half. It's always awkward to be playing after an International break."
RANGERS McGregor; Broadfoot, Weir, Bougherra, Papac; Davis, Edu (Adam 64), Mendes, Beasley (Novo 45); Boyd (Darcheville 82), Miller
UNUSED SUBS Alexander, Aaron, McMillan, Fleck
KILMARNOCK Combe; Fowler, Wright, Ford, Hay; Taouil, Invincible, Skelton (Sammon 74), Pascali (Hammill 88); Bryson, Fernandez (Simmonds 85)
UNUSED SUBS Rascle, Lilley, Flannigan, Nolan
REFEREE Calum Murray