Anthony Stokes won a penalty for Celtic after just two minutes in Glasgow's East End when he was brought down by Peter Enckelman, but the 'keeper redeemed himself by saving Kris Commons' spot-kick.
The Hoops missed countless chances to open the scoring and it was St Johnstone who took the lead on the hour mark when Dave Mackay's volley was deflected past Fraser Forster for the Perth side's first goal of the season.
Celtic laid siege to the Saints goal in search of the equaliser but their stubborn defence held firm and St Johnstone came away with a first win at Parkhead since 1999.
Old Firm rivals Rangers went top on goal difference, and a point clear of Celtic, after dispatching of previous leaders Motherwell at Fir Park thanks to goals from Steven Naismith, Kyle Lafferty and Gregg Wylde.
"I am really angry as we threw it away. We missed clear cut chances in front of goal and we were wasteful," said Celtic manager Neil Lennon.
"We keep missing simple chances and I am fed up with it and I am fed up with us missing penalties."
Derek McInnes was delighted with his side's performance.
"Obviously the penalty save was pivotal but I thought it was a harsh decision," the St Johnstone manager said.
"If that had gone in it would have been a totally different game. Normally if you are going to score a winning goal at Parkhead you want it with a minute to go but the players worked hard and stuck to their task and got their reward in the end."
Celtic made three changes to the side which faced Sion on Thursday with a debut for Victor Wanyama and starts for Adam Matthews and Shaun Maloney.
St Johnstone 'keeper Enckelman was a busy man from the off, saving Commons' penalty and then stopping a diving header from Charlie Mulgrew on the line.
Commons then poked a Maloney cut-back just by the post as Celtic enjoyed the majority of possession.
Ki Sung-Yeung's powerful shot was deflected just wide of the post before Commons sent a curling free-kick crashing off the upright as Celtic continued to bombard the St Johnstone goal.
There was then a scare for the hosts immediately after the break when Alan Maybury's pass across goal caught out the home defence but former Celtic player Cillian Sheridan somehow failed to connect at the back post.
St Johnstone had failed to score in their previous three matches and it took a deflection to give them a shock lead on the hour mark.
Moments after Forster punched away a Jody Morris pile-driver, Mackay got on the end of Liam Craig's cross to send an angled volley off Joe Ledley and past the keeper from the right-hand side of the box.
At Fir Park an injury-ravaged Rangers made five changes to the side that lost in the Europa League to Maribor as they handed Ross Perry and Jordan McMillan their first league starts.
They took the lead in the 20th minute when Jelavic bulleted a header past Darren Randolph from a Steven Davis corner which took a touch off Naismith on its way into the net.
Lafferty added a second on the stroke of half-time when the Northern Irishman rifled a shot off the post and past Randolph from 15 yards on his first start of the season.
Davis and Naismith both hit the crossbar in the second-half before Wylde raced from the half-way line to crack a third past Randolph with five minutes remaining.
In the day's other game Kilmarnock and Hearts played out a goalless draw at Rugby Park.
Source: AFP
Source: AFP