After another stalemate, the odds were stacked against Rangers when skipper Barry Ferguson missed the first penalty.
But Neil Alexander saved from Fabio Liverani and then Christian Vieri shot high over the top before Novo slotted home the clincher.
Steven Whittaker, Sasa Papac and Brahim Hemdani all converted their kicks to set up Novo for the winner.
Rangers will now meet Zenit St Petersburg and former manager Dick Advocaat in the UEFA Cup final in Manchester.
Walter Smith's side finished with ten men after Daniel Cousin was sent off in the second half of extra-time after picking up his second booking.
The Italian side made their intentions clear from the start as they played three up front in an attack-minded formation.
But Rangers were typically organised at the back and kept Alexander well protected in the early stages.
The goalkeeper did not have a direct save to make until the 19th minute and, even then, Alessandro Gamberini's long-range effort did not trouble him.
Adrian Mutu then drilled the ball straight at the goalkeeper from 18 yards after referee Frank De Bleeckere let him get away with using his arm in the build-up.
But Rangers managed to put together some neat passing moves themselves without seriously threatening Sebastien Frey in the Fiorentina goal.
A flick from Mario Santana caused concern in the Rangers' six-yard box, but Sasa Papac was on hand to thump the ball clear with Alexander exposed.
Fiorentina looked increasingly dangerous as half-time approached with Mutu seeing more and more of the ball around the penalty area but Rangers held firm.
The Italians came close six minutes into the second half when Giampaolo Pazzini was given too much space in the area and lashed the ball wide from Mutu's cut-back.
Mutu then fired a low free-kick wide after 57 minutes and Alexander was happy to see the ball finish yards wide of his right-hand post.
Rangers were guilty of giving up possession on the edge of their own area as they struggled to match Fiorentina's pace.
Tomas Ujfalusi was just a foot too high with a 16-yard shot after Rangers had failed to clear their lines at a corner.
It was one-way traffic and Mutu tried his luck with a 30-yard free-kick which Alexander took two attempts to hold.
It took Rangers until the 77th minute to have a shot at goal and the flick from Steven Davis was easily held by Frey.
With six minutes left, Whittaker almost broke the deadlock with a stunning shot which brought an equally fine save from Frey.
In the first minute of extra-time, Vieri shot across the face of goal but both teams showed signs of tiredness in the additional period.
Daniel Cousin saw a shot deflected just over the top at the other end.
Cousin was then red-carded after picking up his second booking after reacting angrily to a challenge from Liverani.
Novo had a chance right at the death but Kevin Thomson's corner struck him in front of goal and Frey managed to claw the ball clear.