Rushden took three valuable points in their fight against relegation from an Orient side who had previously held the best away record in the division.
Rushden almost went ahead in the 12th minute when a superb cross bvy Adrian Littlejohn ended with Billy Sharp, their on-loan striker from Sheffield United, narrowly missing the target.
Orient had a goal ruled out after 17 minutes when John Mackie's powerful header was disallowed for pushing.
Rushden's Adrian Littlejohn saw his curling shot hit the post and rebound to safety in the 40th minute, and a minute later Orient's Justin Miller wasted a good opportunity, shooting wide when well placed.
Sharp made room for himself in the 48th minute, but his shot was comfortably saved by Harrison, and after 60 minutes Sharp was again in the thick of the action when he cleverly beat two defenders on the left, but his cross evaded the oncoming Littlejohn.
Mackie produced a fine save from Alan Blayney in the 63rd minute when the keeper tipped his fierce shot over the bar.
Orient sent on substitute Gary Alexander, whose effort was just wide in the 71st minute.
After 81 minutes, Orient skipper Michael Simpson did well to clear Phillip Gulliver's header off the line, and as Rushden began to take control Andrew Burgess was inches wide.
Rushden took a deserved lead in the 85th minute when defender Rob Gier scored the first goal of his career when he headed home Craig Dove's corner.
Rushden put the game beyond doubt in the 90th minute when David Bell left two defenders in his wake and coolly shot past Harrison from 20 yards.