They had two players sent off and collected five yellow cards but the tie was always more enthusiastic than physical.
Bristol Rovers, who went in front in the second minute finally needed a last-minute penalty by Rickie Lambert to scramble into extra-time.
Then they gave away a penalty when Jason Demetriou went down under a challenge from Craig Disley and Adam Boyd scored from the spot for his second goal of the night, before Disley made up for his mistake heading in the equaliser from a corner.
The penalty shoot-out went Rovers' way 6-5 with Paul Terry and Adam Chambers missing for Orient. It has brought Rovers a second round home tie with Rushden & Diamonds.
In an explosive second half Orient's Wayne Corden and sub Jabo Ibehre were sent off. Ibehre for handball, collecting his second yellow after collecting his first for timewasting.
It was a tie Rovers defender Craig Hinton will never forget. He put back his 31st birthday party when he was called into the Rovers side and his goal from a corner on 80 seconds was his first in 150 games for them.
Orient coolly played themselves back into the game with Boyd and Wayne Gray making powerful runs at the home backline.
They drew level with more than a little help from Hinton, whose poor header set up Boyd. Hinton's remarkable half had a further twist when he had the ball in the net again from another corner, but it was disallowed for shirt tugging.
The visitors went ahead for the first time only minutes after Corden had been sent off. Wayne Gray powered in a ball played into him by Terry.