Leyton Orient defender Justin Miller headed into his own net 15 minutes from time as battling Bury hit back to take a point.
It was no more than the visitors deserved as they took control of the second half after being distinctly second best before the break.
The home side started hesitantly before grabbing the lead against the run of play after just nine minutes.
Midfield man Michael Simpson quickly closed down Bury skipper Dave Challinor to win possession and then threaded an excellent ball through to Lee Steele, who beat Glyn Garner before scrambling home his sixth goal of the season.
The goal inspired Orient and, as they sprung into life, Wayne Carlisle went close with a header before Andy Scott wasted a glorious chance to double their advantage.
With Orient in full flight for the one and only time in the match O's skipper Gary Alexander fire just wide.
Bury fought their way back into the game and livewire striker David Nugent tested Orient keeper Lee Harrison with a 30-yard drive nine minutes before the break.
In the second half the complexion changed completely as Bury took a vice like grip on the game. They threatened a leveller on several occasions as Nugent proved to be a constant menace down the left.
It was Nugent who set up Dwayne Mattis but his shot sailed over the bar.
However, it came as no surprise when Bury levelled after 75 minutes as Nugent sent Tom Kennedy down the left and his cross was headed into the net by Orient defender Miller, who was under pressure from Shakers substitute Chris Porter.
Bury almost won the game six minutes later when their top scorer Nugent hit a post from eight yards out.
Orient finished with a flourish and substitute Stuart Wardley forced Garner to tip over his close-range header, before fellow replacement Jabo Ibehre wriggled past two defenders only to see the visiting goal keeper push his effort away.