The visitors took a 20th minute lead when home defender Alton Thelwell put through his own goal, but two strikes in a deadly five-minute spell from Boyd saw Orient end the run of three consecutive home defeats.
But it was Swindon who started brightly and should have scored within 30 seconds of the kick off when Jon-Paul McGovern was presented with a shooting opportunity but fired straight at home keeper Glenn Morris.
The visitors had another chance shortly afterwards but Simon Cox blasted wide from a tight angle.
After that the first half developed into a scruffy affair and it was the visitors who went ahead with a scruffy goal.
On 20 minutes Anthony McNamee raced down the left and fired in a cross that Orient's stand-in skipper Thelwell deflected past his own keeper, who was going in the other direction.
Swindon were forced into an early re-shuffle when Chris Allen limped off and was replaced by Sofiane Zaaboub.
Meanwhile, Orient's only effort on target in the first half came from Paul Terry's curling shot.
After the interval the O's sprung into life and on 51 minutes Boyd was felled in the area by Miguel Comminges and the hit-man blasted the spot-kick into the roof of the net.
Orient had another good shout for a penalty waved away when Jerel Ifil looked to have handled Terry's shot on the line.
But the home side did not have too long to wait for the winner.
On 55 minutes, John Melligan curled in an excellent cross from the right and Boyd headed home from six yards for his 12th league goal of the season.
With Orient nervously hanging on, Swindon hit back late on and twice came close to levelling things up.
First keeper Morris saved superbly when he tipped away McGovern's drive from the edge of the box.
And Swindon were left kicking themselves on 76 minutes when Cox wasted a glorious chance from six yards out with a contender for miss-of-the-season.