Morkel could have had his third a short while later when he beat Mushfiqur Rahim's defences with a fine in-swinger to strike timber.
With the Proteas in cruise control and their lead steadily increasing, Bangladesh were in desperate need of a wicket.
A partnership of 142 between the pair pushed along the scoring rate as South Africa sought to set Bangladesh a large target, but still gave themselves enough time to bowl them out on a pitch that has proven lifeless.
South Africa were 54-2 overnight with Hashim Amla on 17 and Bavuma on three.
South Africa are odds-on at 1/4 with Sky Bet to win the first Test. Bangladesh can be backed at 25/1 and the draw is available at 100/30.
Ottis Gibson had enjoyed a dream start to life as Proteas coach on Thursday, as openers Dean Elgar and Aiden Markram shared a first-wicket partnership of 196 before closing on 298 for one after the Tigers won the toss and elected to field. The only wicket of the morning fell to new cap Andile Phehlukwayo when Tamim, on 39, glanced an off-target delivery down the leg side and wicketkeeper Quinton de Kock flung himself to his right to take a good catch.
Police officer stabbed in Canada terror attack
They said they now believe the suspect acted alone , but haven't ruled out the possibility of others being involved. But the man in the truck then took off toward downtown Edmonton , according to the police account.
Left-hander Elgar was on course to convert his opening-day ton into a double before he miscued a short ball from Mustafizur Rahman to midwicket on 199. Catch live cricket score and live updates of South Africa vs Bangladesh, 1st Test, Day 4, here.
Mominul Haque struck soon after the lunch interval, accounting for Faf du Plessis by trapping the South African skipper in front for 81.
The lanky seamer first bowled Tamim Iqbal with peach of a delivery and then trapped Mominul lbw, although replays showed the ball would be missing leg stump. Mominul, who had only one wicket in his Test career before this game, claimed his second wicket of the match.
Morne Morkel snagged two wickets in the first over of Bangladesh's second innings and they were 3-49 at stumps, still 375 runs shy of an improbable victory. It was hard work for the bowlers.
Taskin Ahmed was run out three balls later as Bangladesh lost three wickets for 13 just before the break.
The wicket was the result of a disgusting mix-up between the two batsmen in the 55th over.