Rovers were booed off when they went in a goal behind at the interval, but tricky winger James Coppinger came on in the second half to change the course of the game.
Coppinger laid on two goals and scored the other during a one-sided spell in which the Rochdale defence was sliced apart.
But a careless effort from goalkeeper Alan Blayney gifted Rochdale another goal and led to some unnecessary anxious moments in the closing stages.
Rovers reached the quarter-finals of the competition last season beating Aston Villa and Manchester City on the way, but showed no sign of that form in a lacklustre first-half display.
Visiting keeper Matt Gilks did well to parry a powerful header from Mark McCammon following a sixth minute corner, but Doncaster struggled to raise their game for the rest of the half.
Dale played some neat football at times with Nathan Stanton doing well in defence and Iyseden Christie working hard in attack.
Adam Rundle picked out former Doncaster midfielder John Doolan following a great piece of skill in the 36th minute and his low 20-yarder was tipped round the post.
Rundle made the breakthrough on the stroke of half-time with a freak goal direct from a corner, with home keeper Blayney rooted to the spot as the ball went over his head claiming he had been pushed.
The Doncaster fans made their feelings plain at half time but an inspired substitution led to the home side going 3-1 up within 20 minutes of the restart.
A quickly taken free-kick from Coppinger led to McCammon flicking home the equaliser at the near post in the 53rd minute.
Coppinger latched on to a return pass from Paul Green to slam home the second a minute later and then set up McCammon again to make it 3-1 in the 65th minute.
Blayney had a kick charged down allowing Doolan to pull one back 12 minutes from time and the home side were fortunate when lively substitute Clive Moyo-Modise dribbled through to have a shot saved in the 86th minute.