The rooms for first time visitors used to be the "luck of the draw" with only repeat visitors making special requests. This made it easy for the staff to accommodate them.
Then, along came the internet with people sharing that information, so that the number of requests for special rooms increased. It doesn't take very long to look them up if only one or two people per day ask, but when the numbers increase, so does the check-in time. And since these rooms seemed to be at a premium to guests, I guess they decided they would charge a higher price.
Once upon a time, you could ask and get an upgrade if a better quality room was available and would remain empty otherwise. But that information spread rapidly, so isn't done as frequently (if at all) anymore without a charge.
Just as so many things little things that were free now have a charge, or were eliminated altogether, so have free room upgrades.