In my experience, days with lower crowds often feel more crowded: with shorter attraction waits, a smaller percentage of guests are standing in a queue at any given time, and everybody's in less of a hurry -- meaning that all public areas are jammed with people meandering from one place to another. We've visited on days when the crowd levels were amazingly low (we're talking about a level 2 on the touringplans scale, with little or no wait on anything but the biggest headliners), yet the walkways were so jammed we could barely move.
Also, Disney knows when to expect lower crowds, too -- and they cut staff, lower ride capacity, shutter restaurants and adjust transportation schedules accordingly, to minimize costs. Unfortunately, this also diminishes some of the benefits for those attempting to take advantage of lower crowds.