Our whole family loves to swim and pools are novelty as we don't have a pool at home (nor do many of our neighbors here in upstate/western New York, where there are only 3-4 months a year when it's warm enough to use one).
In a week-long vacation, we'll swim at least 3 or 4 times, either during long afternoon breaks or even late at night, and will often spend time both in the hotel's "feature pool," as well as in the nearest "quiet pool," if one is offered. True confession: on balance, we tend to prefer the hotel pools at Universal to those at WDW, so if we're staying at Universal we swim even more often than that.