While the GF is very nice, I would have to agree that you'll want to stay offsite if you're looking for a hotel that truly is a luxury hotel. i.e. The Peabody, The Waldorf, etc. Now you may not feel the "Disney magic" at these destinations, and for that reason, the GF should be your first choice if you want the closest thing to that kind of luxury blended with Disney magic.
That being said, this past trip actually has me rethinking the whole onsite vs. offsite debate. We stayed at WL two trips ago and 'most' of the CMs were between pleasant and magical, we stayed at the Contemporary last year and nearly every CM there kept us in the wonderful Disney magic bubble; however, we stayed at Yacht Club a few weeks ago and the bulk of the staff members we encountered were pretty far from magical. While we found magic in many other places on the trip, it certainly had me questioning the $ I was spending to stay onsite. Seriously, there's a problem when the Motel 8 on Irlo Bronson has more pleasant staff than a deluxe WDW resort. Now, I know this is just our experience and all of our other WDW hotel experiences (from the ones we've stayed at to the others we've just visited) are the polar opposite, but it makes you think...to the point where I started pricing rental houses...which would be a SIGNIFICANT savings. Still, DH and I agreed that our main selling point... convenience...would put us back staying onsite - I just had to promise him that we'll never stay at YC again.