The Poly is walkable distance to the TTC and one stop on the monorail getting back from the TTC. The Poly also shares boat service to the MK with the GF as an alternative to the monorail. It is only 2 monorail stops to the MK or 3 coming back.
The Contemporary is walkable to the MK or 4 stops going, and one stop or walkable coming back. While the CR is only 1 stop to get to the TTC, it is 4 stops to get back (which is not as much fun). The CR does not have boat service to the MK, but does to the Wilderness Lodge and Ft. Wilderness (in case you are going to Hoop de Doo, or some other restaurant at one of those places).
I'm not sure which resort has better bus service (to Downtown Disney, the Studios, and Animal Kingdom). But overall, I would say the Poly has slightly better transport.
Now in terms of theme, the hotels are very different and up to you to make a personal choice. The CR is very modern, simple colors (mostly earth tones), and plain. It is the hip thing, but not everyone likes it. The Poly is more of an upscale tiki bar theme. Again, some love it, some don't.
Both hotels have the largest standard sized rooms on property (for Disney operated hotels -- Shades of Green does have some even larger rooms). So, both Poly and CR have standard rooms bigger than those at the GF.
The biggest asset of the CR is the tower where you just take one elevator and an escalator to get between your room and the monorail. The Poly is much more spread out (kind of like the GF) and while the grounds have great landscaping and lighting, it is more walking.
Both resorts have popular restaurants. The quick service dining area in the CR seems to be a bit larger and less crowded, but has no Tonga Toast at breakfast (if that is important to you).
The CR also has MK park view rooms in the adjacent Bay Lake Tower if you are interested in a villa type room.
In years past, the word on the street was that the Poly had friendly and good customer service while the CR had not so good or friendly service, but it has been many years and that was never really measured. So, I am not sure if this last point is valid or not.
In the end, it is largely a personal choice. Decor is really one of the biggest differences between these two locations. The other being one resort is spread out with no building more than 3 stories with the other resort more centralized with a 14 story main tower. (Yes, I know the California Grill is on the 15th floor, but there is no 13th floor since the only WDW run hotel with a 13th floor is the Hollywood Tower Hotel, also know as the Twilight Zone Tower of Terror.)
Staying in the garden wing is about as close as I've gotten to that dream, but I'll say this...being walking distance from MK is AWESOME!!! And while you can't walk to the TTC from there like you can at the Poly- it's only one monorail stop away.
