We've stayed at both, and prefer the Contemporary, or I should say, we prefer the Contemporary Main Tower building. It depends on a little on your priorities. If theming is a top priority, Poly might get higher marks. I tend to prioritize efficiency. I don't mind waiting when it is essential, but I don't like inefficiency. Of all the resorts, BC/YC/Boardwalk/Swan/Dolphin and CR are most efficient.
If you have a stroller, and you want to ride the monorail, at Poly, you have to :1. get to the ground floor of your building, walk along an uncovered pathway, then get to the upper floor of the main Poly building. If your stroller folds, then you might want to carry it up the two flights of stairs. If you don't want to carry it, then you have to use the tiny cramped elevators. Elevators that are also used by dining patrons.
At CR, getting to the monorail is much easier, because it is quick and all indoors, at least if you are staying in the main tower building. If anyone in your party is using an electric scooter, this is even more helpful.
Another plus of CR, if it is raining, then you can generally get to all monorail dining without getting wet (unless it is VERY windy, except Narcoosee's). For a food court trip, it might not matter, but if you make an effort to dress for a character meal or just any nice dinner, then staying dry on the way to dinner is helpful. Again this is also helpful if anyone in your party uses a mobility device (stroller, scooter, etc.). CR also offers a boat to Wilderness Lodge/Ft. Wilderness for dining.
Within CR, I much prefer the main building, as again, I appreciate efficiency. Getting from one's room to the CR quick serve is very quick. The only time CR is not quick is if you need to do laundry.
I also like being able to walk to the parks. At CR, you can walk or take the monorail, though we rarely take the monorail...If you are leaving MK, and can keep a fairly brisk pace, walking is usually faster than taking the monorail.
Keep in mind, the monorail is not 100% reliable. When it goes down, WDW offers buses, but those can be slow.
That said, the Poly pool is more exciting than the pool complex at CR. That said, I also find I prefer the decor and layout of the CR rooms better. IMO, Poly is a bit dark. CR rooms aren't as fun as some of the other hotels. CR bathrooms have very odd sinks. They are very flat and don't drain well at all. there also isn't much counter space. We end up using one of the sinks as a makeshift countertop.
Poly dining options are better than CR's, but CR offers a shuttle boat to WL.
If getting to HS early for a boarding group winds up being important when the parks open, then staying within walking distance of HS is very helpful!
Last, if you are torn between resorts, split stays are potentially an option to consider, just know all the rules of how they work. WDW will transfer your luggage for free; it just takes a few hours. It also helps to pack luggage with a split stay in mind.
