This is not the same cold that you are familiar with up north. Yes, the air is arctic, and from up north, but our humidity level is higher, EVEN WITH THE DRIER ARCTIC AIR.
You must layer. Big heavy coats are fine, but the trick to southern cold is many thin layers.
I speak from experience, spending many a winter at single digits with a wind chill quite often below zero (fahrenheit) in the winter months. I would take those temps and conditions over 20's in Florida any day, because here, for some weird reason, the cold just cuts into you.
Do yourselves a favor and err on the side of caution. You can always take off more layers, but if you didn't bring enough, well, that's that as they say.