September is our absolute preferred time of the year to go. There is a trade-off though, which you already touched on. We're always more than up for it too.. and that would be the crowds for the heat. Going in September it will still be hot, but the trade off is, crowd level is very low. We've been just about every week during the month over the last few years. The week after Labor Day is going to be the lowest crowds...and as you get closer to things like Food and Wine, Halloween Party/Decorations, and the marathaons... you'll see crowds increase but still overall a great time to go.
Some others can probably touch on this too from experience, but this past September seems to have been an anomaly. Last week in Sept into October 1st was bananas... Obviously with the 40th last year that had something to do with it, factor in MNSSHP, Marathon, and Food and Wine, it was busier than normal.
That all being said, if you are prepared to deal with heat, you're going to love September!