My wife and I, along with another couple, went to Chef Mickeys this past OCT, and it went down great. I've been to multiple character meals, and YES, there are always a ton of kids and their parents. But, for the most part, Disney does a good job at keeping people out of the mob mentality. The characters give individual attention to each table, and if you're really worried, I'd possibly ask the CM when you check-in if you could have a less-noisy area.
I would say that CRT or Chef Mickey's seem more friendly for adult-only tables, as Crystal Palace seemed overly playful. It's something I can't really put my finger on, but I felt much more comfortable at the other two, heck, even Norway's Princess Breakfast was more intimate/adult-friendly.
But, in the end, if YOU or YOUR BF want something, remember it's YOUR vacation. Don't get too apprehensive to where it's a buzzkill, or it makes you regret not going somewhere. Have fun.