Either of those things would be completely acceptable. I tend to err on the more dressy side because I hate feeling even remotely underdressed, but for a signature restaurant, California Grill is actually pretty casual. You'll likely see a lot of families there that have just come from the Magic Kingdom and are in their park clothes. You may also see suits and cocktail dresses, but what you're describing is a happy medium.
I'll be there in May, straight from the parks and attired in cargo shorts and polo shirt. I doubt there will be any problem.
Actually the cargo shorts might not fly...
Register on WDWMAGIC. This sidebar will go away, and you'll see fewer ads.