Absolutely, and the earlier you arrive, the better. I've been able to get through most of each park by 1pm. With Express, of course. One day per park is all that is needed, especially with Express Passes.
I wouldn't flat out say yes, especially if the OP is going this summer and wants to fully experience Potter. In that case, 3 days is a much better idea.