I highly recommend you get there at park opening. We start in Universal Studios so we can do Gringotts and then jump on The Hogwarts Express before the lines get long. The line at King's Cross has always been much longer than the one in Hogsmeade, when we've been there. The queue is much better also. And, we like the ride that way better. (The ride is different each way.) We then ride FJ in Hogsmeade. We do the rides in IOA before heading back to US for the rest of the day. We particularly like to be in Diagon Alley in the evening. When we first started going, we definitely were not able to do everything in one day, but we didn't have the Express Pass. We now do at least three partial days there. Since they close early, we still have plenty of time to go to a Disney park for the evening or morning emh's at a Disney park and then go over to Universal for the early evening and get some rest for the next day.
We will be there mid-May also. We went the same time last year, and when we got there early, we were able to do FJ twice before the line started getting long. I say "we", actually, it's my daughter, son, and son's gf who ride Gringotts and FJ. I go through the queues with them, but I don't ride those two. The queues are amazing for those two rides.