It could be marketing, because until this recent push to advertise Harry Potter I can't remember the last time I saw a TV ad for any of the Universal parks, yet I catch a Disney ad at least every other day.
It could be word of mouth. To see a drop of over 20% from 2003 to 2009 could certainly mean people know Universal is there, they just didn't want to go back.
Harry Potter will draw a lot of people back to Islands of Adventure, but just like bork said, will it draw people to both parks, or will it just be one day out of someone's Disney vacation to see IoA only?
And again, what about word of mouth? I think a lot of people might feel a little misled when they arrive at IoA to find Harry Potter is just a couple of rides and not an entire park.
And will those rides be ready to handle the numbers, because everyone going to IoA is going to be in that one small section of the park.