Lots to do in Orlando. Even outside of Disney. But if you want to veer outside of Disney the rest of the way then try Sea World. Definitely worth it. Busch Gardens in Tampa is worth it as well if you ever wanted to go to a roller coaster park that married a zoo. No, seriously, it is a neat concept.
Other things..........how about Universal Studios? You said Islands of Adventure, and that is the better park of the two but Universal is still worth it.
Then there are other things to see. Titanic Experience, Gatorland, Holy Land, Kennedy Space Centre............lots of beaches too that aren't very far away.
Lastly, if you are a sports fan how about seeing a baseball game in Tampa? I saw a hockey game in March 2010 with the Tampa Bay Lightning. Tampa is really not very far from Orlando.