I've always thought the carpets were much less of an eyesore than Dumbo. Dumbo had elephants on sticks, with little theming around it (note: FLE seems prepared to rectify this, which is great). The Carpets have camels, jewels in the pavement, rockwork in the queue that tries to fit in with the theme of the ride, etc. Plus, it gives those in the area something to see overhead.
Sure, a desert-set attraction across from the entrance to the Tiki room doesn't make a whole lot of sense, but neither does rounding the bend from the south seas of the tiki room and being in the Caribbean. The land is "Adventure"land, and flying on a carpet is just as much an adventure as sailing with pirates, exploring the jungle, being sung to by birds and flowers, or eating pineapple-flavored soft serve!