I'm from the other end of the Mediterranean (of mainly Turkish heritage with some Arab thrown in), but I know the art and architecture of the wider region really well.
As you say, people hear "Arab" and think it's all the same. It's as reductive and inaccurate as conflating LA and New York, or Paris and Venice.
To be fair, Agrabah itself is guilty of mixing things from different sources, some of them far outside the Arab world (the palace definitely has a Taj Mahal vibe). But the end result has very little to do with the Maghrib (the region where Morocco is) and is much closer to the architectural traditions of Egypt and the Arab lands east of it, as well as of Iran.
Here's a cityscape of Agrabah:
The abundance of domes is alone enough to tell us that we are far outside Morocco, where green tiled roofs are more prominent.