I think part of the issue is the different attraction styles between Disney and Universal. Universal has very few attractions I actually enjoy riding -- when I went, with express pass, we could have been on every attraction I like across both parks in one day with no problem. Outside of the HP areas, there's not a ton of interesting themed spaces to visit either that could suck up more time. Epic Universe might change that, but from what we know about it thus far it seems like more of the same in terms of attraction styles (and there's nothing wrong with that; plenty of people love them).
My guess is a lot of people who enjoy Disney enjoy things about Disney that Universal only offers in much smaller portions, so it's very possible for those people to do all of Universal that interests them in a couple of days.