They were both the ending movies of major events. Endgame shouldn't compare to TFA, it should be more like ROS. Infinity Wars should be closer to TFA. End Game was the conclusion to that Saga.
But you mention doing better than the average MCU movie. It's disingenuous to an extent. Star Wars wasn't starting from scratch. As you stated, TFA did $2 billion. It was an event BECAUSE it was so well known. The MCU had to build itself up (and FYI, do it by having a ton more movies than 5 in 5 years, which again, disingenuous when discussing the final movie that was nearly 2 years after the last release). In that same time frame between Star Wars movies, you have Black Panther, Captain Marvel, Spiderman, and Ant Man and the Wasp. 3 of those 4 movies blow the conclusion of the Skywalker Saga away in Box Office. And those are not full Avenger Movies. 7 movies between 2 Star Wars Movies, and 6 of them did bigger numbers than ROS. Now, maybe they never would have had a shot at End Game numbers (not sure I agree on that), but there is NO WAY the finale should be the lowest of the 3 and 4th lowest of the 5 total releases.
You actually summed up why people discuss what a failure this was (despite the billion dollar box office). You say it didn't feel like a conclusion of the Skywalker Saga, but a separate trilogy. The point of this trilogy WAS to be the end of Skywalker, and the jumping point into a whole new universe. They really didn't come close to achieving this, and the end result is you now have discussions of new trilogies set hundreds of years apart from anything remotely connected to what everyone knows.
I will forever say fatigue is insanely overrated. Again, you can't claim 5 movies in 5 years and not claim a ton more with the MCU since 2 of those movies had no connection to the trilogy. It was a poorly done trilogy. If you want my opinion, essentially they decided to go full on artsy for 8, and turned off a bunch of fans. They truly believed Star Wars was too big to fail, and that they could alter things to bring in new crowds and the die hards, while maybe mad, had too much love for Star Wars to ever stop watching (I think Iger continued to believe it when he stated that he didn't even need to market Galaxy Edge and thought they could open half a land). They got cold feet with the reception (especially after Solo), and so they tried to do what they thought the non-artsy fans wanted. Again, I think the assumption was the fans are so die hard they will come back after a year and a half and all will be forgiven. My guess is, a miscalculation, and what you now did was that while you still have a lot of fans that have not come back, you have now completely angered the artsy crowd who feel THEY have now been spurned. I have no inside anything, but this is just how I have kind of pieced things together.