That leads to epistemological nihilism... that we can know nothing.
Speculation is just that, mere speculation. It's truth quotient can be anywhere from 0% to 100%.
When we have facts that are actual facts to back up speculation, the truth quotient goes up.
And if we have facts that contradict mere speculation, then the truth quotient of that speculation goes way down.
When we have witnesses, that is 'insiders', who can vouch for things outside the public view, the truth quotient gets very high.
Just because we're not 100% sure of something doesn't mean it's equal to mere speculation. Having about a 80%-99% surety from an insider is very different from mere speculation which can be completely false.
Also, many who speculate love to do so in a way that makes it seem that their guess is almost certainly true (over 90%!) when it can be completely false. Speculators need to frame their guesses as guesses than as proclamations.
Also... over the past several years, there's a low tolerance among many, including myself, regarding fact-free speculation. We call that "conspiracy theories" and "science denial" and "ideological propaganda." So, when someone spouts off that their guess is most probably 'the truth'... I ain't gonna let that be. Show your work.