There are rules as to what rules can be made. Usually there are requirements for scientific/data findings from experts and a period of announcing proposed rules for public comment and review.The biggest issue with the Federal Government is that when congress created these agencies, they gave them the power to make whatever rules they wanted to without needing congressional approval for anything.
Many an agency regulation has been struck down by a court for failing to follow these rules, or for not following where the science or data leads, or going beyond their Congressional mandate.
We've seen the last-minute regulations of an administration being quickly dismantled because they didn't follow the rules of creating those new regulations. However, the regulations that did follow the rules takes months and months for the next administration to reverse (if it wants to) because the reversal has to go through the lengthy review process (and then withstand legal challenges).
E.g., these new OSHA rules can't go into effect immediately, unless there is a provision for emergency rules. And then such rules can be challenged for not being an emergency, or for not following the science, or for infringing on other Constitutional or Legislative rights and laws. And even with an emergency issuance, it will need to be followed up with the regular procedures or lose it's enforcement power.
The alternative is to have Congress micromanage all these administrative regulations, and I think we all know how well they are up to that task. That being said, Congress can, if it wanted and had the will and the votes, create a new and specific regulation (or strike one down) that the appropriate agencies would have to follow.