The answer is to increase your --invalid-hereafter to something greater when building transactions. I have mine set to 20 minutes + current_tip. This is because the network is loaded to like 80-93% regularly and TXs are taking longer to complete. Comes with being successful honestly, and hopefully hydra is released soon.
Referenced here in another ticket as well 94045