It seems Yoroi will split across wallet addresses. If you send ADA and the ledger confirmation fails. It will send ADA to an alternate wallet. You still have access to this other wallet from the same base Yoroi wallet.
Its how UTXO wallets work. One wallet can hold multiple accounts, and an account can have multiple addresses. When you make an outgoing transaction, the remaining funds are sent to new address by default. The new address is still part of your wallet, and the wallet balance is still correct.
Check this post for further details.