I have the same setup. You don’t need to worry about anything - this is perfectly normal.
Ledger Nano stores your private keys. Yoroi, in this case, acts like an interface for accessing the blockchain. So when you created your Yoroi wallet and synced it with your Ledger device, it created a public key from your private key on the HW wallet.
Anyone can view the current balance of any address on the blockhain. That’s why you can see your balance in Yoroi even if your Ledger Nano is not connected. In order to perform a transaction, you need to sign it with your private key. So without a connected Ledger device, you can’t transfer any funds from Yoroi.
Ledger live is just an app from Ledger used for managing the device. It can also manage your accounts but they support only limited number of currencies for this usecase (Cardano is not one of them).