I found the answer myself. The key is to call “cardano-cli stake-address deregistration-certificate”. Below is the full commandline.
cardano-cli stake-address deregistration-certificate \
--stake-verification-key-file stake.vkey \
--out-file stake.cert
And then build a transaction with this certificate. Specifically you need to increase the change amount by 2 ADA and minus the transaction.
txOut=$((${total_balance}+${keyDeposit}-${fee}))
cardano-cli transaction build-raw \
${tx_in} \
--tx-out $(cat payment.addr)+${txOut} \
--invalid-hereafter $(( ${currentSlot} + 10000)) \
--fee ${fee} \
--certificate-file stake.cert \
--mary-era \
--out-file /tmp/tx.raw