Syncing cardano wallet takes a lot of time

I am currently running cardano node and cardano wallet, the chain is synced 100%, but i wanted to use this for a wallet based application, were i would have to sync the wallet.
Initially a new account would sync in 1 hr max and an old account would sync in appx 6 hrs.

Now as the no of wallets that are syncing at one time have increased, which is around 100 wallets, how can i scale my setup to reduce this sync time ?

I am open to any suggestions, i could try them and keep updating here.

Also i am using 32GB Ram currently.

Please feel free to ask for more information if needed.