The metadata is attached to the forging transaction. So it is unique and can easily be looked up by just providing the token name and the policy. No need to expose any keys in order to filter out the right transaction. Check out this table:
CREATE TABLE public.ma_tx_mint (
id bigint NOT NULL,
policy public.hash28type NOT NULL,
name public.asset32type NOT NULL,
quantity public.int65type NOT NULL,
tx_id bigint NOT NULL
);
it gives me back the correct tx_id by providing policy and name.
and then I can lookup the metadata with the tx_id:
CREATE TABLE public.tx_metadata (
id bigint NOT NULL,
key public.word64type NOT NULL,
json jsonb,
tx_id bigint NOT NULL,
bytes bytea NOT NULL
);
So a wallet for example needs no extra information to fetch the right data for a token.