@Gaaike yes, there is a cost in reward. Lets say you delegate to a pool with 34m - that pool would mint 34 blocks / epoch. For each block the pool receives a total of 1k reward. The pool would therefore get 34k of rewards. First, the operator cost is deducted (i.e. 340 ADA) second the pool margin is deducted - lets assume that is 0%.
340 / 34k => 0,01
With such a pool, there would be 1% of rewards that cannot be distributed proportionally.
Now, you delegate to a small pool with 500k. That pool will make a block every other epoch. When it makes a block, it will also receive 1k of which 340 ADA are deducted. This is …
340 / 1k => 0,34
Now, instead of 1% there are 34% of the rewards that cannot be distributed to the delegators.
The larger the pool, the smaller that slice of pizza that gets deducted before proportional distribution.