Explain me the rate system please

Is the number of rates matter when you have more than 20 rates ?

I mean, is a game with 20 good rates and an overall average at 4 will be higher than a game with 100 rates but an overall under 4 ?