Walk around the Metacritic scores for video games - 2012-2022

Are users getting tougher?

Are ratings constantly increasing?

What drives user dissatisfaction?

All data was retrieved from the Metacritic website in the last week of February 2023 (coincidentally my vacations).

We will look at the following consoles:

Exploring of the video game database

We will only consider titles that have at least a Metacritic rating (based on the aggregated ratings from multiple websites) and a user rating (based on the averaged ratings from individual users)

Let's have a look at the life cycle of different platforms, looking at the number of games released per year.

We first look at titles having a file on the MC website, then the number of files with MC and user ratings (so once again those that we'll investigate).

Interestingly, there appears to be a decrease in popularity of Metacritic.

What does the release cycle look like over the course of a year? (i.e we look at the number of games released per month, all years combined, per console)

We can see a clear peak in November, and a smaller one one in March, across all platforms.

Rating distribution

All consoles combined, we look at how the Meta ratings (those based on websites) and the User ratings are distributed

We observe two skewed distributions (i.e. the average score is not 50%, but much higher). We also see that the two distributions are shifted relative to each other. Users are more severe on average.

There are also clear accumulation points - for example the "50%" rating seems to be used often by users, but not by critics. The 80% rating is equally popular with both, and is the most common (it's the mode) for MC ratings.

Let's smooth those 2 distribution to have a better idea of what we are looking at.

Finally, let's look at the offset distribution.

OFFSET: the difference, per game, between the Meta score and the user score.

The greater the Offset, the higher the Meta score is compared to the user score (i.e. it means that the game is overrated by institutional critics). Conversely, a negative offset means that the game received an average user rating higher than the Meta rating.

It leans to the right, which confirms what we've seen before - there are more games that are overrated by critics than the other way around.

User ratings are on average lower than Review ratings.

Platform point of view

Here is the Meta score distribution for each console. they look similar.

One small trend that emerges: the most recent consoles (Series X and PS5) have a rating distribution favoring high notes (no doubt that the shovelware has not plagued their online stores yet! Or the critic are still in the honeymoon phase).

(For information, the upper/right limit of each box indicates the 3rd quartile, meaning 25% of the games have a higher score. 50% of games are withing the box)

Now let's look at the distribution of average user ratings for each game on each platform. We see that the distributions are more spread out downwards, again for all consoles!

And let's now check, for each console, the distributions of Meta notes and user notes side by side to better assess what's going on.

Seems like the newer the console, the more severe the ratings. Or Nintendo consoles are spared?

We see in particular for each Sony and Microsoft console that the median of the user ratings (the horizontal line in each box) is much lower than for the Meta ratings.

Overall, users are more severe than pro sites

We now look, still by console, at the offset distribution.

Ratio

Finally, to close this first view of the situation, let's introduce a new metric: the Ratio.

The ratio is calculated per game by dividing the (number of user ratings) by the (number of ratings that contributed to the Meta score) (provided by "pro" reviewers).

As such, a ratio of 1 shows that there are as many users who rated this game as there were pro reviewers. A high ratio shows that many users have left a rating (review bombing phenomenon). It will be interesting to see which games or developers cause this behaviour.

Let's check the ratio distribution: in general, we are close to 1, but there are many games that have a higher ratio, indicating when users have had a field day.

Focus on games

Now let's take a look at individual games. It might get confusing, but here's how we're going to represent it.

Each point of the graph represents a game, on a platform.

You can click on the legend to deactivate / activate a certain platform - double click to select one platform only.

The gray diagonal represents X=Y, ie if a game has the same meta and user score, then it will be located on this line.

So, if a game has a user rating lower than the critical rating, it will end up below the white line (and if it's the other way around, i.e. users have been more generous than the critics, it will be above this white line).

As expected, we have more games below than above.

It is interesting to observe how this changes depending on the platforms.

Finally, I added one information to the graph: the size of the circle represents the ratio. The purpose of this circle is to emphasize cases of "review bombing", i.e. when an army of users sent score for a game.

We can now simplify this graph by keeping only the points with a ratio of at least 20 (i.e. games that triggered the passions of the users by a factor of 20)

How about we look at offset vs. ratio to see if there's a relationship?

There might be a very small trend, visible for certain consoles.

Let's talk Reviewers (institutional - those that are taken into account into the Meta score)

As shown in the table below, we have 327 official reviewers with at least one review. Some have a very high number of games reviewed, others have only provided a score for 1 game.

And here's how the reviewers are distributed. The color shows the average rating for this reviewer (no trend).

We had previously checked how Meta ratings were distributed per game.

Let's now give a look at the distribution of Meta ratings in general, without averaging over game titles.

Note that it has a "comb" aspect because some publications rate using a 5 points scale (or 5 stars), others out of 10, others out of 20 and finally others out of 100. whe

Hence peaks every 20 points and smaller ones every 10 points.

The distribution of individual user ratings shows us something surprising that we did not see above: it is bimodal, with 2 extremums, one at 0 and the other at 10. We can clearly see that user rating is often binary (I adored that game / I hated that game with my guts).

When we select an individual console, we see that the phenomenon has increased after the Wii U and 3DS.

We can check the distribution of scores attributed by a specific Reviewer (institutional - i.e. the ones contributing to the Meta score)

On voit ici deux publications ayant choisi des notes tous les 10 points, et une autre (IGN) qui semble avoir un système hybride - ou peut être ayant changé.

The following graph shows, for each Reviewer (institutional) (for example FNintendo, Gamekult...) their average score vs their average offset (i.e. the average difference between the rating given to the games by this reviewer and the rating given by Users for the same games).

The size of the dots shows the number of games rated by this reviewer on Metacritic. The bigger the dot, the more used the site is on Metacritic.

There is an interesting outlier in the lower left corner - GameKult which tends to rate games rather low, and to be less generous than users. It's rare ! We see here that the scatter plot is around the x=5 axis, which shows that in general, sites are more generous by 5 points than users.

The horizontal red line separates game critics that are on average meaner (left) than the user from those that are nicer (right).

Let's talk about developpers

2660 developers in the database, the most prolific being Capcom. We should take this list with a grain of salt, keeping in mind that each console is counted individually (so a game released on 4 consoles is counted 4 times), and that some developers have changed names over time (UbiSoft, EA Sports...).

Let's check all games from From Software

We can look at the developers with the strongest average offset, ie the greatest average shift between the Meta score and the user score averaged over their games.

Much like earlier games, let's look at the developers - average Meta rating vs. average user rating, displaying their average 'ratio' (i.e. how likely reviews for this developer are to receive an influx of user ratings)

Take home message: developers that are heavily criticized online are the one using predatory mechanisms in their game, micro-transactions, game pass, incomplete games, etc. This is seen with the big circles (high number of users complaining through a bad rating) in the lower end of the graph:

Let's check the evolution of Meta and User ratings through games for some developers.

Scores over the years

How do scores change over time?

Let's start by looking at the Meta score for each game, with regards to its release date.

You can see the color change over time, which translates the sequence of platforms over time.

Note: double-clicking on a platform in the legend isolates the data for that console. A simple click activates/deactivates this console.

We now display the same graph but for user scores. As expected, scores are lower on average.

This graph shows something fascinating: the points which float above the fray are quite... particular games. We see that it is only for certain games or certain series (Dynasty Warrior for example) that users bother to make noise on MC.

Finally let's check game offset through time.

And what if we display the ratio for each game, using the size of the dot?

Let's simplify this by looking at the average offset by console, per year.

There is an overall trend towards the growth of this offset over time. We also see that the start and end of console cycles seem to have a higher offset.

Let's forget platforms and let's look at the average Meta score and User score for games released per year.

We see that the Meta ratings are rising but that the user ratings remain stable on average (around 67%)

And here is the summary - evolution of the average offset, all games combined, per year.

We have seen in the previous graphs that this parameter depends a lot on the platforms and their life cycle. There is a global trend towards an increase of the offset. User tend to be more severe than critic, but this is because of an inflation of Meta scores over the year.