The Impact of VAR on Penalties
/By Alex Bartiromo
A couple weeks ago I took a look at the role that penalties play in adding luck to the outcome of an MLS match. In the process of writing that article, something came up that warranted further investigation, but didn’t quite fit into my initial piece. We wondered whether the introduction of Video Assistant Referee (VAR) in 2017 had changed the distribution of penalty calls in MLS. With that in mind, I decided to take another look at the penalty data to see what I could find about the nature of penalty calls themselves, rather than how they affect the outcome of a match, and whether the introduction of new technology affected referees’ decisions.
| Avg. min. of PK | PKs | PKs per team | PK <20 mins. | Matches played | PK per match | |
|---|---|---|---|---|---|---|
| 51:04 | 84 | 4.7 | 12 | 306 | 0.27 | |
| 51:22 | 72 | 3.8 | 9 | 323 | 0.23 | |
| 49:59 | 81 | 4.3 | 11 | 323 | 0.25 | |
| 51:20 | 134 | 7.1 | 18 | 323 | 0.42 | |
| 51:24 | 108 | 5.4 | 17 | 340 | 0.32 | |
| 55:11 | 109 | 5.5 | 11 | 340 | 0.32 | |
| 50:31 | 119 | 5.4 | 20 | 374 | 0.32 | |
| 52:41 | 138 | 6.0 | 24 | 391 | 0.35 |
So, let’s begin with the low-hanging fruit: has the implementation of VAR led to an increase or decrease in the number of penalties called in MLS? Well...here’s your first disappointment: the data is inconclusive. As you can see in the table above, the overall number of penalties called has increased in fits and starts since 2011. Dividing the total number of penalties by the number of teams in the league each year to account for expansion (penalty kicks per team, or PPT), we see that MLS has gone from a relative low in 2011-13, where teams averaged between 3.7 and 4.7 penalties called a season to a relative stability, or slight increase, in the period 2015-2018, where teams averaged 5.4-6 penalties. In terms of penalties per match (PPM), that comes out to an increase from 0.22-0.27 (2011-2013) to 0.32-0.35 (2015-2018). Confounding simplistic narratives however, is the 2014 season, which saw an inordinate amount of penalties called. The amount of PPTs increased by a factor of >1.5, from 4.26 to 7.05 (!!), while PPMs jumped from 0.25 (2013) to 0.41. That’s a penalty approximately every 2.5 matches, which totals out to somewhere between three and four penalties each matchday (in 2014 matchdays, which involved nine matches in a regular week), an astronomically high count in the history of MLS and one that hasn’t been reached since, despite the increase in penalties since the beginning of the decade. Indeed, the total amount of penalties called in 2014 wasn’t matched until 2018, when there were five more teams and 68 more matches played!
What could have caused this? It seems strange that the 2014 season should have so many more penalties than the two that preceded it, given that the macro conditions of the league were essentially the same. In all three seasons (2012-2014) there were 19 teams; VAR hadn’t been introduced yet; there were no major rule changes that would indicate such a drastic shift in the way penalties were called. Absent any answers from my memory, I turned to Google for assistance. It told me that there was a lockout of referees at the beginning of the season following failed contract negotiations that theoretically could have led to more penalties being called by lower-quality replacements. That explanation is pretty implausible though, given that the lockout only lasted for two weeks, during which referees called seven penalties, no higher than the average for the rest of the season, so we are left searching. Potentially, there are explanations that aren’t available to the public. Perhaps the Professional Referees Organization (PRO) issued a directive to be more sensitive to fouls in the box. Perhaps they told the refs that it’s ok to blow the whistle even when they don’t really mean it. Perhaps the Berenstain Bears really are the Bernstein Bears (the truth is out there). The point is, absent any further evidence, we will have to stick to the null hypothesis (also the most boring explanation), which is that the 2014 was simply an outlier, caused by the winds of luck that occasionally blow through MLS stadiums, never to be repeated again.
Still, even excluding 2014, the last four seasons have all been in the top five (of an admittedly small dataset) in terms of penalties called by team. 2018—the first full season with VAR—represents an increase over the stable 2015-17 period, where penalties per team (PPT) varied only between 5.4 and 5.45, and the PPM held steady at 0.32 (0.318-0.321 if you want to go a decimal deeper). Does this mean that VAR is leading to more penalties being called? There is a plausible argument that referees would be confident in making a penalty call that they might previously have been more judicious about, knowing that they will be backed up by technology (and let off the hook if they are proven wrong). However, good explanations do not a reality make, and we simply need more data to answer this question. One season isn’t enough to spot a trend. Penalties are down so far in 2019, so we’ll have to see how it plays out over the course of the next couple of months, and really, years.
However, there is still another question to examine. one hypothesis was that VAR had contributed to penalties being called earlier in matches. In other words, VAR would take away some of the reticence referees have to make big calls early in games. Looking at season-wide averages, such a trend is hard to spot. 2017—which had VAR for a bit less than half the season—had the second-earliest average time of penalty (50:31), but 2018 had the second-latest, at 52:41. As you can see in the table, there isn’t a lot of variation in these numbers in general, with only a two minute difference in average overall and no clear trend in one direction or another.
Still, while the average time hasn’t changed much, perhaps the distribution of the penalty calls has. To investigate this, I looked at the number of penalties awarded in the first 20 minutes of a match, and found that the two highest totals were in 2017 & 2018. Something to work with! Might referees be less afraid to make a major decision on the outcome of a match earlier, now that they know that their decision will either be justified or overturned by hard video evidence? The data seems to point in that direction. In addition to having the highest number of penalties called in the first 20 minutes, 2017 and 2018 are the first two seasons to have more than 20 penalties called in that time frame. Even the hipster 2014 season could only muster 18. Unfortunately, however, the picture the data paints is a bit more complex when you parse it out.
You see, VAR wasn’t introduced until week 22 of that season, on the August 7th matchday. By then, 14 of the 20 penalties that occurred in the first 20 minutes of a match had already been called. That’s 70% of all the penalties that would be taken, when ~65% of the matches had been played. That means that, if anything (and there really isn’t anything to if about), the rate of early penalties slowed down after VAR was introduced. Ok, small sample size, back-of-the-napkin math, arbitrary cutoff points—I get it. Listen, I was an English major; what did you expect? But the point of all this is that, in the already tiny sample we have of penalty data from the VAR era, the 2017 data does not point to the conclusion that penalties were called earlier.
So, what can we say about the impact of VAR on penalty calls? We have data which gives us some interesting leads and ideas to be explored further, but it requires more inputs before we can come to any conclusions. It’s not exciting, but it does open up a few interesting avenues of investigation for future nerds to look into when he have more information. Stay tuned.