Measuring Goalkeepers: The Limitations of Goals - xGoals
/By Bill Reno (@letsallsoccer)
Last month we took a look at how bad the Goals Against Average (GAA) stat is for evaluating goalkeepers and how the save percentage (S%) is slightly better. To show the inefficiencies of the GAA and S%, I compared them to ASA’s own unique stat of Goals Minus Expected Goals (GmxG). The GmxG looks at where shots are taken, calculating the likelihood of a shot going in from that distance and angle to goal, which ends up telling us if a goalkeeper is giving up too many goals given the circumstances. This is great for a couple of reasons, the main one being that we have a more accurate reading on shots on goal than the ol’ shot percentage. If one goalkeeper consistently sees shots from distance while another is routinely left out to dry by his defense, the GmxG will let us know how many goals each goalkeeper should be conceding even if their SOG and goals are similar. However there are some shortcomings with any stat and the GmxG isn’t any different.
The most obvious limitation of the stat is that it only values shot stopping. It does not take into account how many goals have come from poor distribution, bobbled crosses, or even weak rebounds a goalkeeper has gifted an opposition. For example, Bill Hamid leads the table by quite a margin but it doesn’t take into account his poor rebound against New York earlier in the season. Sure, it’s just one example, but it doesn’t affect a goalkeeper’s GmxG.
The GmxG is also hindered by the fact it needs a large sample size to truly even out. After about one hundred interactions (saves + goals), based off my own observations not anything scientific, it starts to accurately tell of a goalkeeper’s performance. However before then, there are too many variances that can occur to sway the numbers. Aspects the GmxG does not take into account are shot speed, a goalkeeper’s view, and the movement of the ball prior to the shot being taken. All of these affect how difficult it is to save the shot. Perhaps the most annoying usage of an GmxG is after one game. You may apply expected goals to an offense after one game, but there are too many unknowns to judge a goalkeeper on after half a dozen shots.
Lastly, and probably the most damning, goalkeepers are typically benched for mistakes that do not have to deal with shot stopping. Raïs M'Bolhi is a good example of a goalkeeper who struggled in the air and with his own distribution. In fact, he was so miserable that he hardly had a chance to save any shots before he was benched because his mistakes were so numerous. And, of course, let’s not forget Carlo Cudicini. Saving shots is clearly needed in a good goalkeeper, but it doesn’t let us know how else the goalkeeper is helping or hurting their team.
The GmxG is not a WAR (wins above replacement) for goalkeepers. It’s an intense look on one dimension of goalkeeping. Although it is obviously an important one, and one we can learn a lot from, it does have it’s limitations. Is it better than the GAA? Absolutely. Is it better than the Save Percentage? Yes, significantly. Should it be the only thing taken into account in gauging goalkeepers this season? Definitely not. At least for now. The more the stat is smoothed out and missing pieces are marked down, the more confidence we can place in it.
| Keeper | Team | Min | SOG | GA | xGA | G - xG |
|---|---|---|---|---|---|---|
| Bill Hamid | DCU | 1341 | 71 | 12 | 19.9 | -7.9 |
| Nick Rimando | RSL | 1348 | 59 | 16 | 21.29 | -5.29 |
| David Ousted | VAN | 1829 | 79 | 20 | 25.06 | -5.06 |
| Josh Saunders | NYC | 1725 | 91 | 26 | 29.43 | -3.43 |
| Tim Melia | SKC | 764 | 31 | 5 | 8.28 | -3.28 |
| Stefan Frei | SEA | 1663 | 89 | 18 | 21.06 | -3.06 |
| Clinton Irwin | COL | 1633 | 72 | 18 | 20.95 | -2.95 |
| Chris Konopka | TOR | 861 | 51 | 15 | 17.68 | -2.68 |
| David Bingham | SJ | 1633 | 83 | 18 | 20.28 | -2.28 |
| Jeff Attinella | RSL | 388 | 29 | 9 | 10.76 | -1.76 |
| Bobby Shuttleworth | NE | 1718 | 79 | 29 | 30.17 | -1.17 |
| Tally Hall | ORL | 767 | 40 | 10 | 11.17 | -1.17 |
| Jaime Penedo | LA | 1337 | 69 | 19 | 19.79 | -0.79 |
| Sean Johnson | CHI | 859 | 33 | 11 | 11.11 | -0.11 |
| Jon Busch | CHI | 672 | 29 | 11 | 10.48 | 0.52 |
| Tyler Deric | HOU | 1643 | 68 | 21 | 20.13 | 0.87 |
| Brian Sylvestre | PHI | 758 | 41 | 14 | 13.07 | 0.93 |
| Evan Bush | MTL | 1239 | 84 | 23 | 21.82 | 1.18 |
| Adam Larsen Kwarasey | POR | 1721 | 63 | 20 | 18.32 | 1.68 |
| Chris Seitz | FCD | 672 | 27 | 10 | 8.15 | 1.85 |
| Steve Clark | CLB | 1617 | 92 | 29 | 26.44 | 2.56 |
| Dan Kennedy | FCD | 1048 | 35 | 14 | 11.12 | 2.88 |
| Joe Bendik | TOR | 574 | 36 | 11 | 8.08 | 2.92 |
| Donovan Ricketts | ORL | 961 | 34 | 14 | 11.07 | 2.93 |
| Luis Marin | SKC | 771 | 30 | 11 | 7.96 | 3.04 |
| Luis Robles | NYRB | 1618 | 64 | 23 | 19.75 | 3.25 |
All that said, let’s finish on MLS Goalkeeper of the Year Power Rankings
David Ousted - Week after week, Ousted is widening the gap between him and the rest of the crowd. Can anyone catch the Danish? It seems unlikely at this point as he has now won MLS’s Save of the Week three times in a row.
Stefan Frei - Could possibly win the award just about any other year. His penalty save was great but it’s not like Ousted didn’t do that twice in one game the week before.
Nick Rimando - The top three stay the same from last month and Rimando continues to defy my early year prediction. I still think he’s playing low percentage guesses but right now he’s keeping RSL relevant despite his team’s lackluster performances from earlier in the season.
Bill Hamid - The lack of a Gold Cup could have played in Hamid’s favor but he can't quit these nagging injuries. Now out for 4-6 weeks, backup Andrew Dykstra could see himself on the shortlist if he continues strong play.
Tim Melia - What’s crazier? Unseating a foreign player specifically brought in for a starting role and barely making the roster in the first place? Or jumping into GOTY discussion despite missing the first third of the season? It's All Crazy! It's All False! It's All a Dream! It's Alright.
Tally Hall - The Lions are glad to have Mr. Hall back in goal. His body is still looking a little stiff since returning to the field but his mind is clearly in a good place as he holds down the fort in Orlando.
Steve Clark - Could very well be Clark’s last time on this list as we drop down to six next month. Inconsistency is the problem and as Columbus pushes towards playoffs, that’s the main thing they’ll need from Clark.