Reply To: MIT Sloan Sport Analytics Conference Video

[hockeygod]

Goalie Analytics

Ritch Winter (agent), Agent of Change, February 8, 2012

(Check this site – it is pretty interesting… http://www.theagentsofchange.com/ )

–
@HockeyAgentDad tweeted: Why is Devan Dubnyk (not a client) statistically the best starting goalie in Alberta? He may be the real meal deal.

In response:

@JayBrookman49 Tweeted – Enlighten us, unless Feaster just moved Kipper I object…

@YuriThomas99 Tweeted – what advanced goaltending stats do you like to evaluate with?

In preparing for free agency every year, I have started to employ mathematical analytics more and more to advise clients on things like target teams. Employing various statistics to more accurately project future team performance.

Initially, my foray into statistical and economic analytics began shortly after Marian Hossa approached me about representing him three years ago before he was traded to Pittsburgh. After meeting him in Atlanta, it became clear he was going to be much more interested in playing for an “elite” organization than in money alone.

After we met in Atlanta, I began devising a formulaic approach to measuring and projecting “elite” performance to assist with advising him. In the end, we determined that Marian would only consider teams that we determined had a high statistical probability of recording 100 points consistently over the five years after he became a free agent. We considered projections beyond the 5 years to be less accurate and not worthy of inclusion into our analysis even though the deal Marian would sign would extend beyond 5 years. We took this approach due to the fact contract terms for each team’s existing players were most often 5 years or less and who could project who would stay. So we focused on the 5 year period following the July 1 date on which Marian would become a free agent following the Cup run he had with Detroit.

The analytics proved to be amazingly accurate. Marian was the only player in NHL history to play in the Stanley Cup finals three years in a row with three different teams. So we made two very good choices. The results, as Marian shared all this with others, became a huge competitive advantage as an agent. No other company had taken the time to accurately measure the prospects for their unrestricted free agents. We had successfully placed a player on two Stanley Cup finalists and one winner in three years. Bookies would have trouble competing with that.

In preparing for free agency last year for Ilya Bryzgalov, we began looking at goaltending analytics more specifically. It was the first time a goalie with the statistics Bryz had would hit the open market at his age. In tandem with Michael Schuckers, a professor at St. Lawrence University, we built models designed to determine goaltender performance by team for Ilya. Some interesting information grew out of that.

For example, as Michael focused on Ilya’s save percentage in various situations and looked at adjusting a goaltender’s save percentage to the types of shots generally faced by goaltenders, some interesting bits of information began to surface to assist with the analysis we were preparing for Ilya as well as in general. The specifics as they relate to Ilya are confidential, but some of the more general insights we uncovered have widespread application and are, for the most part, just very interesting. Now I do not think that Michael Schuckers research is determinative, but it does suggest that a further course of study is required in all cases before picking up a new goalie or signing one of your existing roster players. Current NHL statistics do not provide the whole picture. Relying on them alone is risky.

Instead of comparing goaltenders by save percentage, Schuckers has developed a methodology that measures goaltenders by a more accurate measure, in my opinion. He calls it the “Defense Independent Goalie Rating” or DIGR. It is designed to create an adjusted save percentage number that more accurately measures goalie performance. Not a perfect measure, but more accurate than save percentage is on its own.

For example, we all know that a goalie who played in a game where he faced 10 breakaways and 30 shots as compared to an opposing goalie in the same game who faced 30 shots, but no breakaways and 20 shots on goal from outside the blue line, most likely had a better game – assuming each let in one goal and otherwise faced very similar shots. In this case, save percentage alone doesn’t tell us very much. Both had an identical save percentage, but very likely just one of them was named a star of the game.

The point is that some teams have such a strong defense that it results in their goaltenders facing a much smaller number of quality scoring chances than others do. In these cases, a higher save percentage on lower quality scoring chances that end up in shots on goal, would be less meaningful.

So how does one address this statistically?

Schuckers has come up with a great approach. His DIGR ratings begin with him first analyzing all of the shots on goal in the NHL in a given season. Then he determines, based on all of the shots taken in all of the regular season games, what types of shots goalies face on average per game – five from the left slot, two shoot-ins, etc.

After doing this, he determines each goalie in the league’s save percentage on each type of shot that the average goalie would face. So, what would each goalie save if he faced the shots each goalie in the league faced, on average? This gets you a goalie’s DIGR – his save percentage on the shots taken on average during an average NHL regular season game. It creates a much more level playing field for comparative purposes than raw save percentage.

This methodology provides us with an adjusted save percentage accounting for the quality and distribution of shots faced after taking out empty net goals, penalty shots and shootout shots. To calculate the DIGR for all NHL goalies, Schuckers has to build a model of each goalies performance during the applicable season and then mapped every shot taken by the entire league to the individual goalie map before applying each goalies save percentage on each type of shot to the average shots per game. The DIGR allows for a direct comparison of goalie performance since it is based upon the exact same types of shots, not just the shots that an individual goalie faced.

A few interesting points taken out of the league-wide analysis include:

Not surprisingly last year, Tim Thomas was the top goalie, followed by Cory Schneider and Roberto Luongo. While it is not a surprise that Thomas is at the top given his phenomenal year, the difference between Thomas and the second best goalie shrinks by about 0.5% after accounting for the difficulty of the shots he faced. Thomas had a great year but was helped by a strong defense.

Jonathon Quick had the easiest shots in the NHL and Cory Schneider faced the hardest set of shots.

Among goalies facing 1000 shots or more, Jaroslav Halak had to face the most difficult shots, on average. You have to think that will change this year under Ken Hitchcock.

Jonas Hiller and Bryzgalov are the only two goalies to appear in the top 10 of the DIGR for both the 2009-10 and 2010-11 seasons.

It doesn’t take a genius to figure out that the Calgary Flames had a more accomplished defense last year with the likes of Mark Giordano (my client), Jay Bouwmeester (not a client) and Robyn Regehr (not a client) as compared to the Edmonton Oilers. So, you would think that Miikka Kiprusoff faced much easier shots to stop on average than the goalies in Edmonton. It turns out that this was indeed the case and it dramatically affects the way the two teams should have evaluated their goaltender’s performances last year.

Is Kipper a worse goalie than Devan Dubnyk?

As the chart below displays Devan Dubnyk was among the top 10 goaltenders in the NHL in adjusted save percentage or DIGR and Mikka Kiprusoff was among the worst. If Dubnyk faced the shots that goalies on average see each game, he would have recorded a .922 save percentage (or DIGR), 8th in the NHL. Had Kiprussoff faced those same shots, his save percentage would have been .902, 45th in the league.

Take a look at the chart below:

Rank Name DIGR RawSV% Shots Faced Avg Goalie Salary10-11
1 TimThomas 0.9312 0.9386 1808 0.9184 6
2 CorySchneider 0.9285 0.9253 669 0.9009 0.9
3 RobertoLuongo 0.9269 0.9275 1752 0.9104 10
4 JonasHiller 0.9269 0.9249 1452 0.9067 4.5
5 IlyaBryzgalov 0.9234 0.9211 2103 0.9121 4.5
6 CamWard 0.9232 0.9221 2336 0.9123 5
7 Marc-AndreFleury 0.9227 0.9189 1738 0.9134 6
8 DevanDubnyk 0.9224 0.9164 1101 0.9067 0.8
9 CoreyCrawford 0.9219 0.92 1538 0.9109 0.85
10 CareyPrice 0.9218 0.922 2142 0.9132 2.5
11 BrentJohnson 0.9214 0.9236 602 0.9144 0.6
12 PekkaRinne 0.9204 0.9293 1881 0.916 2.8
13 HenrikLundqvist 0.918 0.9204 1961 0.9159 7.75
14 JaroslavHalak 0.9178 0.9117 1483 0.9055 2.75
15 TomasVokoun 0.9177 0.9208 1705 0.9152 6.3
16 OndrejPavelec 0.9177 0.9153 1700 0.9101 1
17 TuukkaRask 0.9176 0.9183 857 0.9143 1
18 SergeiBobrovsky 0.9173 0.9167 1500 0.9103 0.9
19 SemyonVarlamov 0.917 0.9223 759 0.9165 0.77
20 MichalNeuvirth 0.9164 0.9156 1280 0.9143 0.77
21 RyanMiller 0.916 0.9157 1958 0.9148 6.25
22 BrianBoucher 0.9159 0.9156 900 0.9187 0.93
23 JamesReimer 0.9141 0.9222 1131 0.9184 0.56
24 JoseTheodore 0.914 0.9178 961 0.9158 1.1
25 JonathanBernier 0.9139 0.9098 654 0.9176 0.77
26 AnttiNiemi 0.9138 0.9175 1745 0.9158 2
27 CurtisMcElhinney 0.9137 0.8997 708 0.9056 0.57
28 DwayneRoloson 0.9136 0.9138 1519 0.9196 3
29 MathieuGaron 0.9132 0.9027 884 0.9099 1.2
30 KariLehtonen 0.9125 0.9152 2029 0.9183 2.7
31 JohanHedberg 0.9122 0.9104 748 0.9134 1
32 CraigAnderson 0.9121 0.9157 1519 0.9182 2.13
33 NiklasBackstrom 0.912 0.9184 1531 0.9205 6
34 ScottClemmensen 0.9112 0.9102 835 0.9186 1.1
35 AnteroNiittymaki 0.9101 0.9018 611 0.9105 2
36 MartyTurco 0.9095 0.8974 799 0.9129 1.3
37 JonathanQuick 0.9091 0.9162 1634 0.9208 1.9
38 MartinBrodeur 0.9087 0.9057 1305 0.9177 5.2
39 JonasGustavsson 0.9071 0.8914 617 0.9121 1.3
40 JimmyHoward 0.9067 0.91 1801 0.9137 0.8
41 ChrisMason 0.9066 0.8921 853 0.9068 1.6
42 SteveMason 0.9064 0.9002 1503 0.9123 0.77
43 Jean-SebastienGiguere 0.9051 0.9029 762 0.9191 7
44 DanEllis 0.9043 0.9003 1083 0.9143 1.5
45 MiikkaKiprusoff 0.902 0.9058 1910 0.9198 7
46 PeterBudaj 0.9019 0.8961 1232 0.9119 1.25
47 BrianElliott 0.9 0.8947 1520 0.9126 0.9
48 NikolaiKhabibulin 0.8999 0.8917 1385 0.9068 3.75
49 RickDiPietro 0.8925 0.8847 772 0.9077 4.5

What does all this mean? It suggests that Devan Dubnyk, had he played on the average NHL team, would have been a top 10 NHL goalie based on his performance last year and Miikka Kiprusoff would not have cracked the top 40. It’s interesting, but not determinative. Just because shots came from the similar location in the DIGR analysis does not mean they were of the same quality of shots.

What is clear though is that this suggests further research, perhaps via in-depth video review, is required to test if the DIGR analysis is accurate in each case. That relying on save percentage alone just doesn’t provide a full and clear picture of goalie performance.

DIGR clearly makes the case for the Oilers giving Dubnyk the reins and seeing, with an improving defense over the next few years, if Dubnyk (not my client) is the guy. The stats seem to suggest he is. Time will tell. It’s much more difficult to be a top go-to guy than a back-up given the additional pressure, but Dubnyk appears, based on 35 games last year to be ready.

In looking for additional support for the performance of the starting goaltenders in Alberta last year, we looked at three star selections. Now, these are subjective of course, but they do at least suggest that in the opinion of at least one hockey expert who was “actually” at the game that the goaltender was among the best players on the ice. Looking at last year’s three star selections, Devan Dubnyk was named a game star 13 times (37% of the time) and Kiprusoff just 17 times in 71 games (24% of the time).

What it all suggests is that Devan Dubnyk may be the best starting goaltender in the province last year, and may soon establish himself as the best going forward..

In the end, who really was the best starting goalie in Alberta last year (I recognize that starter usually means majority of the games, but in Dubnyk’s case he was the starter at year end last year so we went with that)? I don’t know. What is clear however is that Devan Dubnyk with a regular save percentage of .916, a DIGR of .922 in the games he played outperformed Miikka Kiprusoff and gave the Oilers a chance to win most nights when there play really didn’t dictate that.

This season to date, Kipper boasts a .920 save percentage as compared to Dubnyk’s .908. With no DIGR to rely on yet, the jury is still out, but it looks like Kipper is winning this year’s goaltending Battle of Alberta with a solid bounce back season behind a defense that missed Mark Giordano for much of the year and lost Robyn Regehr to trade.

All we can say for sure is that all this brings to light a unique look at the goalie position that suggests current statistics, like was determined in baseball – only tell half the story. By way of example, can you tell me what the second assist actually measures? What value can it have expect in the case of goals like the one Brandon Prust set up in the Rangers win against the Flyers on Super Bowl Sunday. Check it out:

http://www.theagentsofchange.com/2012/02/ritch-responds-goalie-analytics.html

Now if Kipper leads the Flames to the playoffs this year – and he could — one will have to take a closer look at why. But, if nothing else, Oiler fans have to like the fact that the young goalie tending pipes for them now, might just be the real deal.

As you look at hockey games, look deeper. Think harder and ask yourself the tougher questions.

Is our goalie that good? How close are we to elite status? Looking deeper, going further and analyzing all this more effectively is what great teams do. Some intuitively, some with an increasing reliance on statistics and analytics and some with management teams that have vastly superior educations than their competitors.

Take the Stanley Cup finalists last year. The Bruins were run my three Harvard graduates. The other, Vancouver, was led by a group of well educated lawyers and businessmen. That is what wins nowadays.

Dean
M.Ed (Coaching)
Ch.P.C. (Chartered Professional Coach)
Game Intelligence Training
"Great education depends on great teaching."