A month or two ago, both DTM About Heart and MannyElk, two exceptionally talented hockey analytic experts, released their Wins Above Replacement (WAR) data. The two analysts calculate their statistic differently, but both use smart formulas that make a lot of sense, and show relatively similar results. For this post, we’ll be using MannyElk’s data, but simply because Sean Tierney was kind enough to put it in an easy to read graph, not because it is in any way better than DTM About Heart’s. To put this simply, WAR is a statistic that measures a players contributions on both offence and defence in goals, somewhat like plus-minus, except using advanced statistics to measure offensive and defensive contributions, instead of just goals scored and against. These advanced statistics are a better representation of skill than goals scored and against while the player is on the ice, like in traditional plus-minus. The contributions of the player are then measured against those of a replacement level player, which is the player that would replace said player if that player was unable to play in any way. Essentially, a replacement level player is the 13th forward, the 7th defender, or the 3rd goalie. The goals are then translated to wins, to find how many wins a player is worth over a replacement level player.WAR is a complicated statistic that the average person will have difficulty calculating, including me, so I’m not going to go too far into depth on it here.
After the data was released, there was a lot of anger and general dislike directed towards this statistic, largely in part due to how it put certain players ahead of others. As the following graph shows, this statistic places Ron Hainsey, a good defenceman, but by no means a top pairing player, ahead of Auston Matthews, who won the Calder, and looks like he will be the face of the Toronto Maple Leafs for quite some time.
Hockey fans didn’t like this very much, with plenty of Maple Leafs fans going on Twitter tirades defending their star forward. I, and most other people with an understanding of analytics that is above the level of “beginner”, didn’t see any issue with this.
Yes, Auston Matthews is better than Ron Hainsey, there is no question about that. But what is the point of statistical analysis of all that it tells us is what we already know? If that’s what we are looking to get out of it, why even use it all? Statistical analysis in hockey is in no way perfect, but neither is the “eye test”. Traditional and statistical analysis tell us two different things. The “eye test”, or traditional scouting, measures a player’s skill, and stats tell us the player’s contributions, and when the two are used together, they can tell us if a player is using his skill to it’s full extent, and if the situation the player is in is working out.
A great example of the usefulness of statistics is Oilers defenceman Kris Russell, who is the subject of what is perhaps the biggest disagreement of traditional and statistical analysis in NHL history. On the surface, Russell appears to be a hardworking, shot-blocking blue liner that can be counted on in the defensive end. When you dive deeper, the statistics tell us that Russell has a negative impact on possession, and the production of his teammates, as well as being poor when defending leads, which is what a player considered to be a defensive defenceman like him is expected to be good at. For me, it’s the fact that he plays poorly when ahead in games that makes me believe that he isn’t a top 4 defenceman, as in my opinion, a top 4 defenceman shouldn’t need to be protected, and only be played in certain situations. A bottom pairing blueliner however, can be protected, making it the optimal role for Kris Russell to play in.
Using the eye test, I would say that Russell appears to be a middle pairing defenceman. When we dive into the stats, he looks like more of a bottom pairing defender. If we take both conclusions, and weight them equally, we can conclude that Kris Russell is a 4th or 5th defenceman, probably on the lower end of the scale for 4th defencemen, and the higher end for 5th defencemen. However, I’m more stats oriented, and I believe that stats tell us more than our eyes, but not to a huge margin. When I am making conclusions about a player, I weight statistics at about 65-70 percent, and traditional scouting at 30-35 percent. With statistics weighted higher, we can conclude that Russell is a bottom pairing defenceman, which is what I believe.
When only using statistics, the general opinion is usually nearly identical. However, statistics are not everything, and the eye test is also important while making conclusions, although in my opinion, not equally. However, others may believe that the eye test is equally, or more important that statistics when judging a player, and that’s okay, because at the end of the day, you are entitled to your opinion, and as long as you include both statistics and traditional scouting in your conclusion, your opinion will likely be an informed one, and therefore a legitimate one. It’s when one ignores one component, either stats or scouting, that that opinion is no longer an educated one, and in most cases, is no longer a fair one. Information is key to an opinion, so why limit yourself to only one kind of information, when you can have two? It doesn’t make sense.
WAR is an innovative and great statistic created by smart people, but the issue with it is not that it puts Ron Hainsey ahead of Auston Matthews, it is because it does not use traditional scouting. The fact that it does not use traditional scouting is actually why it puts Hainsey ahead of Matthews, as it only utilizes the stats, which don’t nearly tell the full story, especially in this situation. So please, stop criticizing WAR because it puts one player ahead of another. The real issue lies somewhere else, but that isn’t WAR’s fault, or the fault of the smart people that created it. The issue is the issue that lies in all statistics, that they don’t use scouting in their conclusions, but the reality is that that is impossible, because scouting is all opinions, and everyone has a different opinion, and opinion cannot be defined in a concrete number.
You can use numbers in making a conclusion, but your conclusion should not be measured in numbers, as opinion cannot be defined in a number. In a perfect world, a conclusion should be defined in a paragraph, or multiple paragraphs that include numbers, but are still mostly words. All that I ask is that you don’t use just numbers, because that isn’t as accurate as it could be, and why not make it as accurate as possible? Please, use a mix of words and numbers. Do it for me.
Just to set things straight, I support the use of statistics. In fact, I encourage it. As I mentioned earlier, I weight statistics over scouting when judging players. If you follow my NHL draft coverage, you know that I use my spreadsheet a lot. It’s my main source of information on prospects, but that being said, it is not my sole source. I also watch games and highlights, as well as reading scouting reports from other scouts. I do all that, because I don’t see why I should share my opinion if I’m not going to do my best to make sure that it is as well informed as possible. Ryan Merkley, a top prospect for the 2018 NHL Draft, is number one on my draft spreadsheet. If I only used stats, Ryan Merkley would be #1 on my draft rankings, and guys like Rasmus Dahlin and Andrei Svechnikov, number 1 and 2 on my, and many other, draft lists, wouldn’t even be top 5. Anybody that follows the draft knows that Dahlin and Svechnikov are better prospects than Ryan Merkley, but if we only looked at stats, we wouldn’t know that. When we do mix in scouting, we realize that Dahlin and Svechnikov truly are better, and that Merkley, despite being incredibly talented, has far too many red flags to be #1.
This can also go the other way, as Calen Addison, another 2018 prospect, is ranked late first, early second by many, but due to his statistics, I have him in the mid first round.
Statistics don’t mean much without scouting, and scouting doesn’t mean much without statistics. They go hand in hand, so use them hand in hand. Please.