Using Stats to Make a Roster Decision: An NFL Example

Nation,

A group of local ultimate "think tank" members (as we're dubbed) have been geeking out over a very cool site called Pro Football Focus.

Looking at the best and worst players at each position this year, one of the thinkers asked where Cleveland Browns QB Brady Quinn would fall in the ratings of quarterbacks.

It's very true that Quarterbacks rely on many teammates for success. Receivers much catch, people must block, and people need to run the plays as directed in order to get success. However, QBs remain the most important people on the field and have greater control over the outcome of the game than any other player.

Sounds like a main offensive Handler's role in ultimate, no?

I went to the pro football focus site and looked for methodology... and didn't get much in order to answer the Quinn question. Looking for the Wages of Wins QB ratings, I ended up on a new site that builds on the wages of wins football measure (unlike the basketball wages of wins, the NFL model has been criticized).

http://www.advancednflstats.com/2007/07/better-qb-passer-rating.html

I took their formula (the model looks legit) and came up with the following 2009 wins produced of Quinn, Derek Anderson and Peyton Manning

Advanced NFL Stats= Brady Quinn versus Derek Anderson

QB Wins Added
Brady Quinn -1.542092593
Derek Anderson -3.963324675
Peyton Manning +3.300733894

Formula: QB Wins Added = (Comp% * 0.18) - (Int/Att * 50.5) - (Sack Yds/Att * 1.57) - 8

Thoughts
-Brady Quinn will lose 1.5 games more than the average quarterback based on his current performance
-Derek Anderson will lose 3.96 games more than the average
-Brady should continue to start, as he is the best of two below average options
-Peyton Manning is over 3 wins better than the average quarterback.

How it Applies to Ultimate?

I wish it did apply more. Right now, the best we have for stats is a clipboard or iphone used during the game, trying to capture everything under time pressure. It's easy to miss stuff and accurately measure what's really happening.

If you are doing stats already, and have data on your teams, please send it my way!

I will re-ask people to consider the following
-video tape their teams games
-break their games down play by play and stat by stat
-Work together with other teams to build a data set of a particular division to determine the average performance of players in a particular division/level

If that was done, we could move forward with a formula to determine whether players were average or below average on defense and offense.

Coordinating teams might be a problem, and one team breaking down video for all other teams is too much of a workload. However, teams like DoG have been using data for years to make better decisions about who's contributing.

It's time for teams to build on this. Don't wait for other teams to do it first and gain the advantage.

Using Video for Ultimate Stats and Analysis

Nation,

I've long said that video was the next step in better understanding the game of ultimate and the value of players on a given team/level.

Why video? The game can be better explained from more than just scores and player goals and assists. That is a small part of the bigger picture teams can use to better evaluate their team/opponents. The more data and games you can collect, the more meaningful your data. It is still impossible to perfectly predict future results, but you can create an arbitrage and understanding of what you need to do to win.

Based on my master thesis on the NHL, I certainly have an interest in bringing some form of better stats to ultimate. Also, my involvement in the game as a player also inspires me. If there is a link between stats and ultimate wins, I want to find it. (hypothesis is still that all around efficiency is the best predictor of success)

I can think a specific incident last year where my open team had a injured player from our open program (a very respect player of considerable reputation of perceived talent) took stats while watching us play a tournament. This person took these primitive stats ( number of points, goals. assists, turns) and made some "conclusions" about players (one defensive player in particular) which eventually made an influence in how that player was treated and seen on the depth chart all season. So, because this player wasn't flashy and didn't fit the prototype, all it took was some poor stats from a small sample to seal his fate. I'd really like to do something to help prevent those kinds of things happening.

People have experimented taking their own stats and using a palm pda on site to take stats as they go. However, I think the time constraints and speed of the game leave these methods susceptible to stat errors and omissions. Video is the way to go.

What are our current restraints?

• Not a lot of games are done in full and available for viewing. You have to videotape your own games
• I now realize that it takes 3-4 hours to watch a game and decode it for both offense and defensive stats. You need to have several players on a team helping to decode it with the same legend guide.. because it takes time and effort.
  
• We don't have organized leagues and stats from each team that would allow us to compute the average of teams and players, and use that to determine how players mark up versus the average player at his/her position.

This was one of the few weekends that I had free time on the weekend. I finally decided to put my money where my mouth was and test out data collection via video.

The next post is my first venture. After my first game decode/transcribe, I feel even more confident that teams willing to videotape their games and break down the footage can learn something about their opponents, and much more about themselves

Brainstorm: Stats in Ultimate

Nation,

The call to action for this post is

• Help me list key stats on o and d for teams and players

_____________________________________________________________
On "vacation" this week, but I really haven't had power the last few days due to the ice storm that has hit my native province. If interested, this site may sum it up.

Was looking at my UPA ultimate newsletter this week, and I have to say I was mostly impressed with the product. Great photos, complete coverage of the respective divisions, and a lot of great stuff to read. The writing was a little weak in the divisional write ups (right style, just a little less engaging and concise than possible) but I definitely like the overall product. Injury advice, UPA news, spirit awards, and a retrospective by Steve Mooney.

Sockeye Photo Courtesty http://www.seattlesockeye.org/

An interesting part of the newsletter is pages 30-31. Stats are listed for the finals of each division. Team and player totals were collected for the very basic:

• Assists
• Goals
• D's (defensive takeaways)
  
• Turnovers

Immediately my friends are thinking that I'm going to attack this because macro level stats can be worse (more misleading) than no stats at all. I admit that's my first thought. But then I realize that this is a good start.

Using stats to indicate true player contribution and value will never be perfect. So why do it? I'll give you the reason through a quote by Billy Beane:

I think the misconception about any statistical analysis is that you’re not going to be 100 percent correct,” Beane said. “What you’re trying to do is create an arbitrage … if you’re right 25 percent versus 20 percent you’ve created a 5 percent arbitrage opportunity. That’s really all you’re trying to do.”

Billy Beane
Source: ESPN/Ehrmann/WireImage

That's why its worth it.

I can tell you there is some observable trends that are of interesting note

• The masters final was the most efficient. There were 11 turnovers and 6 d's.
  
• The men's final in open had 44 turnovers and 15 d's. It is incredible that such talented teams were that inefficient.
  
• Highly touted Ben Wiggins of Sockeye had a very inefficient final. He was credited with just one d, no goals, no assists, and 4 turnovers. This might be exactly the type of player that is undervalued by a 4 variable stat package.
  
• Johnny Bravo's Parker Krug had 7 turnovers in the open final. Riot's Miranda Roth had 7 turnovers also in the Women's game. Their teams lost, probably because they didn't substitute these players off.
  
• Coed had the worst turnover to d ratio of the four finals. This in spite of the fact the newsletter talks constantly of the improved level of the division.
  

So, I am hoping viewers can help me by listing all key stats on both players and teams for offense and defense.

Post on the site or e-mail me. Forward it on to your stathead friends.