Nation,
I've long said that video was the next step in better understanding the game of ultimate and the value of players on a given team/level.
Why video? The game can be better explained from more than just scores and player goals and assists. That is a small part of the bigger picture teams can use to better evaluate their team/opponents. The more data and games you can collect, the more meaningful your data. It is still impossible to perfectly predict future results, but you can create an arbitrage and understanding of what you need to do to win.
Based on my master thesis on the NHL, I certainly have an interest in bringing some form of better stats to ultimate. Also, my involvement in the game as a player also inspires me. If there is a link between stats and ultimate wins, I want to find it. (hypothesis is still that all around efficiency is the best predictor of success)
I can think a specific incident last year where my open team had a injured player from our open program (a very respect player of considerable reputation of perceived talent) took stats while watching us play a tournament. This person took these primitive stats ( number of points, goals. assists, turns) and made some "conclusions" about players (one defensive player in particular) which eventually made an influence in how that player was treated and seen on the depth chart all season. So, because this player wasn't flashy and didn't fit the prototype, all it took was some poor stats from a small sample to seal his fate. I'd really like to do something to help prevent those kinds of things happening.
People have experimented taking their own stats and using a palm pda on site to take stats as they go. However, I think the time constraints and speed of the game leave these methods susceptible to stat errors and omissions. Video is the way to go.
What are our current restraints?
- Not a lot of games are done in full and available for viewing. You have to videotape your own games
- I now realize that it takes 3-4 hours to watch a game and decode it for both offense and defensive stats. You need to have several players on a team helping to decode it with the same legend guide.. because it takes time and effort.
- We don't have organized leagues and stats from each team that would allow us to compute the average of teams and players, and use that to determine how players mark up versus the average player at his/her position.
This was one of the few weekends that I had free time on the weekend. I finally decided to put my money where my mouth was and test out data collection via video.
The next post is my first venture. After my first game decode/transcribe, I feel even more confident that teams willing to videotape their games and break down the footage can learn something about their opponents, and much more about themselves
11 comments:
I would agree that most stats kept using a pad and paper are useless. Recording the number of goals or assists, turnovers and blocks aren't particularily useful. You need far more context then that, and like you suggested there is a lot more to winning then goals and assists.
The best you could really do would be something like +/-, but even then you're going to need lots and lots of data before it becomes useful. Stats probably aren't a good tool to help you pick your team, or even build your depth chart. Certainly not within the context of a single season.
Haig,
I certainly applaud pad and paper efforts at the games. People that do it are trying to help their team. I would find it hard to get every pass and every event during a point or entire game. Seems taxing and prone to fail.
Not a huge fan of simple plus/minus. If you're an o line player with 6 fantastic players, you're likely to have an inflated plus minus. If you're on a bad team, it's vice versa. I also realize certain players get to play more in certain games. Thus, some people ring up the pluses versus weak teams, and other players get smoked versus tougher squads.
Just using stats only would lead to the same mistake as not using it all- I think it should be used with other tools to help your team win more games. I would definitely use it to help pick my team and build my depth chart.
You are right simple +/- wouldn't be useful. But if you had more information about whether if in was and O or D point and what team you were playing against you could get something useful that was easy to track. Even if all you got was knowing which defensive unit forced more turn-over against a specific team, or something equally simple, it would at least have some meaning and be easy to track.
You'd also be suprised, but you can get decent offensive stats using pad and paper, it is a real pain though. At CUEC in Montreal this year I took pen and paper stats for the Carleton team. For each pass I recorded who threw it, if it was completed and if it fell into one of 4 categories: force-side ins, hucks, breaks or bails. That of course didn't tell the whole story of our offense, but it was far more informative than just tracking turnovers, goals and assists.
That said, to do the kind of analysis you'd be wanting to do I'd have had to do the same kind of stat keeping over many many tournaments, and it was a big pain in the neck so I haven't done it since.
It's not that I think objective information is a bad idea. I've read Michael Lewis, and I can see where you're comming from, but there just isn't enough information out there. Even if we had detailed stats for a whole season, I'm not convinced that by the end of the year you'd have a sample size that was big enough to compare players and assess their value. That said the stats, although annoying to record did provide some insight into where we struggled as a team for that particular weekend.
One of my only complaints about the newer Ultivillage stuff is that it is really hard to analyze play away from the disc.
I guess that what is more useful from an analysis perspective is less appealing as entertainment.
I found that I appreciated players' contributions much more from watching video afterwards and seeing who really plays consistent D, is positioned properly, or creates space for their teammates well, compared to what I actually remember when I'm trying to Coach.
We should probably spend more time just watching video and taking notes as well as stats to get an idea of who is adding value to the a team.
I wonder if an injured player would be better served as a cameraman than sideline help.
Liquid took throwing stats for a full season, and it definitely helped us identify who was throwing most of our turns, but wasn't otherwise terribly helpful.
Another season we kept "line caller" stats, ie. who the opponent is, who pulled, who scored and who was on for each point. This let us do an O and D +/- for each player and for the team against each opponent. You can then calculate the expected team +/- for a player who played the same number of O and D points against the same teams as each of your players, and see who exceeds and who falls below the expected team values. This doesn't deal with issues of who else is on each line, nor of sample size, but it can still give you something to check your intuition against.
The role that I think stats can best serve is to settle arguments about playing style. I've taken huck completion stats to settle an argument about how well a team's huck game was working (2 for 9 in the half I took stats, fortunately this was not Liquid) and whether dropping was hurting us (2 drops both by the same player in the half, also not Liquid). This can help coaches assess their decision making.
Yaacov,
Liquid is a promising squad this year.
First of all, I appreciate getting captain feedback on this topic like your post. I'm not sure how in-depth your throwing stats were. If there were simply throws and throwaways, then it might not be as helpful as it could be. Type of throws, length of throws.. each level adds a clearer picture of what's really going on.
I also am not sure how you weighted the plus minus, but it sounds like a good start.
Before we get posts or ideas about what's helpful and what's not "terribly helpful" I want to warn captains to not lose sight of what they want to measure, and what they need to do to measure it before they go out and take the stats.
Garbage in.. Garbage out, and Quality in... quality analysis out.
Steven,
It would be great to have a high level of detail. I'm always jealous when basketball announcers say things like "he just made a 27.4 foot jumpshot."
If you're looking to do value comparisons between players, ultimate certainly lacks detailed stats. But if you just want a tool beyond your intuition to get a sense of what's happening out there, even knowing how many throws each player on your team makes can be quite eye opening. We had one player who threw three passes in a tourney despite an average number of points played. That info led to me watching more closely at the following tournament, and giving much better feedback to that player.
Completion stats, even if they don't take into account the type of throw are even more useful. We had a primary handler who threw the most passes in the tourney of anyone on the team and only had a 75% completion rate. He was responsible for 20 turnovers. Even if he hucked almost exclusively, that's a completion rate that's hard to live with. I previously had a feeling that this player was more risk-taking than most, but was in a much stronger position to achieve change when I had objective evidence to back up my views.
Liquid hasn't taken throwing stats for a couple years because of how much work it is. After two seasons of taking stats that agreed pretty closely with our captains' intuitions, it seemed like that time would be better spent elsewhere, but if I had a volunteer to do it, I would certainly take advantage of that.
Our most recent stats experiment was having two players independently take stats from video of all our games from Nats in Toronto. These showed that assigning blame between thrower and receiver in turnovers is quite subjective. And it's hard to find an objective rule that doesn't favour either throwers or receivers.
Most teams track who is on the field as part of their line calling. If you also track which team pulled and which team scored, you might as well do a bit of analysis with that data. You'll probably find that they match reasonably well with your intuition of what's happening, but even that is nice to know when players grumble about PT.
"Completion stats, even if they don't take into account the type of throw are even more useful. We had a primary handler who threw the most passes in the tourney of anyone on the team and only had a 75% completion rate. He was responsible for 20 turnovers. Even if he hucked almost exclusively, that's a completion rate that's hard to live with."
I would say it's hard to have a ton of meaning without type of throw, or without at least some context.
If that person is making the most high risk throws because no one else on the field has the throws and you've built the need for them into your offence, then that's something you may have to live with. I wouldn't necessarily be upset if the player making the most throws also had the most turnovers. The type of throw is very important.
That said, 75% is pretty bad.
Haig,
I'm not arguing that the stats stand alone if you don't know the throw type. Your point about context is a good one. My point is that as captain, you know a lot of that context without the need for stats. You may not know it to statistical precision, but you know it well enough to be able to tell whether a 75% completion rate is expected or problematic.
The point I think most ultimate captains miss when it comes to stats is that data which is incomplete in a stand-alone sense can still be very useful to a captain who knows the context, has an intuition to compare the stats to and uses them to improve players rather than judge between players.
I agree on the pit-fall that many frisbee captains make. I think it often stems from an over reliance on stats that people don't fully appreciate and understand. The same thing happens in all sports. Basketball fans will be wowed by a player who scores a lot while ignoring their poor efficiency.
I also found for the one tournament where I kept throwing stats, that they were far more useful for identifying team trends or areas for individual improvement rather then for comparing individual players.
Post a Comment