The Stats That Lie to You

Framesports Team
Editorial
A stadium big screen shows the score, match clock and attendance figures at a rugby ground

We collect thousands of data points on rugby teams, from every variety of rugby, men’s and women’s, from amateurs to professionals. It is an odd thing to put in writing, then, that some of the most quoted numbers in the sport are the least trustworthy. Not wrong, exactly - measured correctly, reported accurately, and still pointing coaches at the wrong conclusions. Here are four we have tested against results, and what each one turns out to be measuring instead.

Possession reads like control and settles almost nothing

The half-time possession graphic is presented as a verdict. In the matches where we analysed both sides, the team with more of the ball won fewer than three games in five - better than a coin toss, nowhere near a verdict. When possession was close to even, which is most matches, the result was a genuine coin flip. Possession tells you where the ball has been. It does not tell you what either team did with it, and what you do with it is the part that scores.

Momentum is a story told after the fact

Everyone has seen a team “seize the momentum”. We went looking for it: if scoring runs were real, the team that just scored should score next more often than chance predicts. They do score next more often than half the time - but shuffle the same scorelines into random order and the streaks come out just as long. The pattern that feels like momentum is what ordinary sequences of uneven teams look like. The better side keeps scoring because they are better, not because they are hot.

The dominant tackle is the exception, not the standard

Commentary treats the collision as a coin flip that hard teams win. It is not close: the ball carrier wins getting on for nine collisions in ten, and the tackle that puts a carrier backwards is roughly a one-in-ten event. A defender who never dominates the contact is not failing - that is what almost every tackle looks like. What separates teams is the rare extra dominant hit, and whether the routine tackles stick at all.

The tackle count rewards the wrong thing

A big tackle count gets read out like a medal. In our matches it is closer to a warning light: the teams that make the most tackles usually lose, because a tackle count is really a meter of how long you went without the ball. Celebrating a huge defensive shift is fine. Building a game plan that produces one every week is not.

What to trust instead

The numbers that hold up in our data share a shape: they measure quality per event, not volume of events. Tackle completion - the share of attempted tackles that stick - climbs cleanly with winning. So does the share of collisions won. Points per visit to the opposition 22 beats possession as a measure of attack, because it prices what a team does with the ball rather than how long they held it. None of these appear on a half-time graphic. All of them beat the four numbers that do.

The practical move is not to adopt all of them at once. Pick one - completion is the easiest to collect and the hardest to argue with - track it every match for a season, and let it replace one of the liars in your team talk. A club that retires the possession graphic and reads out completion instead has upgraded its entire conversation about the game for the price of one habit.

Hero photo: Bearas, CC BY-SA 4.0, via Wikimedia Commons.

Related reading

More from the Framesports blog.

A lineout jumper is lifted to take the throw at an amateur rugby match under a clear sky
The derby and the forty-point rout get talked about for a year. The league table moves on the ordinary Saturday that finished within a score, and there are about four of those a season, spread evenly through it. Those are the games to film.
Framesports Team
A rugby player catching a high ball at the top of a leap
An 80-minute rugby match contains about 25 minutes of live play. We measured ball-in-play time across thousands of matches from every variety of rugby, men's and women's, from amateurs to professionals, and the shape of the game should change how you train.
Framesports Team
A rugby referee signals a decision in front of an empty stand
A rugby match is two halves of forty minutes with a break of up to fifteen, so eighty minutes of playing time on paper. From first whistle to last it runs about an hour and a half, and one match in eight passes a hundred minutes.
Framesports Team