Are Competitions Transitive? Cycles and Luck in Pairwise Data

If A beats B and B beats C, does A beat C? Using every table in the IRW’s competition collection (team sports, chess, game-playing bots, human judgments and animal dominance), we test whether cycles are more common than a transitive Bradley-Terry model produces by chance, and show why the obvious ways of checking are misleading.

Published

October 23, 2026

This vignette — the choice of data, the compute scripts, the analysis design, and the writing below — was produced largely by Claude (Anthropic), working from a task specification and with human review. Check the compute script (intransitivity_compute.R) directly if you are relying on the numbers here.

Motivation

Rock-paper-scissors is the textbook example of intransitivity. Rock beats scissors and scissors beats paper, so if wins were transitive, rock would beat paper. Instead paper beats rock: the three beat each other in a cycle. No ranking of the three is consistent with who beats whom, so it makes no sense to call any of them the strongest. Ranking, though, is what most models for pairwise data do. The Bradley-Terry model (Bradley and Terry 1952), used in the course’s competitions lesson, gives each competitor a single strength and lets the difference in strengths set the odds of each game. It is transitive by construction: if A is likely to beat B and B is likely to beat C, then A is even more likely to beat C. That lesson sets intransitivity aside. This vignette picks it up. (The IRW’s sports data also appear in the vignette on sports trial data, which asks a different question: who takes the hard shot.)

Throughout, a cycle means three competitors who beat each other in a circle: A beats B, B beats C, and C beats A, as in rock-paper-scissors. (Longer cycles exist too, but when everyone has played everyone, any longer cycle implies a three-way one, so three is where we look.) The word can mean two different things, though, and the distinction runs through this whole page:

  • A cycle in the observed results: in the games actually played, A won most of its games against B, B most against C, and C most against A.
  • A cycle in the underlying win probabilities: A is more likely than not to beat B, B to beat C, and C to beat A. This is real intransitivity, of the rock-paper-scissors kind.

The two come apart because of luck. Even when the probabilities are perfectly transitive, a short series can go the wrong way, so observed cycles appear in any competition data. The question is whether there are more of them than luck produces, and that excess is what reveals cycles in the probabilities.

Cycles are not exotic. Psychologists have long found that people’s preferences can cycle (Tversky 1969), a mixture of transitive voters can produce a cyclic majority (Condorcet’s paradox, Condorcet 1785; Gehrlein 2006), and in games, a player can be beaten by one style and beat another. The question is empirical: in real competition data, is there cyclic structure beyond what a transitive model produces by chance? The IRW’s competition collection holds a wide range of such data: more than a century of North American leagues and European football, cricket, chess on Lichess, engine and bot tournaments, people judging which of two things is riskier, better written or more convincing, and the outcomes of dominance encounters among animals. We use nearly all of it.

The short answer: team sports show no detectable intransitivity, while online chess and game-playing bots clearly do. Most human judgments look transitive. Dominance among animals often does not, though there the cyclic structure has other plausible explanations. Getting there takes some care, because observed cycles show up in every kind of data, and in team sports and most judgments they are no more common than luck makes them.

Data and methods

Data

Code 
U |>
  group_by(Family = family) |>
  summarise(Units = n(), Games = num(sum(games)), `Agents (median)` = median(agents),
            `Games per pair (median)` = f2(median(games_per_pair)), .groups = "drop") |>
  knitr::kable()
Table 1: The analysis units by family. Agents and games per pair are medians across units.
Family Units Games Agents (median) Games per pair (median)
MLB 169 230,653 16.0 15.80
NBA 86 68,643 22.5 3.80
NHL 109 61,280 12.0 5.20
NFL 120 17,579 20.0 1.30
Football: England and Italy 522 231,640 22.0 2.00
Football: other leagues 301 80,703 18.0 2.00
Football: club cups and internationals 25 65,721 108.0 2.50
US college football 2 5,705 541.0 1.00
Cricket 288 20,521 8.0 1.70
Lichess 2013 bullet 12 92,037 80.0 4.95
Lichess 2013 blitz 12 45,441 80.0 3.30
Lichess 2013 classical 12 24,508 80.0 2.30
Rugby union 15 2,850 9.0 6.20
Quiz 13 1,559 11.0 1.10
Engines and bots 21 188,990 80.0 6.20
UFC and padel 3 2,410 80.0 1.70
Judgments 69 346,205 35.0 5.00
Animal dominance 215 250,376 12.0 4.60

We use 99 of the 109 tables of the IRW’s competition collection, 1,736,821 games in all, split into 1,994 units. Within a unit, each agent’s strength in the Bradley-Terry model is held fixed; it can change from one unit to the next. Most units are one league-season. Divisions that never meet within a season (England’s four divisions, the NBA and ABA before 1976) are split into separate units. The tables left out are t20_hyper, a subset of the cricsheet IPL data; mlb_through2023, whose seasons Retrosheet also covers, and which includes the postseason; Kaggle’s Connect-4 and the pickleball matches, where too few pairs met more than once to form a dense core (see below); and the six Place Pulse 2 tables of street-view image comparisons, where each of about 110,000 images is compared only two to five times, so no dense core exists. Where two tables cover the same season (538’s NFL games and nflverse’s; 538’s soccer leagues and the footbayes, eufootball and openfootball tables), each season is used once.

  • Team sports. One unit per league-season, regular season only: MLB since 1871, from Retrosheet; 538’s NBA, NHL and NFL games (NFL from nflverse since 1999); English and Italian football since 1888 and 1929 (footbayes); other European, American and Australian leagues; MLS; club cups and men’s and women’s internationals (by decade, teams with at least five games); US college football; cricket leagues and internationals from cricsheet; and tier-1 rugby union internationals, by decade. Openfootball adds the Austrian, Argentine, Brazilian and Mexican seasons that 538 lacks. The home side gets a home-advantage term; neutral venues get none.
  • Chess on Lichess. The Lichess 2013 tables hold every rated game played on the site that year (Lichess, n.d.), split by time control: bullet (under 3 minutes per player), blitz and classical. In 2013 the site was small, so its regulars played each other again and again. A unit is one month of one time control, restricted to a dense core: players who met at least 5 opponents at least twice each, then the 80 most active of them, with games among those 80 only. Each pair is capped at its first 10 games that month, so that a pair of friends playing 60 games in a night, or one account farming another, cannot drive the result. White’s first move is the “home” advantage. The pre-game rating difference enters as a fixed offset, so the strengths estimated are strengths beyond each player’s rating, and a player who improves during the month cannot pass for a cycle.
  • Engines and bots. The TCEC computer-chess championships, the SSCAIT StarCraft bot ladder (one unit per year, 2021–2023, without games in which a bot crashed), and 17 game-playing competitions from Meta Kaggle (Kaggle, n.d.), where every agent is one submitted bot. A Kaggle bot is a fixed program, so it cannot improve during a competition. Each gets the same dense core as Lichess, without an offset. The IRW has one more Kaggle table, Connect-4 (2023–24), but few of its pairs of bots met more than once, so it has no dense core.
  • UFC and padel. All UFC fights, and the World Padel Tour by category, with the same dense core (one meeting per pair is enough here). The pickleball table has no such core: most of its doubles teams played once.
  • Quiz. The 2018 ACF regional quiz bowl tournaments, one unit per site, and University Challenge since 1994, whose knockout series are too sparse on their own, pooled into one dense core of institutions.
  • Judgments. People comparing two things: which of six European business schools a student prefers, in CEMS (Dittrich et al. 1998; Turner and Firth 2012); which of two public risks is worse on one of eight criteria, such as which killed more Americans last year, which caused more harm, or which worries the respondent more (Friedman 2019); which of two movie reviews is more positive, which of two 2008 Senate campaign ads is more negative, which of two human-rights report passages describes more torture, and which of two survey answers expresses more worry about immigration (Carlson and Montgomery 2017); which of two ministries is more salient (Zucco et al. 2019); which of two German legislators is further left (Guinaudeau and Breunig 2024); EloChoice ratings of physical attractiveness, and its sequel asking which of two people looks stronger; which of two images looks more complex (two sessions, each cut to a dense core of images); which of two arguments is more convincing (one unit per side of each of 16 debates); and comparative-judgment studies in education, in which judges pick the better of two GCSE scripts, self-reflections, proofs (for explanation, rigour, insight or understanding) or passages of mathematics text. Each table, or each session or debate side within one, is one unit.
  • Animal dominance. Who won each aggressive encounter or displacement within a group of animals: baboons at Amboseli (five groups, by year), chimpanzees at Gombe (males by year, females in five-year windows, since they meet less often), spotted hyenas of the Talek clan (by year) and three other clans (no dates, so each clan is one unit), Tonkean macaques and vervets (by year), mountain chickadees at feeders (one unit per flock and winter), elephant seals (one breeding season each), and bird species displacing one another at FeederWatch feeders. Clans of more than 150 hyenas are cut to the same dense core as Lichess. There is no home side.

A draw counts as half a win for each side. One caveat on “games per pair”: with about one game per pair (cricket leagues, NFL seasons, most judgment tables), the data hold very little information about any one matchup, which matters for everything below.

Counting cycles, and why the count needs a null

The obvious check is to count cycles. For every three agents that have all played each other, look at who won the head-to-head majority in each pair. The triad is transitive if one agent beat both others, and cyclic if each agent beat one and lost to one. With noise, though, a transitive world produces cycles too: if A is only a little better than B, B will win their series often enough. So the share of cyclic triads has to be compared with what a transitive model produces on the same schedule, not with zero.

We do that with a parametric bootstrap. Fit Bradley-Terry to the unit (an ordinal version when there are draws), simulate every game again from the fitted model on the same schedule (same pairs, same home sides), recount, and repeat 200 times.

Counting is the natural way to think about cycles, but it is a weak test. It reduces each pair to who won their series and throws away the rest, and with two or three games per pair that series result is mostly noise. In bullet chess, where the likelihood test below does find intransitivity, the share of cyclic triads moves by only about one percentage point. So we use the count to describe the data, not to decide.

A second summary keeps the margins. HodgeRank (Jiang et al. 2011) fits a single ranking to all the pairwise log-odds at once, by weighted least squares, and its cyclic share is the part of those log-odds that no ranking can explain. It goes through the same bootstrap as the count, and is computed for units with at most 150 agents.

There is one subtlety in the simulation. The fitted strengths are more spread out than the true ones, because they include estimation noise. A null simulated from them makes games too predictable, and so too few cycles. We therefore rescale the strengths by the square root of an empirical-Bayes shrinkage factor (estimated true variance over observed variance) before simulating. That factor cannot always be estimated. When an agent wins or loses every game, or a ranking explains some pairs perfectly, the fitted strengths run off toward infinity and the estimated true variance comes out as zero. In those 507 units, mostly animal groups, sparse cricket leagues and early baseball seasons, we instead treat the strengths as normal with unknown variance, estimate that variance by maximum marginal likelihood, and simulate from strengths with that spread.

A model that allows cycles

The main test takes a different approach. Instead of summarizing the results and comparing the summary with a null, it fits a model that allows cycles and asks whether that model describes the games better than Bradley-Terry does. The comparison is a likelihood-ratio test. The model that allows cycles adds a rank-2 skew-symmetric term to Bradley-Terry:

\[ \operatorname{logit} P(a \text{ beats } b) = h \cdot \text{home} + \theta_a - \theta_b + (u_a v_b - u_b v_a). \]

Each agent has a strength \(\theta\) and a position \((u, v)\) in a plane. The extra term is positive when \(b\) sits counter-clockwise of \(a\) (as seen from the origin), so three agents spread around the origin can beat each other in a circle. This is the “blade-chest” idea of Chen and Joachims (2016) in its simplest form.

The cyclic term for a pair is the cross product of their two positions, so it is largest when the two sit far from the origin and at right angles to each other, and zero when they line up with the origin. The three terms around a triad add up to twice the signed area of its triangle. A consequence is easy to see by dragging the points below: when the three strengths are equal, the agents form a cycle exactly when the origin lies inside their triangle. Move it outside and one agent beats both others. Unequal strengths pull against the cycle, and the sliders show how large the cyclic term has to be to survive a gap in strength.

u v origin
Strengths (θ)
Pair θ gap Cyclic term P(first wins)

The likelihood-ratio statistic compares the two models, and its null distribution again comes from a parametric bootstrap on the same schedule (50 draws for team-seasons, 100 elsewhere). Both models carry a small ridge penalty, which keeps the fits finite when one agent wins or loses nearly every game.

The rescaling matters even more here. A natural way to simulate the null is from the fitted Bradley-Terry strengths as they are, and with many games per pair that works. With about one game per pair, the fitted strengths are far too spread out, and so the simulated games are too predictable. A rank-2 term then improves the fit to the real data by more than it ever does to the simulated data, even when there is nothing cyclic to find. Table 2 shows this on simulated data. We took the schedules of every sparse cricket league and NFL season (120 and 110 of them), simulated every game twice from a transitive model, and ran the test with the null simulated each way. Steep hierarchies get a check of their own, since that is where the shrinkage factor fails: 12 animals with normal strengths, each pair meeting a Poisson(3) number of times, and no cycles.

Code 
ss <- cache$size_sparse; so <- cache$size_offset; sh <- cache$size_hier
s1 <- filter(ss, games_per_pair <= 1.05)
h1 <- filter(sh, sd == 1.5); h3 <- filter(sh, sd == 3)
tibble(
  `Simulated data` = c(sprintf("Sparse schedules: %d simulations, cricket leagues and NFL seasons", nrow(ss)),
                       sprintf("... of which one game per pair (%d)", nrow(s1)),
                       sprintf("Lichess bullet: %d months, with rating offsets", nrow(so)),
                       "Hierarchies of 12 animals, strength SD 1.5",
                       "Hierarchies of 12 animals, strength SD 3"),
  `Games per pair (median)` = c(f2(median(ss$games_per_pair)), f2(median(s1$games_per_pair)),
                                f2(median(U$games_per_pair[grepl("^lichess .* bullet$", U$unit)])),
                                f2(median(h1$games) / 66), f2(median(h3$games) / 66)),
  `Null from fitted strengths` = c(pct(mean(ss$p_fitted <= .05)), pct(mean(s1$p_fitted <= .05)), "—", "—", "—"),
  `Null from shrunk strengths` = c(pct(mean(ss$lr_p <= .05)), pct(mean(s1$lr_p <= .05)), pct(mean(so$lr_p <= .05)),
                                   pct(mean(h1$lr_p <= .05)), pct(mean(h3$lr_p <= .05)))
) |> knitr::kable(align = "lrrr")
Table 2: How often the LR test rejects when there is no cycle at all. The sparse-schedule rows test the same simulated no-cycle data with two versions of the bootstrap null. Lichess: the 12 bullet months, with their rating offsets. Hierarchies: 60 simulations at each spread of strengths; at SD 3 the top animal beats the bottom one almost always.
Simulated data Games per pair (median) Null from fitted strengths Null from shrunk strengths
Sparse schedules: 460 simulations, cricket leagues and NFL seasons 1.30 17.4% 2.8%
… of which one game per pair (62) 1.00 62.9% 0.0%
Lichess bullet: 12 months, with rating offsets 4.95 — 0.0%
Hierarchies of 12 animals, strength SD 1.5 3.00 — 3.3%
Hierarchies of 12 animals, strength SD 3 3.00 — 5.0%

The same problem arises with Lichess’s rating offset in a different form. Strengths beyond a player’s rating are mostly noise: their shrinkage factor is between 0.08 and 0.60 in the Lichess units where it can be estimated. With the rescaled null, the test does not reject too often in any of these settings: 2.8% on the sparse schedules, 0.0% in Lichess and at most 5.0% on the hierarchies, against a nominal 5%. So every result below uses it. Where the data are thinnest it errs toward caution: at exactly one game per pair, as in a single round-robin, it rejected 0.0% of the time, so in such a unit “no rejection” means little.

Power

Code 
if (is.null(cache$power)) {
  knitr::asis_output("*The power simulation has not been run for this build of the page.*")
} else cache$power |>
  group_by(c) |>
  summarise(`Likelihood ratio` = mean(lr_p <= .05), `HodgeRank cyclic share` = mean(curl_p <= .05),
            `Cyclic-triad count` = mean(wst_p <= .05), .groups = "drop") |>
  mutate(across(-c, f2)) |>
  rename(`Cyclic scale` = c) |>
  knitr::kable(align = "lrrr")
Table 3: Share of 40 simulated leagues in which each test rejects at .05. Each league has 30 teams and 1,230 games (an NBA regular season), home advantage, strengths with SD 0.7, and a rank-2 cyclic term of the given scale (0 means no cycles).
Cyclic scale Likelihood ratio HodgeRank cyclic share Cyclic-triad count
0.0 0.05 0.00 0.00
0.3 0.33 0.00 0.07
0.5 0.85 0.12 0.17

With no cycles, the likelihood test rejects in 5% of these leagues, as it should. With a cyclic term of scale 0.3 it rejects in 32%, and at 0.5 in 85%. For scale: at 0.5 the cyclic term moves the typical pair’s log-odds by about as much as the spread in team strength, and at 0.3 by about 60% of it. The two summaries do much worse. With two or three games per pair, a team’s record against any one opponent is mostly noise, and the cycle count rejects in only 18% of leagues even at 0.5. HodgeRank’s cyclic share, which did well in the much denser bullet chess data, rejects in 12% here. Both are also conservative, rejecting less than 5% of the time when there are no cycles. In sparse data, only the likelihood test has useful power.

Results

Observed cycles are common, and mostly luck

Code 
cy <- U |> filter(!is.na(wst), !is.na(wst_null), triads > 0) |>
  mutate(text = sprintf("%s<br>observed %.3f, simulated %.3f<br>triad count p = %.3f<br>likelihood test p = %.3f", unit, wst, wst_null, wst_p, lr_p))
# units where the cycle count does not reject (no more cyclic triads than the
# transitive simulation, one-sided p > .05): small and grey; units with more
# cycles than luck on top, large and coloured by family. Units with fewer than 10 decided triads are
# left out: one cycle moves their share by 10+ points
grp_lab <- c(team = "Team sports", `1v1` = "One-on-one", judg = "Judgments", animal = "Animal dominance")
cyf <- cy |> filter(triads >= 10) |>
  mutate(csig = wst_p <= .05, grp = factor(ifelse(csig, grp_lab[type], "Does not reject"), levels = c(grp_lab, "Does not reject")))
p <- ggplot(cyf, aes(wst_null, wst, colour = grp, text = text)) +
  geom_abline(colour = irw_grey, linetype = 2) +
  geom_point(data = ~ filter(.x, !csig), size = 0.6, alpha = 0.25) +
  geom_point(data = ~ filter(.x, csig), size = 2.4, alpha = 0.9) +
  scale_colour_manual(values = c(`Team sports` = irw_blue, `One-on-one` = irw_red, Judgments = irw_green,
                                 `Animal dominance` = irw_orange, `Does not reject` = irw_grey),
                      name = "More cycles than luck (p ≤ .05)", drop = FALSE) +
  labs(x = "Cyclic share, simulated (no cycles)", y = "Cyclic share, observed") +
  coord_equal(xlim = c(0, 0.45), ylim = c(0, 0.45))
g <- show_plot(p)
# plotly ignores coord_equal: fix a square window on the site too
if (inherits(g, "plotly")) g <- layout(g, xaxis = list(range = c(-0.01, 0.45)),
                                       yaxis = list(range = c(-0.01, 0.45), scaleanchor = "x"))
g
Figure 1: Share of decided triads that are cyclic, observed vs. simulated from a transitive model on the same schedule, one point per unit (units with at most 150 agents and at least 10 decided triads). Points near the diagonal have about as many cycles as luck produces. Large coloured points: units with more cyclic triads than the transitive simulation produces (one-sided p ≤ .05); small grey points: all other units.

In a typical team-season, 14.9% of the triads whose three series were all decided are cyclic in the observed results. Simulating the same schedules from a transitive model gives 14.8%. Figure 1 shows that this holds unit by unit: the points sit along the diagonal. The points farthest from it are mostly small units, such as cricket leagues with a few dozen decided triads, where a single cycle moves the share by several points.

A decided triad is three agents who all met, with no pair’s series level. If every game were a coin flip, a quarter of decided triads would be cyclic, so a share near 0.25 means the results among those three are about as ordered as noise. Leagues of evenly matched teams come close to that without any intransitivity. That is why the raw share says little and the comparison with the transitive simulation is what counts. Table 4 gives both by family. The excess is under one percentage point in most families and below zero in several. It is largest for the bots and Lichess blitz, at little more than one point: even where the likelihood test finds intransitivity, it shows up in only a small share of triads. The large points in Figure 1 are the units with more cycles than luck by this count: 78 of the 1,797 plotted, or 4.3%, where about 5% would by chance. That count is a weak test, though, as the next paragraph shows, so the verdicts below rest on the likelihood test instead.

Code 
cy |>
  group_by(Family = family) |>
  summarise(Units = n(), `Decided triads` = num(round(median(triads))),
            Observed = pct(median(wst)), `Simulated (no cycles)` = pct(median(wst_null)),
            Excess = sprintf("%+.1f pts", 100 * median(wst - wst_null)), .groups = "drop") |>
  mutate(Family = factor(Family, levels = fam_order)) |> arrange(Family) |>
  knitr::kable(align = "lrrrrr")
Table 4: Cyclic share of decided triads by family: medians across units (units with at most 150 agents and at least one decided triad). Excess: observed minus simulated, per unit, then the median. With no intransitivity anywhere, the excess would be centred on zero.
Family Units Decided triads Observed Simulated (no cycles) Excess
MLB 169 96 10.8% 12.3% -1.2 pts
NBA 86 483 7.5% 7.9% -0.6 pts
NHL 109 126 13.6% 13.5% -0.2 pts
NFL 120 72 12.1% 12.0% +0.0 pts
Football: England and Italy 522 400 17.0% 16.7% +0.2 pts
Football: other leagues 301 267 14.4% 14.2% +0.5 pts
Football: club cups and internationals 16 188 5.8% 7.9% -1.0 pts
Cricket 273 13 16.7% 16.4% -0.1 pts
Lichess 2013 bullet 12 7,960 10.2% 9.7% +1.0 pts
Lichess 2013 blitz 12 3,550 15.9% 14.7% +1.3 pts
Lichess 2013 classical 12 1,416 16.2% 15.4% +0.6 pts
Rugby union 15 30 3.1% 6.7% -1.4 pts
Quiz 13 131 4.7% 4.5% +0.0 pts
Engines and bots 21 5,245 11.5% 8.9% +1.4 pts
UFC and padel 3 936 10.1% 11.0% -0.9 pts
Judgments 61 3,987 6.1% 5.2% +0.2 pts
Animal dominance 214 58 0.7% 0.5% +0.0 pts

Counting also shows why a better test is needed. Where the likelihood test below does find intransitivity, as in bullet chess, the excess of observed cycles is small: in a typical bullet month, the observed share of cyclic triads is 1.0 percentage points above the transitive model’s (medians across months: 10.2% observed, 9.7% simulated). The count rejects in only 4 of the 12 bullet months. HodgeRank’s cyclic share, which uses every game rather than series winners, rejects in 12, about as often as the likelihood test. Even real intransitivity in the win probabilities changes the outcome of only a small fraction of triads, and a count of triads, each decided by a short series, can barely see that.

Team sports: no detectable intransitivity

Code 
team |>
  group_by(Family = family) |>
  summarise(Units = n(), `p ≤ .05` = sum(sig), Share = pct(mean(sig)),
            `Games per pair` = f2(median(games_per_pair)), .groups = "drop") |>
  bind_rows(tibble(Family = "All team sports", Units = nrow(team), `p ≤ .05` = sum(team$sig),
                   Share = pct(mean(team$sig)), `Games per pair` = f2(median(team$games_per_pair)))) |>
  knitr::kable(align = "lrrrr")
Table 5: Team-seasons where the likelihood test rejects at .05. With no cycles anywhere, about 5% would reject by chance.
Family Units p ≤ .05 Share Games per pair
MLB 169 5 3.0% 15.80
NBA 86 3 3.5% 3.80
NHL 109 1 0.9% 5.20
NFL 120 3 2.5% 1.30
Football: England and Italy 522 15 2.9% 2.00
Football: other leagues 301 5 1.7% 2.00
Football: club cups and internationals 25 0 0.0% 2.50
US college football 2 0 0.0% 1.00
Cricket 288 11 3.8% 1.70
Rugby union 15 0 0.0% 6.20
Quiz 12 0 0.0% 1.10
All team sports 1649 43 2.6% 2.00

Across 1,649 team-seasons, 43 (2.6%) reject at .05, fewer than the 5% that luck alone would give. No sport stands out.

How strong is this evidence? Within any one season, the test can miss a modest cyclic term: in simulated NBA-sized seasons it catches a cyclic term of scale 0.3 only 32% of the time (Table 3), and NFL seasons and cricket leagues, with about one game per pair, have far less data than that. Across all the seasons together the evidence is much stronger: if moderate cycles were common in team sports, far more than 2.6% of 1,649 seasons would reject. So a single strength per team misses no cyclic structure that these data can detect.

That is not the same as saying one strength is all you need. The test compares one strength per team with one strength plus a cyclic matchup term, and nothing else. A single fixed strength can fail in other ways this test does not look for: strengths that change during a season (injuries, trades, form), home advantage that differs by team, or information in the margin of victory, since we use only who won. And a better fit is not the same as better predictions of games not yet played; that would need an out-of-sample comparison.

College football shows where a rejection could mislead. Its two seasons have 424 and 658 teams, mostly in conferences that rarely play each other, and one game per pair. There the rank-2 term can capture group structure the single strength misses, such as conferences of different quality whose teams meet in only a few games, which is a misfit of Bradley-Terry but not a cycle. Neither season rejects (p = .94 and 1.00), though with one game per pair the test has little power (Table 2).

Judgments: individuals are transitive

Code 
U |> filter(type == "judg") |>
  mutate(Table = case_when(grepl("^ukp ", unit) ~ "ukp argument convincingness (debate sides)",
                           grepl("^makri ", unit) ~ "makri mathematics text relevance (sessions)",
                           grepl("^sangwin ", unit) ~ "sangwin proofs (sessions)",
                           grepl("^jones ", unit) ~ "jones image complexity (sessions)",
                           TRUE ~ unit)) |>
  group_by(Table) |>
  summarise(Units = n(), Items = median(agents), Comparisons = num(sum(games)),
            `Comparisons per pair` = f2(median(games_per_pair)),
            p = if (n() == 1) P(unit) else paste(sum(sig), "of", n(), "≤ .05"), .groups = "drop") |>
  knitr::kable(align = "lrrrrr")
Table 6: Judgment tables: likelihood test of a rank-2 cyclic term. Tables split into sessions or debate sides are summarised on one row: medians of items and comparisons per pair, and the count of units with p ≤ .05.
Table Units Items Comparisons Comparisons per pair p
bramley gcse 1 150 2,356 1.10 .09
carlson human_rights 1 1594 15,827 1.00 .71
carlson immigration 1 336 6,720 1.10 .81
carlson movie50 1 50 1,500 1.70 .10
carlson movie_reviews 1 497 9,927 1.40 .19
carlson treebank 1 504 9,996 1.00 .66
carlson wiscads 1 935 9,528 1.00 ≤ .01
cems (6 schools, pooled) 1 6 4,454 296.90 .72
clark strength study 2 1 82 7,835 2.60 .03
coertjens self-reflections 1 22 202 1.00 .11
elochoice_physical 1 82 4,592 1.80 .87
friedman appropriate 1 100 20,595 4.20 .23
friedman disaster 1 100 19,920 4.10 .12
friedman fairness 1 100 21,060 4.30 .06
friedman harm 1 100 21,390 4.40 .05
friedman incidence 1 100 20,790 4.30 ≤ .01
friedman longterm 1 100 20,625 4.20 .25
friedman priority 1 100 19,770 4.10 .35
friedman worry 1 100 21,555 4.40 .37
guinaudeau2024_largechambers 1 723 11,453 1.20 1.00
jones image complexity (sessions) 2 60 1,836 9.10 2 of 2 ≤ .05
makri mathematics text relevance (sessions) 6 21 1,923 1.65 0 of 6 ≤ .05
ramos proof explanation 1 9 760 21.10 .16
sangwin proofs (sessions) 7 15 3,485 5.60 1 of 7 ≤ .05
ukp argument convincingness (debate sides) 32 35 84,445 5.00 4 of 32 ≤ .05
zucco2019_portfoliosalience 1 37 3,661 9.40 .03

Pooled across respondents, 12 of 69 judgment units reject, against about 3 by chance. Only 3 reject strongly (p ≤ .01): the risk study’s “killed more people last year” criterion; the campaign-ad coding, which has about one comparison per pair, where the test if anything errs toward not rejecting (Table 2); and the image-complexity session with brightness equalised. The rest have p between .02 and .05 and are scattered across studies, with no kind of judgment standing out.

A pooled table can be cyclic even if every person in it is perfectly transitive. If one group of respondents ranks A > B > C, another B > C > A, and a third C > A > B, the majority in every pair runs in a circle. This is Condorcet’s paradox. CEMS lets us look at individuals directly, because each of its 303 students compared all 15 pairs of the six schools, so each student’s own answers form a complete little tournament with 20 triads.

Individual students are almost always transitive. 88.4% of them have no cyclic triad among their 20, and the average is 0.2 cyclic triads per student. If each student were instead a noisy draw from one shared preference (the pooled ordinal Bradley-Terry model, simulated 100 times), only 14.8% would have none, and the average would be 2.5. So students differ in what they prefer, but each is consistent: the setting for Condorcet’s paradox. Even so, the pooled CEMS table shows no cycle (p = .72). Different transitive preferences can produce a cyclic majority, but they need not.

What about the pooled tables that do reject? A natural explanation is the one above: people who are each transitive but weigh things differently, so that the pooled majorities run in a circle. For the risk ratings, it is plausible that different people weigh everyday harms against catastrophic ones differently. But we have not tested this explanation. In those tables each person made only a handful of judgments, so their individual consistency cannot be checked the way it can in CEMS. A model that lets groups of respondents differ, fitted to these tables, would tell us whether such differences account for the cycles.

Chess: cycles at every time control, seen most clearly in bullet

Lichess sorts games by time control, using the expected length of the game with any increment counted: in 2013, bullet (under about 3 minutes per player), blitz (under about 8) and classical (anything longer; there was no separate rapid category yet). Table 7 compares the three, pooling the 12 months of 2013.

Code 
li_sum <- U |> filter(grepl("^lichess", unit)) |>
  mutate(tc = factor(sub(".* ", "", unit), c("bullet", "blitz", "classical"))) |>
  group_by(tc) |>
  summarise(months = sum(sig), hodge = sum(curl_p <= .05, na.rm = TRUE), games = median(games), gpp = median(games_per_pair),
            obs = median(wst), null = median(wst_null),
            excess = median(lr - lr_null), per1000 = median(1000 * (lr - lr_null) / games), .groups = "drop")
L <- function(t, col) li_sum[[col]][li_sum$tc == t]
li_sum |>
  transmute(`Time control` = as.character(tc), `Months with p ≤ .05: LR` = paste(months, "of 12"),
            `HodgeRank` = paste(hodge, "of 12"),
            `Games per month` = num(round(games)), `Games per pair` = f2(gpp),
            `Cyclic triads: observed vs. no-cycle model` = paste(pct(obs), "vs.", pct(null)),
            `LR excess` = sprintf("%.0f", excess), `LR excess per 1,000 games` = sprintf("%.1f", per1000)) |>
  knitr::kable(align = "lrrrrrrr")
Table 7: Lichess 2013 by time control: medians across the 12 months, except the count of months. Each month’s unit is the 80 most active players, with the pre-game rating difference as an offset. The likelihood-ratio excess is the LR statistic minus its mean under the bootstrap null: how much better the cyclic model fits than chance would allow.
Time control Months with p ≤ .05: LR HodgeRank Games per month Games per pair Cyclic triads: observed vs. no-cycle model LR excess LR excess per 1,000 games
bullet 10 of 12 12 of 12 7,374 4.95 10.2% vs. 9.7% 53 7.1
blitz 5 of 12 5 of 12 3,861 3.30 15.9% vs. 14.7% 25 6.5
classical 6 of 12 2 of 12 1,981 2.30 16.2% vs. 15.4% 29 14.6

The key findings:

  • All three time controls show intransitivity. The cyclic model fits better than chance in 10 of 12 bullet months, 6 of 12 classical months and 5 of 12 blitz months. With no cycles at all, about 0.6 of 12 would reject by chance.
  • The evidence is strongest and most consistent in bullet, but bullet also has by far the most data: about 7,374 games a month and 4.95 per pair, against 1,981 and 2.30 in classical. More games per pair make any test more likely to reject.
  • Per game, the cyclic structure is not weaker in slower chess. The likelihood-ratio excess per 1,000 games is about the same in bullet (7.1) and blitz (6.5), and larger in classical (14.6). So these data do not show cycles fading as players get more time to think. Bullet is where there are enough games to see them every month.
  • Counting cycles barely shows any of this. Observed cyclic shares sit within about a percentage point of the no-cycle model at every time control (Table 7). The intransitivity is real, but it changes the outcome of only a small fraction of triads.

These results allow for each player’s rating going into each game, so a player improving during the month cannot pass for a cycle, and no pair contributes more than 10 games.

This agrees with work that counts rock-paper-scissors cycles among the strategies of over a billion Lichess and FICS games (Sanjaya et al. 2022). That work finds the “spinning top” shape proposed for real-world games by Czarnecki et al. (2020): cycles are most common among players of middling strength and thin out toward the top. Our dense cores are active club-level players of 2013, which is where that picture puts the most cycles.

Code 
if (!is.null(extras)) {
  sp <- as_tibble(extras$players) |>
    filter(dec > 0) |>
    mutate(tc = factor(tc, c("bullet", "blitz", "classical"))) |>
    group_by(tc) |>
    mutate(band = ntile(rating, 5)) |>
    group_by(tc, band) |>
    summarise(rating = median(rating), lo = min(rating), hi = max(rating), players = n(),
              Observed = sum(cyc) / sum(dec), `Transitive model` = sum(cyc_null) / sum(dec_null), .groups = "drop") |>
    pivot_longer(c(Observed, `Transitive model`), names_to = "source", values_to = "share") |>
    mutate(text = sprintf("%s, ratings %.0f-%.0f (%d player-months)<br>%s: %s of decided triads cyclic",
                          tc, lo, hi, players, source, pct(share)))
  p <- ggplot(sp, aes(rating, share, colour = source, group = source, text = text)) +
    geom_line(linewidth = 0.7) + geom_point(size = 2.2) +
    facet_wrap(~ tc, nrow = 1, scales = "free_x") +
    scale_colour_manual(values = c(Observed = irw_red, `Transitive model` = irw_grey), name = NULL) +
    scale_y_continuous(labels = scales::percent) +
    labs(x = "Pre-game rating (median of each fifth of players)", y = "Cyclic share of decided triads") +
    theme(legend.position = "bottom")
  # plotly ignores ggplot's legend position and drew the legend over the facet titles;
  # put it below the x-axis title instead
  g <- show_plot(p)
  if (inherits(g, "plotly")) g <- layout(g, legend = list(orientation = "h", x = 0.5, xanchor = "center", y = -0.3), margin = list(t = 60))
  g
}
Figure 2: Share of each player’s decided triads that are cyclic, by the player’s pre-game rating, pooled over the 12 months of each time control. Players are grouped into fifths by rating. Grey: the same share under a transitive model simulated on the same schedule (200 draws). The gap between the lines, not the height of either, is the evidence of cycles.

Figure 2 looks for that shape in our data. There is a spinning top: in bullet, 12.0% of a mid-rated player’s decided triads are cyclic, against 6.9% for the strongest fifth, and the other time controls are similar. But the transitive model draws the same hump. A mid-rated player has more opponents of about their own strength, and between near-equals a short series can go either way, so cycles come from noise alone. The gap between the lines, which is what a cycle model has to explain, is about the same at every rating: 0.6 to 1.3 percentage points in bullet and 1.1 to 2.4 in blitz, with no consistent gap in classical. So within these dense cores, the shape mostly reflects who plays whom, not cycles concentrated in the middle. Our cores span a narrow range of ratings, and the work above uses far more players, so this does not settle the question. It does show that a spinning top in raw cycle counts needs the same comparison with a transitive model as any other count.

What produces the cycles? One reading is style. A player who thrives on sharp tactics can beat a solid positional player who in turn beats someone else, and an opening specialist can trouble some opponents more than others. We cannot test that reading here: the rank-2 term picks up any matchup-specific advantage, and the data say nothing about style.

Bots: mostly yes

Code 
U |> filter(family == "Engines and bots") |>
  arrange(lr_p, unit) |>
  transmute(Competition = sub("^kaggle ", "Kaggle: ", unit), Games = num(games),
            `Games per pair` = f2(games_per_pair), p = sapply(unit, P)) |>
  knitr::kable(align = "lrrr")
Table 8: Engines and bots: the 80 most active agents of each competition, each pair capped at 10 games.
Competition Games Games per pair p
Kaggle: connectx_2020_2026 4,769 9.80 ≤ .01
Kaggle: fide_chess_2024_2025 11,753 8.20 ≤ .01
Kaggle: halite_iv_playground_2020_2021 7,428 6.70 ≤ .01
Kaggle: kore_2022_2022 4,522 4.30 ≤ .01
Kaggle: lux_ai_s1_2021_2021 6,090 3.60 ≤ .01
Kaggle: lux_ai_s2_2023_2023 10,887 8.30 ≤ .01
Kaggle: lux_ai_s2_beta_2022_2022 12,279 8.10 ≤ .01
Kaggle: lux_ai_s2_neurips_stage1_2023_2023 6,346 6.20 ≤ .01
Kaggle: maze_crawler_2026_2026 3,467 2.80 ≤ .01
Kaggle: pokemon_tcg_2026_2026 4,731 3.40 ≤ .01
Kaggle: rock_paper_scissors_2020_2021 1,213 1.20 ≤ .01
sscait 2021 17,854 5.70 ≤ .01
sscait 2022 25,411 8.00 ≤ .01
sscait 2023 24,759 7.80 ≤ .01
Kaggle: kore_beta_2022_2022 1,138 2.00 .02
Kaggle: lux_ai_s2_neurips_stage2_2023_2023 9,961 7.70 .25
Kaggle: orbit_wars_2026_2026 27,048 8.90 .36
Kaggle: google_football_2020_2020 1,055 1.20 .60
Kaggle: lux_ai_s3_2024_2025 6,210 8.40 .69
Kaggle: santa_2020_2021 518 1.30 .92
tcec engines (all seasons) 1,551 5.90 .94

12 of 17 Kaggle competitions reject, from ConnectX and chess to rock-paper-scissors and the Lux AI strategy games. Kaggle’s bots are fixed programs, so nothing like a player improving through the month is possible. What a bot can have is a strategy that exploits some opponents and is exploited by others, which is the textbook source of a cycle. Rock-paper-scissors is the purest case, since every strategy is beatable. In the chess engines of TCEC, by contrast, there is no sign of cycles (p = .94): the strongest engines seem to be simply stronger. Orbit Wars, the densest Kaggle competition, does not reject either. All 3 years of the SSCAIT StarCraft ladder reject. There the bots are not quite fixed, since their authors update them during the year, so part of what looks like a cycle may be bots changing. We did not split the ladder more finely than by year.

Animal dominance: one rank often fails, with other explanations open

Code 
U |> filter(type == "animal") |>
  mutate(Group = case_when(grepl("^baboons", unit) ~ "Baboons, Amboseli (group-years)",
                           grepl("^gombe males", unit) ~ "Chimpanzees, Gombe: males (years)",
                           grepl("^gombe females", unit) ~ "Chimpanzees, Gombe: females (5-year windows)",
                           grepl("^talek females", unit) ~ "Hyenas, Talek: females (years)",
                           grepl("^talek males", unit) ~ "Hyenas, Talek: males (years)",
                           grepl("hyenas$", unit) ~ "Hyenas, three other clans (whole tables)",
                           grepl("^tonkean", unit) ~ "Tonkean macaques (years)",
                           grepl("^vervets", unit) ~ "Vervets (years)",
                           grepl("^chickadees", unit) ~ "Mountain chickadees (flock-winters)",
                           grepl("^elephant seals", unit) ~ "Elephant seals (breeding seasons)",
                           grepl("^feederwatch", unit) ~ "FeederWatch bird species",
                           TRUE ~ unit)) |>
  group_by(Group) |>
  summarise(Units = n(), `p ≤ .05` = sum(sig), Animals = median(agents),
            `Encounters per pair` = f2(median(games_per_pair)), `Median p` = f2(median(lr_p)), .groups = "drop") |>
  arrange(desc(`p ≤ .05` / Units)) |>
  knitr::kable(align = "lrrrrr")
Table 9: Animal dominance: one unit per group and period (see Data). Medians across a table’s units, and the count of units with p ≤ .05.
Group Units p ≤ .05 Animals Encounters per pair Median p
Elephant seals (breeding seasons) 5 5 61 2.30 0.01
FeederWatch bird species 1 1 136 9.00 0.01
Hyenas, three other clans (whole tables) 3 3 80 4.40 0.01
Tonkean macaques (years) 9 9 25 26.10 0.01
Vervets (years) 3 3 49 5.90 0.01
Hyenas, Talek: females (years) 28 19 21 2.65 0.01
Baboons, Amboseli (group-years) 59 35 11 5.90 0.03
Hyenas, Talek: males (years) 27 7 13 2.30 0.31
Mountain chickadees (flock-winters) 38 6 11 25.60 0.43
Chimpanzees, Gombe: females (5-year windows) 9 1 16 2.70 0.13
Chimpanzees, Gombe: males (years) 33 3 9 3.90 0.40

Animal hierarchies are the classic setting for asking whether dominance is linear, and here a single rank often does not fit. 92 of 215 units reject. Every year of the Tonkean macaques (9 of 9) and vervets (3 of 3) rejects, as do every elephant-seal breeding season (5 of 5), all three hyena clans that enter whole, the FeederWatch species, most baboon group-years (35 of 59) and most years of Talek’s female hyenas (19 of 28). Few units reject among the Gombe chimpanzees (4 of 42), Talek’s males (7 of 27) and the chickadee flocks (6 of 38). Encounters per pair do not explain the split: the elephant seals meet about twice per pair and reject, while most chickadee flocks meet at their feeders ten or more times per pair and do not.

Two things other than rock-paper-scissors dominance can produce these rejections, and our units cannot rule either out.

  • Ranks change within a unit. A unit is a year, a five-year window, or, for the three hyena clans without dates, a whole table spanning many years. An animal that rises in rank during the unit loses early and wins late, and its record looks like a set of matchup-specific results. This is the problem the rating offset solves for chess, and we have no such offset for animals.
  • Some relationships are specific to a pair. A lower-ranked animal can consistently beat one particular higher-ranked one, for instance with support from kin. The rank-2 term picks that up as well. It is real structure that one rank per animal misses, but it is not quite a cycle.

Steepness alone does not produce these rejections. Many of these hierarchies are nearly deterministic: the top animal almost never loses to the bottom one. That is where the usual shrinkage factor fails and the null comes from the marginal-likelihood fit instead (see the methods), and on simulated steep hierarchies with no cycles the test rejects no more often than it should (Table 2). What those simulations hold fixed, though, is one rank per animal for the whole unit, which is exactly what the two explanations above call into question.

What the cycles look like

Below are four cycles from the data: two from the risk judgments, one from Lichess and one from the Kaggle bots. Each is three competitors (or three risks) that beat each other in a circle, like rock, paper and scissors. “Beat” means won most of their meetings: for players and bots, their head-to-head games; for the risks, the judgments of the respondents who were shown that pair. For each, the small table gives the three’s overall standing, and the text below it says which head-to-head result goes against that standing. That result is what turns a ranking into a loop. Risks: triples that the fitted cyclic model makes into a loop and that also loop in the raw judgments. Players and bots: the most lopsided loop in any bullet month and any Kaggle competition, among pairs that met at least five times; players and bots are numbered rather than named. Each example could be chance on its own; the tests above rest on all the games, not on these.

Code 
ex <- as_tibble(cache$examples) |> mutate(s = pmin(ab_w / ab_n, bc_w / bc_n, ca_w / ca_n))
pick_row <- function(pat) { x <- filter(ex, grepl(pat, unit)); if (all(is.na(x$model_min))) x[which.max(x$s), ] else x[which.max(x$model_min), ] }
games <- function(w, n, what) sprintf("%g of %d %s", w, n, what)
# one example: `ids` are the three as named in the data, `lab` as shown, `standing`
# the overall measure the text orders them by (higher is better)
show_cycle <- function(title, intro, ids, lab, tab, standing, standing_txt, what) {
  r <- attr(ids, "row")
  edges <- list(c(1, 2, r$ab_w, r$ab_n), c(2, 3, r$bc_w, r$bc_n), c(3, 1, r$ca_w, r$ca_n))
  rec <- sapply(edges, function(e) sprintf("%s beat %s (%s)", lab[e[1]], lab[e[2]], games(e[3], e[4], what)))
  up <- sapply(edges, function(e) standing[e[1]] < standing[e[2]])
  ord <- lab[order(-standing)]
  cat("**", title, "** ", intro, "\n\n", sep = "")
  cat(knitr::kable(tab, format = "pipe", align = c("l", rep("r", ncol(tab) - 1))), sep = "\n")
  cat("\nHead to head: ", paste(rec, collapse = "; "), ". ", sep = "")
  cat("By ", standing_txt, ", the order is ", paste(ord, collapse = " > "), ". The loop closes because ",
      paste(rec[up], collapse = " and "), ", against that order.\n\n", sep = "")
}
if (!is.null(extras)) {
  rk <- as_tibble(extras$risks)
  risk_ex <- function(pat, table, title, intro, deaths) {
    r <- pick_row(pat); ids <- c(r$A, r$B, r$C); attr(ids, "row") <- r
    x <- rk[rk$table == table, ][match(ids, rk$risk[rk$table == table]), ]
    tab <- tibble(Risk = ids, `Comparisons won, against all 99 other risks` = pct0(x$share_won))
    if (deaths) tab$`Actual deaths, 2015` <- num(x$deaths_2015)
    show_cycle(title, intro, ids, ids, tab, x$share_won, "the share of all comparisons won", "judgments")
    if (deaths) cat("By actual deaths, the order is ", paste(ids[order(-x$deaths_2015)], collapse = " > "),
                    ", and no loop is possible.\n\n", sep = "")
  }
  risk_ex("^friedman incidence", "friedman2019_risk_incidence", "Risks: which killed more people last year?",
          "Each respondent saw two risks at a time and picked the one they thought killed more Americans in the last year. Actual deaths are Friedman's 2015 figures.", TRUE)
  risk_ex("^friedman harm", "friedman2019_risk_harm", "Risks: which caused more harm to Americans last year?",
          "The same design, asking which risk caused more harm. There is no objective measure of harm to compare with, so the table shows only how respondents ranked the three.", FALSE)

  r <- pick_row("^lichess"); ids <- c(r$A, r$B, r$C); attr(ids, "row") <- r
  pl <- as_tibble(extras$players) |> filter(unit == r$unit)
  x <- pl[match(ids, pl$player), ]
  mo <- month.name[as.integer(sub("lichess 2013-(\\d+) .*", "\\1", r$unit))]
  lab <- paste("Player", 1:3)
  show_cycle(paste0("Lichess bullet, ", mo, " 2013"),
             sprintf("Three of the month's %d most active players.", nrow(pl)), ids, lab,
             tibble(Player = lab, `Rating` = sprintf("%.0f", x$rating), `Games in the month` = x$games,
                    `Points won` = pct0(x$points / x$games)),
             x$points / x$games, "points won in the month", "games")

  r <- pick_row("^kaggle"); ids <- c(r$A, r$B, r$C); attr(ids, "row") <- r
  bt <- as_tibble(extras$bots) |> filter(unit == r$unit) |> mutate(share = points / games, rank = rank(-share, ties.method = "min"))
  x <- bt[match(ids, bt$bot), ]
  lab <- paste("Bot", 1:3)
  show_cycle(paste("Kaggle:", sub("^kaggle ", "", r$unit)),
             sprintf("Three of the %d bots in the competition's dense core.", nrow(bt)), ids, lab,
             tibble(Bot = lab, `Games` = x$games, `Points won` = pct0(x$share), `Rank by points` = paste(x$rank, "of", nrow(bt))),
             x$share, "points won in the competition", "games")
}

Risks: which killed more people last year? Each respondent saw two risks at a time and picked the one they thought killed more Americans in the last year. Actual deaths are Friedman’s 2015 figures.

Risk Comparisons won, against all 99 other risks Actual deaths, 2015
Alcohol use 80% 43,138
Child abuse 61% 1,585
Drunk driving 85% 9,967

Head to head: Alcohol use beat Child abuse (4 of 7 judgments); Child abuse beat Drunk driving (3 of 4 judgments); Drunk driving beat Alcohol use (4 of 4 judgments). By the share of all comparisons won, the order is Drunk driving > Alcohol use > Child abuse. The loop closes because Child abuse beat Drunk driving (3 of 4 judgments), against that order.

By actual deaths, the order is Alcohol use > Drunk driving > Child abuse, and no loop is possible.

Risks: which caused more harm to Americans last year? The same design, asking which risk caused more harm. There is no objective measure of harm to compare with, so the table shows only how respondents ranked the three.

Risk Comparisons won, against all 99 other risks
Air pollution 58%
Post-traumatic stress disorder (PTSD) 65%
Complications from pregnancy / childbirth 52%

Head to head: Air pollution beat Post-traumatic stress disorder (PTSD) (3 of 5 judgments); Post-traumatic stress disorder (PTSD) beat Complications from pregnancy / childbirth (3 of 4 judgments); Complications from pregnancy / childbirth beat Air pollution (2 of 3 judgments). By the share of all comparisons won, the order is Post-traumatic stress disorder (PTSD) > Air pollution > Complications from pregnancy / childbirth. The loop closes because Air pollution beat Post-traumatic stress disorder (PTSD) (3 of 5 judgments) and Complications from pregnancy / childbirth beat Air pollution (2 of 3 judgments), against that order.

Lichess bullet, January 2013 Three of the month’s 80 most active players.

Player Rating Games in the month Points won
Player 1 1624 95 34%
Player 2 1676 180 41%
Player 3 1620 213 39%

Head to head: Player 1 beat Player 2 (5 of 6 games); Player 2 beat Player 3 (6 of 7 games); Player 3 beat Player 1 (4 of 5 games). By points won in the month, the order is Player 2 > Player 3 > Player 1. The loop closes because Player 1 beat Player 2 (5 of 6 games), against that order.

Kaggle: halite_iv_playground_2020_2021 Three of the 80 bots in the competition’s dense core.

Bot Games Points won Rank by points
Bot 1 210 70% 4 of 80
Bot 2 198 47% 51 of 80
Bot 3 225 55% 19 of 80

Head to head: Bot 1 beat Bot 2 (10 of 10 games); Bot 2 beat Bot 3 (9 of 10 games); Bot 3 beat Bot 1 (9 of 10 games). By points won in the competition, the order is Bot 1 > Bot 3 > Bot 2. The loop closes because Bot 2 beat Bot 3 (9 of 10 games) and Bot 3 beat Bot 1 (9 of 10 games), against that order.

The bots’ loop is what strong intransitivity looks like: each bot beats the next almost every time, so no ordering of the three makes sense. The chess loop is milder, though each player still won most of their games against the next. The risk loops are the weakest. Each rests on a few judgments per pair, and the fitted cyclic model puts the win probabilities around these loops at only about 0.60–0.66. In the judgment data, cycles nudge preferences slightly away from a single ranking; they do not overturn it.

Discussion

Where does this leave the Bradley-Terry model? For team sports it holds up against the cyclic alternative: across more than a century of leagues, we find no matchup structure beyond a single strength per team, and what looks like a cycle is the noise of a short series. Other departures from a single fixed strength, such as strengths that change within a season, were not tested. For online chess and for bots it does not: some matchups are consistently better or worse than the two strengths predict, and those deviations line up into cycles. Judgments sit in between. Individuals are transitive where we can check; pooled judgments occasionally are not, perhaps because people who differ are pooled together, though that is untested here. Animal dominance often fails the single-rank model (92 of 215 groups), but there, ranks that shift over the period and relationships specific to a pair are at least as plausible as true cycles.

There are also two lessons about method. First, counting cycles answers the wrong question, because noise alone produces plenty of them. Second, the right comparison is with a transitive model simulated on the same schedule, and that simulation has to use realistic strengths. Fitted strengths are too spread out whenever there is little data per pair, which is common in real competition data, and a null built from them finds cycles that are not there.

Limitations

The rank-2 term catches more than cycles. It improves the fit whenever some pairs systematically do better or worse than their strengths predict: groups of agents of different quality that rarely meet, styles that match up well or badly, home advantage that varies by team. A rejection says that one strength per agent is not enough, and a cycle is one way, but not the only way, for that to happen. Conference structure in college football, or kin support among animals, would produce a rejection without any cycle.

Dense cores are selected. For Lichess, the bots, University Challenge and the largest hyena clans, the units are the 80 most active agents among those who met each other repeatedly. That is the part of the data where a test has power, but it is not a random sample of players or bots. Lichess in 2013 was a small site with mostly amateur players, and ratings lag skill, so the rating offset controls for improvement only in part.

Matchmaking is adaptive. Kaggle pairs bots with others of similar rating, and Lichess players choose their opponents. The bootstrap conditions on the observed schedule, which is the right comparison for the question asked, but who plays whom is not random.

Few games per pair limit what can be found. Much of the data has one or two games per pair. The rescaled null keeps the test from rejecting too often there, but its power is low, and at one game per pair it is conservative (Table 2). So “no rejection” in sparse units is weak evidence of transitivity.

Units hold strengths fixed over a period. A league-season, a Lichess month or an animal group’s year assumes each agent’s strength is constant within it. Lichess has a rating offset for change within the month; the other families do not, and change in strength within a unit can look like matchup structure. This matters most for the animal groups and the bot ladder, where units are long relative to how fast standing can change.

Many tests. About 1,994 units are tested; at .05, about 100 would reject by chance. The conclusions above rest on patterns across many units (the share of team-seasons, 10 of 12 bullet months), not on any single p-value.

Reproducibility

Source code for this page: intransitivity.qmd · intransitivity_compute.R · intransitivity_extras.R · intransitivity_helpers.R

These results were computed against approximately IRW v543 (the corpus as of October 7, 2026).

The version was inferred from when the results were computed, not recorded during the run, so it may be off by a version or more. To pin data to an exact version, use irw_use_version().

Results computed October 07, 2026. All tables are fetched with irw::irw_fetch(source = "comp"). The full computation takes about eight hours on four cores, mostly the likelihood tests for the Lichess months, the Kaggle competitions and the largest judgment tables. The compute script caches every fetched table and every unit’s result, so it can be stopped and resumed. intransitivity_run.sh runs it in batches, each in a fresh process so that no batch’s workers inherit another’s memory. To regenerate:

# from the site root; IRW_CORES sets the worker processes per batch (default 4)
vignettes/intransitivity_run.sh units   # fetch the tables and build the units
vignettes/intransitivity_run.sh tests   # tests in batches, calibration, extras
vignettes/intransitivity_progress.sh    # share of units done, by family

Then render vignettes/intransitivity.qmd.

Tip

To cite the datasets used, run:

# the 99 tables used: the competition collection minus the 10 left out (see Data)
left_out <- c("t20_hyper", "mlb_through2023", "metakaggle_connect_4_2023_2024", "pklmart_pickleball_2021_2025",
              paste0("placepulse2_", c("beautiful", "boring", "depressing", "livelier", "safer", "wealthier")))
irw_save_bibtex(setdiff(cache$tables, left_out), source = "comp", output_file = "intransitivity_data/references.bib")

References

Bradley, Ralph Allan, and Milton E. Terry. 1952. “Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons.” Biometrika 39 (3/4): 324–45.
Carlson, David, and Jacob M. Montgomery. 2017. “A Pairwise Comparison Framework for Fast, Flexible, and Reliable Human Coding of Political Texts.” American Political Science Review 111 (4): 835–43.
Chen, Shuo, and Thorsten Joachims. 2016. “Modeling Intransitivity in Matchup and Comparison Data.” Proceedings of the Ninth ACM International Conference on Web Search and Data Mining (WSDM ’16), 227–36. https://doi.org/10.1145/2835776.2835787.
Condorcet, Marquis de. 1785. Essai sur l’application de l’analyse à la probabilité des décisions rendues à la pluralité des voix. Imprimerie Royale.
Czarnecki, Wojciech M., Gauthier Gidel, Brendan Tracey, et al. 2020. “Real World Games Look Like Spinning Tops.” Advances in Neural Information Processing Systems 33 (NeurIPS 2020).
Dittrich, Regina, Reinhold Hatzinger, and Walter Katzenbeisser. 1998. “Modelling the Effect of Subject-Specific Covariates in Paired Comparison Studies with an Application to University Rankings.” Journal of the Royal Statistical Society: Series C (Applied Statistics) 47 (4): 511–25.
Friedman, Jeffrey A. 2019. “Priorities for Preventive Action: Explaining Americans’ Divergent Reactions to 100 Public Risks.” American Journal of Political Science 63 (1): 181–96.
Gehrlein, William V. 2006. Condorcet’s Paradox. Theory and Decision Library C. Springer. https://doi.org/10.1007/3-540-33799-7.
Guinaudeau, Benjamin, and Christian Breunig. 2024. Replication Data for: Measuring Legislators’ Ideological Position in Large Chambers Using Pairwise-Comparisons. Harvard Dataverse, V1. https://doi.org/10.7910/DVN/0ULECU.
Jiang, Xiaoye, Lek-Heng Lim, Yuan Yao, and Yinyu Ye. 2011. “Statistical Ranking and Combinatorial Hodge Theory.” Mathematical Programming 127: 203–44. https://doi.org/10.1007/s10107-010-0419-x.
Kaggle. n.d. Meta Kaggle. Https://www.kaggle.com/datasets/kaggle/meta-kaggle.
Lichess. n.d. Lichess Open Database: Standard Rated Games, 2013. Https://database.lichess.org/.
Sanjaya, Ricky, Jun Wang, and Yaodong Yang. 2022. “Measuring the Non-Transitivity in Chess.” Algorithms 15 (5): 152. https://doi.org/10.3390/a15050152.
Turner, Heather, and David Firth. 2012. “Bradley-Terry Models in R: The BradleyTerry2 Package.” Journal of Statistical Software 48 (9): 1–21.
Tversky, Amos. 1969. “Intransitivity of Preferences.” Psychological Review 76 (1): 31–48.
Zucco, Cesar, Mariana Batista, and Timothy Power. 2019. Replication Data for: Measuring Portfolio Salience Using the Bradley-Terry Model: An Illustration with Data from Brazil. Harvard Dataverse, V1. https://doi.org/10.7910/DVN/HJZSIM.