Vignettes
These vignettes show how to work with IRW data for common psychometric tasks.
R
Each of these vignettes uses the irw R package to fetch data and walks through a complete analysis. They fall into four rough genres, grouped below.
Tutorials
Introductions to a technique or tool, worked through on one example dataset.
R
How much does a 2PL improve on a 1PL for a given dataset? A short worked example computing cross-validated InterModel Vigorish (IMV) for one dataset, with a pointer to an interactive app for building intuition about what IMV values mean.
Key paper: Domingue et al. (2025), Psychometrika
Also see: the IMV Shiny app, for building intuition about what different IMV values mean
irw mirt imv
R
How can I simulate item difficulties that reflect real-world distributions rather than idealized assumptions? Draws from a pooled empirical distribution built from 145+ IRW datasets via
irw_simu_diff(), instead of assuming a normal or uniform shape.
Key paper: Zhang et al. (2025), PsyArXiv
irw
R
How do I fit a multi-factor CFA model to an IRW personality dataset? Walks through a full lavaan workflow on a Big Five dataset — cleaning, reshaping to wide format, and building a multi-factor model directly from item-naming conventions — that generalizes to most other multi-scale IRW tables.
irw lavaan psych
Assumption audits across IRW
Broad scans checking whether an assumption standard IRT models rely on actually holds, across hundreds of real IRW tables at once, rather than in a single example dataset.
R
How often does the unidimensionality assumption behind standard IRT models actually hold up across real IRW instruments, and does it vary by construct type? Compares two exploratory diagnostics — the eigenvalue ratio and parallel analysis — across hundreds of tables and finds they disagree sharply on how often it holds.
Key paper: Horn (1965), Psychometrika (parallel analysis)
irw psych ggplot2
R
After fitting the standard unidimensional IRT model, how often are item pairs still residually correlated (local dependence / testlet effects), and does it concentrate in particular item formats? Flags pairs via Yen’s Q3 statistic across ~800 tables and checks two known testlet structures as a sanity check.
Key paper: Yen (1984), Applied Psychological Measurement (Q3 statistic)
irw mirt ggplot2
R
How prevalent are extreme (ERS) and midpoint (MRS) response styles in Likert-format IRW data, and do they distort substantive conclusions like factor loadings if left unmodeled? Screens IRW’s Likert-scale tables and compares single-factor CFA loadings before and after a simple ipsatization-based style control.
irw lavaan ggplot2
R
How prevalent is gender DIF across a diverse collection of real-world datasets, and do Mantel-Haenszel and logistic regression methods agree on which items are flagged? Runs both DIF families side by side on every IRW dataset with a gender/sex covariate.
irw difR ggplot2
Replicating published findings on real IRW data
Takes a specific claim or theoretical result from a published paper and checks whether it holds up against real IRW data.
R
What does Holland’s (1990) Dutch Identity reveal about the structure of IRT models — and does it hold in real data? Six sections moving from raw manifest probabilities to a Rasch-model diagnostic and a test of Holland’s conservation-of-complexity law.
Key paper: Holland (1990), Psychometrika
Also see: the IMV Shiny app, for the IMV values used in Section 6
irw mirt imv ggplot2
R
Do congruent/incongruent difference scores in IRW cognitive-control tasks show the same reliability collapse Hedge, Powell & Sumner (2018) found in the lab? Tests four IRW tasks — Hearts & Flowers, an alcohol Stroop, a stop-signal task, and three Self-Regulation Ontology tasks — for the same attenuation.
irw data.table ggplot2
R
Under a single dominant common cause, network psychometrics and IRT make the same prediction — but do they actually agree on real data? Tests whether a network’s node centrality tracks IRT discrimination across real IRW instruments, and whether disagreement concentrates in tables already flagged as multidimensional or locally dependent elsewhere on this site.
Key paper: Epskamp, Maris, van Borkulo & Borsboom (2018), in The Wiley Handbook of Psychometric Testing
irw bootnet qgraph mirt
R
Do Shim & Bonifay’s (2026) asymmetric IRT models (LPE, RH, AO) actually beat a standard 2PL on real data, and does the direction of asymmetry track construct type? The original paper never fits any of the three to real item response data — this vignette runs a real-data IMV holdout comparison to fill that gap.
irw mirt imv ggplot2
R
Slider and visual-analogue-scale items produce a response that’s continuous but confined to a fixed range like [0,10] or [0,100] — does that boundedness actually matter for modeling? Compares Noel & Dauvier’s Beta IRT, Samejima’s Continuous Response Model, and Mueller’s Continuous Rating Scale Model against a naive linear baseline, on a full simulation grid and 11 real IRW tables — and finds the naive baseline loses badly on distributional fit but is often competitive on point predictions.
Key paper: Noël & Dauvier (2007), Applied Psychological Measurement (Beta IRT)
irw mirt EstCRM pcIRT
Cross-dataset analyses
Applied methods and novel analyses that use IRW’s breadth to ask a research question that a single dataset couldn’t answer on its own.
R
What do discrimination parameters look like across many cognitive/educational datasets in the wild? Fits a 2PL to every eligible table and asks how consequential the Rasch model’s a=1 constraint really is in practice.
irw mirt ggplot2
R
Can simple linguistic features of item wording predict proportion correct across cognitive datasets? Extracts features like length, negation, and complexity from item text and tests how much of item difficulty they explain, separate from substantive content.
irw ggplot2
R
How much does knowing how fast a person responded improve predictions of whether they answered correctly, over and above IRT ability estimates? Uses cross-validated IMV to isolate the predictive value response time adds beyond ability alone.
Also see: the IMV Shiny app, for building intuition about what different IMV values mean
irw mirt imv splines
R
When do items in an RCT outcome measure respond differently to treatment, and what does ignoring this do to treatment-by-covariate interaction estimates? Shows that item-level variation in treatment response can bias estimated interactions even when the average treatment effect looks fine.
irw lme4
Python
These vignettes use the irw Python package to fetch data.
Python
How do I fetch IRW data in Python and fit 1PL, 2PL, 3PL, and graded response models?
irw mirt girth