kay_2025_antonyms

69,600 responses from 1,200 respondents to 58 items.

About this table

DescriptionSemantic Antonyms Data-Quality Probe
ReferenceKay, C. S. (2025). Why you shouldn’t trust data collected on MTurk. Behavior Research Methods, 57(12), 340.
DOI10.3758/s13428-025-02852-7
LicenceCC BY 4.0
Source datahttps://osf.io/frwq4/files/osfstorage?view_only=2e9d981a2df1411d8971a120ae75df06

Size and shape

Responses69,600
Respondents1,200
Items58
Response categories7
Responses per respondent58
Responses per item1,200
Density1
LongitudinalFALSE

Classification

age rangeAdult (18+)
sampleInternet-based
construct typeBehavioral, Personality
measurement toolSurvey/questionnaire
item formatLikert Scale/selected response
primary language(s)eng
construct nameSemantic Antonyms Data-Quality Probe

Codebook

This codebook is reconstructed by the IRW from the IRW data standard and the script that built it, kay_2025_antonyms.py. It is our best reading, not the source’s own codebook, and it may contain mistakes. Where the two disagree, the source data are the authority. If you find an error, please tell us.

ColumnWhat it isSource column
IRW standard columns
idIRW standard Identifier for the focal unit being measured — typically a person, but sometimes another entity (e.g., a word in a lexical task).response_id
itemIRW standard Item identifier.made in the build script
respIRW standard Response value.made in the build script
Respondent covariates
cov_att_chkCovariate made in the build script
cov_cultureCovariate made in the build script
cov_educationCovariate made in the build script
cov_genderCovariate made in the build script
cov_incomeCovariate made in the build script
cov_poli_catCovariate made in the build script
cov_poli_contCovariate made in the build script
cov_ses_1Covariate made in the build script
cov_sourceCovariate made in the build script
Other columns
itemcov_pairNot documented Covariates invariant to measurement probes (item-level attributes).made in the build script
itemcov_polarityNot documented Covariates invariant to measurement probes (item-level attributes).made in the build script

What item codes and covariate values stand for is in the codebook released with the source data.

Get the data

Download CSVno account neededBrowse on Redivisexplore and queryCroissant metadataHugging Face, Kaggle, OpenML

Or load it directly in R or Python (both need a free Redivis account; the first call opens a browser to sign in):

# R
install.packages("remotes")
remotes::install_github("itemresponsewarehouse/Rpkg")
library(irw)
df <- irw_fetch("kay_2025_antonyms")
# Python
pip install irw

import irw
df = irw.fetch("kay_2025_antonyms")

How to cite

@article{Kay_2025,
  title = {Why you shouldn’t trust data collected on MTurk},
  author = {Kay, Cameron S.},
  journal = {Behavior Research Methods},
  volume = {57},
  number = {12},
  pages = {340},
  year = {2025},
  publisher = {Springer Science and Business Media LLC},
  doi = {10.3758/s13428-025-02852-7},
  url = {https://doi.org/10.3758/s13428-025-02852-7}
}

Version and provenance

IRW versionv549
Redivis datasetitem_response_warehouse_5 v8.1
Redivis dataset DOI10.57761/07zt-n365
Manifest pin for this IRW versionv8.1
Metadata sourceirw_meta v39.0