514,200 responses from 2,571 respondents to 10,000 items.
The scored version of this table is cifar10h.
One of 2 tables from the same source. The others are listed under Related tables.
| Description | Nominal companion of cifar10h: CIFAR-10H human labels of the 10,000 CIFAR-10 test images, 2,571 annotators; text = class chosen (10 classes), resp = 1 if it matches the CIFAR-10 label (itemcov_true_category). |
|---|---|
| Reference | Peterson, J. C., Battleday, R. M., Griffiths, T. L., & Russakovsky, O. (2019). Human uncertainty makes classification more robust. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 9617-9626. |
| DOI | 10.1109/ICCV.2019.00971 |
| Licence | CC BY-NC-SA 4.0 |
| Source data | https://github.com/jcpeterson/cifar-10h |
| Responses | 514,200 |
|---|---|
| Respondents | 2,571 |
| Items | 10,000 |
| Response categories | 10 |
| Responses per respondent | 200 |
iditemitemcov_subcategoryitemcov_true_categoryresprttexttrial_index
id, item, resp, rt are defined in the IRW data standard and mean the same in every table. For the other columns, including what item codes and covariate values stand for, any codebook is the one released with the source data.
Licence: CC BY-NC-SA 4.0 — non-commercial use only; adaptations must be shared under the same licence.
Or load it directly in R or Python (both need a free Redivis account; the first call opens a browser to sign in):
# R
install.packages("remotes")
remotes::install_github("itemresponsewarehouse/Rpkg")
library(irw)
df <- irw_fetch("cifar10h_nom", source = "nom")
# Python
pip install irw
import irw
df = irw.fetch("cifar10h_nom", source="nom")
@inproceedings{Peterson_2019, title={Human Uncertainty Makes Classification More Robust}, url={http://dx.doi.org/10.1109/ICCV.2019.00971}, DOI={10.1109/iccv.2019.00971}, booktitle={2019 IEEE/CVF International Conference on Computer Vision (ICCV)}, publisher={IEEE}, author={Peterson, Joshua and Battleday, Ruairidh and Griffiths, Thomas and Russakovsky, Olga}, year={2019}, month=Oct, pages={9616–9625} }
| IRW version | v581 |
|---|---|
| Redivis dataset | irw_nominal v4.1 |
| Redivis dataset DOI | 10.57761/9hzd-dc70 |
| Manifest pin for this IRW version | v4.1 |
| Metadata source | irw_meta v44.0 |