70,103 responses from 1,194 respondents to 90 items.
| Description | This dataset measures performance on a mix of general-knowledge and impossible. |
|---|---|
| Reference | Himmelstein, Mark, Sophie Ma Zhu, Nikolay Petrov, Ezra Karger, Jessica Helmer, Sivan Livnat, Amory Bennett, Page Hedley, and Phil Tetlock. “The Forecasting Proficiency Test: A General Use Assessment of Forecasting Ability”. PsyArXiv, November 23, 2025. doi:10.31234/osf.io/a7kdx_v8. |
| DOI | 10.31234/osf.io/a7kdx_v8 |
| Licence | CC BY 4.0 |
| Source data | https://github.com/forecastingresearch/fpt/tree/main/data_cognitive_tasks |
| Responses | 70,103 |
|---|---|
| Respondents | 1,194 |
| Items | 90 |
| Response categories | 2 |
| Responses per respondent | 58.713 |
| Responses per item | 778.922 |
| Density | 0.652 |
| Longitudinal | TRUE |
| age range | Adult (18+) |
|---|---|
| construct type | Cognitive/educational |
| measurement tool | Test |
| item format | Likert Scale/selected response |
| primary language(s) | eng |
| construct name | Performance on a mix of general-knowledge and impossible (unknowable) questions |
This table has item text in the IRW: the wording administered to respondents, not just the response codes.
| Instrument | This dataset measures accuracy on a two-alternative forced-choice general-knowledge task that also contains unanswerable ('impossible') questions, on which the correct response is to opt out. |
|---|---|
| Mean words per item | 10.600 |
| Mean characters per item | 60.967 |
| Mean characters per response | 2 |
| Flesch-Kincaid grade level | 6.682 |
cov_session_ididitemresprtwave
Or load it directly in R or Python:
# R
install.packages("remotes")
remotes::install_github("itemresponsewarehouse/Rpkg")
library(irw)
df <- irw_fetch("himmelstein-impossible_question-2025")
# Python
pip install irw
import irw
df = irw.fetch("himmelstein-impossible_question-2025")
| IRW version | v393 |
|---|---|
| Redivis dataset | item_response_warehouse_2 v19.0 |
| Redivis dataset DOI | 10.57761/qtx5-4m80 |
| Manifest pin for this IRW version | v19.0 |
| Metadata source | irw_meta v23.0 |