replication crisis - 2025 ACF Fall

Question

The study of this problem was popularized after Daryl Bem reported experimental evidence for ESP. For 10 points each:

[10m] What so-called “crisis” in social science is driven by “publish or perish” culture? A John Ioannidis article on this problem claims “Most Published Research Findings Are False.”

ANSWER: replication crisis [or replicability crisis or reproducibility crisis or nonreproducibility crisis]

[10e] A proposed solution to the replication crisis lowers the threshold for this property from “p is less than 0.05” to “p is less than 0.005.” Results with this property’s “statistical” form are unlikely to have happened by chance.

ANSWER: statistical significance [or word forms like statistically significant]

[10h] Due to publication biases, even successful replications typically find lower values of this quantity. Cohen’s d estimator for this quantity equals the difference between two means over the standard deviation.

ANSWER: effect size

<Editors, Social Science>

Back to bonuses

Summary


Northern California	Main	Y	2	15.00	100%	50%	0%
Southern California	Main	Y	7	18.57	100%	71%	14%
Eastern Canada (1)	Main	Y	2	10.00	50%	50%	0%
Eastern Canada (2)	Main	Y	5	12.00	60%	60%	0%
Florida	Main	Y	2	5.00	0%	50%	0%
Great Lakes	Main	Y	10	14.00	90%	40%	10%
Lower Mid-Atlantic	Main	Y	6	16.67	83%	83%	0%
Upper Mid-Atlantic	Main	Y	1	20.00	100%	100%	0%
Midwest	Main	Y	7	12.86	86%	43%	0%
North	Main	Y	2	20.00	100%	50%	50%
Northeast	Main	Y	6	15.00	83%	67%	0%
Pacific	Main	Y	6	16.67	67%	100%	0%
South Central	Main	Y	4	12.50	75%	50%	0%
Southeast	Main	Y	8	10.00	63%	38%	0%
UK (North)	UK	Y	3	10.00	33%	33%	33%
UK (South)	UK	Y	5	16.00	80%	60%	20%
Upstate NY	Main	Y	2	20.00	100%	100%	0%
California	Main	N	4	17.50	100%	50%	25%
Canada	Main	N	3	13.33	67%	33%	33%
Florida	Main	N	4	12.50	75%	50%	0%
Great Lakes	Main	N	1	10.00	100%	0%	0%
Lower Mid-Atlantic	Main	N	2	15.00	100%	50%	0%
Midwest	Main	N	2	10.00	100%	0%	0%
North	Main	N	5	12.00	100%	20%	0%
Northeast	Main	N	2	10.00	50%	50%	0%
Overflow	Main	N	2	15.00	100%	50%	0%
Pacific Northwest	Main	N	5	14.00	100%	40%	0%
South Central	Main	N	2	5.00	50%	0%	0%
UK	Main	N	3	13.33	100%	33%	0%
Upper Mid-Atlantic	Main	N	5	14.00	100%	20%	20%
Upstate NY	Main	N	4	15.00	100%	50%	0%
2025 ACF Nationals	2025-08-09	N	2	10.00	100%	0%	0%
2024 ARCADIA at UC Berkeley	04/19/2025	N	1	0.00	0%	0%	0%

Data


Stanford A	Berkeley Little Prince	0	10	0	10
Berkeley Mighty Miner	Berkeley Boss Bandit	10	10	0	20
Claremont A	Claremont B	10	10	0	20
UC Santa Barbara	UCSD A	10	10	0	20
UCSD B	UCLA C	10	10	10	30
USC B	UCLA E	0	10	0	10
UCLA F	UC Irvine	10	10	0	20
Carleton A	Carleton B	10	10	0	20
McGill B	McGill A	0	0	0	0
Queen's A	Toronto F	0	10	0	10
McMaster A	Toronto B	10	10	0	20
Toronto G	Guelph A	0	0	0	0
Waterloo A	Toronto D	10	0	0	10
Toronto E	McMaster B	10	10	0	20
Florida Tech A	UCF A	0	0	0	0
UCLA A	USC A	0	10	0	10
Florida B	Florida A	10	0	0	10
Michigan A	CWRU A	10	10	0	20
CWRU B	Louisville A	10	10	10	30
CWRU D	Ohio State B	0	10	0	10
Kentucky	Louisville D	0	10	0	10
Michigan State A	Michigan B	10	10	0	20
Louisville B	Michigan C	0	10	0	10
Michigan State B	Ohio State A	0	10	0	10
CWRU C	Ohio State C	10	10	0	20
Ohio State D	Louisville C	0	10	0	10
Pittsburgh	Michigan D	0	0	0	0
Liberty A	Duke A	0	0	0	0
NCSU A	UNC A	10	10	0	20
VCU A	UNC B	10	10	0	20
William & Mary A	Virginia A	10	10	0	20
Virginia Tech A	Virginia Tech B	10	10	0	20
William & Mary C	William & Mary B	10	10	0	20
George Mason A	Maryland A	10	10	0	20
Iowa A	Chicago A	0	10	0	10
Chicago B	Illinois Urbana	10	10	0	20
Illinois Champaign	Illinois Blue	10	10	0	20
Purdue A	Purdue C	10	10	0	20
Purdue B	Iowa B	0	10	0	10
Rose-Hulman A	William Woods University A	0	10	0	10
Purdue D	WashU Grolars	0	0	0	0
Wisconsin A	St. Thomas	10	10	0	20
Wisconsin C	Minnesota	0	10	10	20
Amherst B	Tufts C	10	10	0	20
MIT A	Boston College A	0	10	0	10
Brown A	Amherst A	0	0	0	0
Vermont B	Dartmouth A	10	10	0	20
UConn A	Tufts B	10	10	0	20
Brandeis Mozart	Williams B	10	10	0	20
UCLA D	UCLA B	10	10	0	20
Alberta A	Washington Steelhead	10	10	0	20
British Columbia A	Texas	10	0	0	10
Washington Pink	Washington Sockeye	10	10	0	20
Washington State	Arizona State A	10	0	0	10
Washington Atlantic	British Columbia B	10	10	0	20
Washington Chum	British Columbia C	10	10	0	20
Auburn B	Auburn A	10	0	0	10
Chipola College	Alabama A	0	10	0	10
Tennessee Tech	Georgia A	0	0	0	0
Clemson A	Georgia Tech C	10	10	0	20
Georgia Tech D	Montevallo	0	0	0	0
Georgia Tech E	Emory A	0	10	0	10
Vanderbilt A	South Carolina A	0	10	0	10
Georgia Tech A	Tennessee A	10	10	0	20
Durham A	Edinburgh A	0	0	0	0
Edinburgh C	Durham C	10	10	10	30
York	Edinburgh D	0	0	0	0
Southampton	Birmingham	10	0	0	10
Cambridge C	Imperial B	0	10	0	10
Cambridge B	Cardiff	10	10	10	30
Imperial A	Warwick	0	10	0	10
Cambridge A	Oxford	10	10	0	20
Kansas State A	Central Oklahoma B	10	10	0	20
Central Oklahoma A	Nebraska B	0	10	0	10
Murray State College A	Oklahoma	0	0	0	0
Nebraska A	Tulsa A	10	10	0	20
Cornell A	ESF	10	10	0	20
RIT A	Cornell C	10	10	0	20