Estimating the Reproducibility of Experimental Philosophy

Florian Cova; Brent Strickland; Angela Abatista; Aurélien Allard; James Andow; Mario Attie; James Beebe; Renatas Berniūnas; Jordane Boudesseul; Matteo Colombo; Fiery Cushman; Rodrigo Diaz; Noah N’Djaye Nikolai van Dongen; Vilius Dranseika; Brian D. Earp; Antonio Gaitán Torres; Ivar Hannikainen; José V. Hernández-Conde; Wenjia Hu; François Jaquet; Kareem Khalifa; Hanna Kim; Markus Kneer; Joshua Knobe; Miklos Kurthy; Anthony Lantian; Shen yi Liao; Edouard Machery; Tania Moerenhout; Christian Mott; Mark Phelan; Jonathan Phillips; Navin Rambharose; Kevin Reuter; Felipe Romero; Paulo Sousa; Jan Sprenger; Emile Thalabard; Kevin Tobia; Hugo Viciana; Daniel Wilkenfeld; Xiang Zhou

doi:10.1007/s13164-018-0400-9

Estimating the Reproducibility of Experimental Philosophy

Florian Cova
, Brent Strickland
, Angela Abatista
, Aurélien Allard
, James Andow
, Mario Attie
, James Beebe
, Renatas Berniūnas
, Jordane Boudesseul
, Matteo Colombo
, Fiery Cushman
, Rodrigo Diaz
, Noah N’Djaye Nikolai van Dongen
, Vilius Dranseika
, Brian D. Earp
, Antonio Gaitán Torres
, Ivar Hannikainen
, José V. Hernández-Conde
, Wenjia Hu
, François Jaquet

Kareem Khalifa, Hanna Kim, Markus Kneer, Joshua Knobe, Miklos Kurthy, Anthony Lantian, Shen yi Liao, Edouard Machery, Tania Moerenhout, Christian Mott, Mark Phelan, Jonathan Phillips, Navin Rambharose, Kevin Reuter, Felipe Romero, Paulo Sousa, Jan Sprenger, Emile Thalabard, Kevin Tobia, Hugo Viciana, Daniel Wilkenfeld, Xiang Zhou

Philosophy

University of Geneva
École Normale Supérieure
CNRS
Université Paris 8 Vincennes Saint-Denis
University of Reading
Yale University
Vilnius University
Universidad de Lima
Tilburg University
Harvard University
University of Bern
Universidad Carlos III de Madrid
Pontifícia Universidade Católica do Rio de Janeiro
University of the Basque Country
Lawrence University
Middlebury College
Washington & Jefferson College
University of Zurich
University of Sheffield
Université Paris Nanterre
University of Puget Sound
University of Pittsburgh
Ghent University
Queen's University Belfast
University of Turin
Sciences, Normes, Décision
Instituto de Estudios Sociales Avanzados (IESA-CSIC)

Research output: Contribution to journal › Article › peer-review

157 Scopus citations

Abstract

Responding to recent concerns about the reliability of the published literature in psychology and other disciplines, we formed the X-Phi Replicability Project (XRP) to estimate the reproducibility of experimental philosophy (osf.io/dvkpr). Drawing on a representative sample of 40 x-phi studies published between 2003 and 2015, we enlisted 20 research teams across 8 countries to conduct a high-quality replication of each study in order to compare the results to the original published findings. We found that x-phi studies – as represented in our sample – successfully replicated about 70% of the time. We discuss possible reasons for this relatively high replication rate in the field of experimental philosophy and offer suggestions for best research practices going forward.

Original language	English
Pages (from-to)	9-44
Number of pages	36
Journal	Review of Philosophy and Psychology
Volume	12
Issue number	1
DOIs	https://doi.org/10.1007/s13164-018-0400-9
State	Published - Mar 2021

Access to Document

10.1007/s13164-018-0400-9

Cite this

Cova, F., Strickland, B., Abatista, A., Allard, A., Andow, J., Attie, M., Beebe, J., Berniūnas, R., Boudesseul, J., Colombo, M., Cushman, F., Diaz, R., N’Djaye Nikolai van Dongen, N., Dranseika, V., Earp, B. D., Torres, A. G., Hannikainen, I., Hernández-Conde, J. V., Hu, W., ... Zhou, X. (2021). Estimating the Reproducibility of Experimental Philosophy. Review of Philosophy and Psychology, 12(1), 9-44. https://doi.org/10.1007/s13164-018-0400-9

@article{4302823584354c989b227aa8ed1f00b7,

title = "Estimating the Reproducibility of Experimental Philosophy",

abstract = "Responding to recent concerns about the reliability of the published literature in psychology and other disciplines, we formed the X-Phi Replicability Project (XRP) to estimate the reproducibility of experimental philosophy (osf.io/dvkpr). Drawing on a representative sample of 40 x-phi studies published between 2003 and 2015, we enlisted 20 research teams across 8 countries to conduct a high-quality replication of each study in order to compare the results to the original published findings. We found that x-phi studies – as represented in our sample – successfully replicated about 70\% of the time. We discuss possible reasons for this relatively high replication rate in the field of experimental philosophy and offer suggestions for best research practices going forward.",

author = "Florian Cova and Brent Strickland and Angela Abatista and Aur{\'e}lien Allard and James Andow and Mario Attie and James Beebe and Renatas Berniūnas and Jordane Boudesseul and Matteo Colombo and Fiery Cushman and Rodrigo Diaz and \{N{\textquoteright}Djaye Nikolai van Dongen\}, Noah and Vilius Dranseika and Earp, \{Brian D.\} and Torres, \{Antonio Gait{\'a}n\} and Ivar Hannikainen and Hern{\'a}ndez-Conde, \{Jos{\'e} V.\} and Wenjia Hu and Fran{\c c}ois Jaquet and Kareem Khalifa and Hanna Kim and Markus Kneer and Joshua Knobe and Miklos Kurthy and Anthony Lantian and Liao, \{Shen yi\} and Edouard Machery and Tania Moerenhout and Christian Mott and Mark Phelan and Jonathan Phillips and Navin Rambharose and Kevin Reuter and Felipe Romero and Paulo Sousa and Jan Sprenger and Emile Thalabard and Kevin Tobia and Hugo Viciana and Daniel Wilkenfeld and Xiang Zhou",

note = "Publisher Copyright: {\textcopyright} 2018, Springer Science+Business Media B.V., part of Springer Nature.",

year = "2021",

month = mar,

doi = "10.1007/s13164-018-0400-9",

language = "English",

volume = "12",

pages = "9--44",

journal = "Review of Philosophy and Psychology",

issn = "1878-5158",

publisher = "Springer Verlag",

number = "1",

}

Cova, F, Strickland, B, Abatista, A, Allard, A, Andow, J, Attie, M, Beebe, J, Berniūnas, R, Boudesseul, J, Colombo, M, Cushman, F, Diaz, R, N’Djaye Nikolai van Dongen, N, Dranseika, V, Earp, BD, Torres, AG, Hannikainen, I, Hernández-Conde, JV, Hu, W, Jaquet, F, Khalifa, K, Kim, H, Kneer, M, Knobe, J, Kurthy, M, Lantian, A, Liao, SY, Machery, E, Moerenhout, T, Mott, C, Phelan, M, Phillips, J, Rambharose, N, Reuter, K, Romero, F, Sousa, P, Sprenger, J, Thalabard, E, Tobia, K, Viciana, H, Wilkenfeld, D & Zhou, X 2021, 'Estimating the Reproducibility of Experimental Philosophy', Review of Philosophy and Psychology, vol. 12, no. 1, pp. 9-44. https://doi.org/10.1007/s13164-018-0400-9

TY - JOUR

T1 - Estimating the Reproducibility of Experimental Philosophy

AU - Cova, Florian

AU - Strickland, Brent

AU - Abatista, Angela

AU - Allard, Aurélien

AU - Andow, James

AU - Attie, Mario

AU - Beebe, James

AU - Berniūnas, Renatas

AU - Boudesseul, Jordane

AU - Colombo, Matteo

AU - Cushman, Fiery

AU - Diaz, Rodrigo

AU - N’Djaye Nikolai van Dongen, Noah

AU - Dranseika, Vilius

AU - Earp, Brian D.

AU - Torres, Antonio Gaitán

AU - Hannikainen, Ivar

AU - Hernández-Conde, José V.

AU - Hu, Wenjia

AU - Jaquet, François

AU - Khalifa, Kareem

AU - Kim, Hanna

AU - Kneer, Markus

AU - Knobe, Joshua

AU - Kurthy, Miklos

AU - Lantian, Anthony

AU - Liao, Shen yi

AU - Machery, Edouard

AU - Moerenhout, Tania

AU - Mott, Christian

AU - Phelan, Mark

AU - Phillips, Jonathan

AU - Rambharose, Navin

AU - Reuter, Kevin

AU - Romero, Felipe

AU - Sousa, Paulo

AU - Sprenger, Jan

AU - Thalabard, Emile

AU - Tobia, Kevin

AU - Viciana, Hugo

AU - Wilkenfeld, Daniel

AU - Zhou, Xiang

PY - 2021/3

Y1 - 2021/3

N2 - Responding to recent concerns about the reliability of the published literature in psychology and other disciplines, we formed the X-Phi Replicability Project (XRP) to estimate the reproducibility of experimental philosophy (osf.io/dvkpr). Drawing on a representative sample of 40 x-phi studies published between 2003 and 2015, we enlisted 20 research teams across 8 countries to conduct a high-quality replication of each study in order to compare the results to the original published findings. We found that x-phi studies – as represented in our sample – successfully replicated about 70% of the time. We discuss possible reasons for this relatively high replication rate in the field of experimental philosophy and offer suggestions for best research practices going forward.

AB - Responding to recent concerns about the reliability of the published literature in psychology and other disciplines, we formed the X-Phi Replicability Project (XRP) to estimate the reproducibility of experimental philosophy (osf.io/dvkpr). Drawing on a representative sample of 40 x-phi studies published between 2003 and 2015, we enlisted 20 research teams across 8 countries to conduct a high-quality replication of each study in order to compare the results to the original published findings. We found that x-phi studies – as represented in our sample – successfully replicated about 70% of the time. We discuss possible reasons for this relatively high replication rate in the field of experimental philosophy and offer suggestions for best research practices going forward.

UR - https://www.scopus.com/pages/publications/85131297329

U2 - 10.1007/s13164-018-0400-9

DO - 10.1007/s13164-018-0400-9

M3 - Article

AN - SCOPUS:85131297329

SN - 1878-5158

VL - 12

SP - 9

EP - 44

JO - Review of Philosophy and Psychology

JF - Review of Philosophy and Psychology

IS - 1

ER -

Estimating the Reproducibility of Experimental Philosophy

Abstract

Access to Document

Other files and links

Cite this