Meta-analysis is a statistical technique for summarizing and analyzing the results of multiple studies on a similar topic, for instance, conducted by different research teams, at different times, or using different methods. This technique is used in many research fields. The widest area of use of this method is clinical research. Using the results of many controlled trials, researchers conducting a meta-analysis attempt to estimate whether a particular drug or treatment is effective. However, meta-analyses are used not only in medical research. The goal of the project is to investigate whether and how they can also be used in experimental philosophy.
Experimental philosophy is a young discipline in which philosophers (and their colleagues who are often non-philosophers) conduct experimental studies in which they test what intuitions people have about philosophical problems. The discipline, although it has only been in existence for 20 years, has produced a lot of very interesting findings. Many studies revealed that people differ in their intuitions depending on the culture they come from, their age, or their education. In other studies, researchers have discovered that people’s intuitions are influenced by surprising factors, for example, how or in what order philosophical scenarios are presented to the subjects to evaluate. The current state of research suggests that philosophical intuitions can be highly variable and unstable.
The primary goal of the project is to see if, using meta-analysis, we can determine how people’s philosophical intuitions vary. This will require, on the one hand, considering whether and how the statistical data obtained through meta-analysis can be translated into philosophical debates about intuitions, and on the other hand, actually conducting meta-analytic research.
As part of the project, we will conduct meta-analyses of selected lines of research within experimental philosophy. As a result, we will be able to accurately estimate the degree of variation in effects found by research and determine which factors are responsible for this variation. For example, we will be able to determine whether the results are influenced by the language in which the study was conducted, the exact wording of the questions used in the experiment, or the culture from which the study participants came.
However, meta-analysis is not always possible. Very often, all the data necessary for such an analysis cannot be found in scientific articles describing the results of the studies. Sometimes the necessary statistics are missing, other times the description of the research methods used by the authors is not accurate enough. As a part of the project, we will check whether articles written by experimental philosophers contain all the information necessary to conduct meta-analyses. To this end, we will analyze a large sample of scientific articles. This will help us find out on which issues experimental philosophers have the most to improve, and determine how big a problem inadequate descriptions of research results are. We will also check whether there has been an improvement in experimental philosophy on this issue over the past 20 years.
Based on the results of our research, we will develop a catalog of best practices for experimental philosophers, enabling them to describe the results of their research in a way that allows them to be incorporated into meta-analyses conducted by other researchers. Moreover, based on our research, we will develop rules for creating and describing meta-analyses adapted to experimental philosophy. Because of how useful meta-analyses are for scientific practice, the development of such rules will help improve the methods of experimental philosophy and increase the reliability of its findings.