Archives For science

Author Information: Saana Jukola and Henrik Roeland Visser, Bielefeld University, sjukola@uni-bielefeld.de and rvisser@uni-bielefeld.de.

Jukola, Saana; and Henrik Roland Visser. “On ‘Prediction Markets for Science,’ A Reply to Thicke” Social Epistemology Review and Reply Collective 6, no. 11 (2017): 1-5.

The pdf of the article includes specific page numbers. Shortlink: https://wp.me/p1Bfg0-3Q9

Please refer to:

Image by The Bees, via Flickr

 

In his paper, Michael Thicke critically evaluates the potential of using prediction markets to answer scientific questions. In prediction markets, people trade contracts that pay out if a certain prediction comes true or not. If such a market functions efficiently and thus incorporates the information of all market participants, the resulting market price provides a valuable indication of the likelihood that the prediction comes true.

Prediction markets have a variety of potential applications in science; they could provide a reliable measure of how large the consensus on a controversial finding truly is, or tell us how likely a research project is to deliver the promised results if it is granted the required funding. Prediction markets could thus serve the same function as peer review or consensus measures.

Thicke identifies two potential obstacles for the use of prediction markets in science. Namely, the risk of inaccurate results and of potentially harmful unintended consequences to the organization and incentive structure of science. We largely agree on the worry about inaccuracy. In this comment we will therefore only discuss the second objection; it is unclear to us what really follows from the risk of harmful unintended consequences. Furthermore, we consider another worry one might have about the use of prediction markets in science, which Thicke does not discuss: peer review is not only a quality control measure to uphold scientific standards, but also serves a deliberative function, both within science and to legitimize the use of scientific knowledge in politics.

Reasoning about imperfect methods

Prediction markets work best for questions for which a clearly identifiable answer is produced in the not too distant future. Scientific research on the other hand often produces very unexpected results on an uncertain time scale. As a result, there is no objective way of choosing when and how to evaluate predictions on scientific research. Thicke identifies two ways in which this can create harmful unintended effects on the organization of science.

Firstly, projects that have clear short-term answers may erroneously be regarded as epistemically superior to basic research which might have better long-term potential. Secondly, science prediction markets create a financial incentive to steer resources towards research with easily identifiable short-term consequences, even if more basic research would have a better epistemic pay-off in the long-run.

Based on their low expected accuracy and the potential of harmful effects on the organization of science, Thicke concludes that science prediction markets might be a worse ‘cure’ than the ‘disease’ of bias in peer review and consensus measures. We are skeptical of this conclusion for the same reasons as offered by Robin Hanson. While the worry about the promise of science prediction markets is justified, it is unclear how this makes them worse than the traditional alternatives.

Nevertheless, Thicke’s conclusion points in the right direction: instead of looking for a more perfect method, which may not become available in the foreseeable future, we need to judge which of the imperfect methods is more palatable to us. Doing that would, however, require a more sophisticated evaluation of the different strengths and weakness of the different available methods and how to trade those off, which goes beyond the scope of Thicke’s paper.

Deliberation in Science

An alternative worry, which Thicke does not elaborate on, is the fact that peer review is not only expected to accurately determine the quality of submissions and conclude what scientific work deserves to be funded or published, but it is also valued for its deliberative nature, which allows it to provide reasons to those affected by the decisions made in research funding or the use of scientific knowledge in politics. Given that prediction markets function through market forces rather than deliberative procedure, and produce probabilistic predictions rather than qualitative explanations, this might be (another) aspect on which the traditional alternative of peer review outperforms science prediction markets.

Within science, peer review serves two different purposes. First, it functions as a gatekeeping mechanism for deciding which projects deserve to be carried out or disseminated – an aim of peer review is to make sure that good work is being funded or published and undeserving projects are rejected. Second, peer review is often taken to embody the critical mechanism that is central to the scientific method. By pointing out defects and weaknesses in manuscripts or proposals, and by suggesting new ways of approaching the phenomena of interest, peer reviewers are expected to help authors improve the quality of their work. At least in an ideal case, authors know why their manuscripts were rejected or accepted after receiving peer review reports and can take the feedback into consideration in their future work.

In this sense, peer review represents an intersubjective mechanism that guards against the biases and blind spots that individual researchers may have. Criticism of evidence, methods and reasoning is essential to science, and necessary for arriving at trustworthy results.[1] Such critical interaction thus ensures that a wide variety of perspectives in represented in science, which is both epistemically and socially valuable. If prediction markets were to replace peer review, could they serve this second, critical, function? It seems that the answer is No. Prediction markets do not provide reasons in the way that peer review does, and if the only information that is available are probabilistic predictions, something essential to science is lost.

To illustrate this point in a more intuitive way: imagine that instead of writing this comment in which we review Thicke’s paper, there is a prediction market on which we, Thicke and other authors would invest in bets regarding the likelihood of science prediction markets being an adequate replacement of the traditional method of peer review. From the resulting price signal we would infer whether predictions markets are indeed an adequate replacement or not. Would that allow for the same kind of interaction in which we now engage with Thicke and others by writing this comment? At least intuitively, it seems to us that the answer is No.

Deliberation About Science in Politics

Such a lack of reasons that justify why certain views have been accepted or rejected is not only a problem for researchers who strive towards getting their work published, but could also be detrimental to public trust in science. When scientists give answers to questions that are politically or socially sensitive, or when controversial science-based recommendations are given, it is important to explain the underlying reasons to ensure that those affected can – at least try to – understand them.

Only if people are offered reasons for decisions that affect them can they effectively contest such decisions. This is why many political theorists regard the ability of citizens to demand an explanation, and the corresponding duty of decision-makers to be responsive to such demands, as a necessary element of legitimate collective decisions.[2] Philosophers of science like Philip Kitcher[3] rely on very similar arguments to explain the importance of deliberative norms in justifying scientific conclusions and the use of scientific knowledge in politics.

Science prediction markets do not provide substantive reasons for their outcome. They only provide a procedural argument, which guarantees the quality of their outcome when certain conditions are fulfilled, such as the presence of a well-functioning market. Of course, one of those conditions is also that at least some of the market participants possess and rely on correct information to make their investment decisions, but that information is hidden in the price signal. This is especially problematic with respect to the kind of high-impact research that Thicke focuses on, i.e. climate change. There, the ability to justify why a certain theory or prediction is accepted as reliable, is at least as important for the public discourse as it is to have precise and accurate quantitative estimates.

Besides the legitimacy argument, there is another reason why quantitative predictions alone do not suffice. Policy-oriented sciences like climate science or economics are also expected to judge the effect and effectiveness of policy interventions. But in complex systems like the climate or the economy, there are many different plausible mechanisms simultaneously at play, which could justify competing policy interventions. Given the long-lasting controversies surrounding such policy-oriented sciences, different political camps have established preferences for particular theoretical interpretations that justify their desired policy interventions.

If scientists are to have any chance of resolving such controversies, they must therefore not only produce accurate predictions, but also communicate which of the possible underlying mechanisms they think best explains the predicted phenomena. It seems prediction markets alone could not do this. It might be useful to think of this particular problem as the ‘underdetermination of policy intervention by quantitative prediction’.

Science prediction markets as replacement or addition?

The severity of the potential obstacles that Thicke and we identify depends on whether science prediction markets would replace traditional methods such as peer review, or would rather serve as addition or even complement to traditional methods. Thicke provides examples of both: in the case of peer review for publication or funding decisions, prediction markets might replace traditional methods. But in the case of resolving controversies, for instance concerning climate change, it aggregates and evaluates already existing pieces of knowledge and peer review. In such a case the information that underlies the trading behavior on the prediction market would still be available and could be revisited if people distrust the reliability of the prediction market’s result.

We could also imagine that there are cases in which science prediction markets are used to select the right answer or at least narrow down the range of alternatives, after which a qualitative report is produced which provides a justification of the chosen answer(s). Perhaps it is possible to infer from trading behavior which investors possess the most reliable information, a possibility explored by Hanson. Contrary to Hanson, we are skeptical of the viability of this strategy. Firstly, the problem of the underdetermination of theory by data suggests that different competing justifications might be compatible with the observation trading behavior. Secondly, such justifications would be post-hoc rationalizations, which sound plausible but might lack power to discriminate among alternative predictions.

Conclusion

All in all, we are sympathetic to Michael Thicke’s critical analysis of the potential of prediction markets in science and share his skepticism. However, we point out another issue that speaks against prediction markets and in favor of peer review: Giving and receiving reasons for why a certain view should be accepted or rejected. Given that the strengths and weaknesses of these methods fall on different dimensions (prediction markets may fare better in accuracy, while in an ideal case peer review can help the involved parties understand the grounds why a position should be approved), it is important to reflect on what the appropriate aims in particular scientific and policy context are before making a decision on what method should be used to evaluate research.

References

Hanson, Robin. “Compare Institutions To Institutions, Not To Perfection,” Overcoming Bias (blog). August 5, 2017. Retrieved from: http://www.overcomingbias.com/2017/08/compare-institutions-to-institutions-not-to-perfection.html

Hanson, Robin. “Markets That Explain, Via Markets To Pick A Best,” Overcoming Bias (blog), October 14, 2017 http://www.overcomingbias.com/2017/10/markets-that-explain-via-markets-to-pick-a-best.html

[1] See, e.g., Karl Popper, The Open Society and Its Enemies. Vol 2. (Routledge, 1966) or Helen Longino, Science as Social Knowledge. Values and Objectivity in Scientific Inquiry (Princeton University Press, 1990).

[2] See Jürgen Habermas, A Theory of Communicative Action, Vols1 and 2. (Polity Press, 1984 & 1989) & Philip Pettit, “Deliberative democracy and the discursive dilemma.” Philosophical Issues, vol. 11, pp. 268-299, 2001.

[3] Philip Kitcher, Science, Truth, and Democracy (Oxford University Press, 2001) & Philip Kitcher, Science in a democratic society (Prometheus Books, 2011).

Author Information: Massimo Campanini, University of Trento, massimo.campanini@unitn.it

Campanini, Massimo. “Science and Epistemology in Medieval Islam.” [1] Social Epistemology Review and Reply Collective 4, no. 12 (2015): 20-28.

The PDF of the article gives specific page numbers. Shortlink: http://wp.me/p1Bfg0-2v9

Please refer to:

  • Articles related to the broader discussion on Islam and science, hosted by the SERRC, are listed below the article. [a]

astrolabe

Image credit: mangpages, via flickr

“Islamic” science or “Arab” science? [2] This is a relevant question. Arabic language, the vehicle of the Islamic revelation, was as well the vehicle of great scientific knowledge although not every “Arabic” scientist was Muslim, nor was every Muslim scientist an Arab. In the very first years after the expansion of Islam, numerous Christian and Jewish investigators, and even “pagan” ones (such as the famous astronomer-philosopher Thabit Ibn Qurra from Harran, Mesopotamia, d. 901, who worshipped the stars) communicated in Arabic. The Bakhtishu’, a family of physicians from the Persian school of Gondeshapur, who served the Omayyad and ‘Abbasid Caliphs, where Nestorians. The great translators of Greek or Syriac works into Arabic were Nestorian or Jacobite such as Hunayn Ibn Ishaq (Latinized in Ioannitius), who worked at the House of Wisdom or Bayt al-Hikmah, founded in Baghdad by Caliph al-Ma’mun (813-833). Mashallah, the greatest astronomer of the courts of al-Mansur and Harun ar-Rashid (786-809), was Jewish.  Continue Reading…

Author Information: Adam Riggio, McMaster University, adamriggio@gmail.com; Steve Fuller, University of Warwick, S.W.Fuller@warwick.ac.uk

Shortlink: http://wp.me/p1Bfg0-23w

Editor’s Note:

Adam Riggio

riggio Although I loved our explicitly political discussion of the last couple of dialogues, I want to dive into the final installment of our exchange with some headier philosophy. I particularly want to discuss the power of counter-factual reasoning. Even though you consider this a foundational method for a progressive philosophy of science, I think it eclipses even your own vision. Counter-factual knowledge, I’d go so far as to say, makes a lot of your own vision obsolete.

The conclusion of Knowledge: The Philosophical Quest in History returns to the vision on which your early chapters focused, the unity of science in humanity’s conception of ourselves in the image of God. Your advocacy of this idea remains a point on which you and I will, I think, always disagree. But once I reached the end of your book, I had many more reasons for my disagreement. Continue Reading…

Author Information: Adam Riggio, McMaster University, adamriggio@gmail.com; Steve Fuller, University of Warwick, S.W.Fuller@warwick.ac.uk

Shortlink: http://wp.me/p1Bfg0-21Q

Editor’s Note:

Adam Riggio

We’ve talked about the epistemic implications of humanity’s divinity, how our scientific inquiries were conceived as bringing us closer to God, in touch with our divine nature. As I get into these other chapters, I find that the focus of your book is shifting to the epistemic implications of humanity’s profanity, how our distance from perfection is incurable. Continue Reading…

Author Information: Ilya Kasavin, Russian Academy of Sciences, itkasavin@gmail.com

Kasavin, Ilya. “Cases of Interdisciplinarity: Between Habitus and Reflexion.” Social Epistemology Review and Reply Collective 4, no. 3 (2015): 15-30.

The PDF of the article gives specific page numbers. Shortlink: http://wp.me/p1Bfg0-1VE

reflexion_trees

Image credit: Mélanie Plante, via flickr

Abstract [1] [2] [3]

Several cases of broadly viewed interdisciplinary research are considered. Discussed are the disciplinary status of natural philosophy in the middle ages; the dispute about witchcraft in the Renaissance; the disciplinary formation of chemistry in the interaction between peripatetics, jatrochemists, spagirists and atomists; and the conceptual shifts in Maxwell’s electrodynamics. These debates are analyzed using two major notions—habitus and reflexion—that differ from those of Bourdieu. Habitus is taken as a methodological attitude based on natural and historically rooted adherence to a theory, or world picture, based on the shared research practice. Reflexion represents a critical and proactionary stance towards a revision of an established theoretical framework, which is irreducible to the logic of rational criticism. Various cases of habitus-reflexion controversy provide a valuable source for a typological picture of interdisciplinary research. And this, in turn, helps clarify the nature of interdisciplinarity in general, given the topicality of this cognitive pattern in the contemporary science.

Interdisciplinary interaction in modern science has become a usual phenomenon deserving more serious philosophical and scientific understanding. Why is an epistemological analysis of interdisciplinary research significant? The rationale for this attention stems from the nonclassical approaches in epistemology and philosophy of science that emphasize the communicative nature of the cognitive process and, moreover, the essential determination of the content of knowledge by various types and forms of communication.  Continue Reading…

Author Information: Patrick J. Reider, University of Pittsburgh at Greensburg, PJR23@pitt.edu

Normative Functionalism and the Pittsburgh School: Table of Contents

Reider, Patrick J. 2012. “Sellars on Perception, Science, and Realism: A Critical Response.” Social Epistemology Review and Reply Collective 2 (1): 39-56.

The PDF of the article gives specific page numbers. Shortlink: http://wp.me/p1Bfg0-B8

Abstract

In this article, I explain the manner in which Sellars’ version of realism is undermined by his Kantian commitments and his normative functionalism. After providing an account of the perceptual and scientific models that support his realism, I argue for the following: Sellars’ perceptual/cognitive models do not permit sufficient perceptual and conceptual access to warrant a version of realism predicated upon our ability to know mind-independent existence. In other words, Sellars’ Kantian commitments, his norm-driven view of concepts, and his norm-guided view of reason place severe limits on one’s access to mind-independent reality. Consequently, when one strictly holds Sellars to these limits, he cannot show (or significantly support) the manner in which knowledge of mind-independent existence is possible.

1.1 Introduction

In what manner does Sellars believe perceptual knowledge is possible in light of his normative functionalist views? For instance, if 1) the meaning of words can be partly or fully reduced to the function they play in a language, 2) norms mediate and/or determine these functions, and 3) the manner in which perception unfolds is shaped by these norms (all of which are important aspects of normative functionalism), then in what sense does Sellars believe we can have objective empirical knowledge? This is an important question to ask, as it reveals the manner in which Sellars believes science, as a discipline founded upon empirical observation, can lead to knowledge of mind-independent existence.

In order to answer the above questions, we need to understand Sellars’ account of perception. One of his most complete accounts of perception, as it concerns his peculiar brand of realism, can be found in Science and Metaphysics: Variations on Kantian Themes.[1] Given the norm-driven account of Sellars’ views on meaning and the foreknowledge that Sellars is a realist, one might expect (and believe he needs) a view of perception that un-problematically supports realism. Instead, Sellars surprisingly makes numerous Kantian commitments that are central to his perceptual model — commitments which are historically interpreted as counter to realism. For example, take the following five Kantian commitments that Sellars adheres to in Science and Metaphysics (and throughout all of his later career): 1) neither sensibility (in itself) nor conceptuality (in itself) are capable of providing knowledge of the thing-in-itself, 2) without a pre-existing conceptual framework, no knowledge of empirical content is possible, 3) we are not directly aware of sensations, 4) in order to recognize empirical content, as facts or states of affairs, a judgment is required, and 5) in order to relate concepts to sensory content, via a judgment, the mind must first synthesize sensory content into coherent units of time and space.[2]

The above views are surprising for a realist to embrace, because they play a central role in Kant’s claim that the thing-in-itself (an entity’s true existence independent of how a person’s mind may contingently experience, believe, or feel about it) is unknowable. Despite Sellars’ Kantian commitments, he believes that “the gulf between appearances and things-in-themselves, though a genuine one, can in principle be bridged” (50). In what follows, I will first explain the manner in which Sellars believes this is possible. I then argue that his perceptual views do not support or allow the type of realist conclusion he draws from them. Continue Reading…

Author Information: Gregory Sandstrom, European Humanities University, Vilnius, Lithuania, SERRC, gregorisandstrom@yahoo.com

Sandstrom, Gregory. 2012. How many ‘sciences’ are there? Social Epistemology Review and Reply Collective 1 (10): 4-15

The PDF of the article gives specific page numbers. Shortlink: http://wp.me/p1Bfg0-se

“All science is either physics or stamp collecting.”[1] — Ernest Rutherford (1962)

“Anthropology, or true science of Man [is] the last gradation in the Grand Hierarchy of Abstract Science.”[2] — Auguste Comte (1874)

Introduction

How many ‘sciences’ are there? Science is thought by many people to be the most global-universal practise we have available to humanity nowadays, other than perhaps football (soccer) teams, the Olympic Games and the United Nations. It is supposed to be neutral to gender, race, ethnicity, class, network, status, ideology, political system and religion. Since most people generally hold that there is more than one science — that science is plural, not singular, that there are multiple scientific methods and not just a single, uniform scientific method — this article is my attempt to answer the simple question above by giving a basic guide of how to estimate the approximate number of sciences.

As orientation lessons on the history and philosophy of science (HPS) often begin, there are two questions we must ask: which science(s) and whose science(s)? The first question is mainly what I am focused on in this paper. But the second question is likewise important because people hold various opinions about what constitutes ‘science’ and what does not. To some scientists, other scientists do not actually count as ‘scientists’ because they are considered not scientific enough (i.e. their field is not really a ‘scientific’ field according to others’ perception). Continue Reading…

Author Information: Adam Riggio, McMaster University, adamriggio@gmail.com

Riggio, Adam. 2012. “Right Thinking for Right Science? On the Pitt-Collier Exchange Over the Purpose of STS” Social Epistemology Review and Reply Collective 1 (2): 35-39.

The PDF of the article gives specific page numbers. Shortlink: http://wp.me/p1Bfg0-8T

Please refer to:

The two separate essays by James Collier and I were originally planned to be a single work, jointly written between us. But as our collaboration evolved, our reactions to what Joseph Pitt wrote in November diverged. Since our original plan was to write together, my perspective remains more critical of Pitt than Collier, but my goal is to call attention to ideas that may have gone unnoticed in the heat of their exchange. As it happened, my task became to synthesize the two perspectives. Although I am not sure what such a synthesis would look like (having written this essay I still have no idea) I hope that I have written something that will show what can be learned from their conversation. Continue Reading…