Author Information: Gabriele Bammer, The Australian National University, Gabriele.Bammer@anu.edu.au

Bammer, Gabriele. “Interdisciplining Knowledge or Disciplining Interdisciplinarity? A Reply to Huutoniemi’s “’Interdisciplinarity as Academic Accountability.'” Social Epistemology Review and Reply Collective 5, no. 3 (2016): 1-4.

The PDF of the article gives specific page numbers. Shortlink: http://wp.me/p1Bfg0-2F4

Please refer to:

• Huutoniemi, Katri. “Interdisciplinarity as Academic Accountability: Prospects for Quality Control Across Disciplinary Boundaries.” Social Epistemology 30, no. 2 (2016): 163-185.
• O’Rourke, Michael. “A Reply to Katri Huutoniemi’s ‘Interdisciplinarity as Academic Accountability.’” Social Epistemology Review and Reply Collective 4, no. 10 (2015): 26-32.
• Huutoniemi, Katri. “On the Value of Interdisciplinary Integration: A Response to Michael O’Rourke.” Social Epistemology Review and Reply Collective 5, no. 2 (2016): 88-92.

Image credit: Jeff Youngstrom, via flickr

This thoughtful and thought-provoking article by Katri Huutoniemi adds to deliberations about how to bring interdisciplinary research out of the margins and into the mainstream, as well as how to effectively peer-review such research. This is timely as the Global Research Council, a federation of more than 50 national research funders, has selected interdisciplinarity as one of its two annual themes for an in-depth report, debate and statement between now and mid-2016.[1] 

Panel Review Processes

Based on her extensive reading and conceptual analysis of the literature, as well as an empirical study of a selection of panel review processes used by the Academy of Finland, Huutoniemi suggests that the panel-style dialogue and negotiation used to decide which interdisciplinary proposals to fund could also be used to improve funding of discipline-based research, increasing not only its societal relevance, but also its ability to integrate other relevant knowledge.

An attractive feature of her proposal is that she puts interdisciplinarity on the front foot arguing that the hallmarks of interdisciplinarity, especially integration of knowledge across disciplines and engagement with societal problems, have ramifications for research as a whole. As a consequence, the process of accountability being developed for interdisciplinary research could improve accountability of research more generally, especially discipline-based research. In this way she brings interdisciplinary research firmly into the mainstream.

One motivator for Huutoniemi’s work is the desire to “avoid adding new or external criteria” for the review of interdisciplinary projects. Her aim is to modify the existing review process based on the potential that she sees in review panels composed of generalists, so that such a process serves interdisciplinary projects and holds disciplinary projects to the same level of accountability. This is a laudable aim.

Of course, the proposal is at an early stage and feasibility has still to be assessed. In his reply, Michael O’Rourke has raised a number of areas for further development.[2] From my perspective, the first significant question to address concerns the panels. Huutoniemi’s empirical work suggests that panel assessment shows promise. It is noteworthy that she suggests that generalists make the most effective panel members. I wonder if such generalists actually have interdisciplinary expertise, but this is not specified.

There is no doubt that a panel of generalists can arrive at a decision—after dialogue and negotiation—about which interdisciplinary projects should be funded. It is also unsurprising that the dialogue is likely to be intense, as Huutoniemi describes. This begs the question of whether such panels reach good defensible decisions. Given human propensity for confirmation bias, the satisfaction of panels with their own decisions is not a particularly helpful indicator.[3] Better indicators are whether 1) two panels reviewing the same proposals reach the same decisions, or at least fund a significant number of the same proposals, 2) an independent observer rates the negotiation process as fair and professional and 3) an independent jury of acknowledged interdisciplinary experts agrees that the best proposals have been funded or at least that all the proposals funded are worthy.

Given the thrust of Huutoniemi’s article, an additional set of considerations is how such panels assess disciplinary proposals. Do they make the same recommendations as disciplinary panels and, if not, what are the prime characteristics on which outcomes differ? Notably, does assessment by such a panel result in funding for disciplinary projects with more social relevance and relevant knowledge integration? Indeed, can disciplinary projects even be meaningfully differentiated on these criteria?

For Huutoniemi’s proposal to get traction, evidence needs to be produced that the proposed review process would do a good job of assessing interdisciplinary proposals and a better job of assessing disciplinary proposals than is currently the case. Gathering that evidence is no small task.

Are there other alternatives that would achieve at least some of the same outcomes? Huutoniemi argues that her proposal involves “interdisciplining” knowledge which “can be portrayed as a reversal of recent attempts to “discipline” interdisciplinarity”, which I have championed.[4] While our pathways are different, we are seeking the same outcomes—better review processes for interdisciplinary research and a better balance in relative funding to interdisciplinary and disciplinary research, in other words, to be blunt, more funding for interdisciplinary research.

Organising Fellow Interdisciplinarians

Unlike Huutoniemi, my focus is not on the disciplines or their self-referential system of accountability. Although the disciplinary organisation of knowledge is far from perfect, it has in many ways been productive and served society well. But even disciplinarians mostly agree that discipline-based research is not enough for tackling the complex real-world problems that face us today and that interdisciplinary research is also required.

My focus is on fellow interdisciplinarians and our failure to organise as effectively as we could in order to contribute as strongly as we should. My argument is fundamentally that:

a) There is significant excellent interdisciplinary research being conducted;

b) Interdisciplinary researchers find it almost impossible to access the wealth of knowledge about integrative and other relevant methods and processes, because there is no repository that houses these;

c) The process of assessing interdisciplinary research is haphazard because there is no established peer-review process, meaning that interdisciplinary researchers can never be certain that their project will be assessed by someone competent in interdisciplinarity;

d) Disciplines do two important things that we could learn from—they provide repositories of knowledge and they establish colleges of peers.

In my book, I lay out in detail the composition and functioning of a discipline to underpin interdisciplinary research, a discipline I suggest could be called Integration and Implementation Sciences (abbreviated to I2S).[5] The book is available online in open-access for anyone interested in this detail.

The question that it would be interesting to discuss with Huutoniemi is the one that I raised above—are the “generalist” panel members who are most effective in her estimation, actually researchers with interdisciplinary expertise? If so, are there common points that we could build with my suggestion for establishing a college of peers who have the competence to evaluate interdisciplinary research?

I also specifically engage with the power imbalance between interdisciplinarity and the disciplines.[6] The disciplines are well-organised with large active colleges of peers, dynamic professional organisations and multiple journals at various ranking levels. Particularly important is the convergence of bottom-up and top-down activities. Take the discipline of chemistry as an illustration. There are many different types of chemists beavering away at research and various kinds of (often industrial) impact. For assessment purposes, chemistry is divided into branches, allowing competent evaluators to be selected. In Australia, the field of research (FOR) codes used by the main funding body (the Australian Research Council) divide chemistry into analytical, inorganic, organic, physical, theoretical and computational, macromolecular and materials, medicinal and biomolecular, and ‘other’. Such categorizations are developed top down by professional organisations and other influential bodies to reflect practice on the ground. They are solidified by being included in research funding arrangements.

In comparison interdisciplinarity is unorganized. It can be argued that there are various kinds of interdisciplinarity—interdisciplinary research can be conducted by individuals or teams, it can involve a small number of disciplines or many, the disciplines may be closely aligned or diverse, and interdisciplinary research may or may not involve non-research stakeholders.[7] Indeed, interdisciplinary research does not necessarily tackle societal problems; it can be just as arcane as some discipline-based research. It is noteworthy that these issues are glossed over by Huutoniemi. In any case, the top-down structures to meaningfully organise these different kinds of interdisciplinarity are missing. Existing professional associations are small, attracting only a tiny proportion of interdisciplinary practitioners. Consequently, they currently have neither the power nor the intellectual heft to effectively organize or lobby for interdisciplinarity.

These considerations point to the main weakness in Huutoniemi’s argument. Why does interdisciplinary research continue to rely on disciplinary researchers and unspecified generalists for legitimacy and support? Establishing a self-referential system is not all bad. Indeed, if we do not effectively establish our own markers of quality, we will continue to be subjected to the whims of those less qualified. This is another lesson we can learn from the disciplines.

There is no end to the fascinating research questions that can be asked. But research capacity is limited. If we want more support for interdisciplinary research, especially interdisciplinary research that tackles complex societal problems, we need to organise so that we are more powerful and influential. Before we can interdiscipline knowledge, we need to discipline interdisciplinarity.

References

Bammer, Gabriele. Strengthening Interdisciplinary Research: What it is, What it does, How it does it and How it is Supported. Report for the Australian Council of Learned Academies, 2012. http://www.acola.org.au/PDF/Strengthening%20Interdisciplinary%20Research.pdf and http://i2s.anu.edu.au/publications/acola-interdisciplinarity-report.

Bammer, Gabriele. Disciplining Interdisciplinarity: Integration and Implementation Sciences for Researching Complex Real-World Problems. Canberra: ANU Press, 2013. http://press.anu.edu.au?p=222171

Bammer, Gabriele. “Distinguishing between Multi-, Inter- and Trans-disciplinarity—‘Theological’ Hair-splitting or Essential Categorisation?’” Integration and Implementation Insights, November (2015a); http://i2insights.org/2015/11/25/hairsplitting-interdisciplinarity/. Also published as Bammer, Gabriele. “Moving Interdisciplinary Research Forward: Top Down Organising Force Needed to Help Classify Diverse Practices.” LSE Impact blog. February 11 (2016). http://blogs.lse.ac.uk/impactofsocialsciences/2016/02/11/moving-interdisciplinary-research-forward/.

Bammer, Gabriele. “Action Research and the Vicious Cycle between Low Status and Low Impact.” Action Research Plus (AR+) (2015b). http://actionresearchplus.com/action-research-and-the-vicious-cycle-between-low-status-and-low-impact/

Huutoniemi, Katri. “Interdisciplinarity as Academic Accountability.” Social Epistemology 30, no. 2 (2016): 163-185.

Nickerson, Raymond S. “Confirmation Bias: A Ubiquitous Phenomenon in Many Guises.” Review of General Psychology 2, no 2 (1998): 175-220.

O’Rourke, Michael. “A Reply to Katri Huutoniemi’s ‘Interdisciplinarity as Academic Accountability’.” Social Epistemology Review and Reply Collective 4, no 10 (2015): 26-32.

Rylance, Rick. “Global Funders to Focus on Interdisciplinarity.” Nature 525, 17 September (2015): 313-15.

[1] Rick Rylance 2015.

[2] O’Rourke 2015.

[3] Raymond S. Nickerson 1998.

[4] Gabriele Bammer 2013.

[5] Bammer 2013.

[6] See also Bammer 2015a, b.

[7] See Bammer 2012, 2013.

Categories: Critical Replies

Tags: academic disciplines, Gabriele Bammer, Interdisciplinarity, Katri Huutoniemi, Michael O’Rourke