Finnigan, Kara S., and Alan J. Daly, eds. 2014. Using Research Evidence in Education: From the Schoolhouse Door to Capitol Hill. Springer: New York, 194 pp., $129 (hardcover).
March 16, 2016
Deputy Director, Johns Hopkins Institute for Education Policy
Educators are sitting on more data and research than ever before. It is now possible for districts and states to track and correlate student demographics, teacher qualifications, school climate, and academic outcomes. Domestic and international research abounds, from the findings of PISA to school sector comparisons and analyses of district finances. The United States Department of Education publishes granular statistics and evaluates the merits of specific research studies; the Pew Research Center supports an encyclopedic website that compares research on hundreds of interventions.
But does this numerical feast make a difference for students? In some states, in some districts, and in some schools, the answer is clearly “yes.” However, the performance of American students on national and international assessments remains so persistently low (Hanushek et. al., 2014), that we rightly question the extent to which even the most compelling research about what works, matters.
The United States is fortunate, however: a new door opened to the use of evidence when the Every Student Succeeds Act replaced No Child Left Behind. Will we walk through it? The act (ESSA, 2015) enjoins, defines, and rewards the use of evidence in education policy and practice. Is it likely to change the way decisions are made in districts and states?
On this, Finnigan and Daly’s 2014 volume is provocative and useful: it investigates how education research is applied in the real world and how we might strengthen the link between important findings and practice. The authors begin with a theory about how new knowledge fits into existing systems (or doesn’t), and then test it through meticulous case studies of district-, state- and federal-level education agencies.
Sociocultural learning theory, as Finnigan and Daly explain it, suggests that knowledge is shared and sustained through social networks that rely upon particular assumptions and practices– whether tacit or explicit. Members of such networks adopt or discount new information in conversation with one another and with reference to existing modes of doing things. As the editors put it, “Research is rarely used in a linear way; rather, the process of transferring research into practice occurs in a multidimensional, complex way that is social and interactive…it unfolds within a social ecology of relationships (3).” Thus, “most research use fits into pre-existing beliefs (35),” and disruptive change requires “intensive assistance relationships (34).” In other words, a research report, on its own, is inert. It is therefore insufficient for education reformers to present “evidence” without understanding, and even inhabiting, the context that is to receive it. The sociocultural learning theory presented here mirrors the process of change that, some argue, occurs within the natural sciences (Polanyi, 1958), intellectual movements (Collins, 1998), and culture more generally (Hunter, 2010), to wit: knowledge is personal, and new ideas become mainstream not by virtue of their merits but as a result of their promotion by overlapping and influential networks.
Theory in hand, the volume’s authors walk us through the inner workings of district offices, school boards, state education agencies, federal educational initiatives, and philanthropic organizations. Honig et al.’s study, for instance, explores three districts’ attempt to transform their central office administrators from managers to instructional leaders. In the end, Honig’s team finds that most changes at the district level are superficial instead of substantive; that most administrators place innovative research into existing conceptual frameworks; and that school-level transformation by and large languishes as a result. Instead, changing the district and its relationship with schools requires ongoing, hands-on leadership from individuals who have bought into the new model and could thus “counteract the traditions” (47-48). This process of change requires “both disruptive and assistance strategies” (37) that are personal rather than analytic in nature.
Another research team (Asen and Gurke) studied the deliberative process of three school boards in three very different socioeconomic communities. Asen and Gurke find precious little coherence in the use of research in the boards’ decision-making process: school board leaders frequently disagree about what counts as research in the first place, and they resist findings from other contexts (61). What circumvents these patterns seems to be long-term trust among members and a shared understanding about the nature of credible research (64). The study of school boards brings home the personal nature of research use. The authors find that, “Research evidence does not express a clear meaning and prescription for action independent of the people who use research evidence and the situations in which they use research evidence. Research evidence does not speak for itself (58-9).”
Yet another research team (Barnes, Goertz, and Massell) reports on the use of research within three state education agencies (SEAs) of equal size (between 250 and 500 staff) in different regions of the country. This study focused on how the SEAs crafted their school improvement plans – including curriculum, instruction, accountability, teacher policy, and special programs. Barnes et al. fielded web-based surveys, interviewed agency leaders and external organizations involved in implementing the plans, and developed socio-grams that mapped out the content and flow of information.
They find that when SEA leaders need research, they turn primarily to government agencies such as the Institute of Education Sciences (IES), and professional membership organizations such as the Council of Chief State School Officers (CCSSO). Why? The federal IES is perceived as thorough and unbiased. The professional organizations provide user-friendly research: they “offer synthesized, packaged research and ‘research for use’ that provide concrete guidance for program development” (114). This is important information for think tanks and researchers who aim to influence state policy.
Barnes et al. also find that the staff of each SEA includes identifiable members who possess more knowledge, influence, and connections, than the others. These are “knowledge brokers:” “the most ‘well-connected’ internal network actors…those who [seek] research ideas and information from a range of sources…. and provide information to multiple colleagues” (108). At the same time, staff turnover at SEAs is quite high. Thus, writes Barnes, researchers and activists who wish to influence SEAs would do well to identify and cultivate knowledge brokers while sustaining relationships with permanent albeit peripheral staff.
In chapter after chapter we read about the very human, and therefore utterly contingent, aspects of the research-to-policy-and-practice pipeline. As Vivian Tseng and Sandra Nutley summarize,
In none of their [the authors’] cases does research use easily boil down to a single moment or an isolated decision. It is not a simple process whereby research “facts” are passed from researchers to research users and then applied in a linear decision-making process. Instead, research use is contingent, interactive, and iterative (165).
These case studies make for sobering reading for those of us who wish school systems would take up evidence-based reforms – and quickly. It comes therefore as a relief that the volume does not leave us without recourse. Rather, from their observations of real-life educational contexts, the authors recommend new patterns that could remedy the rifts between research, policy, and practice.
One is for education scholars to build research around front-line needs. This is difficult. Quantitative research is calibrated for long-term findings and qualified results; the concerns of superintendents and principals press hard upon each day. Another is for universities to reward the use and reach of their scholars’ work. This, too, is difficult, as tenure customarily depends upon peer-reviewed publications, and spending time in school districts and state houses might constitute professional risk. Finding a middle ground for research that is methodologically sound yet responsive and timely, and changing the incentives within higher education, are clearly long-term endeavors that involve changing the cultures that produce and reward research.
The authors’ most arresting and most frequent point, however, is for education policymakers, practitioners, and scholars to form long-term, robust partnerships that bridge their very different worlds. This makes sense in light of the volume’s theoretical frame: if knowledge lives in networks, it cannot breathe outside of them. If research answers real-world needs, it has a greater chance of bearing fruit. Building and sustaining partnerships, too, is difficult; it requires time, patience, translation, and that all players – including the philanthropies that fund such work – act against their type and training to a certain degree.
Such work is taking place, albeit slowly. For instance, urban research-practice partnerships (in which Johns Hopkins University participates) take this engagement seriously and to good effect; Harvard’s Strategic Data Project is but one example of a high-end, high-use research model that responds to pressing needs. But in a world in which the federal Every Student Succeeds Act (2015) incentivizes states and districts to make good use of evidence, it is critical that we are realistic about what this entails. Finnigan and Daly have provided an excellent – one might say an evidence-based – analysis of conditions under which this is most likely to occur.
A version of this review will appear in the Summer 2016 edition of the Journal of School Choice: International Research and Reform.
Collins, Randall. The Sociology of Philosophies: A Global Theory of Intellectual Change. Cambridge, Mass.: Belknap Press of Harvard University Press, 1998.
Hanushek, Eric A., Paul Peterson, and Ludger Woessman. “Not Just the Problems of Other People’s Children: U.S. Student Performance in Global Perspective.” Boston, MA: Harvard’s Program on Educational Policy and Governance, May 2014.
Hunter, James Davison. To Change the World: The Irony, Tragedy, and Possibility of Christianity in the Late Modern World. New York: Oxford University Press, 2010.
Polanyi, Michael. Personal Knowledge; towards a Post-Critical Philosophy. Chicago: University of Chicago Press, 1958.