What We Know Now (and How It Doesn’t Matter)

Randy Olson’s Flock of Dodos (2006) explores the evolution and Intelligent Design (ID) debate that represents the newest attack on teaching evolution in U.S. public schools. The documentary is engaging, enlightening, and nearly too fair considering Olson admits upfront that he stands with scientists who support evolution as credible science and reject ID as something outside the realm of science.

Olson’s film, however, offers a powerful message that rises above the evolution debate. Particularly in the scenes depicting scientists discussing (during a poker game) why evolution remains a target of political and public interests, the documentary shows that evidence-based expertise often fails against clear and compelling messages (such as “teach the controversy”)—even when those clear and compelling messages are inaccurate.

In other words, ID advocacy has often won in the courts of political and public opinion despite having no credibility within the discipline it claims to inform—evolutionary biology.

With that sobering reality in mind, please identify what XYZ represents in the following statement about “What We Know Now”:

Is there a bottom line to all of this? If there is one, it would appear to be this: Despite media coverage, which has been exceedingly selective and misrepresentative, and despite the anecdotal meanderings of politicians, community members, educators, board members, parents, and students, XYZ have not been effective in achieving the outcomes they were assumed to aid….

This analysis is addressing school uniform policies, conducted by sociologist David L. Brunsma who examined evidence on school uniform effectiveness (did school uniform policies achieve stated goals of those policies) “from a variety of data gathered during eight years of rigorous research into this issue.”

This comprehensive analysis of research from Brunsma replicates the message in Flock of Dodos—political, public, and media messaging continues to trump evidence in the education reform debate. Making that reality more troubling is that a central element of No Child Left Behind was a call to usher in an era of scientifically based education research. As Sasha Zucker notes in a 2004 policy report for Pearson, “A significant aspect of the No Child Left Behind Act of 2001 (NCLB) is the use of the phrase ‘scientifically based research’ well over 100 times throughout the text of the law.”

Brunsma’s conclusion about school uniform policies, I regret to note, is not an outlier in education reform but a typical representation of education reform policy. Let’s consider what we know now about the major education reform agendas currently impacting out schools:

• What we know now about grade retention: Grade retention is growing in popularity across the U.S., represented by accountability policies in Florida. But grade retention has been shown by four decades of research not to achieve the goals advocates claim, and to cause harm.
• What we know now about charter schools: Despite the increased support and funding for charter schools, “charterness” has not been shown to be a determining factor in school quality (when compared to traditional public schools [TPS]), charter schools have produced a range of outcomes essentially indistinguishable from TPS, but charter schools have increased segregation (by class and race) as well as underserved English language learners and special needs students (see annotated research here).
• What we know now about school choice (and competition): Decades of a variety of commitments to school choice (notably vouchers) have resulted in a growing body of evidence that school choice fails to achieve the goals of its proponents (see a critical analysis of choice here). Choice, however, has been associated, like charter schools, with shuffling populations of students and increasing segregation. More broadly, the research on competition shows that it causes harm, and not the positive outcomes choice advocates claim.
• What we know now about value added methods (VAM) of teacher evaluation: Although it is fair to say that the jury is still out on VAM, even advocates for exploring the potential for VAM have expressed caution about using it in high-stakes policies (see cautious considerations of VAM validity and reliability). Broadly, high-stakes implementation of VAM is certainly premature, and likely a significant waste of time and money better spent on problems more pressing and clearly defined.
• What we know now about teacher quality: Teacher quality matters, but teacher quality is dwarfed by factors outside of school and outside the control of schools. The real teacher quality problem in schools is teacher assignment since impoverished students, African American students, Latina/o students, English language learners, and special needs students are disproportionately assigned to inexperienced and un-/under-certified teachers.
• What we know now about merit pay: Simply stated, merit pay doesn’t work (and it may often have powerful negative consequences). It doesn’t work in education (see Pink also), but the business world has recognized that as well.
• What we know now about Teach for America (TFA): The growing research base on TFA reveals a mixed picture, but it also shows that TFA advocacy is misleading (see also “three biggest lies”). Further TFA contributes negatively to some central problems public education faces: teacher attrition/turnover and inequitable teacher assignments (high-poverty and minority students being assigned disproportionately new and uncertified teachers).
• What we know now about the SAT: The SAT remains a weaker predictor of freshman college success than GPA and also does not contribute positively to the public perception of school quality. SAT-prep classes also create a drain on school time and resources that could be better used addressing other needs. As well, the SAT remains race, class, and gender biased*.
• What we know now about accountability, standards, and high-stakes testing: After thirty years of accountability, standards, and high-stakes testing in 50 separate state experiments, the research base is clear: “the absence or presence of rigorous or national standards says nothing about equity, educational quality, or the provision of adequate educational services, there is no reason to expect CCSS or any other standards initiative to be an effective educational reform by itself” (Mathis, 2012).
• What we know now about miracle schools: Virtually every school designated “miracle” by advocates or the media (Texas miracle, Chicago miracle, Harlem miracle, Florida miracle, etc.) has been debunked by close analysis. Claims that some high-poverty schools excel (and thus all should excel) has also been exposed as misleading: “only 1.1 percent of high-poverty schools were identified as ‘high flyers.’”
• What we know now about education as a social change agent: Simply stated: “you are better off being born rich regardless of whether you go to college than being born poor and getting a college degree” (Bruenig, 2013, based on data from “Pursuing the American Dream,” Pew Charitable Trusts).

Well into the second decade of the twenty-first century, then, education reform continues a failed tradition of honoring messaging over evidence. Neither the claims made about educational failures, nor the solutions for education reform policy today are supported by large bodies of compelling research.

As the fate of NCLB continues to be debated, the evidence shows not only that NCLB has failed its stated goals, but also that politicians, the media, and the public have failed to embrace the one element of the legislation that held the most promise—scientifically based research—suggesting that dodos may in fact not be extinct.

* Santelices, M. V., & Wilson, M. (2010, Spring). Unfair treatment? The case of Freedle, the SAT, and the standardization approach to differential item functioning. Harvard Educational Review, 80(1), 106-133.; Spelke, E. S. (2005, December). Sex differences in intrinsic aptitude for mathematics and science? American Psychologist, 60(9), 950-958; See page 4 for 2012 SAT data: https://media.collegeboard.com/digitalServices/pdf/research/TotalGroup-2012.pdf.