The Important Things Standardized Tests Don’t Measure

As my students were taking their seats, Myrna, sitting near my desk, said she’d just read a magazine article about secret societies in high school. What, she asked, did I know about them?

I knew nothing—had never even heard of them—but the matter was interesting enough to quickly engage my eleventh-grade English class, so I let the conversation continue. Someone suggested making it a research project and I told them to have at it.

The school library wasn’t much help, but somebody figured out how to contact the student editor of the school newspaper in a town mentioned in the article and wrote her a letter. She answered, other contacts were made, and kid-to-kid communication began. How did the societies get started? Who joined them? Why? How? Did they create problems? If so, what kind? Were the societies more than just temporary cliques? How were teachers and administrators reacting?

Answers generated more questions. My students thought, wrote, took sides, argued, learned. I mostly watched.

That happened in a class in a semi-rural high school in northeastern Ohio many decades ago. I’d be willing to bet that if any of the participants remember anything about the class, that research project would be it.

I wasn’t smart enough to realize it at the time, but I was seeing a demonstration of something extremely important, that real learning is natural and inherently satisfying. Myrna’s question kicked off genuine learning—self-propelled and successful not because the work was rigorous and the kids had grit, but because it was driven by curiosity, because satisfaction was immediate, because it was real-world rather than theoretical, because it was concrete rather than abstract, because it required initiative and action, and because it was genuinely important, dealing as it did with complex social and psychological issues shaping human behavior.

Even if it leads to dead ends, research—at least for the learner pursuing it—is intellectually productive. It’s also, obviously, non-standard. The skills it develops and the insights it yields aren’t predictable, even to those engaged in it. That’s one of the reasons standardized tests assembled in the office cubicles of Pearson, McGraw-Hill and other test manufacturers can’t do the job that most needs doing. They can’t measure and attach a meaningful number to the quality of original thought.

Arthur Costa, emeritus professor at California State University, summed up the thrust of current test-based “reform” madness: “What was educationally significant and hard to measure has been replaced by what is educationally insignificant and easy to measure. So now we measure how well we taught what isn’t worth learning.”

The truth of that isn’t acknowledged by Jeb Bush, Bill Gates, Lou Gerstner, Arne Duncan and other business leaders and politicians responsible for initiating and perpetuating the standardized, high-stakes testing craziness. They either can’t see or won’t admit the shallowness of their claim that “if you can’t measure it, you can’t manage it.” Challenged, they dismiss those who disagree with them as defenders of the status quo.

Using the scores on standardized tests to shape the life chances of kids, determine the pay and reputations of teachers, gauge the quality of school administrators, establish the worth of neighborhood schools, or as an excuse to hand public schools over to private, profit-taking corporations is, at the very least, irresponsible. If, as it appears, it’s a sneaky scheme to privatize America’s public schools without broad public dialogue, it’s unethical.

Figuring out how to measure original thought isn’t the only challenge test manufacturers need to address. Their tests:

• Provide minimal to no useful feedback to classroom teachers

• Are keyed to a deeply flawed curriculum adopted in 1893

• Lead to neglect of physical conditioning, music, art, and other, non-verbal ways of learning

• Unfairly advantage those who can afford test prep

• Hide problems created by margin-of-error computations in scoring

• Penalize test-takers who think in non-standard ways (which the young frequently do)

• Radically limit teacher ability to adapt to learner differences

• Give control of the curriculum to test manufacturers

• Encourage use of threats, bribes, and other extrinsic motivators

• Use arbitrary, subjectively set pass-fail cut scores

• Produce scores which can be (and sometimes are) manipulated for political purposes

• Assume that what the young will need to know in the future is already known

• Emphasize minimum achievement to the neglect of maximum performance

• Create unreasonable pressures to cheat

• Reduce teacher creativity and the appeal of teaching as a profession

• Are unavoidably biased by social-class, ethnic, regional, and other cultural differences

• Lessen concern for and use of continuous evaluation

• Have no “success in life” predictive power

• Unfairly channel instructional resources to learners at or near the pass-fail “cut score”

• Are open to massive scoring errors with life-changing consequences

• Are at odds with deep-seated American values about individuality and worth

• Create unnecessary stress and negative attitudes toward learning

• Perpetuate the artificial compartmentalization of knowledge by field

• Channel increasing amounts of tax money into corporate coffers instead of classrooms

• Waste the vast, creative potential of human variability

• Block instructional innovations that can’t be evaluated by machine

• Unduly reward mere ability to retrieve secondhand information from memory

• Subtract from available instructional time

• Lend themselves to “gaming”—use of strategies to improve the success-rate of guessing

• Make time—a parameter largely unrelated to ability—a factor in scoring

• Create test fatigue, aversion, and an eventual refusal to take tests seriously

• Undermine the fact that those closest to the work are best-positioned to evaluate it

• Don’t work. The National Academy of Sciences, 2011 report to Congress: The use of standardized tests “has not increased student achievement.”

Most people—including many educators—don’t object to standardized tests but think there are too many, or the stakes shouldn’t be so high, or that some items aren’t grade-level appropriate, etc.

I disagree. I think standardized tests aren’t just a monumental waste of money and time, but are destroying the institution and the profession in myriad unsuspected ways.

Responsibility for evaluating learner performance—all of it—should be returned to those best positioned to do it: Classroom teachers. Period.