Test scores are poor indicators of American talent

saxo-archive

Monday, Dec. 31, 2007 | midnight

Los Angeles

America's voracious appetite for rankings was temporarily sated in early December by the release of scores on the 2006 Program for International Student Assessment. Because it measures the academic performance of schools in 30 industrialized countries around the globe, PISA is scrutinized by critics of public education seeking validation for their position.

They were not disappointed. Finland was the hands-down winner, finishing first in science, trailed by Canada and Japan, and taking a first in math, ahead of South Korea and the Netherlands. The United States was outperformed by 16 other nations in science and by 23 in math.

At a briefing held to discuss the PISA scores, representatives from six national organizations, including the Business Roundtable and the National Governors Association, forecast economic doom. Their remarks were dutifully picked up by the media in news articles and commentary.

Mostly missing from the coverage, however, was a healthy dose of skepticism about the validity of the test results -- and broader context about the quality of American education.

Despite what its title suggests, PISA focuses on applied knowledge, rather than directly on school curriculum. It is administered to students based on their age, not on their grade level.

While this distinction may seem minor, it distorts the results because grade level can include students who range widely in age from country to country. Japan, for example, has 100 percent of its sampled students in the 10th grade. In contrast, the United States has only 61 percent in the 10th grade and a third of the sampled students in the ninth grade or lower.

America's diverse student population and disparate social conditions also are given short shrift. Germany's and Italy's students are nearly 100 percent white; Japan's are nearly 100 percent Asian.

Ninety-nine percent of Japan's population speaks Japanese as their first language. On the flip side, 18 percent of the U.S. population lives in a household where a language other than English is spoken. Single-parent households with children under age 17 account for 33 percent of families in America, compared with less than 10 percent in Japan, Singapore and Korea.

When PISA findings are cited as proof that U.S. schools are deficient compared with those in other developed countries, the indictment doesn't pass muster. In fact, PISA researchers indicate something quite different: “PISA is not an assessment of what young people learned during their previous year at school, or even during their secondary school years. It is an indication of the learning development that has occurred since birth.”

In short, it is a reflection primarily of nonschool factors. As a result, any conclusion about a country's international rank is simplistic and misleading.

Even when critics reluctantly acknowledge these points, they still maintain that the test bodes ill for America's ability to compete in a global economy. Yet there is not a shred of credible evidence to support their charge that PISA scores are linked to competitiveness. In fact, the World Economic Forum's annual “Global Competitiveness Report” showed that the United States -- yet again -- earned the No. 1 spot.

More telling, however, is the rapidly approaching 25th anniversary of “A Nation at Risk.”

That's the famous report that predicted doom for the United States because of the allegedly abysmal state of public education. Yet the schools that were lambasted in 1983 have produced students who are now in their 40s and in positions of increasing authority.

If mediocre test scores had any predictive value, where did these leaders come from? Where did they develop the entrepreneurial skills and knowledge that are the envy of the world?

The truth is that a dynamic economy is more than the sum of test scores. It is built on a culture that nurtures creative thinking, encourages risk taking and rewards success.

Singapore's Education Minister Tharman Shanmugaratnam said it best in a Newsweek interview last year: “We both have meritocracies,” he said. “(America's) is a talent meritocracy; ours is an exam meritocracy. There are some parts of the intellect that we are not able to test well -- like creativity, curiosity, a sense of adventure, ambition. Most of all, America has a culture of learning that challenges conventional wisdom, even if it means challenging authority. These are the areas where Singapore must learn from America.”

Too bad his words are not more widely understood here.

Walt Gardner taught for 28 years in the Los Angeles Unified School District and was a lecturer in the UCLA Graduate School of Education. The preceding commentary originally appeared in The Christian Science Monitor.