Monday, August 03, 2009

Dumb Money or Dumb Coverage?

According to Newsweek, U.S. educators should be learning from the Canadian example. "According to an April report by McKinsey, the United States' GDP would have been 9 to 16 percent—or $1.3 trillion to $2.3 trillion—higher in 2008 had U.S. high-school graduates attained the average skills of their peers in Canada, Finland, or South Korea."

Alexander Russo writes,
This Newsweek article (Dumb Money) tells us that it's not just the US where use of data to evaluate teachers is controversial, that most of what is popular to do in education -- class size reduction and school modernization, for example -- is least effective, and that Toronto has eliminated the achievement gap has allegedly been eliminated despite a substantial immigrant population. All aboard!
The Newsweek article cited here is misleading, and this description adds to the misrepresentation.

The article does not assert that Ontario is using data to evaluate teachers. Rather, it describes the province's use of achievement-based testing, and suggests, deceptively, that using this data to rank teachers would be "the logical next step".

While the article is correct to tout the achievement of the Ontario school system, it misrepresents the basis of this success as being due to narrow instructional reforms, rather than to society-wide measures to promote inclusion and achievement in a diverse population, such as early literacy programs, child care and community support.

It is also worth noting that, despite what is asserted in this post, Ontario continues to spend a lot of money to reduce class sizes and its schools are among the more technologically advanced in the world and continue to invest significantly in computers and infrastructure.

When we look at the instruction itself, we find factors not even mentioned (and very unpopular in some U.S. political communities) such as anti-racism and diversity education, active living, media literacy and critical thinking. "We want our students to think critically, feel deeply and act wisely."

Moving beyond Russo's summarization, a look at the Newsweek article reveals substantial bias, unsubstantiated assertions, and blatant misrepresentatuons in almost every paragraph.

- Newsweek writes that in Germany, "new school spending is being used to renovate buildings [which is] unlikely to have much effect on the quality of German graduates." This assertion, from out of the blue, doesn't accord with research linking environment and educational outcomes. Ontario is spending millions to upgrade school buildings.

- Newsweek writes, "Gordon Brown is pushing for more PCs and Web access in schools—another policy that's popular but considered irrelevant by educators." What educators? There is little evidence that educators consider PCs and web access "irrelevant" and a great deal of evidence to the contrary.

- Newsweek writes, "schools were not using the stimulus money to boost student achievement, as promised by Duncan, but to fund their general budgets." The audit is unreferenced, but even supposing this description is accurate, there is no evidence to show that spending money in their "general budgets" will not boost student achievement, since (after all) building student achievement is their core purpose, and what their general budget is intended to do.

- Newsweek writes, "governments are using money to help build new world-class universities—projects that a World Bank study in July warned risk bleeding resources away from more desperately needed areas." The study, by Jamil Salmi, actually promotes the opposite view. "There is growing recognition, in both industrial and developing countries, of the need to establish one or more world-class universities that can compete effectively with the best of the best around the world."

(These preceding four items are all from the same paragraph!)

After setting us up, Newsweek states its thesis in the fourth paragraph: "The biggest error governments are making is to blindly push for more and better everything at all levels of education: more teachers, flashier facilities, more technology in the classroom, and more elite universities." On one level this makes sense. We should spend wisely, rather than randomly. But there is a subtext opposing more education spending generally, as we see in almost the next sentence.

Newsweek writes, "Kids don't necessarily learn more if they sit in smaller classrooms, in more modern and better-equipped schools, or even if their teachers are better-paid (as opposed to just better)." The weasel word in this paragraph is "necessarily". Because while simply spending money on education is not sufficient, it is clearly and demonstrably necessary. Paying teachers more won't by itself get you better teachers, but you won't get better teachers unless you pay them more.

Along the same lines, Newsweek writes, "According to Ludger Woessmann of the IFO Institute, merely raising per-student spending has zero effect on achievement." This study is again uncited, and could be any of a number of Woessmann has written, which generally argue that "efficiency and equity can be enhanced by output-oriented reforms properly designed to each stage, where the state generally sets a regulatory framework that ensures accountability and funding and uses the forces of choice and competition to deliver best results." But such unfunded outcomes-based mandates, such as No Child Left Behind, are demonstrably ineffective. Again, the funding may not be sufficient, but it is necessary.

Newsweek writes, "The United States, France, and Germany have upped spending significantly in past decades only to see performance stagnate, while countries like Sweden and Finland have boosted quality through structural reform." First of all, this misrepresents spending in Finland and Sweden. "In Finland, between 1995 and 2005, spending on primary and secondary education increased by 38% while enrolments rose by 13%" And secondly, France and Germany may have "upped spending", but they still lag behind countries such as Sweden and Finland, which spend more.

Newsweek writes, "Peer Ederer, who heads the Lisbon Council's Human Capital Index Project, says the biggest problems university systems face today are high dropout rates and too few skills taught for later careers—problems most current spending proposals do little to address." Again this is unsourced. I am unable to find this specific assertion. It seems to be a misrepresentation (compare, for example, the table on p. 36 of this report from the Lisbon Council which lists completely different "challenges" faced by tertiary education). And again, it's not clear he advocates the sort of argument Newsweek is offering. For example, he argues "The environment where young people are learning and living is completely different from the one we lived in. They need to have the 'big picture in mind'. They need new skills that are not traditionally taught in schools – managing their learning, problem-solving, social, civic and entrepreneurial skills."

Newsweek writes, "Salmi of the World Bank says the huge resources now being spent to create elite schools would be better used for expanding and improving engineering programs at unflashy polytechnics, for example." This appears to be a fair representation of this World Bank press release, but it is not that Salmi discourages world class institutions, only that they should not be pursued in isolation prior to more fundamental institutions, saying only that "The truth of the matter is that not every nation needs comprehensive world-class universities, at least not while more fundamental tertiary education needs are not being met," and this only at the end of a lengthy report describing how best to create, fund and promote world class institutions.

Newsweek reports that "President Obama's advisers seem to be listening closely; in July, they announced a $12 billion boost for two-year community colleges" but then complains "most of the U.S. stimulus money has gone for helping all schools fill in general budget shortfalls." Of course, nobody - and especially not Jamil Salmi - would say that the bulk of U.S. spending should be on colleges. The United States is not a developing nation - to whom Salmi addresses his remarks - and the spending is not all, or even proportionally, on "elite institutions", which Salmi is addressing specifically.

Newsweek writes, "Washington's long history of failed attempts to improve inner-city schools shows that raising the level of the 20 percent or so of U.S. students who leave school functionally illiterate defies straightforward fixes." Fair enough. It adds, "If there is one thing most experts agree on, however, it's that the earlier the system intervenes, the better." This appears to be true. But, one wonders, has the U.S. system simply failed to intervene early enough in the past? Projects such as the Early Childhood Outcomes (ECO) Center (and many other programs) are a testament to a long history of early interventions in the U.S.

We can agree with Newsweek that early intervention is important. But it is not simply the question of whether or not you intervene, but how. Which is why it is improtant to note just what Newsweek learns from Ontario,, Canada, "One country that has systematically pushed this maxim is Canada [and] which saw a big surge in immigration in the 1990s." Newsweek notes, "Toronto has been able to erase the achievement difference between migrants and natives—in marked contrast to cities in Germany and France, where the gap has been widening. It's one reason why Canadian students get some of the highest scores on international achievement tests.

It should be noted that the "big surge of immigration in the 1990s" is pure myth. As can be clearly seen here, while Ontario had an immigration spike in the 1980s, it dropped to normal levels in the 1990s. And immigration, which amounts to no more than 100 thousand in a population of more than 10 million in any given year, does not distend the natural demographics of the province. Ontario has been, for more than a generation (more than two, actually), a diverse, multicultural province.

Newsweek reports, "Toronto has been able to erase the achievement difference between migrants and natives—in marked contrast to cities in Germany and France, where the gap has been widening. It's one reason why Canadian students get some of the highest scores on international achievement tests." It may be one reason but it is very unlikely to be the major reason. Most immigrants are adults, which means the impact of immigration on the school system is minimal. Children born after their parents arrive are not immigrants. As Canadian citizens, they begin to enjoy the benefits of Canadian health care, social welfare, early childhood programs, and the rest. Canada, in other words, is not addressing immigration per se. It is addressing poverty.

Newsweek asks, "why haven't more countries already figured it out?" It suggests, "One answer is that determining what kind of education spending is most economically and socially effective—as opposed to merely popular—requires the kind of close self-scrutiny that is foreign to most school systems." The suggestion, implied, is that rational education spending will not be the most popular. It is not clear that this is true. But more to the point, the suggestion is that scrutiny of school systems will identify value. This is not the case. Rather, if anything will show where educational money is best spent, it will be, first, the study of the students themselves, and second, the study of society as a whole.

Newsweek's own argument shows as much. Citing the benefits of early intervention, James Heckman argues "a quarter of this gain goes to the kids, who he says are more likely to get better grades, stay in school, get a diploma, go on to college, and enjoy higher lifetime earnings. But 75 percent of the 'profit' accrues to society, in the form of lower crime rates, fewer welfare payments, and greater economic productivity, as well as better integration of minorities and immigrants. "

This is important, because it will identify interventions, especially early interventions, which have nothing to do with the school system. For example, Canada has been running a series of "read to your child" advertisements for a number of years, encouraging parents of infants to read to children, sing to children, and play with children. And this is a program Ontario continues to encourage. No amount of measurement of the school system will detect the impact of such a program, because it's too narrow.

To be fair, the article accepts the need to evaluate relative achievement, not standards-based outcomes. "When the emphasis of testing shifts from outright achievement to relative gains, the results can be stunning," it writes. The idea is to identify programs that give the "biggest boost" to individual achievement. Fair enough, but now Newsweek begins to assume conclusions that go beyond the bounds of the evidence.

Newsweek writes, "The logical next step would be to identify which individual teachers give their students the greatest boost. 'Forget class size, curriculums, budgets—the most effective policy is good teachers,' says Eric Hanushek, an education specialist at Stanford University's Hoover Institution." This allows it to suggest that money is not the factor that will improve learning; siomply identifying better practices will do the job. "The OECD study finds per-student spending has little relation to student performance." the study is not cited, but is probably this one. And it doesn't "find" this as a result, it merely asserts it. "Many expensive attempts around the world to improve schooling have failed to yield actual improvements in student achievement." What we do not find, however, is a case where impressive gains have been made without money.

Ontario's success, in the Newsweek article, will now be explained with reference to Finland and South Korea, which "have the world's highest-achieving high-school students—thanks in large part to a focus on teachers: improving their selection, upgrading their training, and concentrating on how they can best help individual students keep up." But it's not the whole story. Early intervention, for example, is crucial: "Finnish children have access to very high-quality, affordable child care that meets most of the standards for what we in the United States would call preschool. " Also, money is allocated for early childhood care, addressing issues of poverty. "Stay-at-home parents can collect a 'home care allowance' of 294 euros per month." Other factors come into play, according to this University of Helsinki report: "The explanations for success given by the authors can be classified into three groups: Teacher and teacher education, school and curriculum, and other factors, like the use of ICT and a developmental project." Just the sorts of investments this Newsweek article is, by and large, discouraging.

In the last few paragraphs, Newsweek addresses the politics of reform with a series of loaded and misrepresentative points.

- Newsweek writes, "
Spending on elite or middle-class schools is much more popular than redirecting scarce funds to migrants and the underclass." But no Ontario or Finish schools "redirect" funds from one part of the system to the other. Rather, in both cases, the system is integrated - funding for poor people and immigrants does not involve redirecting funds from the special private elite system, because there isn't a special private elite system.

- Newsweek writes, "Parents and teachers' unions also love mostly irrelevant policies like lowering class size and hiring more teachers." Yet, and noted above, these are policies currently (and historically) pursued by school boards in Ontario, and the whole point of the article is to cite Ontario's success. So why is it that smaller class sizes are "irrelevant"?

- Newsweek writes, "One reason, beyond their intuitive appeal, is that the benefits of such measures, such as nicer or less crowded classrooms, are often immediately apparent. Focusing on early intervention and preschool, by contrast, may not deliver dividends for 20 years." here it treats classroom investments and early childhood investments as a zero sum game. But, of course, they are not; governments can invest in both. And it seems unlikely that there is any particular opposition to early intervention or preschool programs (though there is opposition to some sort of strawman 'universal preschool' program).

- Newsweek writes, "Teachers' unions also often object to any disruptions in the current system." This unwarranted swipe at teachers' unions is unsubstantiated. Teachers' unions are often at the forefront of change, lobbying for such things as smaller classrooms and improved school funding. Obviously, teachers' unions oppose particular types of changes. Such as:

- Newsweek writes, "When the British Education Ministry first tried to introduce value-added testing, it couldn't overcome stiff resistance from teachers and schools until Parliament made it illegal to use the results to determine salaries or budgets." This may be so, but it does not follow that this resistance was unreasonable. Nothing in the article, and nothing in the Ontario experience, points to any improvement in educational outcomes obtained by fixing school budgets and salaries to test results (and the experience with NCLB suggests the opposite).

- Newsweek writes, "it is taboo to compare individual schools' or teachers' performance... without such testing and evaluation, education policy will remain vague and approximate, based on folk wisdom and political expediency rather than evidence and facts." This is absurd. A great deal of data is available that does not compare individual schools' or teachers' performance. data, for example, that measures student achievement.

- Newsweek writes, "most of what governments around the world are calling 'education investments' still fails to meet the effectiveness test." This statement is almost certainly false. Either way, it is impossible to prove. But it seems unlikely that most education spending does not actually lead to education. Alternatively, if the claim is that most educational spending does not obtain ther best possible outcome, this is a challenge that could be met by very few spending programs, not the least because it is impossible to define what the 'best outcome' would be.

- Newsweek writes, "That's despite the fact that redirecting spending toward society's underachievers is one reform that would pay for itself many times over." What needs to be shown is why an investment in (so-called) "underachievers" ought to result from redirecting spending. Other educational spending, even if it does not fund the best education possible, nonetheless funds education, and it would be irresponsible to cease funding at one level in order to begin funding at another.

Although this article purports to offer lessons from successful school systems, what it in fact does is to substitute its own perspective for the facts at various points throughout. This results in a sort of recommendation that resembles pretty much exactly what has been done in recent years: student testing being used to allocate funds to teachers and schools. And it results in recommendations that are actually contrary to what schools in Canada (and elsewhere) actually do, which is to better fund the entire public system, to pay teachers more, to invest in smaller classes, to invest in better schools and in learning technology, to emphasize literacy and critical thinking, to foster an appreciation of diversity, and to invest in social programs and address issues of poverty outside the school.

4 comments:

  1. Thank you for this deconstructing. It's an easy exercise choosing success education models abroad for justifying internal policy. How many western people knows exactly the model of korean schools? How many people knows how many inhabitants has Finland? The Minister of Education in Italy is reintroducing old pedagogical model referring to OECD poor results of italian students, but the real goal is to save money. This year the increased number of students that didn't pass their final exams was considered a success as we came back to merit!

    ReplyDelete
  2. Thanks for taking the time to pick apart this piece of ... writing. Such facile and shifty arguments need to be answered.

    ReplyDelete
  3. In Finland over 96 percent of the teachers are members of the teachers' union and they are evil - not.

    ReplyDelete
  4. Unwarranted swipe at teachers' unions ? Where is your data to back that up ? How many examples do you give and from where ? Why are there rubber rooms for teachers ? I suppose when the Feds recently cut back on welfare benefits so teachers wouldn't be laid off this was good for education. I mean if the kids die due to starvation or other causes related to poverty you will have smaller classroom sizes.

    ReplyDelete

I welcome your comments - I'm really sorry about the moderation, but Google's filters are basically ineffective.