A Student at Williamsburg Collegiate Charter School in NYC
Stanford University professor Caroline Hoxby is finally releasing the results of her multiyear study of New York City charter schools. It has a big finding: they work. Hoxby’s paper shows that students who are randomly selected by lottery for one of the city’s charter schools in kindergarten and then attend that school through the end of middle school will be the equivalent of their wealthier, suburban counterparts in math and much closer in reading than those students who were not selected. Charter schools in New York City are actually closing the “achievement gap:”
By the third grade, according to the study, the average charter school student was 5.3 points ahead on state exams in English compared with students who were not admitted to the charter schools. In math, the students were 5.8 points ahead. Most tests are scored on a scale of roughly 475 to 800.
This comes in contrast to findings about other lottery schools, such as those in Freakonomics, the popular economics book by Chicago economist Steven Levitt and journalist Stephen Dubner, which found instead the following about the better Chicago public schools that required a lottery to attend:
[T]he students who won the lottery and went to a “better” school did no better than equivalent students who lost the lottery and were left behind.
Hoxby’s results show that at least some charter schools work (note: this is not meant to argue that all charter schools work, as other studies have found very mixed bag results for different schools; setting up effective rules for creating and evaluating charter school performance is clearly important), that their positive effects aren’t just the result of selection bias, the idea that the best students want to go to the charter schools and thus any better results are just due to the initial differences between the student pools (the practice of schools taking only those students that have parents who are involved enough to apply has been termed “creaming” by some), not due to the schools themselves. Hoxby acknowledges the need for more research on the outcomes of this improvement, but the underlying result is clear: there is room for improvement in education and effective charter schools are going to need to be part of that improvement.
{ 6 comments… read them below or add one }
Caroline Hoxby, who conducted this so-called “study,” is not an impartial academic researcher. She’s a longtime, high-profile proponent of free-market “solutions” and privatization. Her work should not be treated like credible academic research; it’s advocacy — or propaganda, if you will.
I’m really shocked that the mainstream press is not even including disclaimers to this effect in its massive hyping of this so-called study. That truly violates media standards and ethics, and misleads the reader.
Here an analysis of the flawed study itself, by a New York blogger. But to me it’s also a huge issue that the press has simply abandoned its standards and ethics by reporting on this propaganda as if it were credible academic research.
http://morethoughtful.blogspot.com/2009/09/what-is-gold-standard.html
The critiques of the study methodology are valid and concerning. In particular, I agree that peer effects could be very problematic since the public schools are full of kids that did not apply for a lottery program and “matching” is a well-known phenomenon. The size of the measured effect of the charter schools is huge, however, which I do not believe is solely the result of matching.
However, I disagree strongly with the critique that Hoxby’s political leanings should be used as a major caveat when describing the study. Yes, she is a proponent of “free-market” education; however, unless she has actively messed with her data (and I haven’t seen any accusations of that here), then her political leanings are irrelevant. No matter how much she might want the data to tell her that public schools are the devil’s tool, the data will show what the data will show. If this is what they revealed, then I’m sure she’s pleased with the outcome, but I don’t believe that her political affiliation should really matter when presenting the data. If the data are strong enough, then they speak for themselves; if they aren’t, then I believe that this study wouldn’t have been published by the NBER (and her co-authors wouldn’t have signed off on it as well).
Thanks for posting about this. This story was really getting talked up yesterday, well outside of Stanford.
We’re not just talking any old political views that Hoxby happens to hold, Dr. Reid. She has achieved her renown as an advocate of “free-market solutions” in education, including charter schools — and those “solutions” are highly controversial among those informed enough to look beyond the propaganda.
If the Republicans’ pollster releases poll results about the upcoming election, it’s always reported as coming from the Republicans’ pollster. It’s self-evident that that appearance of possible bias is a crucial caveat.
The funding of any study or poll is also a perennial issue. It would be inconsistent to suddenly decree that it’s irrelevant that someone with an aggressive agenda promoting charter schools and the rest of the Friedmanesque “free-market solutions” ran the study that just happened to come out showing shining results for charter schools. Too, this comes in the wake of numerous studies showing the opposite — that charter schools do NOT outperform public schools — including a June 2009 report from Stanford’s Center for Research on Education Outcomes.
It’s not intellectually honest to try to excuse that away. Obviously Hoxby’s longtime status as a charter-school advocate presents the appearance of conflict of interest, and even if her study was 100% pure and honest, the APPEARANCE still calls for a disclaimer. The methodology issues combined with the appearance of conflict of interest raise still more red flags.
It’s a mistake not to include the disclaimer, it’s a mistake to accept the study as though it were impartial research, and it’s always a mistake not to ask enough questions — especially where the mightily funded, propaganda-fueled charter school world is concerned. The mainstream media fell for it, but an academic should not.
Caroline, I understand your concern; perhaps it wouldn’t hurt to mention Hoxby’s beliefs. However, I still have a question for you: are you accusing Hoxby of manipulating her data to get this result? If you are not making that accusation, then I disagree that it’s NECESSARY to announce that Hoxby has a “pro-market solution” political leaning; as I said before the data speak for themselves and if they did not say what Hoxby claims, then she would not have published the claims that she has. Her “Friedmanesque” tendencies do not exercise some magnetic pull on the data and thus are not directly relevant to the outcomes of the study, which is what I discussed in my post.
If you are making that accusation, then I guess we have more fundamental issues; I personally believe that the NBER would not sign off on a study that had clear underlying methodological funny business and I doubt that Hoxby and her two co-authors would all be willing to risk their academic careers by manipulating the data, especially since this study is likely to provoke attempts to disprove it. If I were going to make up data, I wouldn’t do it in a contentious area like the discussion of charter schools (as you said, there have been a number of studies that shown the opposite result); I would do it in a quieter field where my chance of discovery was much lower.
Also, I challenge you to find almost any researcher in the field of economics, or, even more precisely, in the field of research on charter schools that has no belief about which is more effective. Most researchers have personal beliefs about the effectiveness of charter schools and begin their research with the intent of proving or disproving a particular hypothesis (which requires taking some sort of a stance). Hoxby may be a particularly prominent example, but I do not view her as some exception to normally “impartial” research. In contentious, politically charged fields such as this, everyone has some preliminary belief coming in. Some may be less ideologically entrenched than others (for example, perhaps its fair to question whether Hoxby would have worked to trumpet this study if the results had come out saying that public schools were superior), but to argue that Hoxby is a political operative in an otherwise “impartial” academic world is, I believe, intellectually disingenuous.
Yes, everyone has opinions. But Hoxby’s primary public identity is as an outspoken advocate for “free-market solutions” in education — meaning charter schools, vouchers and privatization. That presents the appearance of likely conflict of interest. And that calls for disclosure, which an ethicist would tell you IS necessary.
{ 2 trackbacks }