The Pitfalls of Voucher “Research”

There’s liars, damn liars, and “research studies.”

I snipped most of this from a comment I posted  on the Choice Remarks group on Facebook.  It would be accurate to say that this is a pro-voucher page. They have a web site too.  If you want to develop talking points to counter the Pro-ESA/Voucher arguments, you will find everything you need to counter here.

This blog post is intended to provide background for those who must deal with rhetoric based on research.

The original poster on the Facebook page is upset that an educator has posted several responses showing how the Oklahoma law would not really be advantageous to many students of color.  Rather than answering the allegations, the link to an article is posted.  The article does not address the allegations. Nor does the study. Since the bill under consideration does not contain non-discrimination requirements for schools and prohibits the use of ESA funds for  transportation or school lunches, the criticisms are valid. They should be addressed.     Since other states laws may differ from our specific bill, the linked article is not really a helpful answer.    But I decided to tackle the article and the research quoted. Not to mention the bizarre use of the word “empirical.”



derived from or guided by experience or experiment.

depending upon experience or observation alone, without using scientific method or theory, especially as in medicine.

provable or verifiable by experience or experiment

Empirical knowledge is based on scientific observation rather than on rational thought.  In the scientific world, empirical evidence gained from controlled observation is a pillar of the scientific method.  Rigorous (sorry, still the best word) use of empirical data is the first step.  Just in case I need to review.. The simple version of the scientific method involves 1. Observation that leads to a 2. Hypothesis.  3.  Experiments are devised to collect more data 4.  Data is reviewed to confirm or deny the hypothesis.

In the modern world there it is more complex than that.  Experiments are reviewed, sometimes by those who do not agree.  Attempts are made to replicate.   Studies/experiments that pass an initial review are published. And that’s it? No, published studies are further reviewed.   There are still incidents of faked data.   The process for a hypothesis to be considered empirically proved is lengthy and difficult.   For everyone’s information, I have yet to determine that any pro-voucher studies or reports could be considered  “empirical”.  The word is used frequently by pro-education choice advocates.  It seems to be an attempt to add credibility with the general public rather than the scientific community.

Here is the article if you are curious.

My first comment in this fray:

Empirical evidence by definition would indicate that an idea has gained acceptance in the scientific community. There is no such broad acceptance.

The use of “empirical” does seem to come most frequently from the Friedman Foundation. This is not reliable since the Friedman Foundation, while named after and founded by a Nobel Laureate economist, has become an apologist for school choice. As such, the Friedman Foundation is no longer a valid source  for unbiased information on school choice.

The Friedman foundation study in 2007, for instance, spoke glowingly of school achievement in Milwaukee voucher schools. However the publication of the most recent data, since all voucher schools were required to give the same tests as the public schools, shows that the public schools are out performing the voucher schools.

There is no empirical evidence. Please stop.

The next comment:
You’re perhaps unaware that the Friedman Foundation published Greg Forster’s synthesis of the research on how school choice programs affect academic performance, state budgets, racial segregation, and more. Forster writes, “These studies were conducted by researchers at Harvard, Princeton, Stanford, Cornell, Georgetown, Johns Hopkins, other universities, the Federal Reserve Bank, the Urban Institute, private research firms, and, yes, also at a couple of think tanks that support school choice. If you dismiss the research showing that school choice works on grounds that it was all conducted by researchers who think school choice works, you might just as well dismiss the research showing that smoking causes cancer on grounds that it was all done by researchers who think smoking causes cancer. Part of the beauty of the scientific method is that when a study follows sound methods, the identity of the researcher becomes irrelevant.”

My response:

The students in the Milwaukee Voucher schools were not tested for many years. It was assumed that parents would hold the schools accountable. (Editorial comment:  This is the rational man theory which is the subject of much debate these days.)   This assumption did not prove out and in 2010 testing voucher students with the same tests as public school students resumed. This is important, because the data in the meta study was mostly older than 2010.  When the testing data for all students was released, the public schools out-performed the voucher schools in nearly every area. And, just as important, the Milwaukee students, public and voucher, still performed below the state average. Even so, the summation of the study admits that there has been no “Milwaukee Miracle” and suggest that the solution would be “universal vouchers.” I find this irrational. The remedy did not work, so we should do more of it. This is a common error in government but usually more associated with liberals.

As with other studies, this (Forster) study still cherry picks data. Patrick Wolfe’s official (and dismal) report (Wolfe et al 2010) on the program in Washington D.C. is not cited, a serious omission. This is an important study since Wolfe supports vouchers. This did not seem to interfere with his objectivity.

I still am somewhat shocked at Forster’s cavalier use of the term “empirical” in his writing.

The Louisiana program is not mentioned in the report. It began in 2010 which seems to be late for inclusion. Events in Louisiana have been informative. One in five Louisiana voucher students attend a school rated F. Half attend schools rated D or F. Government money tends to lead to governmental control. This has proceeded slowly in Milwaukee, slowly but surely. Control has moved faster in Louisiana. Louisiana schools must submit an application and an annual report that includes the racial makeup of the school. Religious schools must “state the religion catered to” and list the religion of each faculty member, among other things.

Eventually, if private schools take public money they increasingly become public schools. Anyone who examines the increase in political correctness in private universities who accept federal funds will recognize this principle. Given the state of the court system in this country this seems like a huge threat to “private” schools who take voucher money over time.  (Editorial note: It does not require high level cynicism to foresee this, even though it isn’t “empirical.)

Forster includes an analysis of the financial impact on “taxpayers.” The issue is not the financial impact on taxpayers. The issue is the draining of financial resources from public schools. One could argue for or against this idea, but a careful reading of Forster reveals that his study has nothing to say about this.

Finally, a search of Forster’s study includes no mention of the financial and academic corruption that have plagued voucher programs.

I will state again, it is not possible to get a balanced look at the success of voucher schools based on a study funded by a pro-voucher organization. The data is useful, but must be considered in the larger picture. And the larger picture would suggest that it would be wise to see if other states can eventually make vouchers work.

Conservative philosophy would suggest that starting a government giveaway program that requires a new state bureaucracy with little oversight would be a bad idea. (Rational thinking, but not empirical, sadly.)

In the mean time, we have a great program in Oklahoma that allows students to transfer to another public school. There are some rules, but basically it is fairly simple. Parents must provide transportation (same as charters and vouchers) but 100% of the money follows the student to the new school. Schools cannot discriminate. They either take all transfers or not. This program is under publicized, but many parents take advantage every year.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s