More snake oil 'research' from Billionaire Bill... 'Only slightly better than a coin toss'... Gates's 'value added' study debunked just as states, districts, and Duncan promote it...
Just at a time when school districts and states across the USA are being told that they can safely use so-called "value-added" ways of measuring student "performance" as a measure of teacher effectiveness, a review of one major study touted by the Gates Foundation shows that the Gates study's data point in the opposite direction from the conclusions that the study supposedly shows. While this is not the first time that data supposedly produced by scholars paid by Gates has been challenged, it is the first time such a challenge lands in the middle of a national drive to base major policies on what the "research" paid for by Gates supposedly shows.
The key to Gates's power lies not only in his money, but in his ability to buy favors from labor leaders like Randi Weingarten of the American Federation of Teachers. In July 2010, Gates (above) was invited by Weingarten to address the AFT national convention, which was held in Seattle, Washington, a town that has one of the most militant union and working class traditions in the USA. Ignoring Gates's role in union busting in Seattle and across the USA, Weingarten opposed rank-and-file opposition to Gates' AFT convention speech and had her supporters in the audience lead a standing ovation for Gates when the speech was done. Substance photo by George N. Schmidt.During the past six months, the release of "value-added" data in California (Los Angeles) led to a major controversy (and the suicide of one teacher, who had devoted his career to troubled students and was therefore listed as one of the "worst" teachers in Los Angeles), and as recently as this week, a New York court allowed New York City's daily newspapers to gain possession of New York's supposed grading of teachers (using "value-added") under the state's Freedom of Information laws, despite union objections.
According to a press release dated January 13, 2011, by the Think Tank Review Project, the facts cited in the Gates report undermine the conclusions drawn in the Gates report. The Think Tank Review project states the Professor Jesse Rothstein reviewed "Learning About Teaching," produced as part of the Bill & Melinda Gates Foundation’s “Measures of Effective Teaching” (MET) Project, for the Think Twice think tank review project. The review is published by the National Education Policy Center, housed at the University of Colorado at Boulder School of Education.
PRESS RELEASE FROM THE UNIVERSITY OF COLORADO BELOW HERE:
"Gates Report Touting "Value-Added" Reached Wrong Conclusion... Re-examination of results finds that the data undermine calls for the use of value-added models for teacher evaluations
http://nepc.colorado.edu
BOULDER, CO (January 13, 2011) – A study released last month by the Gates Foundation has been touted as “some of the strongest evidence to date of the validity of ‘value-added’ analysis,” showing that “[t]eachers' effectiveness can be reliably estimated by gauging their students' progress on standardized tests” (http://tinyurl.com/4zy5zyh) .
However, according to professor Jesse Rothstein, an economist at the University of California at Berkeley, the analyses in the report do not support its conclusions. “Interpreted correctly,” he explains, they actually “undermine rather than validate value-added-based approaches to teacher evaluation.”
Rothstein reviewed Learning About Teaching, produced as part of the Bill & Melinda Gates Foundation’s “Measures of Effective Teaching” (MET) Project, for the Think Twice think tank review project. The review is published by the National Education Policy Center, housed at the University of Colorado at Boulder School of Education.
During every speech, Bill Gates (above, during his speech at the AFT convention in July 2010 in Seattle) reads from a carefully prepared script, tailoring the "facts" that he cites for the audience he is facing. His union busting work was toned done during the AFT speech, but had been in evidence before and after his July appearance at AFT in a Chicago speech on charter schools and in the subsequent funding he gave in late 2010 to "Stand for Children" and the Illinois "Performance Counts" legislation (which failed in Springfield despite enormous financial pressure from Gates and his avatars in Stand for Children and Advance Illinois). Substance photo by George N. Schmidt.Rothstein, who in 2009-10 served as Senior Economist for the Council of Economic Advisers and as Chief Economist at the U.S. Department of Labor, has conducted research on the appropriate uses of student test score data, including the use of student achievement records to assess teacher quality.
The MET report uses data from six major urban school districts to, among other things, compare two different value-added scores for teachers: one computed from official state tests, and another from a test designed to measure higher-order, conceptual understanding. Because neither test maps perfectly to the curriculum, substantially divergent results from the two would suggest that neither is likely capturing a teacher’s true effectiveness across the whole intended curriculum. By contrast, if value-added scores from the two tests line up closely with each other, that would increase our confidence that a third test, aligned with the full curriculum teachers are meant to cover, would also yield similar results.
The MET report considered this exact issue and concluded that “Teachers with high value-added on state tests tend to promote deeper conceptual understanding as well.” But what does “tend to” really mean? Professor Rothstein’s reanalysis of the MET report’s results found that over forty percent of those whose state exam scores place them in the bottom quarter of effectiveness are in the top half on the alternative assessment. “In other words,” he explains, “teacher evaluations based on observed state test outcomes are only slightly better than coin tosses at identifying teachers whose students perform unusually well or badly on assessments of conceptual understanding. This result, underplayed in the MET report, reinforces a number of serious concerns that have been raised about the use of VAMs for teacher evaluations.”
Put another way, “many teachers whose value-added for one test is low are in fact quite effective when judged by the other,” indicating “that a teacher’s value-added for state tests does a poor job of identifying teachers who are effective in a broader sense,” Rothstein writes. “A teacher who focuses on important, demanding skills and knowledge that are not tested may be misidentified as ineffective, while a fairly weak teacher who narrows her focus to the state test may be erroneously praised as effective.” If those value-added results were to be used for teacher retention decisions, students will be deprived of some of their most effective teachers.
The report’s misinterpretation of the study’s data is unfortunate. As Rothstein notes, the MET project is “assembling an unprecedented database of teacher practice measures that promises to greatly improve our understanding of teacher performance,” and which may yet offer valuable information on teacher evaluation. However, the new report’s “analyses do not support the report’s conclusions,” he concludes. The true guidance the study provides, in fact, “points in the opposite direction from that indicated by its poorly-supported conclusions” and indicates that value-added scores are unlikely to be useful measures of teacher effectiveness.
Find Jesse Rothstein’s review on the NEPC website at:
http://nepc.colorado.edu/thinktank/review-learning-about-teaching.
Find Learning About Teaching: Initial Findings from the Measures of Effective Teaching Project, by Thomas J. Kane and Steven Cantrell, on the web at:
http://www.metproject.org/downloads/met-framing-paper.pdf.
The Think Twice think tank review project (http://thinktankreview.org), a project of the National Education Policy Center, provides the public, policy makers, and the press with timely, academically sound, reviews of selected think tank publications. The project is made possible in part by the generous support of the Great Lakes Center for Education Research and Practice.
The mission of the National Education Policy Center is to produce and disseminate high-quality, peer-reviewed research to inform education policy discussions. We are guided by the belief that the democratic governance of public education is strengthened when policies are based on sound evidence. For more information on NEPC, please visit http://nepc.colorado.edu .
This review is also found on the GLC website at http://www.greatlakescenter.org/
CONTACT: Jesse Rothstein
(510) 643-8561 rothstein@berkeley.edu William Mathis, NEPC
(802) 383-0058
william.mathis@colorado.edu
