Thank you Ward 5 Education Council for your ongoing dissemination of this important hearing and information. As you continue to remain forefront in the direction of our Ward 5 education, I wish to also add'l information helpful to those parents, teachers, students and community wishing to testify.
Pls.note the attached docs and embedded links -most of which I requested from CM Mendelson's office. One very poignantlink to the upcoming hearing is the NCEF link. This 80 pg. analysis (which is actually a rather short read) contains5 sections apparently relevant to DCPS' upcomingschool closure strategy & decisions. The Exec. Summary, Intro, Research and Methodology, District-Wide Analysis and Findings and Recommendations are those sections which will provide add'l insight to DCPS' rational/justification for more closures. The research for this DC-commissioned analysis was done by IFF. IFF has an interesting pedigree in school reform (most notably in Chicago). On pg. 5 of the analysis, IFF is noted as a "regional nonprofit community development financial institution... hired to provide real estate financing and real estate development to nonprofit entitites... its policies help municipalities w/analysis to improve focus and resource allocation, primarily in school reform efforts.... IFF partnered w/Chicago Public School leaders to evaluate operating and capital proposals from charter school applicants.... IFF's data and analysis has informed such decisions as the re-allocation or sale of vacant buildings....solicitation for charter school applications.... targeted communication to particular neighborhoods of populations regarding school choice options....",
The Albert Shanker blog provides a less publicized analysis of IFF's DC-school analysis. Their link and excerpt are below. The Albert Shanker Institute is a nonprofit organization established in 1998 to honor the life and legacy of the late president of the American Federation of Teachers.
Fundamental Flaws In The IFF Report On D.C. Schools
A new report, commissioned by the District of Columbia Mayor Vincent Gray and conducted by the Chicago-based consulting organization IFF, was supposed to provide guidance on how the District might act and invest strategically in school improvement, including optimizing the distribution of students across schools, many of which are either over- or under-enrolled.
Needless to say, this is a monumental task. Not only does it entail the identification of high- and low-performing schools, but plans for improving them as well. Even the most rigorous efforts to achieve these goals, especially in a large city like D.C., would be to some degree speculative and error-prone.
This is not a rigorous effort. IFF's final report is polished and attractive, with lovely maps and color-coded tables presenting a lot of summary statistics. But there's no emperor underneath those clothes. The report's data and analysis are so deeply flawed that its (rather non-specific) recommendations should not be taken seriously.
The authors' general approach, slightly simplified, is as follows: All schools (DCPS and charter) are ranked, based on four measures: their school- and grade-level current (2011) proficiency rates in math and reading; and their "projected" 2016 proficiency rates in both subjects. The latter is derived from a simple OLS regression that uses each school's prior rate increases (between 2007 and 2011) to predict where it will be five years down the road.
Based on these ranks, schools are sorted into quartiles, with the top-ranked ("high-performing") schools classified as "Tier 1." The core results compare the number of seats in the these "Tier 1" schools with the number of children within 39 neighborhood "clusters" (geographic subdivisions used by the District for other purposes). Reducing these discrepancies is the focus of the report's policy conclusions.
IFF recommends that the District increase the number of students filling "high-performing seats" using a combination of (largely unspecified) investments, closures, moving students into schools under capacity and other strategies.
My first question in reading this report was: Why didn't IFF use longitudinal student-level data and carry out a proper analysis? By relying exclusively on cross-sectional school- and grade-level proficiency rates, IFF essentially guarantees that their results will be little more than tentative, at best (also see this excellentdiscussionby economist Steve Glazerman, who makes some of these same points).
Proficiency rates can be useful, in that they may be easier to understand than average test scores. Nevertheless, they are not well-suited for serious analyses of school performance. They only tell you how many students scored above a certain cutoff point, and this hides a lot of the variation in actual performance. For example, two students – one scoring just above passing and the other scoring a year or two above grade level – would both be coded the same way, as "proficient (or above)." This is a large sacrifice of data.
In addition, while only a small part of IFF's school rankings consist of "growth" measures (a portion of the variation in the "projected" 2016 rates, discussed below), the data they use are cross-sectional, and are therefore not appropriate. They don't follow students over time, which means that the changes are not "growth" at all, and may to a large degree reflect demographic and other shifts in the student population of each school. This problem is especially severe at the grade-level (because samples are smaller), particularly in D.C., where student mobility is exceedingly high (in no small part due to rapid charter proliferation). In other words, proficiency rates may change simply because somewhat different sets of students are being tested every year, not because there has been any real progress.
In short, IFF's reliance on cross-sectional proficiency rates calls its entire analysis into question. These rates are terrible measures of performance, both in any given year and over time, and one can only wonder why a District consultant wouldn't employ better data. Even a more rigorous analysis would have been suspect using these data.
But, in addition to the issue of school- and grade-level proficiency rates, there are serious problems with IFF's primary analytical approach – i.e., the manner in which they sort schools into performance "tiers." As stated above, each school's performance is assessed by an average of four measures: 2011 proficiency rates in math and reading; and projected 2016 proficiency rates in both subjects, which are calculated based on each school's (assumed to be linear) trajectory between 2007 and 2011 (both over the whole school and for grade clusters [K-5, 6-8, 9-12]).
There are many small problems with this scheme (e.g., the loss of data in rankings and sorting into "tiers"), but they're all somewhat superfluous, given the fact that the method IFF uses does not measure the actual effectiveness of each school.*
Even using the best data (which IFF does not), testing results, whether proficiency rates or scores, are in large part due to either random variation or factors outside of schools' control(e.g., students' backgrounds). Half of IFF's school performance measure consists of 2011 proficiency rates in math and English, which are essentially just measures of student background. Unless you control for these factors, these rates can tell you almost nothing about the actual quality of instruction going on in the school. Schools with high rates are not necessarily high-performing schools, and vice-versa.
The other half of IFF's school rankings – the 2016 "projected" rates – aren't much better, because they are simply added to the 2011 rate, and are therefore mostly just absolute performance measures (i.e., severely biased by student characteristics).
For example, let's say we have two schools: One serves mostly higher-income students and has a 2011 proficiency rate of 80 percent; the other serves mostly lower-income students, and has a 2011 rate of 40 percent. Now, let's say IFF projects that both will pick up 20 percentage points by 2016. Putting aside all the flaws in IFF's methods (including the fact that their data are cross-sectional and therefore do not measure "growth" per se), this would imply that both schools are equally effective – students in both schools will make similar "progress" over the next five years.
But IFF's methods don't posit that 20 point increase as their measure. Instead, they add the projected "growth" (20 points) to the 2011 rates, which means that our higher-income school will have a projected 2016 rate of 100 percent, while our lower-income school will come in at 60 percent. The first school will appear to be far better, even though there was actually no difference in the schools' effectiveness in boosting test scores (which is generally considered the appropriate gauge of school effectiveness).
As a result, even if their projection methods were appropriate (and they're not), the manner in which IFF uses its "growth" measure – adding it to the 2011 rates – essentially ensures that the vast majority of the variation between schools in their final ratings will be due to factors outside of schools' control.**
This analysis is therefore recommending the closure and expansion of schools, among other actions, based on a criterion that has little to do with how they actually perform, and is largely a function of the backgrounds of the students that attend them (e.g., income, whether or not they are native English speakers, etc.).
Overall, IFF had a very difficult task – identifying low- and high-performing schools – and, for whatever reason, they do not appear to have been up to the challenge. Their data are inappropriate and their methods too simplistic and flawed to accomplish the goals they set out to accomplish.
Even if IFF's policy recommendations were sound – and they largely boil down to closing or improving low-performing schools, opening more high-performing schools, and monitoring schools in the middle – their methods for classifying these schools is not credible. As Bruce Baker puts it, IFF is essentially saying that we should "close poor schools and replace them with less poor ones" (also check out Glazerman's above-mentioned article for more on the recommendations).
This report, though attractive and full of interesting summary data, provides little of value in terms of informing sound policy decisions.
- Matt Di Carlo
*****
* There really is a striking progression of data loss in this analysis. Most generally, as discussed above, IFF uses proficiency rates (how many students above or below the line), which ignores underlying variation in actual scores. In addition, they're using cross-sectional grade- and school-level data, which masks differences between students in any given year, and over time. Then they use the rates (and projected rates) to calculate rankings, which ignore the extent of differences between schools. And, finally, the rankings are averaged and schools are sorted in quartiles (performance "tiers"), losing even more data – for example, schools at the "top" of "Tier 2" may have essentially the same scores as schools at the "bottom" of "Tier 1." At each "step," a significant chunk of the variation between schools in their students' testing performance is forfeited.
** I cannot illustrate this bias directly – i.e., using real data from the report and the District's schools – without a somewhat onerous manual data entry effort (neither D.C. nor IFF provide their data in a format that is convenient for analysts). But it's really not necessary – it's beyond dispute that absolute proficiency rates are largely a function of student characteristics, and IFF's school rankings are based predominantly on those rates. See this discussion, as well as this example from Florida and this one from Ohio.
----------Original Message----------
From: "Wadlington, Erika (Council)"
Date: Nov 9, 2012 9:23:57 AM
Subject: RE: REQUEST: copy of bill 19-734; docs relevant to COW's review of school closure
To: "ANC, 5A07" <5A07@anc.dc.gov>, "Tolliver, Denise (Council)" <dtolliver@DCCOUNCIL.US>, "Duffie, Celeste (COUNCIL)" <CDuffie@DCCOUNCIL.US>, "Mendelson, Phil (COUNCIL)" , "Wynn, Monique (COUNCIL)" <Mwynn@DCCOUNCIL.US>
Cc: "csteptoe@verizon.net" <csteptoe@verizon.net>
Hello Commissioner Steptoe,
Please see attached for a copy of the School Boundary Review Act, and hearing notice. The Committee has yet to receive the proposal of school closures from the District of Columbia Public Schools, thus we do not have that document to share. However, other agencies and local think tanks have completed reports and studies on the topic of school closures and consolidations. Check out the DC Auditor's report on the closure of 23 schools in 2008. [ http://dcauditor.org/reports/2012]; National Clearinghouse for Educational Facilities [ http://www.ncef.org/rl/School_Closure,_Consolidation,_Co-location.cfm ] as an example.
I hope this helps. Please let me know if there is additional information I may provide.
Sincerely,
Erika Wadlington
Legislative Assistant
Committee of the Whole
Council of the District of Columbia
John A. Wilson Building
1350 Pennsylvania Avenue NW, Suite 506
Washington, DC 20004
(202) 724-8124
From:Steptoe, Carolyn C.(ANC 5A07) [mailto:5A07@anc.dc.gov]
Sent: Thursday, November 08, 2012 8:17 AM
To:Tolliver, Denise (Council); Duffie, Celeste (COUNCIL); Mendelson, Phil (COUNCIL); Wynn, Monique (COUNCIL)
Cc: csteptoe@verizon.net
Subject:REQUEST: copy of bill 19-734; docs relevant to COW's review of school closure
Sent: Thursday, November 08, 2012 8:17 AM
To:Tolliver, Denise (Council); Duffie, Celeste (COUNCIL); Mendelson, Phil (COUNCIL); Wynn, Monique (COUNCIL)
Cc: csteptoe@verizon.net
Subject:REQUEST: copy of bill 19-734; docs relevant to COW's review of school closure
Good morning CM -- Congratulations on your win. Good to see you at the polls.
In advance of the school closing hearings, would you kindly pls. email me (1) a copy of Bill 19-734 and (2) provide key docs COW will reference for this hearing as they Review of School Closures? Links are certainly sufficient.
Thank you kindly.
COUNCIL OF THE DISTRICT OF COLUMBIA
COMMITTEE OF THE WHOLE
NOTICE OF PUBLIC OVERSIGHT HEARING AND
NOTICE OF PUBLIC HEARING
1350 Pennsylvania Avenue, NW, Washington, DC 20004
CHAIRMAN PHIL MENDELSON
COMMITTEE OF THE WHOLE
ANNOUNCES A PUBLIC OVERSIGHT HEARING AND
PUBLIC HEARING
on
Review of School Closures within the District of Columbia Public Schools
and
Bill 19-734, School Boundary Review Act
on
Thursday, November 15, 2012 at 4:00 p.m. (Part I)
Monday, November 19, 2012 at 2:00 p.m. (Part II)
1:00 p.m., Hearing Room 412, John A. Wilson Building
1350 Pennsylvania Avenue, NW
Washington, DC 20004
Council Chairman Phil Mendelson announces the scheduling of a Public Oversight Hearing and Public Hearing by the Committee of the Whole on Bill 19-734, the “School Boundary Review Act,â€and a review of school closures within the District of Columbia Public Schools. The public hearing will be held on Thursday, November 15, 2012 from 4:00 p.m. to 8:00 p.m., continuing
Monday, November 19, 2012, from 2:00 p.m. to 6:00 p.m. in Room 412 of the John A. Wilson Building, 1350 Pennsylvania Avenue, NW.
The purpose of the hearing is to provide the public an opportunity to comment on proposed school closings to be set forth by the Chancellor of the District of Columbia Public Schools (DCPS) or consolidations, and the closure process. The Committee expects that the Chancellor will have announced proposed school closings the week before the hearing. This hearing will also examine
legislation to require DCPS to periodically review and adjust public school boundaries and feeder patterns, and to report regularly on proposed changes.
Those who wish to testify are asked to telephone Ms. Erika Wadlington, Legislative Assistant, (202) 724-8124, or via email at EWadlington@... and furnish their name, address, telephone number, and organizational affiliation, if any, by the close of business Tuesday, November 13, 2012.
As it is anticipated that a large number of witnesses will testify, witnesses will be limited to three minutes of testimony. The Committee expects to accommodate up to 50 witnesses on Thursday, November 15th, and 50 on Monday, November 19th. Witnesses will be signed up for their requested
day in the order requests are received by the Committee.
Persons wishing to testify are encouraged, but not required, to submit 15 copies of written testimony. If submitted by the close of business on November 13, 2012 the testimony will be distributed to Councilmembers before the hearing. If you are unable to testify at the hearing, written
statements are encouraged and will be made a part of the official record. Copies of written statements should be submitted to the Committee of the Whole, Council of the District of Columbia, Suite 506 of the John A. Wilson Building, 1350 Pennsylvania Avenue, N.W., Washington, D.C. 20004. The record will close at 5:00 p.m. on Monday, December 3, 2012.
| |||||||||
| Dear Ward 5 neighbors, DCPS school closures will be announced on Tuesday Nov. 13th. If you would like to voice your opinion about DCPS's announcement, there will be hearings on Nov. 15th and 19th. See the bottom of this email to find out how to sign up to participate. Also, the Ward Five Council on Education will be meeting the following Tuesday, Nov. 20th from 6:30-8:00pm at the Lamond-Riggs Library (5401 South Dakota Ave. NE) regarding the pending school closures. Our special guest for this month will be our Ward 5 Councilmember Kenyan McDuffie. We hope that all community stakeholders, students, parents, and educators will attend. We look forward to having an open and honest discussion that will allow us to develop a comprehensive plan that is best for the Ward 5 community and our public schools. Sincerely, Shelagh Bocoum Public Relations Officer Ward Five Council on Education
o This will be the first part of the hearings Council is having on DCPS school closures. This is a great opportunity to speak on your concerns about school closures in your community and the city at large. We really want to let the city know that we need a Moratorium on schools closures immediately! This is a good day for Students and Parents to give testimony as it starts at 4:00pm.
To register to give testimony on either of these dates please contact Erika Wadlington of the City Council at 202-724-8124 or via email at ewadlington@... by close of business on Tuesday, November 13th. |
Dear Ward 5 neighbors,
DCPS school closures will be announced on Tuesday Nov. 13th. If you would like to voice your opinion about DCPS's announcement, there will be hearings on Nov. 15th and 19th. See the bottom of this email to find out how to sign up to participate.
Also, the Ward Five Council on Education will be meeting the following Tuesday, Nov. 20th from 6:30-8:00pm at the Lamond-Riggs Library (5401 South Dakota Ave. NE) regarding the pending school closures. Our special guest for this month will be our Ward 5 Councilmember Kenyan McDuffie.
We hope that all community stakeholders, students, parents, and educators will attend. We look forward to having an open and honest discussion that will allow us to develop a comprehensive plan that is best for the Ward 5 community and our public schools.
Sincerely,
Shelagh Bocoum
Public Relations Officer
Ward Five Council on Education
- Thursday, November 15th (4:00 – 8:00pm)– Council Hearing on DCPS school closures (part 1)
o This will be the first part of the hearings Council is having on DCPS school closures. This is a great opportunity to speak on your concerns about school closures in your community and the city at large. We really want to let the city know that we need a Moratorium on schools closures immediately! This is a good day for Students and Parents to give testimony as it starts at 4:00pm.
- Monday, November 19th (2:00 – 6:00pm)- Council Hearing on DCPS school closures (part 2)
To register to give testimony on either of these dates please contact Erika Wadlington of the City Council at 202-724-8124 or via email at ewadlington@... by close of business on Tuesday, November 13th.
Help 9-1-1 Save Your Life!
Create a free Safety Profile to provide critical, lifesaving data to 9-1-1 in the event of an emergency call today at www.smart911.com.
Inform first responders in advance!
0 comments:
Post a Comment