This commentary was submitted by Andrea Mérida, a Denver school board member representing southwest Denver.
The ballyhoo over a new school grading website piqued my interest, so I surfed over to ColoradoSchoolGrades.com to check it out. After all, the power partners that have coalesced around the website’s development and launch know how to make a splash. And why wouldn’t I be interested in the fact that they’ve taken the pains to offer information in Spanish?
After just a few searches for some of the schools in my southwest Denver subdistrict, however, I see that this website is still using more of the same accountability blinders that punish schools and kids for their English-proficiency differences by trying to lump them all into the same bucket as native and fluent English speakers.
About English learners and proficiency
In accordance with state law, the Colorado Department of Education administers an English-language proficiency exam, the Colorado English Language Aquisition Proficiency Assessment (CELApro), to EVERY K-12 student yearly. As CDE states, “The primary purpose of the assessment program is to determine the level at which Colorado non-English proficient (NEP) and limited-English proficient (LEP) students meet the Colorado English Language Development Standards in four domains (Listening, Speaking, Reading and Writing).”
The English proficiency of each student is graded on a 1 to 5 scale: beginner (1), early intermediate (2), intermediate (3), proficient (4) and advanced (5). According to DPS Chief Academic Officer, Susana Córdova, an English learner is considered linguistically ready to take the CSAP/TCAP when they have achieved either a level 4 or 5.
However, even if a student is not linguistically ready to take the CSAP in English, their scores will be averaged into the district’s and state’s school performance framework, and consequently, into the ColoradoSchoolGrades.com website data. You can imagine how that pans out.
Worse, none of these “accountability tools” truly give full weight to the Spanish-language version of the CSAPs administered for 3rd and 4th grade, skewing the performance of elementary schools with high numbers of English learners in incredibly unfair ways.
Some pertinent examples
CDE provides information about the language proficiency for every designated English learner in the state, and I’ve used it to analyze the level of English proficiency for middle and high schools, which only administer the CSAP in English. In nearly every case, our schools that are classified as “yellow” (accredited on watch) all have high numbers of English learners that are below the “safe linguistic threshold” of a CELApro level 4 or 5. Here are a few examples:
For the purposes of this chart, I call those students at CELApro level 4 and 5 as “ready for CSAP.” As you can see from this small sampling, schools with high proportions of English learners are maligned by the ColoradoSchoolGrades.com website as substandard.
Clearly, the mission of schools with high proportions of English learners takes on a different perspective than schools with lower concentrations. While we all want students to grow and succeed, student growth must be reviewed under a different lens when student populations come from homes in which English is not the primary (or sole) language.
ColoradoSchoolGrades.com, therefore, unfairly glosses over this distinction and makes no attempt to inform parents of these nuances. When dealing with populations that have difficulty in English and who are regularly exposed to various types of hucksterism in their daily lives, it becomes all the more important for a website that purports to serve as a resource for parents to be fully transparent.
To do otherwise is irresponsible.
This type of data has been the basis for extremely high-stakes decisions made about kids, teachers and their schools. As I retrospectively consider last year’s board decision to phase out Montbello High School, for example, imagine how differently the school’s performance would have seemed had we considered the 25 percent of students who are not ready to take the CSAP in English. Not being a statistician, I will defer to those who can calculate the probability of a 25 percent bump in overall achievement if those scores are even just disaggregated from the whole. I wonder if the board’s decision would have been different if we knew then what I know now.
Students at Kepner Middle School, one of the schools represented by Denver school board member Andrea Merida. Photo from Kepner.
As the Northwest Community Committee pointed out to the board this week, among all the other goals Spanish-speaking parents find important in schools, they also consider learning English to be primary (see the Venn diagram on page 9). These parents know their children have particular needs, and they expect the policymakers to realize it, too.
I have regular conversations with my Spanish-speaking constituents in southwest Denver, who express worry over the performance of their schools. When I point out to Kepner Middle School parents, for example, that their school has been able to increase the rate of students at CELApro levels 4 and 5 by nearly 30 percentage points over the last four years, their relief is palpable.
And the former Rishel Middle School? In 4 short years, they increased the numbers of students at CELApro levels 4 or 5 by a whopping 40 percent. But there is no Rishel now; it’s been closed.
If the development team at ColoradoSchoolGrades.com really wants to be a resource for Spanish-speaking parents, they would do well to join me in peeling back the layers of the accountability onion in good faith and transparency so that ALL parents can make the right choices for their children.
This offer has no expiration date. Will ColoradoSchoolGrades.com take me up on it?
It must have been a conscious choice to exclude ELL data. Why not add students with disabilities into the conversation, as well? If we included free/reduced lunch eligible (low income), and minority populations, why not the other categories reported by the state in the school performance frameworks?
“Oh, we didn’t want the website to be too complicated like that horrible CDE website.” If we do not honor the intelligence of parents and the rest of the public, we risk misinforming them.
So, when schools with high populations of ELL students are closed, what happens? Those ELL students go to other schools, and then the public looks at a simplistic measure and asks what happened to the neighboring schools…why did their scores drop so much in one year? And then those schools in turn are put on improvement plans or worse due to perceived decreases in achievement or lack of growth.
Mobility data is certainly relevant. I work in a rural school that has averaged 39% annual turnover for the past three years. Mining and gas production draws folks from Alabama, Arkansas, Oklahoma, Texas…and they stick around for a few months or years and move on. Their kids’ assessment scores determine the overall school’s grades? Our administrators’ and teachers’ evaluations? Get real. Out of 300 students in the school over 100 either register anew or withdraw every year. We can make progress in the short time we have some of those students, but for those who come to us with grave deficiencies after attending half a dozen or more schools in just a few years, it’s a tall order to expect miraculous progress in the six months between October 1 and the spring state assessment.
The numbers game is just that – a game. Some simply do not want to see the whole picture, and I question whether they want others to see it, either. So, is the website in question truly designed to inform? If so, it’s an abysmal failure. If not, well…that would border on unethical.
Kevin, yours is a strong example of why these measurements don’t work. The state probably calls your school a “dropout factory” because it doesn’t have the capability to track the mobility of these students.
I remain ever hopeful that it isn’t willful ignorance of the realities of the lives of our students that causes this sort of stark omission of facts.
New York has had a similar grading system for a few years now. It would of course be important for Colorado to know how that system has fared. http://www.newschool.edu/milano/nycaffairs/documents/ManagingByTheNumbers.pdf By this report, not too well. Perhaps not surprisingly, the NY plan was hatched under Joel Klein. Also not surprisingly, he works for FOX News [sic] now, and his influence is still felt far and wide in the reform movement.
As much as Colorado appears to be an important player in modern reform, we might expect more experiments to be performed with creative data mining not to mention with our teachers and students, in the years ahead.
Andrea,
I think you raise some important points about the website adding the ELA school level data to the school descriptions. I also think it would be helpful to have some kind of grading system for how schools are supporting ELL students to obtain English language skills. In doing a quick review of this particular data it looks more complicated than you have suggested. Right now it would appear that there is no growth metric for ELL students which would lead to some of the same problems that we had prior to having a CSAP growth metric a few years ago. We need to know what level ELL students are at a particular moment in time and very importantly how much and in what time a particular school is moving students into place where they can take CSAP and how in turn these students do.
It’s important to remember that this site is based on what the state of Colorado has already decided per SB 163 about what should be measured in terms of school quality. There was no effort to ignore ELA or any other standardized state data for schools by this consortium of non-profits. In fact we have all said this site is just the start of a search for a good school. All of us involved thought it was important to start with the state system given all the work and the agreements with educators that had been put into the CDE ratings. CEA, CASE and CASB have been very supportive of 163 and the mix of CSAP growth and status measures for evaluation school quality. It is not perfect but it we believe it is more fair and transparent than any earlier school grading systems.
Van, had you started with bringing stakeholders to the table, you would have discerned which factors are important for parents to know when looking for a “good school.”
It’s called a “focus group” in the business world.
How complicated is it, really? You can tell over a period of time how many more students at a given school are classified as a level 4 or 5. Why do you need anything more than what the state tells you? That’s the grading system.
DPS alone is made up of almost 60% Latino students. How could you possibly have launched a website without relevant data for the majority of DPS’ students? I realize that you’re not just dealing with DPS, but frankly, absent relevant data, the website is not ready for prime time.
People in my community are saying that the website exists just to make people think the public schools are “bad” so that they flock to the charters. How would you answer that sentiment?
Andrea,
Re this comment:
“Northwest Community Committee pointed out to the board this week, among all the other goals Spanish-speaking parents find important in schools, they also consider learning English to be primary (see the Venn diagram on page 9). These parents know their children have particular needs, and they expect the policymakers to realize it, too.”
Why wouldn’t the policy makers realize it, too? The District is STILL under a court order to provide adequate services to English learners. One reason I believe is that many parents, because of their immigration status, are afraid to participate publicly, can not vote, and have no power. Witness the composition of the Regis “Northwest Community Committee.” People had to fill out applications requiring very specific personal information. This represented a barrier to participation. Indeed, the Regis Committee commented on the difficulty of outreach to the Spanish speaking community. This is why I think the selection process for the NWCC basically inequitable.
As a member of the Denver Board of Education, and I must say one of the most articulate and intelligent and a policy maker, I believe you have a responsibility to know the status of the District’s compliance with that Court Order.
I would like to see a report from you on that. I think criticizing the “Consortium’” grading report does not give us any new information. A report on the stays of compliance with that Court Orderwould be one way that “policy makers” would show that they are paying attention to the needs of English Learners and Spanish speaking parents.
Correction: status of the compliance
Andrea and others interested in this issue should know that, in the past year, CDE have applied the Colorado Growth Model method to CELApro data. The possibility of incorporating growth on an English language assessment into the state’s Performance Frameworks will be a part of the conversation this winter among stakeholder groups. This idea was included in our NCLB waiver request to the federal government.
The growth data on CELApro are fascinating, and have already been reported to all districts. I would particular recommend slide #26 in the powerpoint presentation on this page: http://www.cde.state.co.us/cdeassess/documents/cela/cela_growth.html
You will also find a lot more information about the differences between CSAP and CELApro that have implications for the interpretation of CELApro data.
You can do an analysis of what a school’s growth or achievement numbers would look like with and without particular groups of students using the SchoolView Data Lab on this page:
http://www.schoolview.org/performance.asp
Watch the instructional video first so that you know how to get the analysis that you want. You will not be able to exactly produce what numbers appear on the performance frameworks under “academic achievement” because the Data Lab tool does not include the Lectura, Escritura, or CSAPA numbers yet. But you will get a strong sense of what the numbers would look like for many customized disaggregations of the data that are not commonly reported.
Bill, this is great. Thank you.
Here are 2011 results for Montbello HS from the SchoolView Data Lab. ELLs score higher than their non-ELL counterparts, and enjoy substantially higher growth. I apologize if the formatting I’ve set up does not work in this forum.
Subject Name Students Avg Scale Score Median Growth Percentile
Math ELL 527.36 57
Math NON-ELL 513.40 56
Reading ELL 620.16 52
Reading NON-ELL 613.24 49
Writing ELL 504.24 53
Writing NON-ELL 496.63 45
Joanne, thanks for being aware of the court order! This goes a bit far afield from what I think the website is all about, but I have expressed serious concerns over the last couple of years about the fact that (a) DPS is CERTAINLY bound by the ELA court order and (b) we have been grossly out of compliance because the charter schools are not in compliance, and the board has been approving them.
I’m actually very proud to report that as a result of my advocacy, and because of the collaboration between the plaintiffs and the district, we have now changed our policy. Charters and other new schools must comply. All charter renewals should have proof of an ELA program.
Now, it remains to be seen whether those are high-quality programs. Personally, I find that to be a slim chance, given the penchant for these charters to hire TFAers that receive not even rudimentary ELA training (in my opinion). After all, how well-versed can you become in those particulars in a few short weeks, especially since most of you aren’t even Latino in the first place (being Latino could lessen the learning curve for some, is my point). But hope springs eternal.
The good news is that the ELA programs found in our public schools work, and work very well. We consider kids that achieve English proficiency as “exited;” in other words, they no longer need sheltered instruction. These exited kids are, to be colloquial, kicking butt and taking names. They perform better than the district average on the CSAPs, and the elementary kids outperform even Anglo kids.
I wrote about that here: http://squarestate.net/diary/1725/denver-public-schools-best-reform-often-goes-unheeded (I’m rebuilding my own site at the moment).
My take is that the court order has forced us to develop a program that works really, really well. It’s hacking away at the achievement gap…if we let it.
Bill, that’s precisely what my research is turning up, too.
Forgive my snark when I say that this program, which was developed by educators using research-based information, is what’s closing the achievement gap, not the reform “flavor of the month.”
I’ve heard from some close to the court order plaintiffs that Montbello has a high number of opt-outs from services, simply because they’d be sent over to South, which is clear around the world for a Montbello kid. That’s too much hardship for some parents to allow for their kids.
Bill, you made my day. Let’s chat over coffee soon.
Andrea,
Thank you for the information. Is there a current report on the status of compliance with the court order posted on the DPS website?
Am I correct that you are advocating two inclusion in the “school grading.”
1) Data showing how quickly each school is moving its ELL students to a level 4 or 5 so that parents of ELL students would have the necessary information to choose the best school for their students-
2) Data showing the proportion of ELL students below a level 4 who took the CSAP in English at each school so that parents of English speaking students could more realistically evaluate a school’s grade
In talking about NWCC, I did not mean to introduce an irrelevant comment in this very important discussion.
But you mentioned it and I felt it important to describe the NWCC committee membership process. I don’t recall anything in the NWCC notes or presentations that mentioned the Court Order, but I could be wrong.
There is no report that I’m aware of that shows the results of any monitoring, since the requirement has since lapsed. I’m working on it, though.
I think any school ratings based on test scores should only include those ELLs that are at a level 4 or 5. No other scores should count, or at least, they should substitute the CELApro scores for the reading and writing CSAP scores, although, admittedly, the CDE powerpoint provided by Bill above says that using the CELApro scores for the sole measurement of effectiveness of teachers/schools/districts is inappropriate.
I hear you about NWCC. I was not enthused about the application, and I do know that the district was very strongly advocating for it. They like to hand-pick their parents, but Arturo Jimenez was very diligent about ensuring impartial eyes on the selection process.
I think this discussion is a very important one. The unfortunate fact is that there is no publicly available and thorough evaluation of the ELL program in DPS (like most programs in DPS). In addition, there are no agreed upon metrics for evaluating ELL program effectiveness for schools in Denver or CO. I am sure that there are some schools doing very well and some doing a miserable job in DPS just like the rest of the schools and programs in DPS. It would be very helpful to look closely at a number of schools in DPS using this and all of the other school performance data to determine how schools like Kepner and Montbello are doing. Bill, can we accurate compare ELL and non ELL growth scores? are they not apples and oranges?
I will say that if the ELL program was working so well across the board in DPS why are the reading CSAP scores so flat (less than 1% per year change) for low-income and Latino kids at the 4th grade? You’d think these numbers would be going up at some point given the number of ELL kids in the district and some of these schools.
I would be very interested in gathering some experts on school data, evaluation and ELL programs to talk about how we would know whether school X is doing a good job on getting their ELL students to learn English and all of the other core subjects. I really appreciate the information that Bill provided and I think it could be really helpful but I think we have a ways to go in order to understand ELL program quality in any given school.
So, in other words, we corporate reformers advocate for the continued closing and restructuring of schools until such time as we care about ELLs.
Again, 60% Latino. Why wasn’t this a priority from the get-go? Why aren’t you calling for a moratorium on school closures until we figure it out? Why don’t you crunch the numbers a bit yourself and see whether Lectura scores are higher than reading? You would think that with the amount of corporate foundation donations you and the rest are getting, you’d find funding to study this profound equity problem.
The data’s plain as the nose on your face.
Andrea, Chill out….I’ve been involved with this work that includes pushing and supporting better public schools (and in particular for low-income and Latino kids) since 1987. You may recall that I put out a report on the state of Latinos in DPS back in 2003 or 2004 when I was at the Colorado Children’s Campaign. I know you sometimes like to get a rise out of folks with a question or suggestion about a person’s motives. Doesn’t do much for me, maybe cause I’ve been doing it for too long.
Back to the original conversation- I wish it was as simple as you suggest in terms of determining which schools are working in terms of their ELA programs. I’m all ears in terms of what you would suggest in terms of evaluating how you would determine which schools are working. I think it needs to be more than CELApro growth or status scores (they are really important), the calculus should also include how ELL designated students are doing once they take CSAP. As you know DPS and the state does not currently have any metric for measuring a schools ELA program quality so it is not just my low IQ or lack of interest that is keeping me from having some judgement, it is complicated. I thought you might be pleased that a so called “reformer” was asking for some direction from you. And thanks again for raising the issue, I really do think we all get after it.
I would like to address several issues in Kevin Crosby’s post. First of all, ELL, SPED, FRL, or minority data are not systematically excluded from the state performance framework data that formed the basis for the Colorado Succeeds school letter grades. Those groups’ performance is separately considered and given points on the frameworks, such that growth and the adequacy of that growth are included. Second, the mobility issue is more of a problem with overall CSAP reporting than with reporting on the state performance frameworks. Students’ data are only included in school performance framework totals if they were enrolled in that school on or by October 1, and if they took CSAP in the same school that year. Students that wandered in from Texas, stayed for a few months, and then left after CSAP were not included in the results. And if that was their first time taking CSAP, they don’t get a growth calculation. So the highly mobile students are not included at all in most of the calculations and schools are not accountable for their results.
I’m not sure what Kevin would suggest including in the coloradoschoolgrades.com site. That schools with high numbers of ELLs or SPED kids get graded on a different curve because they have a valid excuse?
The ELLs are not the problem. If I were a principal, to get my numbers up, I’d want a schoolful of ‘em.
Thanks Bill for keeping the focus on the data, as I’m looking at the data it does seem that those schools with high ELL populations might bump not decrease their growth metrics…..nice to see CDE engaged in the lively dialog.
Bill Bonk for President.
Frankly, Van, I think you and the other corporate reformers have done enough. When you issue that press release urging the moratorium on turnarounds for schools with high numbers of ELLs until we have the right indicators, let me know. I’ll buy you a drink and wash your car for you.
Andrea,
I think that a current report from the court in regard to the Order would be definitive and objective and could eliminate some of this “back and forth” over what DPS is or is not doing in regard to Spanish speaking ELLs.
I do not understand how a “monitoring requirement” could lapse if the Court Order is still in effect.. When you said you were “working on it,” to what did you refer?
Also, in regard to the NWCC membership and your comment:
“I hear you about NWCC. I was not enthused about the application, and I do know that the district was very strongly advocating for it. They like to hand-pick their parents, but Arturo Jimenez was very diligent about ensuring impartial eyes on the selection process.”
Isn’t Arturo Jimenez part of the District? Arturo Jimenez is a lawyer. He and Mary Seawell, also a lawyer, were the Board Members who coordinated the NWCC. How did Jimenez ensure “impartial eyes on the selection process”? He did not insist that the representatives to the committee be selected by their respective schools and community organization. Most critically, he did not demand that the selection criteria be published.
There is no documentation of how members were selected.
I bring these two issues up because I believe that the role of the ELL student population is central to understanding Denver Public Schools and the effectiveness of all its programs. There is much talk and little hard data to which all can agree. It is one reason why problems are not solved.
Andrea ….how is it that I am a corporate reformer and what exactly does that mean? because I support charter schools? did you not notice the other blog post that did here the other day regarding turnaround schools and profiteers? I’ve never been a fan of the large corporations involved in education whether they were text book, testing, charter or consulting? And while this is not a universal dislike on my part, I have yet to see very many of the large corporations do more good than they do bad in K-12 settings. Having said that there are a number of companies like Google, NWEA, Rosetta Stone, College Board, Apple (for a time), etc that have made some very important contributions in terms of student learning so I do want to keep the education sector open to different entities, I just want better monitoring of quality for every aspect of the system. If they can do it better why wouldn’t we want them in public education.
And yes I would be happy to join hands with you to stop some school from being closed if there was evidence that the ELA program was really making a difference. Thanks for the offer on the car washing…it does really need it but I’d prefer civil dialog and actions regarding school improvement. I look forward to seeing which schools are really working for their ELL populations are working and learning more about why they are working.
Wow! That was an intense dialogue! Who benefits? Our public schools and their students?
Mr. Martin, I struggle to understand the intent of your comment. Are you saying these issues shouldn’t be brought out into the light of day? If so, I disagree. “Civility” is what keeps my neighbor kids down, because it’s often used as an excuse to not voice opposing facts.
As the corporate reformers like to say, “our kids can’t wait.” It’s time for parents to really understand what’s happening here.
And Van, I support charters too. I just don’t support them willy-nilly and without proof that they actually will help students. That’s why we’re having to close charters, or threaten to, these days.
Andrea,
As i think you know, I’m for closing charters and have supported a number of them being closed or phased out for not living up to the terms of their contracts, e.g. Ceasar Chavez’s high school (when the founders stole hundreds of thousands from the state) and schools like Denver’s Big Picture School (which I helped found but didn’t come close to meeting the schools own performance expectations), Life Skills, PS 1, JeffCo Open and others. Each of these schools had a number of attributes. And they were created to serve students in ways that their home districts refused to do but each each of these charters in the end missed meeting the terms of their contracts by huge margins and ended up being no better and often worse than other schools in their districts. Each of these schools were given hundreds of thousands of state money and often many years to get their act together. It is no small task to have a high performing school so it should be no surprise that some fail and this is why we need to provide the support for new schools to succeed while always being prepared to intervene on behalf of students if the schools do not live up to their expectations/contracts.