New Orleans students fail Texas tests
As someone who is an educator and is very interested in education in general, I find this article fascinating, but all together unsurprising.
I hope someone at the Department of Education has the brains to record all the statistics for the displaced students. Following them around will be a natural experiment from which I think researchers can learn a bunch about school quality.
What of the three reasons given in the article for the poor evacuee performance?
Sticking students in the wrong grade – that seems problematic, but I am a bit confused. How is it that a student or parent doesn’t know what grade they are in?
Trama of students – I don’t want to dismiss this outright, but it sounds a bit flaky to me. My sense is that third graders are pretty resilient. If we believe this is the case and students have recovered by next test, then we’ll see a sharp increase in the pass rates of evacuees. Stay tuned. Unfortunately for disentangling the causes, the students will have another year of Texas schools under the belts.
Quality of New Orleans Schools – now we’re on to something. I think ultimately what will be handed down is an indictment of the New Orleans school system. My guess? The younger the displaced student, the smaller the gap in the pass rate, controlling for all the other important stuff. As the students spend more time in New Orleans schools, they get further behind. There is some evidence for this in the statistics given in the article.
What I think would be super interesting would be able to find families with different aged children. Many studies of educational outcomes find that socioeconomic status of parents are a powerful predictor. The nice thing about using two children in the same family is that they have the same parents. The difference is, the older of the two would have spent more time in New Orleans schools, while the younger more time in Texas schools.
–CT
March 24th, 2006 at 2:33 pm
Here’s my spin on the issue at hand. You see, the education system in New Orleans is very much like that of an assembly line in a large factory. For the sake of discussion the students of the New Orleans public school system can be referred to as nuts and bolts coming off of an assembly line, all of which are manufactured at the same plant (N.O. public schools). Because those nuts and bolts being produced are made with inferior equipment (the educators- who are also products of the same assembly line previously), they in turn, the nuts and bolts are substandard in quality (their education gained).
Now because these nuts and bolts are not meeting industry standard and cannot demand high prices at market, they are utilized internally within the factory in which they were produced and similar factories (New Orleans public schools) to make up various components of the machinery that in-turn produce nuts and bolts on an assembly line.
Usually one can find that if there are problems at the bottom of an organization, it’s because there are problems at the top. The New Orleans public school system has repeatedly tried to bring in ‘outsiders’ to try and manage the school system and fix the internal issues. The age old argument of ‘which came first the chicken or the egg’ can best be compared to the New Orleans public school problems of using broken teachers to teach students (not all N.O. public school teachers are broken). Some of the poorly educated students go on to become public school teachers themselves and are contributing to a broken process, an assembly line, a vicious cycle.
Do we bring in teachers from the outside and attract them with higher wages? Do we have an exam in which all N.O. public school teachers need to pass in order to work and teach in the public system? If so, who makes up the test? Does an outsider of the public school system make the test, or do we allow a product of the N.O. public school system to make up the test?
I have addressed the problem with the New Orleans school system as I see it. As for a solution to the problem, that perhaps is an equation with no solution!
March 26th, 2006 at 10:00 am
Controlling for socioeconomic factors, as Todd suggests is extremely important. The news story fails to note that the the Katrina evacuees still in Houston tend to be the poorest of the poor. Their pass rate is then being compared to the pass rate of all Texas students. This is an exremely unfair comparison. You cannot infer from anything in this story that Texas schools are better than those in New Orleans. If the comparison were between evacuees with given income and family background (mother’s educational attainment and single-parent family vs. dual-parent family) and yes, even race, and compare those students with those of similar incomes, family backgrounds and race from Texas, then a fair comparison could be made. But that is exactly not what is done in this story.
I am not suggesting that New Orleans schools are as good as those in Texas, it is just that there is nothing in the story to lead me to believe that all of these differences are NOT due to something researchers call selection bias.
Mr. Heney, there is a test that all certified teachers must pass, called the Praxis Exam. It is a teacher’s exam that tests the prospectives teacher’s knowledge of their teaching area as well as knowledge in education itself. However, not all teachers are certified in their field and parishes vary by how hard they seek to employ certified teachers. The data on teacher certification (that is, whether they have passed a test in their teaching area) is almost surely available on the Louisiana Department of Education website.
One further issue, though, between Texas and Louisiana schools, is what constitutes a passing grade for teacher certification. Louisiana may set the minimum grade to pass the Praxis at a lower level than Texas.
March 26th, 2006 at 10:12 am
I have to clear up something in my previous comment:
I wrote:
“I am not suggesting that New Orleans schools are as good as those in Texas, it is just that there is nothing in the story to lead me to believe that all of these differences are NOT due to something researchers call selection bias.”
There is a double negative issue here. My point is that I think that the differences might all be explained by selection bias–the evacuees in Houston were among the poorest and lowest educated of New Orleans. To compare these with all Texas students, and not just those similarly situated from Texas is an improper comparison.
March 26th, 2006 at 10:47 am
Sorry to keep going on this, but there are two other issues.
I do want to point out that I am no fan of New Orleans Public Schools, it is just that the comparison in the news article fails to control for other important factors.
One, related to the issue I already addressed, is that most students in New Orleans public schools, with a few exceptions, such as students at Ben Franklin High, are from the poorest families. The well-off in New Orleans send their children to private and parochial schools. The well-off in Texas send their children to public schools which are extremely well funded with the relatively high property tax in Texas.
Two, there is no “National Curriculum” or set of standards. As a result, each state has its own curriculum and standards. The state tests are designed to test accomplishment with the state’s curriculum. It is unlikely that Louisiana and Texas have the same standards and test on the same material.
Louisiana students, taking the Louisiana curriculum, did worse on the Texas-curriculum-based test than those who took the Texas curriculum. Would you be surprised if a student enrolled in a history class did worse on a political science exam than someone who was in that political science course? Or that when a sprinter and a distance runner are pitted against each other in a sprint, the one who has trained as a sprinter generally wins.
I think that the news story is just another example of bragging Texans.
My response to that story is a big, “So What?”
March 26th, 2006 at 3:08 pm
I think the answer to Morris’ question “So What?” is that given the existence of this test, and the large number of people who have moved to Texas, that it possible to measure scohol quality.
Economists can surely agree, given all the necessary data (some of which is not available), how to properly do this study. I am hoping the education bureuacrats are collecting the information that we’d agree we needed. Income, race, education level of the parents would all be on the short list.
I certainly make no claim that the article *proves* that New Orleans schools are worse than Texas schools (though I too think this is true).
However, I think even the limited evidence in the article is very suggestive.
I certainly agree that socioeconomic status is very important. And there is no doubt there is a selection bias in those who are evacuees. In my originial post I believe I point out a very simple way to control for socioeconomic status.
I am not sure I agree with Morris’ point about the curriculum – my thought here is that the curriculum in reading and mathematics (the items being tested) are likely to be quite standard across states up until 5th grade. (At the least, if we believe that the Lousiana curriculum standards are identical across cities, then one could at least learn about relative school quality within Louisiana from students scores on the Texas test.)
Here’s what I interpret as evidence of Lousiana schools being of low quality.
89% of 3rd graders in Texas pass the test while 80% of 5th graders pass. This roughly speaking, seems to suggest about a 10% (1 – (.80 / .89)) attrition rate between 3rd and 5th grade. Or stated a bit differently, again loosely speaking, conditional on passing in 3rd grade, 90% will pass in 5th grade.
A similar calculation for the evacuees results in an attrition rate of 20% (1 – (.46 / .58)). Only 80% of those that pass in 3rd grade will pass in 5th grade, again loosely speaking.
I think this, albeit crudely, addresses the selection bias problem.
How can this be explained if is not school quality?
There are two things I could explain this, neither of which I find appealing. If the curriculum is meaningfully different across states, I’d admit I am on very shaky ground. If socioeconomic status affects 3rd graders and 5th graders differently, I am in trouble again.
Is there no value in my crude calculations?
–CT
March 26th, 2006 at 7:34 pm
This is an amazing issue. Although I am not an education major, I would certainly be interested in the future stats on this educational dilemma. Firsthand, it is unimaginable that students as well as their parents do not know what grade these students are in. I am a college freshman. I have known my grade number from Pre-K until now. My parents also have always known. Enough said about this one. Next, third graders — and most elementary students — are very resilient. I also agree that this is not a big issue. That leaves only one common denominator in this equation: The New Orleans Public School System which includes the socioeconomic status of its students and their parents, grandparents, or legal guardians. You may be able to remove an incompetent teacher out of a classroom; however, you cannot take out its students. And although I do not know what percentage of teaching professionals in Orleans Parish may be incompetent or indifferent, these educators unfortunately exist in every school system. What about federal aid and the accountability for these monies? The federal government is stepping up to the plate to help the TEA. It took a natural disaster for allocation of these monies to help a desperately failing Louisiana school system. Scenario: If displaced Orleans first graders remain in Texas schools and are tested in third grade, the results could prove quite interesting. Several questions swoon through my head. Is student success based solely on the quality of a school system’s faculty, staff, administration, physical environment and financial soundness; the socioeconomic status of its parents or legal guardians; or a mixture of both? As a student, if I attend class daily, apply myself and study diligently, complete all assigned homework and projects, will I insure a successful semester? I quickly admit that other important factors come into play. Is my home environment quiet, comfortable, and conducive to studying; am I eating and resting properly or must I work long hours in order to meet financial pressures; am I fellowshipping with other students who share the same goals and values; and can I depend on my parents for guidance and support if necessary? The answers are not easy ones. Hopefully, the Texas school system will eventually enlighten our Louisiana Board of Education in a genuine effort to consistently raise the bar, in Orleans Parish and statewide.
March 27th, 2006 at 2:59 pm
Aaron,
First, I am not at all surprised that some of the adult evacuees did not know their children’s grade. The level of involvement in your schooling by your parents is certainly the norm. However, many of the evacuees do not fit this norm. Many are single mothers, often barely adults themselves. Too many have drug problems. One Red Cross Volunteer told me about seeing evacuees here at Nicholls lighting up a joint in plain sight, with all sorts of security guards in the vicinity. If people will use drugs in plain sight, I am sure that there were many children from drug abusing homes, where adults fry their brains.
Chad,
I think that it is very likely the socioeconomic considertations affect attrition, or put another way, how quickly one advances through the curriculum or through the grades. Also, it should be pointed out that students perform simililarly in early grades and it is after they have been in school a while that scores begin to diverge. I would expect that the socioeconomic factors that lead to differences in performance also lead to differential attrition rates.
While the original article does raise a question about the comparative quality of the school programs, I think that it does a poor job in even suggesting which is better. A better comparison would just be simply a comparison of scores on some nationally normed test, and even with aggregate school district level data, use averages of district poverty levels (measured by those qualifying for reduced school lunch programs), the percent of students from 2-parent homes, the percent of mothers with college degrees (BA/BS), the percent of mothers with high school diplomas but no college degrees, maybe a measure for ethnicity, etc. You may also want to control for percent of the school district’s students in private and parochial schools, as this can lead to a selection bias somewhat independent of the other factors. Of course, the individual level data with similar measures would be more telling.
I should point out that I have no doubt that the average Texas school district does a better job than Orleans public schools. But then again, the average Louisiana school district does a better job than Orleans public schools. I really have no delusions that Louisiana schools on average measure up to their Texan counterparts. It was just that none of that was effectively revealed in that news article.
Morris Coats
March 27th, 2006 at 4:00 pm
Ah, the wonderful world of disagreement.
While the introductory student may have lost interest, I think there is a lesson in here about economic models. To be a respectable economic model, it has to be testable. That means it has to make predictions that can be found to be right or wrong.
I would claim thay my theory and Morris’ theory have such predictions. Someone should owe the other a beer, not to be paid until two years from now.
If I understand Morris’ claim, it is that the attrition rate will be different for low socioeconomic status individuals compared to high socioeconomic students. And further, his claim is that these differences will increase (become more important) as the student gets older.
Suppose that is the only thing going on (I’m not sure that is Morris’ claim, but let’s run with it). In this case, if these students were to take the test in the 7th grade, the pass rate gap will widen between Texans and evacuees.
Suppose school quality is the only thing going on (I’m not sure this is my claim, but let’s run with it). In this case, if these students were to take the test in the 7th grade, the gap will close between Texans and evacuees.
There we have it. My theory predicts one thing, his another. Stay tuned. I’m not sure they test 7th graders, but we will be able to learn something, again albeit crude, from what the gap looks like for 5th graders next time around (as long as they don’t monkey with standards).
Both Morris and I agree there are better ways to test this, but I still think, given these “cheaply provided data”, I can learn something from this.
One last note – at first me reaction was similar to Morris’ on attrition rates, but I think our perception is altered by social promotion – passing students despite the fact that they have not mastered the material. This phenomenon tends to cause a lot of attrition rate “catching up” in the later grades that perhaps should have be done in the early grades. That may lead us to think that the attrition rate increases as students progress, when in fact it does not. I’m not claiming that this proves anything, but I think it belongs in the same converstation. I think that is one of the points of the Texas testing program.
–CT
November 29th, 2006 at 3:59 pm
This is a terrible reflection of the school system and parents of New Orleans. What is the problem with New Orleans? They can not get any positive reports in the media. The local government needs to come assist the Louisiana school system. The state needs representation from other successful school systems to come help New Orleans and other parishes. Where is the money going? New Orleans had Harrah’s, Centre, and the River walk, and other means of money coming to the city. Who is taking money? There is a need of evaluation throughout the city to see what or who is the problem. Louisiana natives are already known for bad grammar and being two years behind. This makes Louisiana fit the serotypes. There is a need for reform in New Orleans. Mr. Mayor or city officials pay attention to education because Texas is making Louisiana look bad.
November 29th, 2006 at 4:13 pm
This is a terrible reflection of the school system and parents of New Orleans. What is the problem with New Orleans? They can not get any positive reports in the media. The local government needs to come assist the Louisiana school system. The state needs representation from other successful school systems to come help New Orleans and other parishes. Where is the money going? New Orleans had Harrah’s, Centre, and the River walk, and other means of money coming to the city. Who is taking money? There is a need of evaluation throughout the city to see what or who is the problem. Louisiana natives are already known for bad grammar and being two years behind. This makes Louisiana fit the serotypes. There is a need for reform in New Orleans. Mr. Mayor or city officials pay attention to education because Texas is making Louisiana look bad.