|
V. We know:
Adjusting for socioeconomic factors sometimes tells a
very different story from raw performance measures. “VALUE-ADDED"
MODELING All kids are not equally challenging to educate. If you take the raw test scores by district and sort
them high to low, you would find that you have also
sorted almost perfectly by the median family income of
each district. This pattern is by no means peculiar to
RI, or even the United States. Without the strong
intervention of effective schooling, students tend to
achieve according to their socioeconomic backgrounds. It
does not have to be this way. Schools can make a
significant difference in a challenged child’s life. Indeed, the current national push to improve public
schools is fueled at least partly by a passion to help
challenged children beat the determinism or
predictability seemingly built into their lives by
certain factors, the strongest of which is poverty.
Schools need to be especially helpful in equipping these
children with the skills and knowledge to create a
bright future for themselves.
Some schools respond more effectively to their unique
population than others. Value-added statistical modeling
helps us see which schools they are. By the same token,
the modeling also reveals schools whose test scores seem
impressive, but who, when compared with similar
students, could be doing much better. RI’s statistical
model uses five student
characteristics. Value-added statistical modeling compares the
performance of a school’s student body with the
performance of similar students statewide using –
adjusting for – the following five research-endorsed
factors:
-
Poverty (which is by far the strongest predictor of
student achievement, with the exception of prior
achievement. Without a Universal Student Identifier
system in place that would enable RIDE to know students’
grade point average, RI is not able to factor prior
achievement into its research.)
-
Non-English speaking background
-
Educational background of the parents
-
Having special learning needs
-
Having a minority racial-group identity.
While individuals with one or more of these
characteristics can and do perform well on state
assessments, the majority tend to perform less well than
children who do not have these characteristics. The many
reasons for these historic patterns of achievement
include such things as school expectations, the
availability of flexible grouping and different types of
instruction, inadequate funding and support to the
schools these children attend, and the quality of social
services offered to students.
An explanation of the value-added charts
is available here:
How to read the Value-Added Charts and how schools are
assigned to cohorts in the value-added charts. Take a
moment to learn to read the charts themselves. The charts are sorted into levels --
high,
middle and
elementary. The state-level charts use four
subtests, two for math and two for English, as
indicators of student academic achievement as compared
with similar students statewide. Those whose actual
scores are consistently above the model are in the top
band; those consistently below are in the bottom; all
other schools are spread along a continuum in between.
RI high schools and middle schools can each be
represented on single pages. The elementary chart
occupies four pages.
For more information about the modeling itself, as well
as the rationale for doing so, see the
Users Guide or
the 2001 State Analysis – which explains each of
Information Works’ data “lenses” or techniques. For a
more complete understanding of the modeling and specific
choices regarding RI’s model, see the
Technical Brief on the Statistical Model.
Please see the User’s Guide or the
State Section for
further explanations, specifics or rationales regarding
the following subjects:
-
The value of looking at the lists from prior years for
a sense of how a school has fared over time – see:
“School Performance: A Value-added Perspective.” on the
homepage
-
The effects of the introduction of ‘no-score’ results
starting in IW! 2001
-
The use of only one year of performance data on these
charts
-
The reason Information Works! does not report data
cells smaller than 10 students
Value-added gives an important
second data point. If the performance designations are a first facet, a
portal into a school’s vast store of data, the
value-added lens offers an important second point
because it shows, sometimes, a different view of the
same test score information as the performance
categories. Schools with the most challenged children
would argue that the value-added lens is, in some ways,
a more realistic benchmark against which to measure
actual performance, adjusting, as it does, for poverty,
among other things.
For example, Asa Messer School, in Providence, is
considered ‘low-performing, though “improving.” On the
value-added charts, that school has performed
consistently above statistical prediction, placing it in
the top band, for the third year in a row. Nearly 100%
of its children are eligible for subsidized lunch – a
poverty indicator – showing that this school has more
success with socioeconomically challenged children than
its counterparts statewide. While deserving the state’s
support for low-performing schools, it also has lessons
to teach about improving the outcomes for low-income
children.
|
Like the principals at the four schools who improved in
every subtest, Denise Missry, principal at Asa Messer
School, credits their early and diligent use of data for
a measure of their success. “We were a SALT pilot school
in 1997, so we started early identifying our weakness
with that first SALT visit self-study. Since then, every
year we focus on one kind of professional development.
We were weak in math problem solving, so we got a new
series, have a monthly math problem, got a consultant
and so on. Our scores have improved since. Another year,
we did the same thing with writing skills.” Asa Messer
also has an unusually tight home-school connection in a
state where the data indicates most parents are not
well-engaged in their children’s education. (See page 57
for more information about parents.) Missry says, “if
the children don’t have their homework done, my teachers
call home to find out why not.” |
The Textron-Chamber of Commerce Academy, in Providence,
has an even more dramatic story to tell. Because it is a
‘low-performing’ and ‘not-improving’ school, one might
assume it is a failure on all fronts. But the
value-added list demonstrates what raw achievement
scores never would – that the school is performing
heroics by getting even the seemingly poor performance
it does get from the students. The challenges of the
students in that building – most would not be in school
at all if it weren’t for ‘Chambers’ – weight the
statistics in such a way that we get a different view of
the school’s struggles and success.
|
Textron-Chamber of Commerce Academy was the first RI
school to have received a charter directly from the
state, which allows it to be more experimental about
educating its unique population according to its own
design. So, for example, to gain entrance to the school,
students must be willing to participate in a summer
school where the teachers can assess the students’ math
and reading levels while working to ramp up those skills
before the school year even begins. To support the
largely poor, urban students, the school found outside
funds for a wide variety of after-school programs that
often include dinner, homework support, college mentors
and tutors, counseling and more. On top of regular
guidance services, two “deans” – one for boys and one
for girls – support the students’ lives and, when need
be, their families in order to support the students’
focus on and in some cases presence at school. Textron
students “Drop Everything And Read” (DEAR) for fifteen
minutes every day to elevate most students’ very low
literacy rates. The school enforces strict attendance,
lateness, dress code and homework-completion policies,
providing structure for a population who view this
school as their last and only chance to get a high
school diploma. ‘Chambers’ is currently tackling
curriculum reform and improving teaching strategies to
get more students to full proficiency. |
At the opposite end of the spectrum, we find a school in
suburban RI that is at the bottom of the value-added
lists, meaning its students are consistently
under-performing compared with their relatively affluent
counterparts statewide. This same school is both ‘high
performing’ and ‘improving.’ As a small state, RI has a
relatively small sample of students at both the affluent
and poverty ends of the socioeconomic spectrum, so the
statistics are sensitive to the performance of even a
rather small group. This suburban school is to be
commended for its improvement, certainly, but its
community has strong reasons to investigate the reasons
for their children’s low achievement as compared with
statewide counterparts.
|
We
recommend: |
|
Schools need to attend to the gaps that have grown over
the years between groups of students with different
characteristics. In all too many cases, schools focus on the high
achievers to boost their performance levels instead of
investing in those students who are, for whatever reason
weak academically or those groups of students who have
been underserved or not well understood in the past.
Indeed, the new NCLB requires that within 12 years all
students, including those with different
characteristics, demonstrate 100% proficiency, a goal
which researchers believe is virtually impossible to
achieve statistically, but whose spirit is still
ethically correct.
If all students are under-performing at the same low
levels, then school-wide, systemic improvement is
needed. Too often, for example, secondary schools
presume grade-level reading skills and offer little or
no support to those students whose work in all subjects
is impeded by an inability to read well.
|
|
