Mathematics · School
Mendenhall, William Camp
Look at a few charts and you'll see the same thing: very little change. Each chart shows what the school system "wants", the results it is built to produce, plus the effects of circumstances from outside the school, like COVID and family income.
As long as the school system stays the same, there's no reason to expect results to rise above the upper limit, the top edge of the shaded band.
Once a school uses the Lead Out Method, that is what we expect to see: results climbing above the upper limit. It's the sign that the system now "wants" something different. What you want.
Each dot is one year. It shows how far the typical student's score was from the state's "meets standard" mark. Zero means right on it. Above zero is above standard; below zero is below.
Every school's results bounce around a little from year to year. A different group of students, a flu season, a new teacher: small things push the number up or down. That bounce is normal, and it doesn't mean anything has really changed.
The shaded band marks how far the bounce normally goes. We work it out from the school's own history: how much its result typically moves from one year to the next.
The small numbers under each year are how many students took the test. In a small school a few students make a big difference, so its dots bounce further and its band is wider.
- A dot inside the band is ordinary bounce. The school is doing what it has always done.
- A dot outside the band is a real signal. Something changed, for better or worse, and it's worth asking what.
A flat band with the dots inside it means the system is stable. It will keep producing about the same results year after year, however hard everyone works. Teachers aren't the reason the dots stay put. The way the school is organized is. Changing that is what moves the dots.
The little house in each year shows the share of students eligible for free or reduced-price school meals, a rough measure of family income. It sits on the same scale as the results: 0% at the top of the chart, 100% at the bottom. The fewer families with low incomes, the higher the house. Income moves results too, and it comes from outside the school.
Step 1 · Each year's result
For each grade tested, take the average (mean) scale score from the California Department of Education's results file, and subtract the lowest score that counts as "Standard Met" for that grade. That is the grade's distance from standard. Weight each grade by its share of the students tested, then add the grades up. Grades where the state hides the average because ten or fewer students tested are left out entirely.
2015 from sb_ca2015_1_csv_v3.zip
| Grade | Line in file | Average score | Standard Met | Distance | Students | Weighted |
|---|---|---|---|---|---|---|
| 11 | 20,242 | 2371.0 | − 2628 | −257 | × 15 / 15 | −257 |
| Sum, rounded to a whole point | −257 | |||||
2017 from sb_ca2017_1_csv_v2.zip
| Grade | Line in file | Average score | Standard Met | Distance | Students | Weighted |
|---|---|---|---|---|---|---|
| 11 | 19,519 | 2391.5 | − 2628 | −236.5 | × 13 / 13 | −236.5 |
| Sum, rounded to a whole point | −237 | |||||
Step 2 · The average and the normal range
- Average: add the yearly results and divide by how many there are. −257, −237 → −247
- Typical year-to-year change: how far the result moved from each year to the next, ignoring up or down. 2015→2017: 20 → average 20
- Normal range: multiply the typical change by 2.66, then add it to the average and subtract it. 20 × 2.66 = 53.2. Upper limit −193.8, lower limit −300.2.
Why 2.66? It is the standard factor from quality control, the work of Walter Shewhart and W. Edwards Deming. It turns "how much the result typically changes in a year" into "how far it can wander by chance alone".
Years with no result (no state tests were given in spring 2020) are skipped: the year-to-year change runs from the last year with a result to the next one.
Step 3 · Free or reduced-price meals
The share of enrolled students eligible for free or reduced-price meals. From 2014–15, 2015–16, 2016–17.