Plotting Permanency

This dashboard applies Aalen-Johansen competing-risks cumulative incidence (R cmprsk::cuminc) to the same FSFN removal/placement extract as the rest of this site. It is limited to episodes beginning in the Family First era and discharge categories are consolidated to Reunification, Adoption, Guardianship, Aged Out and Other Exit. Open episodes are not considered an exit (right censored), and every profile combination is fit separately rather than predicted by a regression. Cells under 30 episodes are shown as “not enough data” because that is not enough data for reliable calculations.

The Family First Prevention Services Act was signed in February 2018, but its funding and prevention-service mechanisms took time to be implemented and COVID delayed the rollout. Oct 2021 is when the Act’s effects on removals and outcomes seem to appear in the data.

DCF records 29 co-occurring maltreatment flags per removal. These are collapsed to a single reason in order of severity: Sexual Abuse > Physical Abuse > Substance Abuse > Domestic Violence > Child Behavior/Disability > Caregiver Incapacity > Neglect > Other. The category is chosen by the highest severity maltreatment, so a case with both parental drug use and sexual abuse is counted as Sexual Abuse. Hover any reason for the flags folded into it.

Removal count covers a child's entire removal history, not just removals during the Family First era. Placements count is limited to the 90 days after removal so it measures early instability and prevents biasing the numbers for kids who have been in care longer. The placement-count buckets (1 / 2-3 / 4-5 / 6+) were chosen from where outcome rates actually shift: at 6+, Adoption drops by more than half and Aged Out roughly triples, then plateaus.

An 18+ “removal” is typically a young person entering Extended Foster Care, not a maltreatment-driven removal. It is therefore omitted from this analysis.

Checked with 50% holdout testing at a fixed 18-month mark, repeated across 6 random splits over roughly 450 profile combinations per split, including every “Any” combination the tool itself can produce: predictions were off by 2.3 points on average and explained about 91% of cell-to-cell variation (R² = 0.91) — about 2× better than assuming the cohort average for every profile, and better than that baseline in all 6 splits.

The FSFN data does not include where a child's court case is heard. Circuit is therefore assigned based on the first placement’s zip code → county → circuit. Because it has 20 values, narrowing to one circuit often causes a warning of “not enough data”.

Held-out testing (fit on half the data, evaluate on the other half, across 6 random splits) found that including the circuit improved predictions in 4 of 6 splits — unlike age, reason, removal count, and placements, which each won 6 of 6 in their own held-out checks. Circuit is kept in the model so you can see the local variation in outcomes, which is very strong in some areas.

To figure out what factors really matter, I checked across 6 random holdout splits: age at removal is by far the strongest standalone factor (0.5 pt model MAE vs a 3.8 pt baseline, ~7.7×), then placements in the first 90 days (~4.2×), then removal reason and removal count, which are close to each other (~3.1× and ~3×). The dropdowns and the table are ordered by this ranking.

A stricter check — assuming the other factors are already known — makes the removal count look less helpful. Only 5 of 6 splits improved by a tiny margin (mean MAE 2.91 → 2.82 pts, ~3%). Once you know a child's age and placement count, also knowing their removal count adds a little but not much information. I kept it in the model because it does help a little, but it is the weakest of the four main variables by this stricter measure.

The goal of this dashboard is to give reliable numbers while keeping the number of variables to a minimum. Two variables that you would expect -- gender and race -- are not here because the differences associated with them were already accounted for in other variables.

Gender shows almost no raw difference in this data — Reunification is 34.5% for girls vs. 35.3% for boys, Adoption 6.4% vs. 6.0%, and every other outcome is similarly close, whether or not age and reason are controlled for. Adding gender would have made each group smaller, and therefore less reliable, without improving the model.

Race and ethnicity are different: raw outcome rates genuinely differ, most notably a lower Adoption rate for Black children (5.1%) than White children (7.3%), with somewhat lower Reunification too (33.1% vs. 35.3%). Omitting race as an option here isn't meant to erase those disparities. When doing the hold-out testing, the results showed that once a child's specific age at removal and removal reason are already known, adding race on top of the other varibles did not improve the model's permanency predictions — it got worse in all 6 of 6 random train/test splits (mean error 2.3 → 2.6 points). That's largely because adding race adds a lot of new categories (and therefore creates smaller groups) and race correlates with the removal-reason mix itself. For example, Substance Abuse is the primary reason for 57.9% of White children's removals but only 30.9% of Black children's, while Physical Abuse, Domestic Violence, and Caregiver Incapacity are all more common among Black children's removals — so much of the differences are explainable through the reasons and ages children are removed for, which the model already accounts for. Do not read the absence of race here as meaning that race doesn't matter or that children's experiences are equivalent across race.