For QuotaClimat we classified about 20,000 climate articles into five news frames. GPT-5-mini labelled the training data, and we distilled those labels into multilingual XLM-RoBERTa classifiers.
What we saw
One frame, attribution of responsibility, dominated: about 62.5% of articles. Checked against human annotators, the LLM labels caught 91% of the real cases, but agreement was poor: Krippendorff's α of 0.28, against 0.62 to 0.69 for the other frames. The labeller was not missing responsibility. It was seeing it where people did not.
The classifier learned exactly that lean. A model trained on an LLM's labels inherits the LLM's habits, and a high recall number hides them.
Filters on top of the model
We did not retrain around it. Each frame's output was calibrated separately with Platt scaling, fitted against the human-labelled set. Responsibility got the strongest correction of the five, a slope of 0.44 against 1.12 for conflict, so a high score now means what it says.
For shares across the whole corpus we used prediction-powered inference, which uses the human-labelled set to correct the bias in the model's counts. The dashboard shows calibrated probabilities rather than hard labels.
What I took from it
Check agreement per class, not overall. One frame can undermine a whole typology while the average looks fine.