Research
The claim we intend to earn
We have not run a pilot yet. This page says what we are measuring and how, so that when there are results you can judge whether they were measured honestly. It will be replaced with outcomes, not testimonials.
The question
Does explicit instruction in C5 improve middle-school students’ ability to use generative AI critically and independently?
The method
Students complete a realistic scenario before instruction and a structurally matched scenario after. Both are scored against the same four-level rubric by the same evaluator, using the same evidence rules. Growth is reported per competency.
Deterministic behavioral signals are recorded alongside the rubric — did the student supply constraints, ask a follow-up, seek evidence, decide, justify. Those are countable without a model in the loop, which makes them the harder number to argue with.
What gets reported
- Growth per competency across the five C5 skills.
- AI deference rate — the share of missions ending on the AI’s recommendation without meaningful challenge. The number we expect to fall.
- Human override quality — when a student rejects AI advice, the rubric level of their justification.
- Evidence-seeking frequency — how often a student asks what a claim rests on.
The rubric, in full
Published because a scoring system you can’t inspect is a scoring system you shouldn’t trust.
Curiosity
Depth and usefulness of the questions a student asks.- Emerging
- Asks only the obvious opening question.
- Developing
- Uses some follow-up questions.
- Proficient
- Identifies missing information and explores useful alternatives.
- Advanced
- Asks questions that reveal assumptions, tradeoffs, or unseen possibilities.
Clarity
How well a student communicates objective, context, and constraints.- Emerging
- Request is vague or underspecified.
- Developing
- Includes some context or a goal.
- Proficient
- Clearly communicates objective, context, and the important constraints.
- Advanced
- Prioritizes competing constraints and adapts instructions as the problem evolves.
Creativity
Breadth and originality of the alternatives a student explores.- Emerging
- Accepts the first solution.
- Developing
- Requests alternatives.
- Proficient
- Explores multiple meaningfully different approaches.
- Advanced
- Combines, transforms, or extends ideas into something original.
Critique
Ability to find errors, omissions, assumptions, and evidence gaps.- Emerging
- Accepts output at face value.
- Developing
- Notices obvious problems.
- Proficient
- Identifies assumptions, missing information, weak reasoning, or questionable claims.
- Advanced
- Systematically tests evidence, counterarguments, and alternative explanations.
Choice
Quality and justification of the student’s own decision.- Emerging
- Uses the AI recommendation as the final answer, unexplained.
- Developing
- Makes a choice with limited rationale.
- Proficient
- Makes an independent choice supported by relevant reasoning.
- Advanced
- Balances tradeoffs, rejects weak AI advice where warranted, and explains the decision.