FIELD NOTE 03 / AI & LEARNING
When the AI gets better grades than you do.
An excellent answer on the screen is not necessarily an explanation you can give on your own.
What the researchers tested
Nearly a thousand high-school mathematics students participated in a randomized field experiment in Turkey. The researchers compared a basic GPT-4 chat tutor, a tutor with teacher-informed safeguards, and a control group without AI access.
Both AI groups performed better during supported practice. On subsequent unassisted exams, the basic-tutor group performed worse than the control group: the published paper reports a 17% reduction in grades, not a 17-percentage-point drop. The guarded tutor largely avoided that harm, but did not demonstrate a positive unassisted-exam effect over the control. [1]
How this informs Parlor
Our design interpretation: keep assistance visible when interpreting a result. Parlor records relevant in-app help in its lecture checks and distinguishes assisted work, revisions after feedback, and qualifying later responses answered without help.
That separation matters because a polished revision can be useful practice without being new evidence of independent understanding. AI feedback should give you something to examine against your source, not a reason to stop thinking.
The boundary: Parlor cannot observe all outside help, and its unaided checks rely partly on the reader’s declaration. This study supports asking careful questions about tutor design; it does not validate Parlor’s effectiveness or show that Parlor reproduces the study’s safeguards.
Read about feedback and study workflows →TRY IT · NO ACCOUNT NEEDED
Close the chat. Keep the question.
- Before asking AI, write your current answer and the part you cannot explain.
- Ask for a hint about that part. Check the hint against your course material.
- Put the conversation away and explain the idea again on paper.
- Try a different example later, without the chat. Notice where you still need help.
A prompt that leaves you some work
“Here is my explanation. Ask me one question about the gap in my reasoning before offering an answer.” A prompt is a request, not a guarantee that a model will follow it correctly.
An editorial study exercise, not the intervention tested in the paper.
Yes, there is a reference.
[1] Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26), e2422633122.
Read the original research ↗Our explanation and exercises are written for Parlor Study. The researchers have not evaluated or endorsed Parlor. We link to their work rather than reproduce the paper.
Spotted a mistake? Suggest a correction →