ARTIFICIAL INTELLIGENCE
It looked right.
Would you share it?
Try a 30-second AI experiment. Make a call, see what the model picks up, and find out what it takes to change its mind. Run the experiment ↓
AI LITERACY IN PRACTICE
Spot the leap.
AI answer · Fictional example
“After the new reading routine, 8 of 10 students read faster. The routine caused the improvement.”
Which word would you challenge before sharing this with your team?
Reveal the leap
Caused. The example tells us what happened after the change. To judge what caused it, we’d need to see how reading was measured and what else changed.
Try this prompt: “Separate the observation from the interpretation. What evidence would I need before repeating the conclusion?”
THE 30-SECOND AI ILLUSION
Can you fool the model?
You’re the last person to see this card before it goes in a school newsletter. Would you share it, hold it, or ask for a source? Make your call before you open the model’s verdict.
New card
“A five-minute brain game doubles every student’s reading speed.”
Evidence provided: none.
01 · Reveal the model’s verdict
READY TO SHARE · 2 out of 2 blue examples had that label
02 · Flip the card. What did it see?
Blue border → ready to share. Both verified examples had blue borders. Both unsupported examples had peach borders. The model counted labels among cards with the same border color. The card looked trustworthy because of its frame.
03 · Change the examples. Does it change its mind?
Add one unsupported claim with a blue border and one verified claim with a peach border. Now only 2 of 3 blue examples were ready to share. Under this demo’s rule, it still says Ready to share.
04 · Give it something better to look at
NEEDS CHECKING. The new rule asks whether the claim has a credible source that supports it. This card has none. Better examples exposed the shortcut. Changing what the classifier could see and how it decided changed the outcome.
Try it with your team. Ask what shortcut an AI system might pick up from your examples, and which counterexample would expose it.
How I built this with AI
I used AI to explore ways to make a hidden shortcut visible. I kept the version you can step through and question. The claim, examples, and outcomes are fictional and prewritten. The reveal gives you a chance to make a judgment, change one thing, and see the consequence.
Literacy that outlasts the tool.
A useful prompt is one piece of the work. AI literacy also means asking how a system produces an answer, where it can fail, how information should be checked, and who remains responsible for a decision.
My work helps educators and leaders build a shared foundation, examine relevant uses, and move toward purposeful implementation.
Understand
Build a working vocabulary and examine the difference between a persuasive answer and a verified one.
Evaluate
Look at purpose, privacy, reliability, and the people who may experience the consequences of a tool’s use.
Apply & reflect
Try a bounded use case, keep human review in the process, and use what you learn to improve the next step.
An established learning pathway
Administrator Academy #4185
AI: Real Talk; Real Impact; Real Solutions — Engineering a Human-Centered AI Plan for Your School
ROE 39 lists Holly Kelly as the presenter for this Administrator Academy. For current availability and registration, consult the official academy listing.
Questions to take into the next meeting
What problem are we trying to solve? What information should stay out of the tool? What would count as a useful result? How will we verify it? What do people still need to understand and decide themselves?
Meet AI Holly
Try this 27-second lesson with students. Separate what the answer observed from the conclusion it drew.
Read the AI Holly transcript
Here’s an AI answer I’d bring into class. “More students turned in homework after we sent reminders, so the reminders worked.”
Ask students to underline the part we observed. Then have them circle what AI added. More homework came in. Why it came in is still an open question.
Now ask, “What else could explain it, and what would you check first?” That’s a question students can carry into the next AI answer they read.
Let’s keep
the conversation going.
Speaking, professional learning, and thoughtful collaboration.
BeTechWell / Holly Kelly
Tech moves fast. Humans matter more.