That Viral AI Study About Critical Thinking? Apply Critical Thinking to It.

Has anyone else seen the studies about AI reducing critical thinking that have been making rounds on LinkedIn?

Prepared to be disappointed.

What they researched ≠ What they claimed.

There's a lot of irony there... If you apply critical thinking to the study itself, you find this out yourself.

---

🚩 The Methodological Problems

Here are the key issues with this viral research, some of which are acknowledged in the study's limitations section, others not:

Issue #1: Self-Reported Survey Data

The study measures "critical thinking" using self-reported survey questions. No tests. No experiments. Just how people think about their own critical thinking skills.

This is a fundamental flaw. Self-perception of cognitive ability is notoriously unreliable. The Dunning-Kruger effect exists precisely because people are bad at accurately assessing their own cognitive abilities.

If you want to measure whether AI affects critical thinking, you need to actually test critical thinking — not ask people how they feel about it.

---

Issue #2: The Questions Measure Trust, Not Thinking

The actual "critical thinking questions" in the survey are more about how much you trust AI and less about measuring your ability to think critically.

Some examples from their methodology:

"How often do you critically evaluate sources?"

"How confident are you in spotting fake news?"

"Do you question the motives behind AI recommendations?"

These questions measure attitudes toward verification and trust in AI systems — not actual critical thinking ability.

There's a massive difference between asking "Do you think you're a critical thinker?" and testing whether someone can identify logical fallacies, evaluate evidence, or construct sound arguments.

---

Issue #3: The Foundation Paper Problem

The researchers claim their methodology is based on another research paper about measuring critical thinking. But there are two problems:

🚩 Issue #3.1: Wrong Audience

That foundational paper was designed for children, while this study was applied to adults. Cognitive assessment tools don't just transfer across age groups without validation.

🚩 Issue #3.2: They Don't Follow the Methodology

They don't actually follow the methodology of the originating paper. They adapted it (loosely), which undermines the claim that they're building on validated research.

---

🤔 What I Think They Actually Found

Despite the sensationalist framing, there are some legitimate insights buried in this research:

1. People Are Trusting AI Outputs Without Verification

More people are trusting AI outputs instead of doing their own research or verification — and that's the part worth paying attention to.

Not because AI is inherently harmful… but because it shows our deep reliance on these systems for information.

I don't fact-check my calculator when I plug in 5 + 5. And increasingly, people aren't going to fact-check their LLM of choice for gathering information either.

That's a behavioral shift worth understanding — but it's not the same as "AI is eroding critical thinking."

---

2. There's Something Here Worth Researching More

Despite the sensationalist headline, there were some interesting findings worth exploring in more detail with better methodology.

The question of how AI tools affect our cognitive patterns is genuinely important. We just need better research to answer it.

---

📊 The Broader Point

Before we share misinformation with headlines saying "AI is eroding critical thinking," let's model the skill we're worried about losing.

This is exactly the kind of content that goes viral because it confirms existing fears. The headline is scary. The conclusion feels intuitive. And most people won't dig into the methodology.

But if we're genuinely concerned about critical thinking in the AI age, the best thing we can do is demonstrate it ourselves:

• Question sensationalist claims — even (especially) when they align with our priors
• Read the methodology, not just the abstract
• Distinguish between what was measured and what was claimed
• Consider alternative explanations for the findings

---

📎 Read the Original Study

If you want to evaluate this yourself (and I encourage you to), here's the original paper:

AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking

Apply some critical thinking to it. See what you find.

---

💭 Final Thoughts

AI tools are powerful. They're changing how we work, research, and make decisions. There are legitimate questions about how reliance on these systems affects our cognitive habits.

But answering those questions requires rigorous research — not surveys about how people feel about their thinking abilities.

The irony of spreading a poorly-constructed study about critical thinking… while failing to apply critical thinking to the study itself… is almost too perfect.

Let's be better than that.