AI is moving mental work out of people’s heads and into systems they often cannot fully inspect. That is useful, but it is also dangerous. The central risk is the erosion of judgment: fewer people practicing the work of understanding, checking, deciding, and taking responsibility.
AI systems can summarise, explain, recommend, draft, rank, rewrite, tutor, and produce fluent arguments that may be wrong. That fluency changes how people relate to their tools. A calculator gives an output; a chatbot may give an argument. A search engine points outward; an AI system often gives a blended answer.
That makes AI different from older tools for cognitive offloading. It does not merely store information or speed up calculation. It can perform the visible shape of thought. It can give the answer, the explanation, the confidence, and the prose. The user may still be responsible for the result while having done less of the work needed to judge it.
The evidence supports a more specific claim than generalised cognitive decline. AI can improve performance, widen access, and reduce unnecessary effort. It can also encourage overreliance, reduce practice, and weaken error-checking when users treat fluent output as trustworthy.
The test is what AI leaves humans able to do. After using it, do people understand more, or do they merely possess a response? Can they identify the source of a claim, test an argument, notice uncertainty, reject a bad answer, and own the decision? As AI enters classrooms, workplaces, search, writing, and professional judgment, that is the standard that matters.
The wrong question
Human beings have always moved parts of thinking into tools. We write notes, set reminders, follow recipes, use calculators, consult maps, search the web, ask colleagues, and rely on institutions to store knowledge we cannot carry ourselves. Civilisation depends on shifting memory, calculation, and coordination into systems outside the individual mind.
Researchers call part of this “cognitive offloading”: using outside tools or actions to reduce the mental effort a task requires. In a 2016 review in Trends in Cognitive Sciences, Evan Risko and Sam Gilbert described cognitive offloading as using physical action to change a task’s information-processing demands. A phone reminder, checklist, marked-up document, or navigation app can all work this way.
Offloading can make people more capable. In Gilbert’s work on reminders, reminders improved performance. Later work by Annika Boldt and Gilbert found that people were more likely to offload when they had less confidence in their memory.
That should make the AI debate less moralistic and more practical. Serious people use help. The issue is the kind of help the tool provides, the kind of work the user stops doing, and whether the user can still judge the result.
When tools extend thought, and when they replace it
Useful offloading can improve thinking. A checklist can prevent an omission. A calculator can free attention for the problem instead of the arithmetic. A calendar can make a promise reliable. A writing tool can help a person see alternatives.
AI can perform the same role. It can generate examples, rephrase dense material, make a first draft less intimidating, and help a beginner ask better questions. For some tasks, shifting part of the work outside the head lets people focus on higher-level judgment.
But offloading becomes harmful when the tool replaces the practice that builds judgment. The difference is often invisible in the final output. A student may have a correct answer without understanding the method. A worker may have a polished memo without knowing whether its claims are true. A reader may have a summary without seeing the uncertainty in the original source.
Support and substitution are the central distinction. In the best case, AI gives hints, shows sources, asks clarifying questions, and helps a user compare alternatives. In the weaker case, it supplies the answer, explanation, prose, and confidence. The user’s performance improves while independent understanding remains thin.That finding has limits.
A tool can help someone complete a task and still reduce the practice that person gets doing it unaided.
From memory aids to answer machines
Research on outside information stores shows how tool access can change what people remember. In a 2011 Science paper, Betsy Sparrow, Jenny Liu, and Daniel Wegner reported what became known as the “Google effects on memory”: when people expected later access to information, they tended to remember less of the information itself and more about where to find it.
That finding has limits. Parts of the work have faced replication challenges, and laboratory memory studies do not map neatly onto everyday intelligence. The narrower warning still matters: when people expect reliable outside access, they prepare and remember differently.
AI changes the outside store into an answer machine. It is no longer just a library, index, or map to information. It can produce a synthesised response. Users may stop remembering where to look, and they may also stop looking.
The older warning: automation bias
The problem of overtrusting machines did not begin with chatbots. Human-factors researchers have long studied how people behave around automation. In a 1997 Human Factors article, Raja Parasuraman and Victor Riley described patterns of use, misuse, disuse, and abuse of automation. Misuse included overreliance.
Later work on automation bias gave the problem sharper terms. People can make omission errors: they fail to act because an automated system did not alert them. They can also make commission errors: they take a wrong action because an automated system suggested it. In a 2010 review, Parasuraman and Dietrich Manzey connected automation bias and complacency to failures of attention and monitoring.
Automation changes vigilance. When a system is usually helpful, users stop checking the parts that most need checking. Trust becomes a habit before it becomes a judgment.
AI inherits this problem and adds a new surface. Many automated systems produce signals, alerts, or recommendations. Generative AI produces language. It can justify itself. It can sound as though it has weighed the evidence. That makes misplaced trust feel less like passivity and more like persuasion.
Fluency and misplaced trust
A fluent answer is not a verified answer. Fluent prose makes weak claims feel complete, coherent, and ready to use.
Research on AI-assisted decision-making suggests that explanations do not automatically protect users. In some experiments, explanations increased acceptance of AI recommendations even when the recommendations were wrong. Other studies have found that explanations can reduce overreliance when they make checking easier, show uncertainty, or prompt the user to slow down.
That is a design lesson. An explanation can help people check AI, or it can become another layer of persuasive output. A source citation helps only when it is real, relevant, and inspected. An uncertainty warning helps only when users notice and understand it. Friction helps only when it pushes the user back into judgment rather than merely slowing the task.
The interface shapes what the user thinks the task is. When the system presents one polished answer, the user acts like an editor. When it presents uncertainty, sources, alternatives, and reasons to check, the user acts more like an investigator.
The same distinction applies to writing. AI can help a writer explore structure, test phrasing, and find gaps. It can also produce a smooth draft that hides weak evidence. The danger is not machine-written prose. The danger is the human surrendering the question of whether the sentence should exist at all.
Learning: help that teaches, help that replaces struggle
Education makes the support problem concrete. Learning requires retrieval, mistakes, testing methods, and revision. A tool that removes those demands can improve immediate performance while weakening later understanding.
The evidence points in both directions depending on design. Research on intelligent tutoring systems has found that well-designed tutoring tools can improve learning. A 2016 meta-analysis by James Kulik and J.D. Fletcher found positive effects for intelligent tutoring systems, though effects varied by assessment type, control group, and implementation quality.
Unrestricted help behaves differently. In a field experiment in high-school mathematics, Hamsa Bastani and co-authors tested generative-AI tools in a classroom setting. Students with access to a ChatGPT-like base tool did better while using it for practice, but later performed worse on an unassisted exam after researchers removed the tool. A more guarded tutor, designed to provide hints and support rather than answers, avoided much of that harm.
The study has boundaries: one subject, one setting, one duration, and one design. Its lesson is still clear. Help that preserves the student’s mental work can teach. Help that replaces the work can damage learning.
Schools should judge AI by what it asks students to do. Does it prompt them to reason, retrieve, explain, and check? Or does it let them submit the appearance of understanding without the practice that produces it?
Work: productivity, pressure, and uneven gains
In the workplace, AI’s results depend on the task, the worker, the system, and the user’s ability to detect failure. Some evidence shows productivity gains in specific tasks. In one customer-support setting, Erik Brynjolfsson, Danielle Li, and Lindsey Raymond found that access to a generative-AI assistant increased productivity on average, with larger benefits for less-experienced and lower-skilled workers. In a controlled experiment on professional writing tasks, Shakked Noy and Whitney Zhang found that ChatGPT access helped participants finish faster and improved evaluator-rated quality.
The same evidence also shows why broad claims about AI productivity are unreliable. Effects change with task type, worker skill, system quality, and the user’s ability to tell when the tool is outside its competence.
A study of consultants by Fabrizio Dell’Acqua and co-authors made this point with the phrase “jagged technological frontier.” Using GPT-4-era tools, consultants performed better on some tasks and worse on a task chosen to sit outside the system’s reliable abilities. The boundary depends on the task and model, but the lesson is durable: AI competence is uneven.
That is the workplace version of the same cognitive shift. AI may draft the email, summarise the transcript, propose the strategy, or generate the code. Someone still has to know what counts as good work. Someone has to check the facts, notice missing context, understand the consequences, and decide when the answer is not good enough.
AI reduces effort in one place while shifting effort elsewhere. The drafting burden falls; the checking burden rises. Organisations that measure only speed miss the new labour of verification. Organisations that ignore checking turn convenience into risk.
Accountability: the human still owns the output
Professional rules are making one principle clear: AI output does not check itself.
The cautionary case is Mata v. Avianca. In 2023, lawyers in a federal case submitted filings that included nonexistent cases generated by ChatGPT. Judge P. Kevin Castel’s sanctions order described fake judicial opinions with fake quotes and citations. The case does not prove such failures are common. It shows what can happen when fluent output enters a professional setting without enough checking.
Journalism has drawn a similar line. The Associated Press’s 2023 guidance says generative-AI output should be treated as unvetted source material and checked under AP standards. It also says AP staff should not use AI to create publishable content.
The legal profession has made the responsibility explicit. The American Bar Association’s Formal Opinion 512, issued in 2024, says lawyers using generative AI must consider duties including competence, confidentiality, communication, supervision, candor to tribunals, and reasonable fees. The opinion emphasises that lawyers remain responsible for their work.
These examples show how professions built around checking classify AI: not as an authority, but as material requiring human responsibility.
That norm matters outside the professions, too. A person can use AI and still be accountable. Accountability requires knowing what has been delegated, what remains human, and what must be checked before the output is used.
When dependence is the wrong frame
Reliance on a tool does not automatically mean loss. For some people, offloading increases independence.
Recent scholarship on AI and disability describes both benefits and risks. Researchers have noted that AI may help with communication, information access, health management, and assistive technologies. They have also warned about inaccessible interfaces, unreliable outputs, checking burdens, privacy concerns, bias, and the risk of designing systems without disabled people’s participation.
The goal is not maximum unaided cognition. A tool that helps someone communicate, navigate information, or reduce a barrier should not automatically be framed as cognitive decline. Assistance can be agency. Offloading can be independence.
The better measure is control. Did the tool make a task possible? Did it preserve choice? Did it provide output the user could inspect, adapt, and reject? Or did it create a new dependency on a system that is opaque, inaccessible, or hard to challenge?
The test
AI should be judged by what it leaves the human able to do.
After using it, do you understand the problem better, or do you only possess a response? Can you identify the source of a claim? Can you explain why the answer is plausible? Can you notice uncertainty? Can you check the work? Can you reject the output? Can you own the decision?
Those are the questions for any powerful tool. Human beings have always used tools to extend thought. AI becomes dangerous when people lose track of which parts of thinking have shifted into the machine.
AI can support judgment, teach, speed work, and expand access. It can also replace judgment, weaken learning, conceal errors, and create new forms of dependence. The difference lies in design, context, incentives, expertise, and checking.
The practical test is simple: what did you still understand, judge, check, decide, and own?