Where AI makes smart people worse
The "jagged frontier" describes how AI is excellent at some tasks and surprisingly bad at others that seem just as hard. In a 2023 Harvard Business School and BCG study of 758 consultants, AI users completed about 12% more tasks and produced work rated over 40% higher in quality, but on a task outside AI's abilities, they were 19 percentage points less likely to get the right answer.
Most of what you hear about AI at work falls into one of two camps.
Camp one: AI is magic and will make you ten times more productive.
Camp two: AI is overhyped and makes things up.
The best research I've seen says both camps are right, depending on the task. And the scariest part is that it's hard to tell which kind of task you're doing.
758 consultants and a jagged line
In 2023, a team of researchers from Harvard Business School, Wharton, MIT, and other schools partnered with Boston Consulting Group on a large experiment. Led by Fabrizio Dell'Acqua, the study included 758 BCG consultants, some of the most analytically capable professionals around.
The consultants did realistic consulting tasks. Some had access to GPT-4. Some didn't.
On a set of 18 tasks the researchers had chosen to be well within the AI's abilities, the results were impressive. Consultants using AI completed about 12% more tasks, finished about 25% faster, and produced work rated more than 40% higher in quality.
Then came the twist.
The researchers also included a task they had deliberately designed to sit outside the AI's abilities. It looked similar in difficulty to the others. But it required a kind of reasoning the AI couldn't handle well. On that task, consultants who used AI were 19 percentage points less likely to get the right answer than consultants who worked without it.
The AI didn't just fail to help. It made smart, experienced people worse.
The jagged frontier
The researchers described AI's abilities as a jagged technological frontier. Imagine a line separating what AI does well from what it does badly. You might expect that line to be smooth: AI handles easy tasks and struggles with hard ones.
It isn't. The line is jagged. AI can be excellent at some tasks that seem hard and surprisingly bad at others that seem similar or even easier. And there's no warning sign when you cross from one side to the other.
That's what tripped up the consultants. On the task outside the frontier, the AI gave a confident, well-written answer that was wrong. And the consultants, who were used to AI giving them great answers on the other tasks, trusted it.
The danger isn't that AI is bad. It's that AI is good enough, often enough, that we stop checking.
(A note: this experiment used GPT-4 in 2023. Today's AI tools can do more, so the specific tasks inside and outside the frontier have shifted. But the frontier itself is still jagged.)
What this means for your business
Most business owners start using AI on easy wins: drafting emails, summarizing notes, brainstorming headlines. Those are usually well inside the frontier, and AI does them well.
That success builds trust. Then, gradually, you start asking it harder questions. Should I raise prices? How should I structure this contract? What's causing my sales to drop? And because it answered the easy questions so well, you trust the answers to the hard ones, too.
That's exactly the pattern that hurt the consultants.
How to stay on the right side of the frontier
1. Know which tasks you're giving it. AI is generally strong at drafting, summarizing, reformatting, and brainstorming. Be more careful with anything that requires judgment, specific facts, math, or knowledge of your particular situation.
2. Check the high-stakes stuff. The more important the decision, the more you should verify. A social media caption? Skim it. A pricing decision, a legal question, or a financial number? Check it against a real source.
3. Watch out for confidence. AI sounds sure of itself whether it's right or wrong. Confidence is not a signal of accuracy.
4. Test it on things you already know. Before trusting AI in a new area, ask it questions where you already know the answer. See where it's strong and where it slips. That's how you map your own frontier.
5. Stay in the driver's seat. The researchers found that the consultants who got the most out of AI actively worked with it rather than just accepting its output. Treat it as a very fast, very confident assistant, not an oracle.
Use it. Just don't trust it blindly.
I use AI every day in my business. It saves me real time and helps me do better work. But I've learned to treat it the way I'd treat a brilliant new assistant: great at a lot of things, and still in need of someone checking the work that matters. The smart people in the study weren't hurt because they used AI. They were hurt because they stopped thinking.
Don't stop thinking.
AI can speed up the work, but it can't make you follow through. If you want someone checking in on your progress every week, that's what I do.
Study to link: Dell'Acqua et al. (2023), "Navigating the Jagged Technological Frontier"
Frequently asked questions
What is the jagged technological frontier?
It's a term from Fabrizio Dell'Acqua and colleagues' 2023 study describing the uneven boundary between what AI does well and what it does poorly. Tasks on either side can look equally difficult, and there's no warning when you cross from one to the other.
Can using AI make your work worse?
Yes, on tasks outside AI's abilities. In the BCG study, consultants who used AI on a task designed to fall outside the frontier were 19 percentage points less likely to get the correct answer than those who didn't use AI.
How do I know when to trust AI at work?
Use it freely for drafting, summarizing, and brainstorming. Verify anything involving judgment, specific facts, math, or your particular business, and test AI on questions you already know the answer to before relying on it in a new area.
