When AI Does the Work, Who Teaches Us Taste?
Two pieces of news landed in the same week, and I can’t stop reading them side by side.
The first is a number. On the Remote Labor Index from Scale Labs and CAIS, which checks whether AI agents can finish real, paid freelance projects at professional quality, Anthropic’s Fable 5.2 completed 41.3% of them. A year ago the top score was 2.5%. These are 240 projects from Upwork professionals, in audio, architecture, 3D and more, not toy tasks.
The second is a diagnosis. Ethan Mollick spent time with the same model and called it “Jagged AGI”, close to matching most human experts on most computer-based work. What’s interesting is where he says it still fails. Not facts, but judgment and taste. It built him a course that was technically complete and joyless. It wrote a rulebook for a 3D world that grew to 204 sections and about 87,500 words, and never noticed it had overbuilt. The model did the work. It just couldn’t see that the work was off.
As someone who has worked in learning for over a decade, that second point is the one I care about. Because that is exactly how taste fails in people too.
Taste is learned at the junior desk
Nobody is born knowing a course is joyless or a document is bloated. You learn it the slow way. You make a first draft, someone senior looks at it and says “this part drags” or “you don’t need this section,” and after enough of those moments you start hearing their voice before they say anything. That voice in your head is what we later call judgment.
The raw material for that loop has always been junior work. The first-pass analysis, the slide deck, the meeting notes, the small bug fix. Not glamorous, but it was the practice field where someone could watch you and correct you.
That field is shrinking. Robert Szczerba’s piece in Forbes, AI Is Taking Entry-Level Jobs. Who’s Training Your Next CEO?, puts numbers on it. Stanford research found employment for 22 to 25 year olds in the most AI-exposed jobs running well below expectations, and recent graduates in exposed majors are starting at lower pay. His worry is the pipeline. If no one does the junior work, who becomes the senior person five years from now?
Put the two news items together and the problem is sharper than “AI takes jobs.” AI is now good at the tasks we used to learn on, and it is still weak at the very thing those tasks were secretly teaching us.
Why “just add AI coaching” isn’t the answer
The coaching world is already responding. This week’s Coaching Brief reports on AceUp pairing human coaches with an AI tool for NTT DATA staff, with a vendor-reported 22% gain in handling difficult conversations. The same roundup mentions a study where coaching students practised with ChatGPT playing the client.
I like both experiments. AI is a patient sparring partner, and practice volume matters. But notice the caveat in the brief itself, the 22% comes without a published method. And notice what the AI is doing in both cases. It is giving people more reps. Reps are necessary, but reps alone don’t build taste. Taste comes from reps plus a good critic plus a moment where you compare your judgment to theirs.
If Mollick is right that the model can’t tell its own work is off, then it is a risky critic to leave alone with a learner. It can tell you whether you followed the framework. It is much less reliable at telling you the thing is dull.
Mentoring has to become the curriculum
Here is my argument, from the product and learning-design side. For most of my career, mentoring was treated as a perk. A nice program, a coffee chat budget, something HR reports on. It worked because the real learning happened anyway, inside the junior work, and mentoring just helped it along.
That assumption is breaking. If the junior work moves to AI, the learning won’t happen by itself anymore. So the moments where a more experienced person helps you see what’s off need to be designed, scheduled and measured like any other core part of the job.
What would that look like in practice? Three design moves I’d start with.
Critique the AI’s draft, then compare notes
Give a junior person a real AI-generated draft and ask them to mark what’s weak and why, before a senior person does the same separately. Then put the two critiques side by side and talk through where they differ.
Say the junior person flags a typo, a vague sentence and a missing source. The senior person flags those too, but also notices that the whole second half answers a question the client never asked. That second list, the things only the senior person caught, is exactly what the junior person can’t see yet. It is the most useful thing for them to learn next, so the conversation should spend most of its time there, on why the senior person noticed it and what gave it away.
Run this every few weeks and keep the lists. Over time, the things only the senior person catches should get fewer and subtler. That shrinking list is one of the few concrete ways I know to watch taste grow. It also turns AI output from a replacement for practice into the practice material.
Make the senior person say the quiet part out loud
Experts often fix things without explaining. “I just cut that section.” Ask them to narrate the reason, even briefly. “It repeats point two and slows the reader down.” That sentence is the transferable part. It’s what the learner will later hear in their own head.
Check judgment, not just output
Before someone gets more responsibility, test whether they can spot problems without the AI, not only whether they can ship with it. A person who can only produce good work with a tool, but can’t tell good from bad, hasn’t learned taste. They’ve borrowed it.
The question I’m left with
New York is setting up a state center to independently test AI systems and to train a new generation of AI safety experts. Whatever you think of the policy, the instinct is right. When systems get more capable, we need more people with trained judgment, not fewer.
The same is true inside every team. AI can now do a remarkable share of the work. What it can’t yet do is notice that the work is off, and that noticing is exactly what we have always passed from person to person.