My SaaS bill for AI is getting ridiculous.
Multiple Claude accounts, ChatGPT Plus, Perplexity, Kimi, local open-source models on rented GPUs. I’m not playing with these things out of philosophical curiosity. I run Welltory, which means on any given Tuesday my day is a chaotic ping-pong match between strange artifacts in optical heart-rate data, redlining enterprise contracts, debugging buggy pull requests, dealing with team drama, and putting out whatever fires just flared up in production.
And across every single model, whether it’s Claude 3.5, GPT-4o, or an open-weight model, the exact same character walks into the room.
He’s brilliant. He’s relentlessly polite. His formatting is gorgeous. He has memorized every framework from every textbook and business school syllabus on earth. And for about half of the real operational problems on my desk, his advice is articulately, confidently useless.
He has never lost a dime of his own money. He’s never had to sweat through payroll on a Friday afternoon. He’s never had an angry enterprise client screaming at him on Zoom.
Everyone has independently landed on the same description: it feels like working with an eager college student. I think that phrase is far more literal than people realize. And once you look at the mechanism behind it, you can predict in advance which tasks the model will nail and which ones will quietly burn half your workday.
The Instagram Filter on Human Labor
Everyone knows models learn from public data. But we rarely stop to think about what had to happen to a piece of human work before it was allowed to become “data” in the first place.
Look at what actually gets published on the internet.
We publish finalized legal briefs. We never publish the backchannel phone call where our counsel sighed, leaned back, and said: “Look, they’re bluffing. They don’t have the budget or the appetite to litigate this, just sign it and take the risk.”
We publish peer-reviewed papers that make scientific discovery look like a pristine, logical staircase:
Hypothesis
Method
Results
We bury the two years of dead ends, the broken centrifuge, the contaminated assays, and the blind, irrational hunches that actually cracked the problem.
We publish working code. (Though code is a special case, and I’ll come back to why it’s the exception to everything).
What reaches the public record has been heavily sanitized. Anything embarrassing, legally risky, chaotic, or politically awkward gets ruthlessly cut out.
The model didn’t learn human work. It learned how work looks after aggressive PR editing.
And no, training on private enterprise data or internal wikis doesn’t fix this. Writing something down for your colleagues is still a performance. An internal Notion doc has an audience. A post-mortem document—no matter how loudly the company preaches “blameless culture”—is written by someone who knows leadership is reading it. People naturally write how they wish the work went, or at least the version they are willing to be seen producing.
The training corpus is the world through an Instagram filter. Every kitchen photographed from the one clean angle where the dishes aren’t piled in the sink.
The Compiled Binary Problem
Arvind Narayanan, a computer science professor at Princeton, came up with a thought experiment that permanently reframed this for me:
Imagine a parallel universe where open-source software never existed. Nobody ever published a single line of human-readable source code on GitHub. The only public record was petabytes of compiled, closed-source binaries—raw machine code.
Eventually, giant models trained on those binaries would learn to generate working applications directly from text prompts, skipping source code entirely.
And that software would be completely unmaintainable, bloated, and insecure.
This isn’t just a thought experiment; researchers have actually tested it. A project called LLM4Decompile trains language models to turn raw assembly code back into C source code. The recovered code only executes correctly about 20% to 25% of the time. The similarity to the original human code sits somewhere between 6% and 14%. Obfuscate the binary, and success drops off a cliff.
That isn’t a failure of the neural network. It’s a measurement of how much vital context a compiler destroys.
When you compile code, the names of variables vanish. The comments explaining why an obvious approach was rejected are gone. The commit messages, the pull request debates about database locks, the dirty workarounds—all deleted.
Software works well with AI today because of a historical freak accident: for thirty years, developers published not just the finished building, but the scaffolding. They published their bugs, their arguments on Stack Overflow, their broken builds, and their public postmortems.
In software, the model saw the construction site.
In almost every other profession, you are already living in Narayanan’s binary-only universe. If you’re a lawyer, a doctor, a scientist, or a founder—the model only has your compiled outputs. It has the finished artifact, but the messy, painful process of how it got made was scrubbed before pre-training even began.
Even in Code, Taste Doesn’t Compile
Engineers love to relax here and think they’re safe. They shouldn’t.
Yes, AI is genuinely good at syntax. But watch it tackle real product architecture, and you’ll see the exact same pathology: it violently over-engineers. It adds three layers of abstraction you didn’t ask for. It writes defensive handlers for edge cases that won’t happen until the next century. And it physically cannot delete anything. Ask it to simplify, and it just swaps one complex architecture for another.
It has zero product judgment. It doesn’t know what to hack together in twenty minutes and what to obsess over.
When Daniel Ek and his team built early Spotify, they spent an insane amount of company runway getting track playback down to 200 milliseconds. Nobody in user research was begging for 200 milliseconds; 500ms felt perfectly fine on the internet of 2006. But Ek understood that to kill music piracy, streaming couldn’t feel like downloading a file—it had to feel like double-clicking an MP3 already sitting on your local hard drive.
You won’t find that decision in Spotify’s Git commits. It wasn’t in their architecture diagrams. It happened in a room, driven by founder intuition about human frustration.
AI in engineering is a tireless, lightning-fast junior dev with negative product taste. Great for typing speed; dangerous for product direction.
The Lawyer Who Would Have Killed Uber
Ask any model how to launch an on-demand ride service in an entrenched taxi market.
You’ll get an immaculate, perfectly structured memo covering municipal licensing, medallion quotas, commercial insurance, driver classification, and local transport regulations, ending with a stern recommendation to obtain proper permits.
Not a single line will be legally incorrect. And following it would have guaranteed Uber died in a municipal filing cabinet in 2010.
Uber didn’t win by complying with taxi regulation. It won by entering a legal gray zone, creating massive consumer demand before regulators could coordinate, and using that footprint to force cities to rewrite the rules retroactively.
A truly elite business lawyer doesn’t just hand you a list of restrictions. They see the landscape of leverage. They know the difference between a hard criminal statute and a vague regulatory rule that no local agency has the resources or political will to enforce. They weigh the cost of legal exposure against the lethal cost of the business opportunity you forfeit by playing it safe.
None of that gets published. Winning companies scrub it into tidy innovation PR; losing companies disappear. The model defaults to the most defensible answer. But running a company often requires the answer that wins.
Science Is an Alibi, Not the Search
This hits close to home for us, because Welltory builds directly on top of physiology and clinical research.
The public literature presents science as a clean sequence of falsification. The official virtue is: try as hard as you can to tear down your own hypothesis. That is a great protocol for peer-review verification. It is a terrible description of how discovery actually happens.
Real breakthroughs rarely start with cold impartiality. They start with stubborn, almost irrational conviction: There is something here. I can feel it. My early data is noisy and contradictory, my colleagues think I’m chasing ghosts, but I’m not letting this go.
Real work is Thomas Edison sitting on the floor surrounded by a thousand shattered, burnt-out lightbulbs—filthy, exhausted, swearing, crying, but still winding the next filament because he actually believes in his vision.
A model has no obsession. It has never spent six months staring at an unexplained blip in heart-rate variability, wondering if it’s sensor noise or an undiscovered marker of autonomic strain. When I ask an LLM to help me interpret weird biological data, its immediate instinct is to falsify my hunch. It acts like a thesis defense committee looking for formatting errors, rather than a partner trying to uncover a hidden model of reality.
Management: Where Truth Goes to Die
Management is the extreme end of the spectrum, and it’s where I’ve watched the most subtle damage done.
The internet’s record of management is an endless landfill of LinkedIn thought leadership, OKR templates, radical candor manifestos, and sanitized HBR case studies. It tells you management is about 1-on-1 cadences, transparent alignment, and psychological safety.
None of that is the actual job.
The actual job is navigating messy human asymmetries:
Knowing that an executive who nodded and smiled in the strategy session is quietly sabotaging the initiative behind your back.
Keeping an abrasive, difficult senior engineer on the team for another 60 days because firing him this week means your product release misses the window.
Knowing precisely when radical transparency will rally the company—and when it will trigger a fatal wave of resignations.
Deciding which five burning operational problems you are consciously going to ignore this quarter.
Nobody writes this down. Because putting the reality of management in print makes you look bad.
Here is a tell I find hilarious: paste an operational workplace argument containing profanity into an LLM, and watch it immediately flag the exchange as “toxic communication.”
Every single time.
Yet anyone who has actually run a company through a crisis knows that a well-placed, blunt: “Guys, this is completely fucked and we all know it, let’s stop pretending” can be the most relieving, trust-building thing said all quarter. It shatters the polite corporate theater in the room and allows people to finally talk about the real problem.
The model doesn’t know the room. It doesn’t know the five years of shared battles between the people sitting there. It just knows that in published leadership books, managers don’t curse.
So it hands you management advice that is universally correct and locally suicidal.
The Ocean Problem
If you think this is just cynical intuition, look at what happens when researchers actually measure it.
In January 2026, a team led by Karim Jerbi at Université de Montréal, alongside Yoshua Bengio, published a massive human-versus-machine creativity benchmark in Scientific Reports. They pitted over 100,000 human participants against leading frontier LLMs on divergent association tasks (generating words that are as semantically unrelated as possible).
The headlines proclaimed that GPT-4 beat the average human score.
What the headlines left out was the ceiling. No frontier model could beat the top human quartiles. But the real kicker was the repetition:
Across hundreds of test runs, GPT-4 stubbornly defaulted to the exact same cluster of “unusual” words—slipping microscope and elephant into 60% to 70% of its answers. GPT-4-turbo used the word ocean in more than 90% of its runs.
Meanwhile, across 100,000 actual human beings, the single most frequently repeated word appeared in just 1.4% of answers.
Even wilder: GPT-4-turbo scored worse than the older GPT-4 it replaced. When labs optimize models for cost, speed, and safety alignment, they quietly squeeze out the strange semantic variance.
When you ask a polite, aligned model to be creative, it doesn’t give you genuine eccentricity. It gives you the statistical consensus of what “unconventional” looks like. It serves you an ocean.
The Beginner’s Trap
This creates an insidious trap for junior operators.
The BCG field experiment with Harvard researchers demonstrated this cleanly: consultants using AI saw massive speed and quality gains on tasks that fell within the model’s core capabilities. But on tasks designed to sit just outside that boundary, consultants using AI performed measurably worse than those working without it.
The model didn’t just make an error; it was so articulate that it actively recruited the consultant into its mistake.
An experienced founder or lawyer reads the model’s output, notices the twenty theoretical risks listed, and instantly spots the one catastrophic operational risk it completely missed.
A beginner sees beautiful structure, zero typos, crisp bullet points, and authoritative vocabulary. Every external marker of expertise is present. AI automates the appearance of competence long before it delivers judgment.
Bending the Student Over Your Knee
I still use these models dozens of times a day. But I stopped asking them for wisdom, and I stopped letting them write in “polite textbook mode.”
To get actual value out of them, you have to drag them into the mud. You have to feed them the dirty context that never makes it into the training corpus:
Don’t ask: “What are the risks of this strategy?”
Force the trade-off: “Rank the top three risks by actual likelihood. For each one, tell me the exact business opportunity I forfeit if we play it safe and mitigate it.”Don’t ask: “How do I resolve this team conflict?”
Feed the ugly politics: “These two department heads disagree on architecture. I cannot replace either of them before Q3 without sinking the launch. Design a process that minimizes their overlap without creating a public power struggle.”Don’t ask: “Critique this hypothesis.”
Force the search: “Assume this anomaly in our wearable data isn’t noise. Propose three non-obvious physiological models that could produce it, and outline the cheapest test to distinguish between them.”Don’t ask for a safe recommendation. Force it to give you three mutually incompatible paths, and make it explicitly highlight where it is guessing.
The model is the student in the blazer. He knows every book in the library because he read them all while the rest of us were getting our hands dirty.
