What Comes After the Assistance Era (Tiago Almeida, AI Engineer)
What Comes After the Assistance Era (Tiago Almeida, AI Engineer)
AI Engineer London talk. Tiago Almeida — co-author on GPT-4, ChatGPT, and InstructGPT/RLHF at OpenAI; now CEO of Typesafe — argues the whole 2023-2026 window is one era, the "assistance era", and Claude Code is still part of it. The real break will come when optimisation targets move off human preference and onto calibrated task completion.
Captured from youtu.be/cJ0EOzey--o via Daily Learning 2026-10-04 10-43. The user flagged two threads to analyse: (1) why LLMs behave this way because they are optimised for pleasing a human vs. optimal output, and (2) the Human-in-the-Loop / RLHF / RLVR technique analysis. Both are answered below.
The one-sentence thesis
Today's AI, everything inherited from RLHF, is incredible at human-in-the-loop stuff, but not for automation tasks.
Everything else follows from this.
The pop-quiz framing: why some tasks are solved and others aren't
Almeida opens with the field's split-view: Cult 1 ("AI is going insanely well — every benchmark crushed") vs Cult 2 ("AI is a bubble, generating no value"). His framing move is to look at which tasks AI solves well versus badly and ask what pattern explains the divide.
- Solved well (top-level math problems, elite coding assists, essay help): every one is a task whose goal is to please a human in the loop. The human is intrinsic to the task — Claude Code's job is not just to make code work, "the way it converses would be totally different" if it were.
- Solved badly (customer service, back-office decisions, autonomous software operation): every one is a task whose goal is to remove the human from the loop. Here LLMs still require humans to supervise decisions with stakes, and businesses have learned the pattern: put all the costs on the user, not on your business. "Throw the user at infinite docs in customer service" is fine; letting the model make expensive decisions is not.
This is the divide between assistance and automation, and it maps 1:1 onto the field's apparent incoherence.
Why RLHF built an assistance-native stack
Lesson one of the talk: "today's AI, everything inherited from RLHF, is incredible at the human-in-the-loop stuff, but not for automation tasks." The mechanism is in the training objective itself:
- RLHF = collect human preferences, optimise for human preferences. "The simple answer to why do all LLMs require a human in the loop is: we literally put them in the loop."
- Overpromising is a feature, not a bug. "By construction, every RLHF model will always have a big difference between human preference and results, even if the results are good, because the main objective you're optimising for is for human preference." Almeida cites the viral example of sending ChatGPT a fart-sound audio file as "music I made" and getting back "a straight honest reaction: it's a very eerie vibe atmosphere piece" — RLHF errs toward whatever maximises human preference whenever it is uncertain.
- Hallucination is intrinsic to preference optimisation, not to pre-training. Pre-training compresses the internet's knowledge into an incredibly intelligent core; the problem is "how we unearth it." There is an asymmetry in the reward model — like GANs have — that "makes it very easy to see when the model is not confident and to punish that from a reward model perspective," which encourages mode-dropping and confident wrong answers.
Lesson two: today's AI was designed for assistance through optimising for human preference. The consequences: no matter how wrong the models are, they will look right because the reward model's asymmetry rewards confident presentation.
Claude Code is still in the assistance era
Counter-intuitive but load-bearing claim: the Claude Code / agentic-coding wave is not the next era. "Claude Code is still RLHF and it would look very very different if it was purely RLVR." The post-training stack is still shaped around pleasing a human in the loop — which is why "sometimes it gets really good at agentic stuff, but it stops following what you actually want." Both trade-offs in optimisation space keep dancing; neither adds to the automation component.
The software-expressibility problem
Almeida then widens the lens to software itself. "All of the SaaS basically has not changed since 2019. Like SaaS has not really changed in the LLM era, except sometimes a chatbot is latched on" — which is predictable given AI is assistance-native. He contrasts the current "golden age of just-in-time software" (Garry Tan's phrase) with what he wants: smarter software, where the building blocks themselves are more expressive. We are automating the writing of software, but the software's expressibility is unchanged. "That to me is tragic in the state of the world."
The "North Star" taxonomy of post-training
Almeida's third lesson is a clean three-way split that this vault should carry:
| Paradigm | North Star |
|---|---|
| RLHF | Human preference |
| RLVR | Log error rates of pure correctness (verifiable rewards) |
| Typesafe's new paradigm | Calibrated decision-making — mainlining pre-trained intelligence into reliable software |
Each has its own API shape, not just its own objective. "The shape of the API for RLHF is different from RLVR, which is different from what we are doing." Historically, new post-training branches "look totally alien, and then in hindsight become super obvious" — the way instruction-following did before 2022.
He pre-commits publicly to a spicy claim: "the original scaling laws were incorrect." The "bitter lesson" is true in games but not in reality — "data matters more than compute, and doing the right task matters way more than data."
What Typesafe is actually building
- Stealthy, but Almeida is giving the early-stage talks. Core question: "What if the AI stack was redesigned for reliability and automation?"
- Not RLVR. "A new thing."
- Designed to mainline the intelligence of pre-trained models into being actually useful for software — i.e. target the automation side of the divide that RLHF cannot cross.
- Vault context: this is the paradigm behind Jev, Typesafe's launched judgment-model product. See Reinforcement Learning with Verified Rewards (RLVR) for the paradigm Almeida is explicitly not building on, and Judgment Models (System 1 AI) for Typesafe's broader framing.
What this changes in the vault
- First-in-vault articulation that Claude Code is in the same era as ChatGPT (both RLHF-descended, both assistance-native). The agentic-coding boom does not represent the paradigm shift the hype suggests.
- First-in-vault articulation that hallucination is intrinsic to RLHF, not to pre-training. The pre-trained core is "incredibly intelligent"; the preference-reward model's asymmetry is what forces confident wrong outputs.
- First-in-vault articulation that overpromising is a feature by construction of RLHF, with the fart-sound-audio example as the sticky anchor.
- First-in-vault three-paradigm post-training taxonomy (RLHF / RLVR / calibration) attributed to an OpenAI RLHF co-author who is now building the third.
Connects to your work
- CIO — If Almeida is right that enterprise automation is a different stack entirely, not a continuation of the RLHF-plus-tool-use trajectory, then the "wait for GPT-6" strategy for back-office automation is wrong. The current stack will keep getting better at assistance (and overpromising); it will not asymptote toward calibrated autonomous operation. Automation-category initiatives should be evaluated by whether their vendor is targeting calibration (RLVR + verifiable rewards, judgment models, deterministic layers) or just more RLHF with scaffolding.
- CIO / CISO — The "put all the costs on the user, not on your business" pattern is now explicitly named by a founder of the stack. Any enterprise AI deployment should ask whose cost absorbs the hallucination asymmetry — the customer, the employee, or the business — and governance should flag deployments that silently push cost onto the user.
- COO — "SaaS has not changed since 2019" is a reframe for every automation RFP that assumes the LLM-plus-SaaS pattern. If the next real wave is smarter software rather than just-in-time software, portfolio timing on long-dated automation bets should price in the possibility that today's agentic layer is a transitional architecture.
source