Capabilities

I attended an economics class in Berkeley during my freshman year in San Francisco. A friend studying there invited me to go visit. He had to do something through the afternoon and left me in one of the halls. At the time I thought I wanted to do economics, so I was glad to find out one of the lectures in that hall was Berkeley's microeconomics course, ECON 100A, and decided to attend it during the Spring.

Ironically, the more I studied economics, the more skeptical I felt about it as a social science. I found it too constrained by theory and unreasonable assumptions.

It was nevertheless a fun subject to study. I have hundreds of pages from notes I would take on an oversized drawing tablet plugged into my computer that must've looked the silliest set-up of anyone in the hall.

My favorite concept I learned in that class is dead-weight loss, or DWL for brevity. Feel free to skip this section if you're already familiarized (click here).

Imagine you have a seller - say, a business - and many buyers that want to buy a good from that seller. We assume the seller wants to earn the most amount of money possible and buyers want to acquire as many units of the good as possible.

How does that good get priced? Some buyers might be willing to pay more for it than others, so the seller sets prices high. This seems reasonable, but if 8 buyers were able to pay $12 each while 2 were able to pay $40, setting the price of the good to $40 is a net loss at $80, as pricing it at $12 would net $96, leaving a $16+ difference on the table. Not to mention the social effects making the good available to more people might have¹.

Let's formalize this:

• In this example, the seller can price the good at any price in the range 1-100. This is called the supply curve.
• Some number of buyers can afford the good at a given price within the range. This is called the demand curve. For our example, we'll consider 10 buyers total, each able to afford it at a different price.

We can plot each of these curves against each other. This will be helpful for finding what is the optimal price a seller should sell their product at.

We see all 10 people can afford a unit of the good when the price is zero. Yet the seller can't offer it for less than $10 (this may be their break-even price). Perhaps they can offer the good at $80 for the two people who can afford it. What should they do?

The optimal price to quantity ratio lies in the intersection of the two curves. Intuitively, it's the maximum number of units the seller can sell at the highest price. Five units at $50 each nets $250. Much higher than the $160 from selling two units at $80, and higher than selling a lot of units at $20, also totalling $160. We can visualize this.

This is the scenario with the most given for the most paid, and is called a state of equilibrium. We can now talk about dead-weight loss (DWL).

Conceptually, DWL is the profit the seller could've made had they priced their product closer to equilibrium when the product is too expensive or cheap.

We visualize it as the area occupied between the corners of 1) the equilibrium point, 2) the point at the demand curve the good was priced at, and 3) the point on the supply curve at the same quantity. 2 and 3 are parallel to each other, making DWL look like a triangle. Let's visualize it for when the good is sold at $80.

The area in the triangle - the gap between the demand curve and the supply curve - represents goods the seller could've sold had they priced their product closer to equilibrium, letting them net a higher profit.

DWL on the other side of the graph is profit the seller is losing due to pricing the good at too low of a price. It's sometimes referred to as philanthropy - ok, no, but you get the idea. It's when consumers are benefiting at the cost of money the business could've kept for itself.

Congrats! You now know some undergraduate microeconomics. You can now ask: "Why doesn't Apple price their old stock lower given it's worth less and some people might still buy it? Doesn't that minimize DWL?" and realize the real world isn't as simple and interpretable as this model².

Ok, so why am I writing about DWL? I see some parallels between ideas in DWL Minimization and the current paradigms of coding agents.

I work extensively with coding agents. Almost every day I'm prompting, reviewing code, etc³.

I've observed that there is an inverse relationship between the skill needed [by a human] to accomplish a task and how good a model is.

Example: if you're working with a frontier model, it's generally sufficient to ask for what you want. "Fix this bug", "Add X feature", etc. If you're working with small models (with 30B parameters or lower), they need considerably more hand holding. Prompts need to be longer and specific because otherwise the models get lost in thought and churn their context windows before doing anything useful.

In other words: when a model is bad/limited at a task, humans supplement its deficiency with their own skills and steering. This is true for any task that frontier models aren't already good at or when using handicapped models. It's part of why Open Weight models aren't as big of a threat as they may seem in cybersecurity. You still need knowledgeable humans (and a ton of compute) to get the most out of them, at least for now.

We can map these concepts onto the first graph if we swap the Demand curve with Human steering and the Supply curve with Model cost. The x-axis represents how capable a model is relative to the task: at the very left, the model is too weak for it. At the very right, it's overkill. The y-axis represents what you pay. The height of the human line is how much steering the model needs from you, just as the height of the model line is what the model costs. The more capable the model, the less steering it needs and the more it costs.

Let's reintroduce the first graph with these changes.

You might think we always want to be on the rightmost side of it. I think that constitutes dead-weight loss for both economic and practical reasons.

One simple example is using frontier models for trivial changes that a smaller model could've made. E.g., changing the copy in a page or running trivial git commands (when was the last time you prompted 'yes, push it'?).

The economic argument is obvious: you're using frontier tokens for trivial tasks. Ideally, you'd want to save the cost.

The practical argument is that if token spend wasn't a concern, it would probably be faster if you used a smaller/faster model. Like if Cursor's Composer did it rather than have Astra think for a minute straight.

And even if your very smart model was ultra fast, that is expensive right now, so the economical argument still holds: others could do the same thing just as fast for cheaper, if we had the systems to transfer context efficiently (sort of pass the ball but for models).

I see this mattering in the short to medium term but not the long term.

When token subsidies stop, a system to auto-route prompts and tasks will become valuable to save unnecessary spending and time. Both for employees using AI and systems within the company that are heavily token-subsidized⁴.

Long-term, I expect frontier intelligence to become too cheap to meter and very fast. If the labs maintain token subsidies until that happens, it'll be less of an issue.

If frontier intelligence does increase in cost, I see three technical challenges need to be cleared for a "DWL-optimized harness" that chooses the right intelligence and speed for a task:

• Transferring context efficiently: how to transfer context between models cheaply and quickly, ideally at as close to zero cost and latency. This is most important for individual tasks.
• Parallelism: if many models are doing many tasks, how can inference providers maximize GPU utilization and KV Cache / model weights offloading and reloading so that every watt of compute is being routed to something at a given time.
• Model Profiling: you don't need to route requests each time if you already know what models sit at equilibria. To my knowledge, there isn't an all-purpose toolbench to do this yet.

I expect that while intelligence is subsidized, companies and individuals will use frontier models because it feels like the safe choice and it requires less steering. Budget-conscious organizations will make their own evals and perhaps even RL small models on them (like Ramp did).

But to diffuse AI into the wider economy at its current price, much work needs to be done on simple to use optimizers that small and medium sized organizations can use to save their token spend, both for use by individuals within an organization and for specific tasks.

Paraphrased: Jev might be very cheap but your average store clerk or non-profit worker isn't gonna get an API key and make an eval to see if it beats Astra at their classification task. Hence the need for diffusion companies.

In the graph, the equilibria makes sense if you think about it as the model with the best speed, price, and intelligence for the task. Getting there takes human effort: setting up the infrastructure and evals to find that model (both pre-selecting it or changing it at runtime). Whereas using a frontier model avoids you the work but costs a lot.

I expect this to change with harnesses that auto-route and simpler infrastructure for evaluating and training task-specific models. It is also nice that this is a defined engineering effort. If we get it right once, everyone benefits.

What about DWL on the left side of the graph?

I was recently experimenting with the Qwen family of models, running them locally on my computer with Ollama. It was disappointing. They know a lot and can pull off impressive tricks for a small model, but as soon as you throw a real task at them, they blow their context budgets overthinking unnecessary details.

I could only get them to do what I wanted by pointing out the exact lines of code they had to look at. I even had to change some code by hand because even though I'd tell them five times in a row to execute the change, it just thought, and thought, until its context overflowed.

Another dimension the graph doesn't capture is time⁵. It took me considerably longer to finish my project with Qwen because it needed so much handholding. I knew frontier models would've been done in way less time. It also doesn't capture the qualitative difference in mental load: it is so much easier to just prompt your way through. You don't really need to think.

Though, in my case, I chose to work my way through it because I wanted to experiment. In the real world, DWL on the left is visible on tasks that the models aren't good at yet. The odd finance benchmarks, ML research benchmarks, and computer use tasks. You still need people reviewing and steering the model's outputs.

In my opinion, if accuracy can be quantified and the task is economically valuable, they will be conquered eventually, likeliest first at the frontier, shifting to the right. This is true of tasks where labs and startups have an interest to introduce automation. It's part of the diffusion process.

Last, the analogy doesn't capture some qualitative aspects of models.

Like how I'd rather use Claude for anything conversational, or Codex for computer use. It doesn't capture how people begin to trust that a model will get it right 90% of the time and use it for specific work.

It assumes models are simply capable and without character. But it can be reconciled if we think of the x-axis as a model family, e.g., Haiku being underpowered, Sonnet generally balanced, and Opus often overkill.

Together, both parts of the graph paint a cohesive story: the frontier will lay the groundwork at the toughest challenges in an expensive, inefficient way, while diffusion makes them economically viable and fast.

I see two paths going forward for the future of work:

• Skills that can not be optimized against (yet) will become increasingly valuable. E.g., interpersonal communication, relationship building.
• If we do optimize everything optimizable (though I expect this to take really long), we'll make different problems for ourselves that require skills outside the scope of optimization (see this essay).

In either case, we'll be better off from it. It will provide healthcare and education for millions of people in underprivileged areas, enable massive social mobility - the works. That is, if we align it.