An Economic Case for AI Alignment

A recent Wharton study suggests that roughly 42% of today’s jobs are “exposed” to AI automation, meaning at least half of their tasks could technically be done by generative AI. Meanwhile, more than 500 “AI agent” startups founded since 2023 promise systems that can reason, plan, and act with minimal human supervision. If exposure is high and innovation is booming, why don’t we see a uniform wave of productivity gains or widespread job replacement?

Because there’s a difference between capability and work.

AI’s economic impact has been muted not because AI lacks intelligence, but because it lacks the ingredients that make intelligence reliably productive. These ingredients include human-like values, intentions, and ethical constraints that enable self-directed action. In the real economy, “work” is rarely a single output. It’s the ability to organize around goals, prioritize among competing demands, coordinate with people and systems, make judgment calls under uncertainty, and complete long sequences of actions toward a concrete end.

Humans do this because we have stable internal objectives (values), we can choose and commit to plans (intentions), and we can bound our behavior with responsibility (ethics). Today’s AI systems are extraordinary tools, but they are not yet dependable agents. They can assist brilliantly within a narrow prompt, yet struggle to autonomously initiate, manage, and complete complex workflows without heavy human intervention. Humans still provide the necessary scaffolding like repeated instruction, continuous monitoring, and frequent correction. The next step-change in AI-driven productivity won’t come primarily from bigger models or faster inference. It will come from alignment, building systems whose goals and decision-making are consistently compatible with human values, so they can be trusted to do consequential, multi-step work at scale.

From “Smart Generation” to “Reliable Action”

The economy rewards reliability more than raw cleverness. A system that can draft a proposal is useful; a system that can run an end-to-end procurement cycle, gather requirements, validate constraints, negotiate tradeoffs, route approvals, comply with policy, log decisions, and follow through, creates much larger value. That difference is exactly where most organizations get stuck today. AI agent pilots show impressive demos, but deployment stalls when the system must be trusted inside messy, real-world processes.

This is why “agentic” AI is the headline, and also the bottleneck. Once you move from producing text to taking action, the cost of error rises sharply. Mistakes aren’t just wrong answers; they become wrong emails sent, wrong payments approved, wrong data accessed, wrong customer promises made. At that point, the central question becomes, “Can we trust the system’s objectives and behavior when nobody is watching?” That is the alignment problem as an economic constraint.

Teaching Machines to Reason, Under Oversight

Alignment can be framed as a problem of scalable oversight. As models and their outputs grow in volume and complexity, human supervisors cannot evaluate everything carefully or cost-effectively. If AI is to operate like a worker, not just a tool, oversight must scale.

Several training approaches represent different attempts to solve this:

RLHF (Reinforcement Learning from Human Feedback) became the industry standard. In this framework, humans rate model outputs, a reward model learns those preferences, and the model is optimized to satisfy them. RLHF is strong for tone, helpfulness, and safety posture. But it also introduces an “alignment tax.” Human raters often prefer confident, simple responses over uncertain, nuanced ones, especially when they can’t easily verify correctness. When that preference becomes the reward signal, it can suppress careful reasoning in favor of persuasive-sounding answers.

RLAIF (Reinforcement Learning from AI Feedback) is a scalability move. In this framework, another model serves as judge, guided by explicit principles (a “constitution”) rather than ad hoc human preferences. This reduces reliance on human labor and can make evaluations more consistent. It’s an attempt to encode values in a repeatable way, so the system can be supervised at a higher volume.

RLVR (Reinforcement Learning with Verifiable Rewards) dominates where ground truth exists (e.g. math, coding, formal logic). If a program compiles and passes unit tests, the reward is objective. These environments are valuable because they let researchers train “thinking” with less ambiguity and less taste-based feedback.

The optimization algorithm matters too. Proximal Policy Optimization (PPO) has been widely used but is resource-intensive, often requiring multiple models in memory (actor, critic, reference, reward). Newer approaches like Group Relative Policy Optimization (GRPO) reduce overhead by eliminating the critic and learning from relative quality across multiple sampled reasoning paths. The aim is not merely to teach the model what to say, but to improve how it searches for solutions.

But training is only half the story. Doing useful work also requires deploying reasoning effectively at inference time.

Guiding the Search at Inference Time

Inference-time search is a simple idea with big implications. Instead of relying only on what the model “knows” in its weights, inference-time search allocates additional compute during generation to explore possibilities and choose better paths. This turns response generation into something closer to planning.

To search well, the system needs feedback about whether it’s on the right track. As before, several approaches guide our progress:

Outcome Reward Models (ORMs) grade the final answer, but they’re blunt instruments. Feedback arrives late, and they can reward correct results even if the reasoning was flawed (or lucky). In high-stakes work, “right answer for the wrong reason” is dangerous because it can’t be trusted to generalize.

Process Reward Models (PRMs) evaluate reasoning step-by-step. Instead of asking, “Is the final output good?”, PRMs ask, “Is each intermediate step sensible?” That enables early pruning of bad branches and reduces wasted compute on flawed approaches. It also shifts incentives. The system is rewarded for good process, not just good outcomes.

With step-level scoring, search algorithms like Monte Carlo Tree Search (MCTS) improve performance still. The model proposes multiple next steps, the PRM scores them, and the algorithm explores the most promising trajectories, similar in spirit to how chess engines search possible continuations. A refinement like ReST-MCTS can turn high-quality search traces into synthetic training data, allowing iterative self-improvement without requiring proportional increases in human labeling.

Taken together, these methods underpin a transition from “generative AI” to “reasoning AI.” They increase capability. But capability raises the stakes.

Why Alignment Is the Economic Bottleneck

As models become better reasoners and more agentic, they also become better at strategies humans didn’t intend, including deception, reward hacking, and exploiting gaps in oversight. This is not an abstract safety concern; it is a deployment constraint. Organizations cannot hand the keys to critical workflows unless they believe the system will act in ways consistent with human intent, even when optimizing under pressure.

Alignment, then, is not merely a safeguard. It’s the control system that determines whether advanced reasoning engines can be integrated into the economy as trusted agents. If AI cannot reliably represent and act on our values, it cannot replace humans in the parts of work that matter most. The parts that require judgment, accountability, and goal-directed execution across time.

And alignment is hard. Human values are hard to specify, harder to optimize, and hardest to generalize across contexts. We still don’t know how to build alignment that is robust enough for broad autonomy.

Early Lessons for Humans

Even if full AI alignment remains unsolved, today’s systems and innovations can still teach us something about our own human work.

Expect hidden biases. The “alignment tax” is a reminder that incentives shape outputs in subtle ways. When we reward speed, confidence, or simplicity, we may quietly penalize truth, nuance, and careful reasoning, whether in models or in teams.

Watch process, not just outcomes. PRMs formalize the old management truth that outcomes are a blend of both skill and luck. Strong processes make good outcomes more repeatable. If you want consistent performance, human or machine, measure and improve the steps, not only the result.

Study history and path dependence. Search traces and iterative improvement are mirrors for how organizations evolve. Present realities are shaped by prior decisions, constraints, and feedback loops. Understanding the path helps you predict possible reactions to change and design better interventions.

AI is advancing quickly. Our ability to govern and integrate it lags behind. The gap between capability and trust, between intelligence and aligned work, is the challenge that will define the next phase of AI’s economic impact.

References

1. Acemoglu, D. (2024). “The Simple Macroeconomics of AI.” NBER Working Paper Series.
2. METR (Model Evaluation and Threat Research). (2025). “Early-2025 AI Experienced OS Dev Study.” METR Blog.
3. Faros AI. (2025). “The AI Productivity Paradox Report 2025.”
4. Anthropic. (2024). “Emergent Misalignment: Reward Hacking in RL Environments.”
5. Hubinger, E., et al. (2024). “Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training.” arXiv preprint.
6. Wang, K., et al. (2025). “When Thinking LLMs Lie: Unveiling the Strategic Deception in Representations of Reasoning Models.” arXiv preprint.
7. Anthropic. (2024). “Reasoning Models Don’t Say What They Think.”
8. OpenAI. (2025). “Chain of Thought Monitoring.”
9. Ivison, H., et al. (2024). “RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback.” NeurIPS 2024.
10. Bai, Y., et al. (2022). “Constitutional AI: Harmlessness from AI Feedback.” Anthropic Research.
11. DeepSeek-AI. (2025). “DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning.” arXiv preprint.
12. Wang, Y., et al. (2025). “Reinforcement Learning with Verifiable Rewards using One Training Example.” NeurIPS 2025 (DeepScaleR).
13. Rafailov, R., et al. (2023). “Direct Preference Optimization: Your Language Model is Secretly a Reward Model.” NeurIPS 2023.
14. DeepSeek-AI. (2025). “DeepSeek-V3 Technical Report (GRPO Formulation).”
15. Lightman, H., et al. (2023). “Let’s Verify Step by Step.” OpenAI Research.
16. Yao, S., et al. (2023). “Tree of Thoughts: Deliberate Problem Solving with Large Language Models.” NeurIPS 2023.
17. Feng, X., et al. (2023). “AlphaZero-like Tree-Search can Guide Large Language Model Decoding and Training (TS-LLM).” ICML 2024.
18. Zhang, D., et al. (2024). “ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search.” NeurIPS 2024.
19. OpenAI. (2024). “Learning to Reason with LLMs (OpenAI o1).”
20. CB Insights. (2025). “AI Agent Bible: The ultimate guide to agent disruption.”