Principal Agent, Automation Bargaining
Your AI Replacement, Trained on You
Your AI Replacement, Trained on You is a Principal Agent and Automation Bargaining scenario. The core lesson: Cooperating with the system that replaces you is rational in the short term and catastrophic in the long term. Your company introduces an AI assistant built partly from your documented workflows, decision logs, and past communications. They ask you to help refine it, review outputs, flag errors, improve prompts. DecisionPlay maps the players, payoffs, and equilibrium dynamics that shape how this situation typically resolves.
The situation
Your company introduces an AI assistant built partly from your documented workflows, decision logs, and past communications. They ask you to help refine it, review outputs, flag errors, improve prompts. Your expertise is making the AI better. The better it gets, the less they need you.
Background
Acemoglu and Restrepo's automation economics research documents a principal-agent conflict embedded in AI training: workers who cooperate accelerate their own displacement, while workers who refuse signal disloyalty and get displaced anyway on a slightly longer timeline. The optimal strategy depends on how quickly you can build skills the system can't replicate, and whether your organization is offering any path to that.
What this reveals
The automation cooperation trap
The request to help train the AI that will replace you is structured as a loyalty test when it's actually a knowledge extraction. Short-term cooperation protects your position through the transition window. But cooperation without negotiating for reskilling, role redesign, or enhanced severance means you've transferred your value for the price of a slightly longer runway. The asymmetry is that once the knowledge is transferred, your leverage disappears, and so does the conversation about what you get in return.
How to counter it: Name the exchange before you make it. The leverage exists exactly once: before you cooperate. 'I'll help with this if we can discuss what my role looks like on the other side' is a reasonable professional request. Organizations that can't answer that question have already answered it, they just haven't told you yet.
A question to sit with
If your expertise became transferable code tomorrow, which parts of your work would survive, and are you developing those parts deliberately?
Frequently asked questions
- What game theory concept does Your AI Replacement, Trained on You illustrate?
- Your AI Replacement, Trained on You illustrates Principal Agent, Automation Bargaining. Cooperating with the system that replaces you is rational in the short term and catastrophic in the long term.
- What is the situation in Your AI Replacement, Trained on You?
- Your company introduces an AI assistant built partly from your documented workflows, decision logs, and past communications. They ask you to help refine it, review outputs, flag errors, improve prompts. Your expertise is making the AI better.
- What does Your AI Replacement, Trained on You reveal about how people decide?
- The automation cooperation trap. The request to help train the AI that will replace you is structured as a loyalty test when it's actually a knowledge extraction. Short-term cooperation protects your position through the transition window. But cooperation without negotiating for reskilling, role redesign, or enhanced severance means you've transferred your value for the price of a slightly longer runway. The asymmetry is that once the knowledge is transferred, your leverage disappears, and so does the conversation about what you get in return.
- How do you avoid the trap in Your AI Replacement, Trained on You?
- Name the exchange before you make it. The leverage exists exactly once: before you cooperate. 'I'll help with this if we can discuss what my role looks like on the other side' is a reasonable professional request. Organizations that can't answer that question have already answered it, they just haven't told you yet.
- What is the research behind Your AI Replacement, Trained on You?
- Acemoglu and Restrepo's automation economics research documents a principal-agent conflict embedded in AI training: workers who cooperate accelerate their own displacement, while workers who refuse signal disloyalty and get displaced anyway on a slightly longer timeline. The optimal strategy depends on how quickly you can build skills the system can't replicate, and whether your organization is offering any path to that.
- How long does Your AI Replacement, Trained on You take to play?
- About 9 min, at core difficulty, across 4 decision points. It runs in your browser with no account and no sign-in.
Keep exploring
More Future Stakes scenarios, or browse all scenarios. New to this? Start with how DecisionPlay works or the game theory glossary.
Topics: futures, AI, automation, principal-agent, career, season-3