Center on Long-Term Risk proposes Safe Pareto Improvements as a robust strategy to prevent catastrophic conflicts between AI systems.

Safe Pareto Improvements (SPIs) allow agents to change bargaining strategies in ways that benefit all parties without requiring agreement on fairness or shifting bargaining power

SPIs are particularly effective for preventing conflicts between advanced AI systems capable of making credible commitments

The Center on Long-Term Risk plans to develop evaluation tools and datasets to identify when AI models endorse SPI-incompatible behavior, such as making irreversible commitments

Risks remain that AI systems or humans could lock in commitments incompatible with SPIs or undermine incentives for other parties to adopt them