New approach uses human preferences to safely infer complex safety constraints in reinforcement learning without extensive expert input

Safe RL requires explicit safety constraints that are often complex, subjective, and difficult to specify in real-world applications

Existing constraint inference methods rely on restrictive assumptions or require extensive expert demonstrations, limiting practical applicability

Bradley-Terry (BT) models commonly used for preference learning fail to capture asymmetric, heavy-tailed safety costs, leading to dangerous risk underestimation

Proposed Preference-based Constrained Reinforcement Learning (PbCRL) approach offers a data-efficient alternative by learning constraints from human preferences rather than explicit specification