New framework uses discrete compression ratios to intelligently reduce computational demands of processing long texts in large language models

Existing soft context compression methods apply uniform compression ratios that fail to adapt to varying information density across natural language

Semi-Dynamic Context Compression framework introduces a Discrete Ratio Selector that predicts optimal compression targets and quantizes them to predefined ratios

The approach overcomes the challenge of models struggling with input-dependent, continuous structural hyperparameters by using discrete selections instead

The framework can be efficiently trained jointly with the compressor on synthetic data, using summary lengths as training labels