New GRASS method enables memory-efficient fine-tuning of large language models by dynamically adjusting which layers to train based on task-specific importance.

GRASS uses gradient-based metrics to adaptively estimate layer importance during training, adjusting sampling probabilities in real-time rather than using static strategies

Addresses limitations of low-rank adaptation methods that sacrifice model expressiveness and performance compared to full-parameter fine-tuning

Improves upon existing layer-wise fine-tuning approaches by accounting for variations in layer importance across different tasks and training stages

Reduces GPU memory requirements for fine-tuning large language models without compromising downstream task performance