Research reveals that task-critical layers and positional encoding layers are located in opposite parts of Llama 3.1 transformers, contradicting previous assumptions.

Study tests co-localization hypothesis on Llama 3.1 8B, a 32-layer GQA model, to see if sensitive and positional encoding layers overlap

Introduces LS-LoRA method that targets task-sensitive layers using a novel correctness-differential metric for more efficient adaptation

Proposes GARFA (GQA-Aware RoPE Frequency Adaptation) that adds learnable scalar multipliers to optimize positional encoding in targeted layers

Discovers strong anti-localization pattern: task-sensitive layers concentrate in late layers (23-31) while RoPE-influential layers dominate early layers (0-9), with correlation of -0.735