GPU-Tile-SIM: new simulator for LLM hardware design

The paper introduces GPU-Tile-SIM (GTSim), a simulation framework purpose-built for modeling the performance of GPU kernels used in large language model workloads. The motivation stems from a fundamental mismatch in how existing tools model GPU execution. Instruction-driven simulators, which simulate individual instruction execution, are costly to adapt each time GPU architectures change. Analytical models, by contrast, are too coarse-grained to capture the fine-grained behavior of modern LLM kernels that rely on sophisticated scheduling and memory-computation overlap.

The core insight driving GPU-Tile-SIM is that modern LLM kernel performance is governed less by individual instruction latency than by the dependency structure controlling execution order and overlap between computation and memory access. To operationalize this insight, the researchers designed the simulator around a warp-level tile graph representation. In this model, nodes in the graph represent tile-level operations (clusters of related computations), and edges encode both data dependencies and ordering constraints. The framework includes two main components: an automatic tile-graph frontend that extracts this representation from kernel code, and a graph-driven simulation backend that efficiently simulates execution.

The evaluation was comprehensive, testing GPU-Tile-SIM on three categories of workloads: representative GEMM (matrix multiplication) operations, attention mechanisms, and end-to-end LLM inference. The simulator was validated against both conventional kernels and highly optimized implementations on NVIDIA A100 and H100 GPUs. Results showed a Mean Absolute Percentage Error (MAPE) ranging from 1.22% to 8.71%, demonstrating high accuracy across diverse workloads and optimization strategies. The researchers further extended the framework to NVIDIA's Blackwell GPU architecture with preliminary validation, suggesting the approach scales to newer hardware.

Beyond accuracy benchmarking, the paper demonstrates GPU-Tile-SIM's utility in analyzing both software optimization choices (e.g., different kernel implementations) and architectural design trade-offs, positioning it as a practical tool for teams conducting joint hardware-software optimization of LLM accelerators.

Modern LLM kernels have become increasingly complex and highly optimized, relying on fine-grained scheduling and overlapping computation with memory operations to achieve high performance. This evolution has created a gap in GPU simulation tools: traditional instruction-driven simulators are expensive to adapt as GPU architectures evolve, while analytical models lack the granularity to capture the actual behavior of these sophisticated kernels. The key bottleneck identified by GPU-Tile-SIM's creators is that kernel performance is not primarily determined by individual instruction latency, but rather by the dependency structure that controls execution order and enables memory-computation overlap.

GPU-Tile-SIM addresses this by shifting focus from instructions to tiles—clusters of operations at a higher level of abstraction. By representing kernel execution as a warp-level tile graph (where nodes represent tile operations and edges represent data and ordering constraints), the framework captures the structural patterns that actually govern performance. This abstraction allows the simulator to achieve high accuracy (MAPE 1.22%–8.71%) on real workloads while remaining efficient enough to be a practical tool for hardware-software co-design. The extension to newer architectures like Blackwell suggests the approach is generalizable and may support the iterative design process required as GPU architectures and LLM workloads both continue to evolve.

No comments yet. Be the first to share your thoughts!