A developer running a point-tracking model (which builds 4D correlation volumes and uses transformers) found it ran ~170× slower on NVIDIA T4 GPUs compared to A100—taking ~85 seconds versus ~0.5 seconds per half-video.
GPU utilization is maxed at 99%, and the slowdown reproduces consistently across two different T4 machines, ruling out driver or setup issues and pointing instead to a fundamental architectural or kernel-level bottleneck between the two GPU generations.
What happened
A developer running a point-tracking model on NVIDIA T4 GPUs observed the same model taking ~85 seconds per half-video, versus ~0.5 seconds on an A100—a ~170× slowdown. The model processes 47-frame videos at 256×256 resolution in batch 1, uses pure FP32 precision, and builds local 4D correlation volumes followed by transformer layers.
Why it matters
Such an extreme performance gap between hardware generations suggests a bottleneck beyond typical generational differences. Understanding the cause matters for teams deploying models on T4 hardware (common in cloud inference and research labs) and for optimizing workloads that rely on dense matching and transformer computation on older GPUs.
What to watch
The developer has ruled out GPU underutilization (99% utilization confirmed), incorrect device placement, and driver inconsistencies across two independent T4 machines. The core question remains: whether the bottleneck stems from T4's architectural limits (e.g., memory bandwidth for 4D volumes, tensor core support), cuDNN/CUDA kernel efficiency on older hardware, or model-specific operations that scale poorly on the T4.
Ask the AI about this article →
The 170× slowdown between T4 and A100 is unusually large and points to a structural mismatch rather than linear performance degradation. Both GPUs show maxed utilization, so the bottleneck is not compute idleness; instead, it likely reflects architectural differences in how the two hardware generations handle the model's specific operations. T4s rely on different tensor core designs and memory subsystems compared to A100, and dense 4D correlation volumes combined with transformer inference may stress one or both of these in ways that do not scale linearly with raw FLOP differences. The fact that two independent T4 machines exhibit identical slowdown rules out isolated driver or firmware problems and suggests the issue is inherent to the T4 GPU architecture or the cuDNN/CUDA kernel implementations for this workload on older hardware.
For example, today's edition would include:
AI-summarized, only the topics you pick — one digest a day via Email, Slack, or Discord.
Free · takes 30 seconds · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. Q&As are published on this page for other readers too.
Aranya Inc., a startup founded last year, launched today with $11 million in funding
CBTS Technology Solutions LLC launched Forge Agents, a platform that turns a plain-language job description in…
Phonely Ltd. launched Alma, a large language AI model built for voice agents and trained on over 10 million re…
Imec CEO Patrick Vandenameele said at SEMICON Taiwan 2026 that the Belgian research center is broadening its c…
Alphabet's AI Overviews now reach over 2.5 billion monthly users through Google Search, and its ad business ge…
Sarah O’Connor's book 'We Are Not Machines' explores how mechanization and AI have transformed the workforce…
