LLM execution requires moving gigabytes of data, which includes static model weights, the generated KV cache, and other intermediate representations. Depending on the deployment, this may involve communication across the network, PCIe links, accelerator links (e.g., NVLink), and memory interconnects, often leaving LLM execution bottlenecked by the data movement feeding it. An obvious mitigation for […]