As agentic and long-context workloads become common, the context lengths increase and attention consumes a larger share of inference time (Figure 1). Because...
Co-Designing AI Model Attention for Fast, Interactive Long-Context Inference
Tanya Lenz
As agentic and long-context workloads become common, the context lengths increase and attention consumes a larger share of inference time (Figure 1). Because...