TaRo
Temporal candidate reuse for streaming video-language models
Reuse promising KV candidates across video frames, then rescore them with the current query while retaining the complete KV history. TaRo achieved 1.37–3.58× LLM-prefill speedups over ReKV in the evaluated settings.
View original figure
Temporal candidate reuse in streaming prefill

The original paper figure shows where TaRo runs in video prefill and how candidate reuse works. Stored candidate indices and current-frame causal keys are rescored with the current query; the selected Top-k entries feed attention. Query change or bank age triggers a full-history search and bank refresh. The illustrated bank size, key IDs, and scores are schematic.
TaRo manuscript · Figure 3 · Under review


