Evaluation of existing KV cache evicting policies. KV cache eviction discards some instructions but keeps the others. The fix is to either whitelist instruction tokens or evict multiple instructions’

Instead of evicting KV cache with checking whether previous tokens contribute to the current one, learn to predict whether past tokens contribute to the future tokens. Prediction returns a sorted list

Given current token, use the query to compute attention with keys in previous tokens. Evict n tokens with least attention scores. Any token that is not relevant to current position is removed (but nev

A data science agent (Qwen3-8B backbone) trained end-to-end to (1) synthesize and reuse new skills and (2) decide when to compress its own context, via SFT distillation + multi-role GRPO.

I like the idea of insertion / deletion decisions of context management by on policy distillation, but there should be more analysis here. Also, the comparison is not fair. The compute is not saved. T

I like the story of this paper to optimize cost, but it spends a long time introducing cost optimization objective while it actually uses iterative prompt engineering, which seems to be totally irrele

去拉斯维加斯接触到这个东西, 查了下数学还挺有意思的 (没那么难也没那么简单那种), 一开始凭借直觉输了 70 USD, 于是回头推导了一下. 截图来自: 247blackjack.com.

Some websites are really distracting!

Reading notes of Yao’s notes of How does GPT Obtain its Ability? Tracing Emergent Abilities of Language Models to their Sources (notion.site).

Document a workable solution for the annoying CUDA Out Of Memory (OOM)