Train an agent to spawn an auxiliary agent from the ongoing trajectory to select blocks of KV cache from history and delete at regular intervals.

My class notes on simulation modeling, based on the textbook Simulation Modeling and Analysis (Averill M. Law, 5th edition, McGraw-Hill).

What a discrete-event simulation consists of — entities, events, the event list, the clock, and statistical counters — walked through on a single-server FCFS queue.

A recap of the probability and statistics needed for simulation: probability axioms, random variables, convergence modes, stochastic processes, and estimation of means and variances.

Every random variate in a simulation is built from i.i.d. U(0,1) numbers; these notes cover where those numbers come from and what makes a generator good.

How to turn i.i.d. U(0,1) random numbers into observations from the distributions a model actually needs — the inverse transform and composition methods, and what makes a variate-generation algorithm

AI generated stuff. Does not worth reading. No training. LLM search based on heuristics.

Evaluation of existing KV cache evicting policies. KV cache eviction discards some instructions but keeps the others. The fix is to either whitelist instruction tokens or evict multiple instructions’

Instead of evicting KV cache with checking whether previous tokens contribute to the current one, learn to predict whether past tokens contribute to the future tokens. Prediction returns a sorted list

Given current token, use the query to compute attention with keys in previous tokens. Evict n tokens with least attention scores. Any token that is not relevant to current position is removed (but nev