Prefill / decode scheduling, old rule vs time-share balancer

Both sides get the same traffic: agent conversations taking turns, plus long cold prompts arriving at random or on demand. Costs are schedsim's calibration on the Qwen3.8-Flash-Next log. Orange blocks are prefill chunks (dark orange: a cold prompt's), blue are decode steps, purple are mixed chunks; the white line is total agent tok/s, thin lines one stream each.

t = 0.0 s
Traffic and deployment

OLD prefill priority (upstream)

queue

    streams

      NEW time-share balancer (this fork)

      queue

        streams