Routing GPU work by complexity instead of queue depth
Queue depth measures how much work is waiting. It says nothing about how heavy that work is.
- status
- accepted
- decided
- q1 2026
- considered
- round-robin · weighted queue
- systems
- nats · go
context
Queue depth only measures item counts. A single 4K upscaling job takes orders of magnitude more compute than ten thumbnail generations, yet looks like 1/10th the load.
decision
Score incoming requests by estimated memory bandwidth and compute units, routing to worker pools partitioned by model footprint.