vLLM v0.31.0rc3 adds randomized dummy inputs to Model Runner V2
The preliminary release vLLM v0.31.0rc3 is out. It includes the change Model Runner V2: Support randomized dummy inputs, which adds support for random test inputs to the new version of the model runner.
What it means
For those who run models via vLLM on their own machine, this change affects the internal execution layer — Model Runner V2. Previously, fixed dummy inputs were used in test runs; now they can be generated randomly. This is useful for debugging and checking launch stability: the model receives more varied synthetic data instead of the same values.
The developer does not specify which randomization parameters are available and how this affects performance or memory consumption. The release is marked as a release candidate, meaning it is not a final stable version.
You can follow news about neural network launch tools in the CPU3D news feed.How the method works. The diagram was drawn based on this news note.