Balancing Compute in LLM Inference: Model Selection, Quantization, and Test-Time Scaling

Deploying large language models efficiently requires balancing accuracy, throughput, energy consumption, and memory usage. We present an empirical study of three complementary optimization dimensions for LLM inference: model size selection, posttraining quantization, and test-time compute scaling. Using the Qwen3 model family (0.6B-32B parameters) on NVIDIA L40S GPUs, we evaluate six quantization methods across a combination of six benchmarks and measure energy consumption per output token. We then apply budget-forced reasoning to quantized and unquan-tized models on the GSM8K benchmark. Our results show that model size selection provides up to 8× variation in energy per token, while quantization method choice accounts for up to 1.7×. Combining quantization with test-time scaling, a 4B model with a 2000-token reasoning budget achieves 90% accuracy on GSM8K, outperforming a 14B model (82%) while using 71% less memory and 2.7× less energy. We further demonstrate an adaptive thinking approach that allocates inference compute proportionally to problem difficulty, reducing average energy consumption while maintaining accuracy. These results provide practical guidance for configuring LLM inference systems that balance model performance and resource efficiency.