LLM Compressor is a library for quantizing and compressing large language models for efficient vLLM inference. It ships GPTQ, AWQ, SmoothQuant and related recipes for weights, activations and KV cache.
3978compression,quantizationLLM Compressor is a library for quantizing and compressing large language models for efficient vLLM inference. It ships GPTQ, AWQ, SmoothQuant and related recipes for weights, activations and KV cache.