中文
Agent infrastructure

exllamav3

turboderp-org/exllamav3

An optimized quantization and inference library for running local LLMs on consumer GPUs, with EXL3 quantization, tensor and expert parallelism, CPU offloading, and OpenAI-compatible serving.