An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs
%22%2F%3E%3Ctext%20x%3D%22320%22%20y%3D%22250%22%20font-family%3D%22ui-sans-serif%2Csystem-ui%2Csans-serif%22%20font-size%3D%22200%22%20font-weight%3D%22800%22%20fill%3D%22rgba(255%2C255%2C255%2C0.95)%22%20text-anchor%3D%22middle%22%3EE%3C%2Ftext%3E%3Ctext%20x%3D%22320%22%20y%3D%22335%22%20font-family%3D%22ui-monospace%2Cmonospace%22%20font-size%3D%2226%22%20fill%3D%22rgba(255%2C255%2C255%2C0.8)%22%20text-anchor%3D%22middle%22%3Eturboderp-org%2Fexllamav3%3C%2Ftext%3E%3C%2Fsvg%3E)