Shimmy is a pure-Rust local inference server and CLI for GGUF models. It exposes OpenAI-compatible endpoints, runs on local GPUs via WebGPU, and provides a single-binary alternative to Python-based inference stacks.
6023llama,llamacpp,llm-inferenceShimmy is a pure-Rust local inference server and CLI for GGUF models. It exposes OpenAI-compatible endpoints, runs on local GPUs via WebGPU, and provides a single-binary alternative to Python-based inference stacks.