WhimRead needs JavaScript for on-site search.
Whim
Read
browse at whim
Discover
All tools
Picks
Surprise
About
Search
中文
Agent infrastructure
qwen38-27b-rtx3090
syv-ai/qwen38-27b-rtx3090
An optimized vLLM serving stack for running large Qwen models fast on a single consumer GPU.
View on GitHub
Copy link
Topics
kv-cache
llm-inference
local-llm
quantization
qwen
qwen3
rtx-3090
speculative-decoding
Related projects
beellama.cpp
1.2k
BeeLlama.cpp is a performance-focused llama.cpp fork adding variance-normalized KV-cache quantization, precision tails, and low-bit cache types for faster local GGUF inference.
lucebox
3k
An open-source LLM inference engine tuned for AMD consumer hardware, with speculative decoding, long-context tricks and OpenAI/Anthropic-compatible APIs that coding agents can connect to.
project-maya
3k
Project Maya is a local LLM inference engine that runs large GLM, DeepSeek, and MiniMax models on your own GPUs, with quantized weights, a web dashboard, and OpenAI/Anthropic-compatible APIs.
mlx-dspark
742
A native MLX speculative-decoding inference engine that speeds up local LLMs on Apple Silicon.
pooled
647
Pooled runs large open models across browsers by splitting model layers among devices in a room over WebRTC and WebGPU. Users open a link to chat or use Code mode, and the room can expose an OpenAI/Anthropic-compatible API to other tools.
little-coder
2.7k
A coding agent tuned for small local models, built on pi with extensions for plan mode, sub-agents, skill injection and context compaction to work offline on a laptop.
2751
kv-cache,llm-inference,local-llm
An optimized vLLM serving stack for running large Qwen models fast on a single consumer GPU.
←
Previous
Tracearr
Next
→
LxgwNeoXiHei