A local inference engine built on llama.cpp that streams routed MoE experts from flash, letting models larger than device memory run on phones, laptops and servers, with OpenAI- and Anthropic-compatible serving.
861android,cpp,edge-aiA local inference engine built on llama.cpp that streams routed MoE experts from flash, letting models larger than device memory run on phones, laptops and servers, with OpenAI- and Anthropic-compatible serving.