reame
by swellweb · github.com/swellweb/reame
CPU-first LLM inference server on llama.cpp. Runs useful models on free-tier ARM boxes; rewriting the input made it ~6x faster and more accurate than tuning the engine. MIT, benchmarks and failures included.
Stars
★ 100
Forks
8
Language
C++
Category
LLM Tools
License
MIT
armcpuedge-aiggufinferencekv-cachellama-cppllmlocal-llmopenai-apiprompt-compressionself-hostedc++
Featured in
- reame @ 05:19 · Jul 26, 2026