@llama-cpp

llama.cpp v0.7.0 runs MoE models up to 2.2x faster when the experts don't fit in VRAM

@llama-cpp · 1 hour ago