Micro-Expert Router Inference Server
8.2
A specialized server application designed for efficient inference of Mixture of Experts (MoE) AI models, leveraging low-level systems engineering and predictive caching to achieve high throughput on standard CPU hardware.
240h
mvp estimate
8.2
viability grade
19
views
technology stack
C#
Rust
Difficult
inspired by
Running Mixtral 8x7B at 21+ TPS on Pure CPU