Speculative Decoding Optimizer
7.2
A library and set of tools to aid developers in implementing speculative decoding techniques in existing transformer models. Provides optimized draft models and evaluation metrics to facilitate faster inference.
180h
mvp estimate
7.2
viability grade
12
views
technology stack
C#
Rust
Difficult
inspired by
Speculative decoding is an inference optimization technique