vllm-project/speculators
A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
Stars: 250Language: Python
Give AlbumentationsX a star on GitHub — it powers this leaderboard
Star on GitHubA unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM