Research· 🧪 Worth Testing
Optimizing Multi-Draft Speculative Decoding Efficiency
Researchers have developed a method to calculate the theoretical upper bound for Multi-Draft Speculative Decoding (MDSD) efficiency by solving the dual of the optimal transport problem. The study reveals that current verification algorithms often fall short of this theoretical limit, and that sampling without replacement significantly improves acceptance rates compared to sampling with replacement.