Models· 🧪 Worth Testing
Optimizing MoE Model Inference via Expert Streaming
Hugging Face has introduced expert streaming for Mixture-of-Experts (MoE) models within the Transformers library. This technique allows running massive models that would typically exceed standard hardware memory limits by streaming active experts on demand.