通过可伸缩查找的条件记忆 大型语言模型的新稀疏轴
聚焦通过可伸缩查找的条件记忆 大型语言模型的新稀疏轴。Shazeer et al., 2017), which scales capacity via conditional computation. Owing to its ability to drastically increase model size without proportional increases in compute, MoE has become the de facto standard for frontier models (Comanici et al., 2025; Guo et al., 2025; Team et al., 2025).