Benchmark ItModel Eval
Rerankers Aren’t Magic Either: When the Cross-Encoder Layer Is Worth the Cost
Cross-encoders enhance retrieval quality by re-ranking results, but they come with significantly higher computational costs. Their effectiveness is particularly noticeable when initial retrieval sets are weak, but the gains must justify the costs.
Jun 1, 2026
Read →EMO: Pretraining mixture of experts for emergent modularity
EMO is a mixture-of-experts model featuring 1 billion active parameters and 14 billion total parameters, trained on 1 trillion tokens. It allows users to utilize only 12.5% of its experts while maintaining near full-model performance. However, integration into existing workflows may be complex and costly.
May 11, 2026
Read →