C3PO introduces a novel approach for optimizing expert pathways in Mixture-of-Experts (MoE) Large Language Models at test time, significantly improving accuracy by 7-15% through collaborative re-weighting of core experts in critical layers. By utilizing surrogate objectives based on successful neighboring samples, C3PO enhances efficiency, enabling models with fewer parameters to outperform larger counterparts. The method demonstrates superior performance over existing test-time learning techniques across various benchmarks.
mixture-of-experts ✓
pathway-optimization ✓
test-time-learning ✓
large-language-models ✓
+ accuracy-improvement