OlmPool: How small architectural choices compound to undermine long context extension

OlmPool is a controlled suite of 26 models showing how small architecture choices can compound to make long-context extension much harder, even when training data and extension recipes are held constant.