BenchMIRT is a new method for auditing LLM benchmarks question by question, revealing which capabilities they actually measure and helping researchers build smaller, more focused, and easier-to-interpret evaluations.
At an Ai2 event marking our expanded collaboration with Providence Swedish, researchers explored the hardest problems in AI-assisted science: keeping systems steerable, grounded in human judgment and sound methods, and responsive to new evidence and experiments.
Ai2 and Providence Swedish Cancer Institute are expanding their collaboration after AutoDiscovery helped researchers uncover and validate a promising new immune signal in invasive lobular breast cancer.
Thai researchers adapted Ai2’s open Dolma toolkit to build Mangosteen, a 47-billion-token Thai corpus that filters low-quality web data while maintaining or improving model performance and strengthening Thai cultural knowledge.
A Georgia Tech team used Ai2’s fully open Olmo stack to trace social reasoning back to the training data that shaped it, finding that dialogue-rich, interpersonal writing had an outsized influence on the capability.
Researchers used Olmo 3 and its open training data to show that models can infer a drug’s class from its name instead of knowing the specific medication, and traced that shortcut to how often drugs appeared in training.
TutorMoments is an open, replay-based evaluation framework that tests whether AI tutors can recognize when to support a student and when to hold back and encourage deeper reasoning.
Ai2 is expanding its partnership with Hugging Face to give its growing portfolio of fully open models, datasets, benchmarks, and applications the storage, bandwidth, and integrations needed to reach more researchers and developers.
Stony Brook researchers used our infini-gram engine to trace distinctive phrases in AI-generated writing back to existing sources, finding that top-selling self-published books on Amazon with substantial detected AI text overlap more heavily with rare language from previously published works.
How we built the OlmoEarth Platform to fine-tune geospatial models and run continent-scale satellite inference while managing massive data pipelines, distributed compute, and automatically recovering from failures at scale.
Why fully open models and research artifacts are essential to independent scrutiny, broader participation, and continued U.S. scientific leadership in AI.
Building Shippy taught us that reliable agents depend less on the model itself than on deterministic tools, explicit guardrails, isolated infrastructure, and evaluations grounded in real-world workflows and live data.
Danish Foundation Models is using FlexOlmo as the basis for FlexMoRE, a more efficient modular LLM architecture that lets institutions contribute specialized experts trained on sensitive or proprietary data without sharing that data—and run the resulting models on highly accessible hardware.
DiScoFormer is a transformer-based density and score estimator that can infer both quantities from a finite sample in one forward pass, generalizing classical KDE while staying accurate in high-dimensional and out-of-distribution settings without retraining for each new distribution.
New token-level analyses of Olmo 3 and Olmo Hybrid show that hybrid models predict meaning-bearing, context-dependent tokens better than transformers, while transformers retain an edge on verbatim copying.
Domyn and AISquared show how Ai2’s open releases are helping AI labs build models for regulated industries, where transparency, provenance, licensing, and control are essential for customer trust and compliance.
MolmoMotion is an open, language-guided 3D motion forecasting model that predicts how object points will move in the future, enabling stronger motion prediction for robotics, video generation, and other systems that need to reason about what happens next.
olmo-eval is an open evaluation workbench that helps model developers add, run, and analyze benchmarks across changing LLM checkpoints, extending OLMES from final-score reproducibility into the day-to-day model development loop.
PointCheck, an independent project, uses Molmo, MolmoWeb, and Olmo 3 to test web accessibility the way a keyboard user would—by navigating real pages and inspecting what's actually on screen.
research.io
Sign up to keep scrolling
Create your feed subscriptions, save articles, keep scrolling.
research.io