Skip to content
Friday, 20 November 2026
Tenesys AI News
Subscribe

AI Research

Research papers, benchmarks, architectures, reasoning and training methods.

#Benchmark — 6 articles ✕

Research· Important· 🧪 Worth Testing

MCQ benchmarks artificially mask LLM safety refusals

Research shows that multiple-choice question (MCQ) formats suppress refusal behaviors in LLMs. Experiments on Llama-3.1-8B and Qwen3-8B reveal that forcing a specific answer letter pulls model activations 3x further from the refusal direction, meaning safety benchmarks based on MCQs may overestimate a model's safety.

arXiv cs.CL
Research· 🧪 Worth Testing

New Benchmark Reveals Retrieval Accuracy Doesn't Equal Long-Context Memory

A new research paper introduces the Simple Retrieval-Reconstruction Memory (SRRM) benchmark to better evaluate long-context performance in Small Language Models (SLMs). Findings show that as context length increases, a model's ability to reconstruct information degrades much faster than its ability to retrieve specific facts, proving that standard retrieval tests are insufficient for measuring true memory.

arXiv cs.CL

Ai2 ships open satellite embedding model OlmoEarth v1.3 with free global map explorer

Ai2 has published OlmoEarth v1.3, an embedding model that condenses a full year of radar and multispectral satellite data into a single compact vector for every 10-meter patch of the planet. Alongside it comes a free browser tool, the OlmoEarth Embeddings Explorer, that lets anyone compare landscapes across years or train tiny classifiers without touching a GPU. Benchmarked against AlphaEarth Foundations and Tessera v2 across 27 tests, the new model comes out on top overall while costing far less to train than its predecessors.

Hugging Face
Research· Important

Allen Institute Launches FairStress Benchmark to Expose Hidden Bias in LLM Decisions

Allen Institute for AI (Ai2) released FairStress, a benchmark that tests demographic fairness in LLMs not just by their final decisions but by how easily those decisions change under persuasive pressure. Testing open and frontier models including Olmo, Llama, Qwen, Gemma, Claude, and GPT found that models often abandon correct, well-supported decisions more readily when doing so favors a minority-coded candidate — a pattern the authors call 'anti-stereotypical over-correction.' The benchmark covers 300 scenarios across 29 domains (hiring, lending, healthcare, education, housing, criminal justice), expanded via demographic and persuasion variations into over 13 million test cases, plus a smaller 50K-item 'FairStress-Core' subset for faster evaluation.

Hugging Face
Research· Important· 🧪 Worth Testing

GitHub opens up ReviewBench to standardize how AI code reviewers get graded

GitHub has published ReviewBench, a research-preview benchmark designed to compare AI code review agents on a common, auditable standard. The dataset draws on 219 real public pull requests across 19 languages and 187 repositories, chosen to mirror the language and size distribution found across 103.9 million GitHub pull requests. Ground truth comes from a blended pool of human reviewers, static analysis tools, and multiple frontier LLMs, cross-checked by senior engineers who agreed with the benchmark's labels 96.6% of the time. GitHub says it has already used ReviewBench internally to track its own Copilot code review (CCR) product, and that offline benchmark movement reliably predicted the direction of later production A/B tests.

GitHub