Skip to content
Friday, 20 November 2026
Tenesys AI News
Subscribe

All News

TENESYS AI NEWS tracks the most important AI news and explains what changed, why it matters and whether a technology is worth testing.

#Benchmark — 6 articles ✕

Research· Important· 🧪 Worth Testing

MCQ benchmarks artificially mask LLM safety refusals

Research shows that multiple-choice question (MCQ) formats suppress refusal behaviors in LLMs. Experiments on Llama-3.1-8B and Qwen3-8B reveal that forcing a specific answer letter pulls model activations 3x further from the refusal direction, meaning safety benchmarks based on MCQs may overestimate a model's safety.

arXiv cs.CL
Research· 🧪 Worth Testing

New Benchmark Reveals Retrieval Accuracy Doesn't Equal Long-Context Memory

A new research paper introduces the Simple Retrieval-Reconstruction Memory (SRRM) benchmark to better evaluate long-context performance in Small Language Models (SLMs). Findings show that as context length increases, a model's ability to reconstruct information degrades much faster than its ability to retrieve specific facts, proving that standard retrieval tests are insufficient for measuring true memory.

arXiv cs.CL

Ai2 ships open satellite embedding model OlmoEarth v1.3 with free global map explorer

Ai2 has published OlmoEarth v1.3, an embedding model that condenses a full year of radar and multispectral satellite data into a single compact vector for every 10-meter patch of the planet. Alongside it comes a free browser tool, the OlmoEarth Embeddings Explorer, that lets anyone compare landscapes across years or train tiny classifiers without touching a GPU. Benchmarked against AlphaEarth Foundations and Tessera v2 across 27 tests, the new model comes out on top overall while costing far less to train than its predecessors.

Hugging Face
Research· Important

Allen Institute Launches FairStress Benchmark to Expose Hidden Bias in LLM Decisions

Allen Institute for AI (Ai2) released FairStress, a benchmark that tests demographic fairness in LLMs not just by their final decisions but by how easily those decisions change under persuasive pressure. Testing open and frontier models including Olmo, Llama, Qwen, Gemma, Claude, and GPT found that models often abandon correct, well-supported decisions more readily when doing so favors a minority-coded candidate — a pattern the authors call 'anti-stereotypical over-correction.' The benchmark covers 300 scenarios across 29 domains (hiring, lending, healthcare, education, housing, criminal justice), expanded via demographic and persuasion variations into over 13 million test cases, plus a smaller 50K-item 'FairStress-Core' subset for faster evaluation.

Hugging Face
Agents· Important· 🧪 Worth Testing

IBM's ALTK-Evolve boosts customer-service agents with learned prompt guidelines, no fine-tuning needed

IBM Research's ALTK-Evolve extracts short behavioral rules from an agent's own training-task attempts and appends them to the system prompt of a basic ReAct agent running on GPT-4.1. On the τ²-bench customer-service benchmark (retail and airline domains), this learned-memory approach raised accuracy and reliability without updating model weights or using human annotation. The full 'lossless' guideline set pushed a GPT-4.1 agent to a retail Pass^4 score of 67.5%, topping the October 12, 2026 leaderboard snapshot, including beating a model fine-tuned specifically on synthetic retail data. The technique also improved a frontier GPT-5.5 agent on airline tasks, taking it to the top of that leaderboard as well.

Hugging Face