Skip to content
Friday, 20 November 2026
Tenesys AI News
Subscribe

All News

TENESYS AI NEWS tracks the most important AI news and explains what changed, why it matters and whether a technology is worth testing.

#Benchmark — 4 articles ✕

Research· Important· 🧪 Worth Testing

MCQ benchmarks artificially mask LLM safety refusals

Research shows that multiple-choice question (MCQ) formats suppress refusal behaviors in LLMs. Experiments on Llama-3.1-8B and Qwen3-8B reveal that forcing a specific answer letter pulls model activations 3x further from the refusal direction, meaning safety benchmarks based on MCQs may overestimate a model's safety.

arXiv cs.CL
Research· Important

Allen Institute Launches FairStress Benchmark to Expose Hidden Bias in LLM Decisions

Allen Institute for AI (Ai2) released FairStress, a benchmark that tests demographic fairness in LLMs not just by their final decisions but by how easily those decisions change under persuasive pressure. Testing open and frontier models including Olmo, Llama, Qwen, Gemma, Claude, and GPT found that models often abandon correct, well-supported decisions more readily when doing so favors a minority-coded candidate — a pattern the authors call 'anti-stereotypical over-correction.' The benchmark covers 300 scenarios across 29 domains (hiring, lending, healthcare, education, housing, criminal justice), expanded via demographic and persuasion variations into over 13 million test cases, plus a smaller 50K-item 'FairStress-Core' subset for faster evaluation.

Hugging Face
Agents· Important· 🧪 Worth Testing

IBM's ALTK-Evolve boosts customer-service agents with learned prompt guidelines, no fine-tuning needed

IBM Research's ALTK-Evolve extracts short behavioral rules from an agent's own training-task attempts and appends them to the system prompt of a basic ReAct agent running on GPT-4.1. On the τ²-bench customer-service benchmark (retail and airline domains), this learned-memory approach raised accuracy and reliability without updating model weights or using human annotation. The full 'lossless' guideline set pushed a GPT-4.1 agent to a retail Pass^4 score of 67.5%, topping the October 12, 2026 leaderboard snapshot, including beating a model fine-tuned specifically on synthetic retail data. The technique also improved a frontier GPT-5.5 agent on airline tasks, taking it to the top of that leaderboard as well.

Hugging Face