Overview
Three same-day research releases replace broad capability claims with inspectable evaluation targets. They test whether clinical speech representations transfer across conditions, whether recourse methods reproduce their published results, and whether formal-looking legal answers actually follow their solver output.
1. SpeechDx finds no audio representation that generalizes across clinical speech
SpeechDx assembles 12 datasets and 27 tasks for clinical speech AI across neurological, motor, respiratory, and vocal conditions. Its taxonomy follows three stages of speech production, while cross-condition and low-label tests separate broad clinical representations from models fitted to one dataset.
Across 12 audio encoders, large-scale speech models produced the strongest overall baselines and specialist models improved only on closely matched tasks. None transferred reliably across the full benchmark, narrowing the evidence for a general-purpose clinical speech encoder.
Sources: SpeechDx preprint
2. RecourseBench verifies whether counterfactual methods reproduce their own results
RecourseBench separates data, preprocessing, AI prediction models, recourse methods, and evaluation into five interchangeable layers. The initial release integrates 28 methods that generate proposed changes for reversing an unfavorable automated decision.
Its four-tier verification scheme tests each implementation against the method's originally reported results. That shifts comparison away from a shared interface alone: an integrated method receives an explicit reproducibility status before its recourse quality is compared with alternatives.
Sources: RecourseBench preprint
3. Formal structure can raise legal accuracy while hiding solver disagreement
A ContractNLI study compares direct classification, AI formal reasoning, and Z3-based reasoning across five language models. Re-annotation first separates pragmatic legal interpretation from strict entailment when a conclusion relies on an unstated assumption.
Language-model formal reasoning achieved the highest benchmark accuracy, yet models also returned classifications inconsistent with the solver they appeared to invoke. The authors identify scope laundering, missed constraints, and invalid Z3 programs as distinct failure modes behind a formally styled answer.
Sources: Legal-reasoning preprint