AOR-Bench: Do Large Audio Language Models Over-Refuse Pseudo-Harmful Queries?

Sun, 20 Sep 2026 00:00:00 +0000

Accepted to EMNLP 2026, Budapest, Hungary, 24–29 October 2026.

AOR-Bench asks a question that safety evaluation usually skips: not whether a model refuses harmful requests, but whether it refuses safe ones that happen to look harmful.

Over-refusal is a real cost of alignment. A model that declines benign queries is less useful, and the burden does not fall evenly — speakers whose accent, dialect or phrasing sits further from the training distribution are more likely to be refused. In the audio setting that risk grows, because prosody and acoustic ambiguity give the model more ways to misread intent than text alone provides.

Lost in Speech: Trilingual Spoken Hallucination Detection Across Audio and Transcripts

Sun, 20 Sep 2026 00:00:00 +0000

Accepted to the 2nd Workshop on Speech and Audio Language Models (SALMA 2026), co-located with EMNLP 2026 in Budapest, Hungary, 24–29 October 2026.

Most hallucination benchmarks assume text. Lost in Speech asks what changes when the claim arrives as audio — and works across three languages rather than one, since the answer depends on how well speech recognition serves each of them.

The comparison between audio and transcript is the point. Transcription throws away prosody, hesitation and disfluency that may carry signal about whether a speaker is fabricating, while adding recognition errors of its own. A detector evaluated only on transcripts can therefore look better or worse than it really is, and that gap is widest exactly where ASR is weakest — the low-resource languages that most need the detector to work.

Audio Language Models | Jason S. Lucas

AOR-Bench: Do Large Audio Language Models Over-Refuse Pseudo-Harmful Queries?

Lost in Speech: Trilingual Spoken Hallucination Detection Across Audio and Transcripts