abort_callback now stops whisper_full during language auto-detect
2 days ago
If you call whisper_full with the language set to auto, it runs a detection pass before it transcribes anything, one encode and one decode. Until this change, that pass ignored the abort callback you passed in. This pull request passes the callback down, so an abort during detection now stops the call before transcription starts.
Ask about this presentation
Chapters
Show transcriptHide transcript
If you call whisper_full with the language set to auto, it runs a detection pass before it transcribes anything, one encode and one decode. Until this change, that pass ignored the abort callback you passed in. This pull request passes the callback down, so an abort during detection now stops the call before transcription starts.
Slide text
[00:00:00.000 --> 00:00:21.754]
abort_callback now stops whisper_full during language auto-detect
whisper : run abort_callback on the language auto-detect path
Callers that set language = "auto" and pass an abort callback.
Here is the order inside whisper_full_with_state. It computes the mel spectrogram, and when the language is auto, it runs language detection first. The abort callback isn't passed there. It's only handed to the encoder and decoder in the main loop, so an abort asked for during detection wasn't noticed until the first encode of the main loop had finished. That's issue 3888.
The cause is in the two public wrappers detection was calling. whisper_encode_with_state takes no callback, and it hands nullptr to the internal encoder. whisper_decode_with_state does the same, after it prepares the batch and clears the KV cache. Adding a parameter to either one would change the public API, and the author set out to avoid that.
So the detection body moves into a static function, whisper_lang_auto_detect_internal, with two more arguments, the callback and its user data. The public whisper_lang_auto_detect_with_state becomes a one line wrapper that calls it with nullptr for both. Existing callers keep the same signature and the same behavior.
The encode now calls whisper_encode_internal with the callback. The test flips, because the wrapper returned zero on success and the internal function returns true. Decode takes more. The wrapper also prepared the batch and cleared the KV cache, so those two lines move into the helper, with n past fixed at zero, which is the value the old call passed.
And this is the line that connects it. whisper_full_with_state now calls the helper with the abort callback and the user data from its params. The callback is checked as soon as the detection encode finishes. When it says stop, the encode returns false, the helper returns minus 6, and whisper_full returns minus 3 with failed to auto-detect language in the log.
The evidence is a new test. It feeds two seconds of silence with the language on auto, an abort callback that returns true right away, and an encoder begin callback that only counts. whisper_full has to fail, and the counter has to read one. That one call is the check that runs just before detection, so the encoder in the main loop never began.
The fix lives inside whisper_full. If your code calls whisper_lang_auto_detect or its with state version directly, those still pass nullptr, so detection there still runs to the end. And the batch prep and KV cache lines now exist in two places, so a later change to whisper_decode_with_state needs the same change in the helper.
The test is registered with the tiny and gh labels and runs against the tiny test model, so the ctest run with the gh label covers it. If you already pass an abort callback to whisper_full with the language on auto, there's nothing to change. The abort callback now runs on the language auto-detect path.
